The Science of a DREAM Challenge

DREAM Challenges aim to address fundamental questions in the area of biomedical research. Questions can be in any field as long as they are of broad interest to the research community and finding solutions can have a high impact on human health or understanding biological questions. Researchers from around the world can propose questions, which the DREAM Challenge Directors then evaluate.

Once we identify a Challenge question, we assemble a team of international experts to serve as an organizing team. This team rigorously assesses the issues that might arise with addressing the question, determines the best way to evaluate the solutions, and most importantly, identifies the appropriate data to be used for the Challenge. Importantly, a subset of the Challenge data is not public and is used to evaluate the solutions of participants so that a rigorous and unbiased assessment of the Challenge solutions can be made.

After the Challenge web page is live, the Challenge is open to the world. Anyone with a computer and internet access can participate and we encourage individuals to assemble into teams. During the open phase of the Challenge, teams develop their solutions and submit them to a “leader board” where they are scored and ranked on performance. Teams get time to learn from their solutions and then are allowed a final submission.

Upon close of a Challenge, the final solutions from the participating teams are evaluated, ranked, and assessed by the Challenge Organizing Team. The goal is to identify methods that both solve the Challenge and provide novel insight into the proposed question.