The Science of a DREAM ChallengeAdmin2021-03-23T17:56:04+00:00
DREAM Challenges aim to address fundamental questions in the area of biomedical research. Questions can be in any field as long as they are of broad interest to the research community and finding solutions can have a high impact on human health or understanding biological questions. Researchers from around the world can propose questions, which the DREAM Challenge Directors then evaluate.
Once we identify a Challenge question, we assemble a team of international experts to serve as an organizing team. This team rigorously assesses the issues that might arise with addressing the question, determines the best way to evaluate the solutions, and most importantly, identifies the appropriate data to be used for the Challenge. Importantly, a subset of the Challenge data is not public and is used to evaluate the solutions of participants so that a rigorous and unbiased assessment of the Challenge solutions can be made.
After the Challenge web page is live, the Challenge is open to the world. Anyone with a computer and internet access can participate and we encourage individuals to assemble into teams. During the open phase of the Challenge, teams develop their solutions and submit them to a “leader board” where they are scored and ranked on performance. Teams get time to learn from their solutions and then are allowed a final submission.
Upon close of a Challenge, the final solutions from the participating teams are evaluated, ranked, and assessed by the Challenge Organizing Team. The goal is to identify methods that both solve the Challenge and provide novel insight into the proposed question.
Fundamental goal of the DREAM Challenges are to accelerate research, democratize data, and provide robust and unbiased assessments of Challenge solutions. Thus, the next major goal is to disseminate the data, insights, and solutions to the world through academic publications. The life of a DREAM Challenge does not stop at publication. By providing the question, the data, and the software to evaluate solutions, we are providing a living system that allows for researchers from around the world to continue to improve upon solutions. DREAM Challenges are also an excellent mechanism for educators, where teachers can use the material provided through Synapse in their classrooms.