Outlier Research & Evaluation Joins DSI, Deepening Focus on pK-16+ Data Science Education Research
Since 2005, researchers at the University of Chicago’s Outlier Research & Evaluation have closely examined STEM education. Outlier conducts research on equity and the implementation, scaling and durability of educational innovations, evaluates programs in K16 school and informal settings such as museums, and supports planning and policy development at all levels. With funding from federal agencies, foundations, corporations and educational organizations, Outlier has helped dozens of partners determine how well their programs work for diverse populations in different environments.
In late 2022, Outlier joined the Data Science Institute to expand their work into the emerging field of data science. The new partnership positions Outlier to assess new data science curricula and educational programs sprouting around the country, while also developing and incorporating new data science approaches into their research process. Additionally, Outlier will apply its expertise to evaluating DSI programs that aim to bring data science education to a broader population of students.
“One reason why I’m excited to join the DSI is because the questions we want to ask and answer are far beyond the methodologies we currently have available to us,” said Jeanne Century, director of Outlier Research & Evaluation and research associate professor. “We’re less interested in knowing only if an innovation works, and more interested in knowing what parts of an innovation work for whom, under what conditions.”
Outlier’s core approach is to measure implementation, spread and sustainability of innovations, going beyond binary conclusions of whether a program is successful or not into deeper understandings of its adaptability, and the extent to which it serves students equitably. The team uses a range of data sources, including surveys, focus groups, and observations, as well as administrative data such as student demographics, grades and test scores to do research and create in-depth evaluations for partners. Outlier produces reports, data visualizations, peer-review publications, and other research products that help disseminate findings and recommendations.
Over their history, Outlier has worked with organizations as varied as IEEE, the Lincoln Park Zoo, Google, and the U.S. Department of Education. In 2014 and 2015, they worked with non-profit Code.org to evaluate the organization’s efforts to expand computer science education in K-12 schools nationwide.
That work led to a long-standing partnership with the Broward County (FL) Public Schools. In 2016, with funding from the National Science Foundation, they partnered to develop and research the “Time for CS” program for integrating computer science instruction into elementary school days using problem-based interdisciplinary units. This work continues in collaboration with curriculum writers at UChicago STEM Education and the University of Florida focusing on improving the materials by including specific teaching practices that are culturally responsive and that support neuroatypical learners. Outlier is also working with Broward County on a study called “DIG” that is examining how to spread teaching practices that support critical thinking.
Locally, they have worked in Chicago Public Schools, with the Network for College Success, the Chicago Learning Exchange and Lumity on programs promoting STEM education, digital media and technology use, and equitable student outcomes.
Over the course of these and dozens more projects, Century has developed a paradigm of “component-based research” (CBR). Taking inspiration from methods used in public health, medicine, and economics, CBR breaks down educational innovations into smaller parts, such that researchers can determine which aspects of a program are most effective, in different contexts or with different populations of students.
“It is a way of doing research that can enable researchers to rigorously answer the challenging questions that will move the field toward improvement and more equitable practice: What parts of a reform work, for whom, under what conditions and for how long?,” Century wrote in a recent white paper. “A CBR paradigm positions researchers to embrace the complexity of improving education — rather than reduce it to fit the limitations of our current research approaches.”
Fulfilling this vision will require advances in data collection, statistical analysis, and modeling, which the Outlier team hopes to explore in partnership with DSI researchers. The team is also applying its evaluation framework to DSI education programs, including the Data Science Clinic course, Summer Lab, and Data4All. Century is part of the data.org Capacity Accelerator Network project, which is developing flexible data science curricula and tools for use in various educational and regional settings, and has started a UChicago K-12 Data Science working group to gather similar-minded groups around campus.
In addition to Century and her over 35 years of research experience, the Outlier team includes Dr. Steven Baker, a Senior Evaluation & Research Scientist with over 30 years of experience and expertise in evaluation and the implementation and impacts of educational innovations in school and out-of-school settings; Lead Research & Evaluation Associate Dr. Joanna Schiffman, whose expertise lies in cognitive development with a focus on how experiences and environments impact STEM learning; and Lead Research & Evaluation Associate Dr. Huifang Zuo, who is highly trained in educational measurement, statistics, and GIS.
Under the new partnership with DSI, Century sees an opportunity for the team to deepen its commitment to equitable education and expand its network into new areas.
“All of our work is focused on equity in education” Century said. “I would love to see us working with anybody interested in doing education research differently, who is willing to innovate, and who is interested in exploring data science and education research.”