Open Science and Education Research
The motto of the Royal Society, “Nullius in verba” translates as “Take nobody’s word for it.” However, when educators read journal articles reporting research findings, transparency is limited. For example, educators only see authors’ reports of findings, and do not have access to data (to verify reported findings) or to the details of the research procedures (to examine, for example, whether researchers developed their hypotheses after knowing the results). This lack of transparency is potentially problematic, because researchers may be driven to find and report positive and significant findings to enhance the odds of publication. Indeed, many educational researchers report engaging in questionable research practices that might help generate positive, but potentially biased, research findings (Makel, Hodges, Cook, & Plucker, 2019). Open science is an umbrella term, encompassing diverse initiatives that aim to increase transparency in research. We briefly describe three open-science practices here: pre-registration, Registered Reports, and open data.
Pre-registration involves researchers publicly posting (e.g., on the Registry of Efficacy and Effectiveness Studies) their hypotheses, research plans, and planned data analysis beforeconducting the study. In this way, changes made to a study’s hypotheses, research methods, and data analyses are discoverable by research consumers. Pre-registration does not mean that researchers cannot make changes to their studies. Rather, pre-registration encourages researchers to only make justifiable changes and to be transparent about them. Presumably because it minimizes the occurrence of questionable research practices, pre-registration has been shown to increase the likelihood of researchers reporting null findings (Kaplan & Irving, 2015).
Registered Reportstakes the idea of pre-registration and uses it as the basis for peer review. With Registered Reports, researchers submit their research plans to a journal before a study is conducted. The decision whether to accept the study for publication, then, rests on the quality of study design and the importance of the research questions, not the direction of the findings (because findings are not yet known). Reviewers of Registered Reports are able to give prospective feedback to improve a study, rather than simply critiquing a study after it is completed. After a Registered Report is granted in-principle acceptance, researchers conduct the study, write it up, and submit it for a second round of review, which simply involves reviewers ensuring that the study was conducted as designed (or any changes were clearly described and justified). Like pre-registration, Registered Reports are associated with a higher prevalence of null findings than traditional research (Schijen, Scheel, & Lakens, 2019).
Finally, open data involves researchers sharing, to the degree that it is ethically feasible, study data (with any potentially identifying information masked) on repositories such as the ICPSR. Sharing data allows others to (a) reproduce and verify study analyses, (b) extend the study by analyzing data to explore new research questions, and (c) combine data with that from other studies in research syntheses. Without shared data, research consumers are forced to take the researcher’s word for their findings, in contrast to the Royal Society’s motto.
Open-science reforms are not without obstacles. However, they may help increase the transparency and trustworthiness of the research that guide policy and practice in education and teacher preparation. As such, we hope that researchers will engage in open practices, and that professional organizations—such as AACTE—will encourage and reward open practices.
Tags: research
