The Wisdom of Crowds: Collaborative Student Teaching Instrument Development
This report highlights the use of an AACTE State Chapter Support Grant by the Ohio Association of Colleges for Teacher Education (OACTE). The views expressed in this post do not necessarily reflect the views of AACTE.
Educator preparation providers (EPPs) in Ohio have a longstanding history of collaboration. The 51 public and private institutions embrace the philosophy of the “wisdom of crowds,” that is, the power of decisions made by groups through collective sharing of information and resources (see this 2005 book by James Surowiecki). One of our ongoing collaborative efforts is the “VARI-EPP” (Valid and Reliable Instruments for Educator Preparation Programs) project, which aims to develop assessment instruments for use by any EPP in the nation to empower them with valid, reliable, and comparable data that may be used for program improvement. These types of instruments also address the Council for the Accreditation of Educator Preparation (CAEP) call for accreditation evidence collected from instruments that have been analyzed for validity and reliability.
One such instrument we recently developed and implemented is the Candidate Preservice Assessment of Student Teaching (CPAST) Form, made possible in part by a $9,000 AACTE State Chapter Support Grant awarded to OACTE. (The project was also supported by funding from Race to the Top and the University Center for the Advancement of Teaching at our institution.)
What Is the CPAST Form?
The CPAST Form is a coaching-oriented instrument that serves as a formative and summative assessment during the student teaching practicum. Based on InTASC, Marzano, Danielson, and others, the CPAST Form is an application rubric of research-informed best practices.
The 21-row rubric includes two subscales: pedagogy and dispositions. The rating scale includes four levels of performance, each with a detailed descriptor of observable, measurable behaviors, to guide scoring decisions. University supervisors take a self-paced, 90-minute training module on the scoring and use of the form. Programs also may use an introduction to the form designed for cooperating teachers (mentors) and student teachers (candidates). In addition, the form is accompanied by a “Look Fors” document that provides additional examples of the qualities and behaviors for a given level of performance.
Institutions using the CPAST Form are required to sign a memorandum of understanding and agree to submit their candidates’ de-identified data to a central database of scores. In return, they will receive access to comparison data (aggregated and disaggregated by race, gender, degree level, program area, and placement setting).
How Was the CPAST Form Developed?
The form development was recursive and occurred over 3 years. The form was developed and revised by a team of faculty and staff members from eight public and private EPPs. After a second year of implementation, validity and reliability analyses (including inter-rater reliability studies and content validity analysis, among others) were conducted. The results of these analyses informed revisions to the form, and in Year 3, the instrument was implemented with 1,203 teacher candidates in 23 EPPs. With these data, another round of validity and reliability analyses were completed.
How Did the AACTE Grant to OACTE Further This Project?
The AACTE-OACTE funds were instrumental to allow for the expansion of the project’s data set to include a greater representation of student teachers from around the state. Specifically, the money funded inter-rater reliability (IRR) studies on the form. University supervisors from public and private institutions participated in the IRR portion of the study. The IRR supervisors served as “secondary supervisors” for student teachers in a range of disciplines, teaching in a range of school contexts (urban, suburban, and rural). The quality IRR study would not have been possible without the AACTE-OACTE funding.
What Are Next Steps for the VARI-EPP Project?
Because educator preparation is a field engaged in continuous improvement through collaboration, we are sharing the CPAST Form with other EPPs across the nation; currently 40 institutions from eight states plan to use the form in 2017-2018. Our newest instrument development focus is for specialized professional association assessments. We are starting the validity and reliability studies for an assessment for secondary mathematics, National Council of Teachers of Mathematics (NCTM) Assessment #3, Planning.
Knowing the whole is greater than the sum of its parts, we anticipate collaborative efforts such as the VARI-EPP Project will be essential for using quality data to improve educator preparation programs.
Is your institution interested in using the CPAST Form or NCTM Assessment #3? We are happy to have you join our team! Please contact Erica Brownstein (Brownstein.2@osu.edu) for more information.
For more information on the VARI-EPP project, visit www.go.osu.edu/variepp.
For more information on the CPAST form, visit www.go.osu.edu/cpast.
Erica M. Brownstein is assistant dean for educator preparation and Carolyn Shemwell Kaplan is a postdoctoral researcher in the Ohio State University College of Education and Human Ecology.
Tags: accreditation, assessment, state affiliate
