JTE Author Interview: Fives and Barnes on Teaching Assessment Construction
Have you seen the JTE Insider blog managed by the Journal of Teacher Education (JTE) editorial team? Check out the latest author interview below.
This interview features insights from the JTE article, “Informed and Uninformed Naïve Assessment Constructors’ Strategies for Item Selection,” written by Helenrose Fives and Nicole Barnes of Montclair State University (NJ). The article is featured in the January/February 2017 issue of JTE; you can read the article by going to this link (and AACTE members have free access through this login link).
Q: What motivated you to pursue this particular research topic?
A: The genesis of this project was an argument we had with each other one Friday evening at a local watering hole. We were discussing what content should be included in our master’s-level assessment course for preservice teachers in our MAT program. Specifically, we were debating over the merits of including instruction on the table of specifications in the course – something we both agreed took substantial class time to teach, but we were unclear if the rewards of that time were worth it.
In this discussion of what to teach in a finite amount of time, a second issue – learning to write items at varied cognitive levels – was offered as something that teachers really needed to know and had a very difficult time learning how to do. Thus, there we were in a circular argument, one of us arguing that writing items was irrelevant if the test was poorly planned and the other arguing that a well-planned test with poor items was useless. Stymied in our particular perspectives, one of us looked up (we both claim to be the genius here) and announced – “This is an empirical question; we can find out who is right!”
Two studies later it turns out we both were. In our first study (DiDonato-Barnes, Fives, & Krause, 2013) we were able to ascertain that preservice teachers who read an article on the table of specifications were able to construct tests with better content validity evidence than those who did not read the article. However, no differences emerged across the two groups with respect to response process evidence, that is selecting items at the correct cognitive level for the lesson objective. This JTE article is an extension of this work where we tried to identify what strategies these preservice teachers actively employed when constructing their test. Other than textbook suggestions for common recommendations for item construction and using a table of specifications (Fives, Barnes, Dace, & Gillis, 2016), there is very little empirical research on the strategies teachers use when constructing classroom tests and less evidence indicating which, if any, strategies would be more or less effective in practice. This article provides some initial insight into this area of research and practice.
Q: Were there any specific external events (political, social, economic) that influenced your decision to engage in this research study?
A: Assessment has received a great deal of attention, but the focus has been primarily on standardized or other high-stakes, external assessments. This often ignores, or undervalues, the work teachers do in their classrooms around assessment, which (we believe) is equally (maybe even more) important. Classroom assessment does, in many ways, carry high stakes for students in terms of their grades, access to advanced placement courses, and class rank. Teachers are more likely to rely on the results of classroom assessments to make decisions about student performance and their next steps in teaching. In this study, we were interested in delving deeper into the strategies teachers use to evaluate the quality of test items, so as teacher educators we could build on and refine these processes in our courses and share these ideas with the teacher education research and practice community.
Q: What were some difficulties you encountered with the research?
A: In this experiment, participants in the treatment condition read an article on how to construct the table of specifications (Fives & DiDonato-Barnes, 2013). As teacher educators we are aware that this was not the best way to teach this material to participants. In our future extensions of this work, we plan to teach participants to use the table of specifications using a more constructivist-inspired format.
Q: Writing, by necessity, requires leaving certain things on the cutting room floor. What didn’t make it into the article that you want to talk about?
A: Nothing. We publish everything … eventually :). Seriously though, we found the review process with JTE while challenging and rigorous to also be incredibly helpful to us in presenting our research findings in a coherent and streamlined fashion. As we reflect back on our initial draft of the work and the final product, we did not really lose any of our core findings or recommendations, and in some instances, due to the feedback we received from the reviewers, we were able to better justify and explain our work.
Q: What current areas of research are you pursuing?
A: Our research is dedicated to examining and improving teaching and learning in K-12 settings through the study of classroom assessment, learning, and motivation applied to theory and practice. We see assessment, learning, and motivation as fundamentally integrated in teaching and learning, and while typically explored independently, our overriding goal is to develop deep understandings of how these processes interact and can be enhanced in actual classroom practice.
Q: What new challenges do you see for the field of teacher education?
A: We are uncertain if the challenges facing teacher education are really “new”; many just have a new name but have been a part of this system for years. For instance, the relationship between research and practice is one that seems exacerbated by the availability of vast amounts of information and limited skill in discerning the value and potential use of that information. Many of the introductory courses and texts offered for preservice teachers tend to provide general overviews of teaching practices that seem to lead to a tendency towards relativity – everything is equally important or not – and little of the nuanced findings from research are expressed in ways that teachers can use them in their work in classrooms. This is further challenged by limited opportunities to engage with teachers in classroom-level research where the application of theory must be modified and adapted to work within the complex context of school life. As teacher educators, we need to continue to explore ways to make content both accessible and meaningful to teachers without diluting the intricacies of the research findings.
Q: What advice would you give to new scholars in teacher education?
A: Find good research partners and nurture those relationships in ways that honor both the work and your own humanity. Collaboration requires a unique relationship that must be based first and foremost on trust between the collaborators. Trust allows for disagreement about ideas and a willingness to challenge each other on the design, analysis, and theoretical developments. Without trust and our shared commitment to good research, our scholarship would not be all that good. Moreover, frequent discussions about work effort, authorship, and goals have helped us to develop a meaningful partnership.
Tags: assessment, JTE, research
