Providing Assessment Feedback via Multiple Model-based Diagnoses

Presented by Laine Bradshaw, Assistant Professor, Department of Educational Psychology, The University of Georgia, Athens

Diagnostic classification models (DCMs) are a newer class of psychometric models well-suited for providing multidimensional feedback under realistic testing conditions. In the literature, data from many tests have been fit retrospectively to DCMs, often with inadequate model-data fit. This presentation will share results from a multi-year NSF-funded grant where our interdisciplinary team of researchers developed a multidimensional test built from the ground up for use with DCMs. The test diagnoses middle grades mathematics teachers’ mastery with respect to four components critical for reasoning about fraction arithmetic, especially multiplication and division. These components, or attributes, are very fine-grained and were identified through an analysis of the extensive mathematics education research literature on teachers’ and students’ reasoning in this content area. We administered the test to a national sample of 990 middle grades teachers in the US and analyzed the item responses using a DCM. This presentation will briefly overview DCMS; provide results describing the diagnostic quality of the test at the item level, mastery classifications for teachers, and attribute relationships; and discuss implications from both a mathematics education and psychometric perspective.

Related reference: http://onlinelibrary.wiley.com/doi/10.1111/emip.12020/abstract

Related podcast: http://mathed.podomatic.com/entry/2014-01-26T13_39_12-08_00