AbstractsPsychology

This study investigates the extent to which an assessment can diagnose learner misconceptions in the domain of statistics. A Statistics Concept Inventory (StatCI) was created using an evidence-centered design framework (Mislevy, Steinberg, & Almond, 2003). This assessment draws from a comprehensive literature review of student thinking in statistics, in which clusters of items correspond to a major concept, and each distractor maps onto a misconception. Professors and graduate students with expertise in statistics checked the assessment for face validity. Four studies were run to validate aspects of this assessment. First, a student protocol study was conducted to examine how students interpreted the items. Second, a 30-item pilot version was administered to 100 participants on Amazon Mechanical Turk. Preliminary psychometric tests were run on these data to identify items to modify and delete. A beta version was administered to another 100 participants. Finally, an updated was administered to 750 participants. Participant performance data was analyzed for response patterns demonstrating conceptual and errorful thinking. In particular, data was analyzed by items, conceptual structure, distractors, and demographic groups. These results provided evidence that the assessment is measuring the targeted constructs and is able to identify learner misconceptions and errors. The outcomes of this program of research included: (1) a design pattern template that can be broadly applied to create other assessments in statistics; (2) a final assessment instrument that can be used in undergraduate first-year statistics courses; (3) a methodology for applying ECD to concept inventory design. Advisors/Committee Members: Pellegrino, James (advisor).