Morehead State University
Item Response Theory: Implications for the Assessment of Pre-Service Teachers' Scientific Knowledge: STUDY 1 (Fugate): How Much Science Pre-Service Teachers Retained After Completing a Science Methods Course? An Approach from Item Response Theory
Institution
Morehead State University
Faculty Advisor/ Mentor
Wilson J. González-Espada
Abstract
Educational assessment is a process for obtaining information that is used for making decisions about the extent to which students gained knowledge, skills, and attitudes after the implementation of an instructional experience. Unlike the process of measuring overt variables in science, measuring covert attributes, like learning, requires an indirect approach. A discipline called Item Response Theory (IRT) has emerged to improve tests as a cognitive measurement tool. Specifically, it can be used to improve the quality of locally-made assessments used to evaluate pre-service science educators enrolled in college courses. This study applied IRT strategies to measure test and item parameters for a diagnostic pre- and post-test that is completed by pre-service educators enrolled in SCI 490, Science Methods for Elementary Teachers, between 2008 and 2013. This test measured content knowledge acquired in three pre-requisite courses, BIO 110 (Inquiry Biology for Elementary Teachers), SCI 111 (Inquiry Physical Science for Elementary Teachers), and ESS 112 (Inquiry Earth and Space Sciences for Elementary Teachers). Using SPSS, test reliability, test scores, test averages and standard deviations, item difficulty, item discrimination, item discrimination index, item means and standard deviations, item characteristic curves, and item distractor analysis were calculated and analyzed. The findings of the study informed the identification of life, physical, earth, and space science topics that pre-service teachers learned the best and those topics that need remediation prior to graduation. In addition, test items that do not meet the minimum quality requirements were modified or removed, increasing the overall validity and reliability of the SCI 490 diagnostic assessment. The availability of powerful statistical software packages allows college faculty to use IRT strategies, normally seen only in the preparation of large-scale standardized tests, to create more valid and reliable diagnostic assessments, like the one used in SCI 490.
Item Response Theory: Implications for the Assessment of Pre-Service Teachers' Scientific Knowledge: STUDY 1 (Fugate): How Much Science Pre-Service Teachers Retained After Completing a Science Methods Course? An Approach from Item Response Theory
Educational assessment is a process for obtaining information that is used for making decisions about the extent to which students gained knowledge, skills, and attitudes after the implementation of an instructional experience. Unlike the process of measuring overt variables in science, measuring covert attributes, like learning, requires an indirect approach. A discipline called Item Response Theory (IRT) has emerged to improve tests as a cognitive measurement tool. Specifically, it can be used to improve the quality of locally-made assessments used to evaluate pre-service science educators enrolled in college courses. This study applied IRT strategies to measure test and item parameters for a diagnostic pre- and post-test that is completed by pre-service educators enrolled in SCI 490, Science Methods for Elementary Teachers, between 2008 and 2013. This test measured content knowledge acquired in three pre-requisite courses, BIO 110 (Inquiry Biology for Elementary Teachers), SCI 111 (Inquiry Physical Science for Elementary Teachers), and ESS 112 (Inquiry Earth and Space Sciences for Elementary Teachers). Using SPSS, test reliability, test scores, test averages and standard deviations, item difficulty, item discrimination, item discrimination index, item means and standard deviations, item characteristic curves, and item distractor analysis were calculated and analyzed. The findings of the study informed the identification of life, physical, earth, and space science topics that pre-service teachers learned the best and those topics that need remediation prior to graduation. In addition, test items that do not meet the minimum quality requirements were modified or removed, increasing the overall validity and reliability of the SCI 490 diagnostic assessment. The availability of powerful statistical software packages allows college faculty to use IRT strategies, normally seen only in the preparation of large-scale standardized tests, to create more valid and reliable diagnostic assessments, like the one used in SCI 490.