Lorrie Shepard - US grants

The project will assess the validity of the NAEP in mathematics by testing the adequacy of the NAEP for detecting effects of mathematics education reform (using the "Connected Mathematics Program") by comparing student achievement on the NAEP, state assessments (Arkansas' use of the ITBS and the Colorado state assessment), and the NSF supported "Balanced Assessment in Mathematics (BAM)." The project hopes to determine whether large-scale assessments are sensitive enough to monitor instructional impacts on higher-level cognitive abilities (what the BAM purports to assess).

This work is a joint effort between the National Center for Education Statistics (NCES) and two divisions (REC and ESIE) within the NSF. The project hypothesis is that the NAEP will be more effective than traditional multiple choice tests (ITBS in Arkansas and a state assessment program in Colorado), but not as effective as BAM in capturing student learning gains (and performance in process areas) in Connected Mathematics Program (CMP) classrooms. Such results have the potential to advise if not revise the NAEP and state assessment programs. Given the November, 2004, Loveless report (Brookings Institute) that critiques existing assessments for having an insufficient number of arithmetic NAEP items, this research may provide useful scientific evidence about the role and importance of such items within assessments and curriculum.

This project examines published and non-published education research studies of instructional innovations and alternative approaches to teaching undergraduate science and will synthesize the results using statistical methods. The study would conduct a systematic examination of undergraduate science course innovations to determine: (1) The range of course innovations that are being implemented in undergraduate science courses; (2) How the features of innovations are aligned with cognitive science learning theory and principles of effective instruction; (3) How course innovation approaches are associated with differences in student learning gains within each specific discipline; and (4) what methods are found to be critical for the effective implementation of course innovations. The researchers will develop a theoretical framework on the basis of cognitive science research and effective learning environments of undergraduate science courses from projects that have an instructional strategy, refer to classrooms, and have reported results. They will conduct a document analysis of course innovations in science courses as discovered in published papers and unpublished documents such as the NSF website. A meta-analysis of the statistical data available in each paper would be conducted depending on the suitable number of studies identified.

The broader impact of the study is in its contribution to researchers who would benefit from a systematic analysis of existing studies of undergraduate teaching practices and for educators who wish for advice on how to apply new methods of teaching. The study will produce two documents: (1) a framework for characterizing and evaluating innovations in undergraduate science courses, and (2) a statistical synthesis of the types of course transformations observed in the field, their effectiveness, and the design and implementation characteristics that are associated with their effectiveness. The project would produce published papers that will be submitted to journals in each disciplinary area and presented at professional society meetings.

The University of Colorado is establishing a Distributed Center for STEM Education Research and Transformation that integrates education projects across the campus. The Center addresses the three themes outlined in the National Academy of Science report "Rising Above the Gathering Storm". These are: (1) Teachers in K-12 education (10,000 Teachers, 10 Million Minds), (2) Research (Sowing Seeds), and (3) Higher Education (Best and Brightest). This Distributed Center involves eight traditional departments in three colleges and schools, including: Education, Life Sciences, Mathematics, Physical Sciences, and Engineering. Existing education projects being integrated into the center include: ADVANCE (Increasing the Participation and Advancement of Women in Academic Science and Engineering Careers); Course Curriculum and Laboratory Improvement projects; Robert Noyce Teaching Scholarship project; Integrative Graduate Education and Research Traineeship projects; and Research and Evaluation on Education in Science and Engineering projects.