Paper Authors

Abstract

NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract

Concept Inventories Meet Cognitive Psychology: Using Beta Testing as a Mechanism for Identifying Engineering Student Misconceptions

Ronald L. Miller, Ruth A. Streveler Colorado School of Mines

Mary A. Nelson, Monica R. Geist University of Colorado/University of Northern Colorado

Barbara M. Olds National Science Foundation and Colorado School of Mines

Abstract

This paper summarizes our analysis of beta test results collected at four U.S. engineering schools using the Thermal and Transport Concept Inventory (TTCI) currently under development. The instrument consists of questions in heat transfer, fluid mechanics, and thermodynamics and is designed to elicit information about the nature of undergraduate engineering students’ misconceptions in these disciplines. In addition to using conventional methods for determining TTCI reliability with correlation coefficients, we have applied cross-tabulation data analysis for six heat transfer questions to identify robust student misconceptions. The results of these analyses identified two categories of misconceptions which persist across heat transfer problems and contexts: 1) confusion about differences between energy and temperature, and 2) confusion about differences in steady-state and equilibrium processes.

Introduction

With funding from the National Science Foundation (DUE-0127806), a team of researchers at the Colorado School of Mines (CSM) is creating a concept inventory to measure engineering students’ understanding of difficult concepts in thermal and transport science, the Thermal and Transport Concept Inventory (TTCI). [1] Our project began with a Delphi survey of experts to determine the concepts that were the most difficult and important for engineering students studying thermodynamics, heat transfer, and fluid mechanics. [2] Once the key concepts were identified, we created open-ended questions related to each concept and asked students to think aloud as they answered the questions. Multiple-choice questions were then developed using students’ misunderstandings as documented in the think-alouds as the basis for writing distractors (incorrect but plausible answers). The resulting inventory was alpha tested at CSM [3], expanded, and beta tested at four other engineering institutions.

Standard psychometric procedures include testing to establish various forms of reliability (how repeatable are the results?) and validity (are we really testing the concepts that we intend to test?) for an instrument under development. We are currently analyzing beta test data from the TTCI to establish these parameters, and more detailed findings will be reported elsewhere. However, we are also using the beta test data to develop methods for identifying the kinds of misunderstandings or misconceptions students hold about concepts included in the TTCI. In

Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition Copyright © 2005, American Society for Engineering Education

• Citation

• Format
  • APA
  • APA - LaTeX bibitem
  • MLA
  • MLA - LaTeX bibitem
  • Bibtex
  • EndNote - RIS

Nelson, M., & Olds, B., & Geist, M., & Miller, R., & Streveler, R. (2005, June), Concept Inventories Meet Cognitive Psychology: Using Beta Testing As A Mechanism For Identifying Engineering Student Misconceptions Paper presented at 2005 Annual Conference, Portland, Oregon. 10.18260/1-2--14620