Mapping Engineering Students' Learning Outcomes from International Experiences: Designing an Instrument to Measure Attainment of Knowledge, Skills, and Attitudes

Contribution: This paper developed a diagnostic tool to target various learning outcomes of international programs for engineering students, addressing both broad competencies and engineering specific outcomes. Background: Research has shown that international engineering learning activities provide students with opportunities to develop global competencies. Multiple instruments have been developed to target either students' broad competencies or engineering-related outcomes. While these previous instruments proved to be helpful in linking students' intercultural competencies with outcomes, in measuring engineering specific outcomes, or in assessing engineering students' products or projects, no tool was available to measure the full range of possible learning outcomes from engineering students' international learning experiences. Research Questions: (1) How can an instrument be developed to target various possible outcomes of engineering students from international learning experiences?; (2) How does the instrument perform against established standards for validity and reliability? Methodology: Framed in the dimensions of knowledge, skills, and attitudes, a learning outcome survey was compiled based upon previous qualitative research. The instrument was assessed for structural validity by conducting an exploratory factor analysis, and its reliability by testing Cronbach's alpha (n = 407). Results from two sample programs were presented to demonstrate applicability. Fourteen follow-up interviews were analyzed to verify the survey content. Findings: The resulting survey showed a three-factor structure, each corresponding to knowledge, skills, and attitudes, and having high reliability in each dimension (alpha>0.85). Qualitative evidence verified the survey validity by demonstrating the close link between interview results and survey items, and the impact of learning activities on outcomes.