Search

Rubric-based admissions are claimed to help make the graduate admissions process more equitable, possibly helping to address the historical and ongoing inequities in the U.S. physics graduate school admissions process that have often excluded applicants from minoritized races, ethnicities, genders, and backgrounds. Yet, no studies have examined whether rubric-based admissions methods represent a fundamental change in the admissions process or simply represent a new tool that achieves the same outcome. To address that, we developed supervised machine learning models of graduate admissions data collected from our department over a seven-year period. During the first four years, our department used a traditional admission process and switched to a rubric-based process for the following three years, allowing us to compare which parts of the applications were driving admissions decisions. We find that faculty focused on applicants' physics GRE scores and grade-point averages when making admissions decisions before the implementation of the rubric. While we were able to develop a sufficiently good model whose results we could trust for the data before the implementation of the rubric, we were unable to do so for the data collected after the implementation of the rubric, despite multiple modifications to the algorithms and data such as implementing Tomek Links. Our inability to model the second dataset despite being able to model the first combined with model comparison analyses suggests that rubric-based admission does change the underlying process. These results suggest that rubric-based holistic review is a method that could make the graduate admission process in physics more equitable.

High school science classrooms across the United States are answering calls to make computation a part of science learning. The problem is that there is little known about the barriers to learning that computation might bring to a science classroom or about how to help students overcome these challenges. This case study explores these challenges from the perspectives of students in a high school physics classroom with a newly revamped, computation-integrated curriculum. Focusing mainly on interviews to center the perspectives of students, we found that computation is a double-edged sword: It can make science learning more authentic for students who are familiar with it, but it can also generate frustration and an aversion towards physics for students who are not.

Physics classes with computation integrated into the curriculum are a fitting setting for investigating computational thinking. In this paper, we present a framework for exploring this topic in introductory physics courses. The framework, which was developed by reviewing relevant literature and acquiring video data from high school classrooms, comprises 14 practices that students could engage in when working with Glowscript VPython activities. For every practice, we provide in-class video data to exemplify the practice. In doing this work, we hope to provide ways for teachers to assess their students' development of computational thinking and give physics education researchers a foundation to study the topic in greater depth.

As systematic inequities in higher education and society have been brought to the forefront, graduate programs are interested in increasing the diversity of their applicants and enrollees. Yet, structures in place to evaluate applicants may not support such aims. One potential solution to support those aims is rubricbased holistic review. Starting in 2018, our physics department implemented a rubric-based holistic review process for all applicants to our graduate program. The rubric assessed applicants on 18 metrics covering their grades, test scores, research experiences, noncognitive competencies, and fit with the program. We then compared faculty's ratings of applicants by admission status, sex, and undergraduate program over a three-year period. We find that the rubric scores show statistically significant differences between admitted and nonadmitted students as hoped. We also find that differences in rubric scores based on sex or undergraduate program reflected known systematic inequities such as applicants from smaller and less prestigious undergraduate universities scoring lower on the physics GRE and women performing more volunteer work in academia. Our results then suggest rubric-based holistic review as a possible route to making graduate admissions in physics more equitable.

Despite limiting access to applicants from underrepresented racial and ethnic groups, the practice of using hard or soft Graduate Record Examination (GRE) cutoff scores in physics graduate program admissions is still a popular method for reducing the pool of applicants. The present study considers whether the undergraduate institutions of applicants have any influence on the admissions process by modeling a physics GRE cutoff score with application data from admissions offices of five universities. Two distinct approaches based on inferential and predictive modeling are conducted. While there is some disagreement regarding the relative importance between features, the two approaches largely agree that including institutional information significantly aids the analysis. Both models identify cases where the institutional effects are comparable to factors of known importance such as gender and undergraduate GPA. As the results are stable across many cutoff scores, we advocate against the practice of employing physics GRE cutoff scores in admissions.

Introductory physics lab courses serve as the starting point for students to learn and experience experimental physics at the undergraduate level. They often focus on measurement uncertainty, an essential topic for practicing physicists and a foundation for more advanced lab learning. As such, measurement uncertainty has been a focus when studying and improving introductory physics lab courses. There is a need for a research-based assessment explicitly focused on measurement uncertainty that captures the breadth of learning related to the topic, and that has been developed and documented in an evidence-centered way. In this work, we present the first step in the development of such an assessment, with the goal of establishing the breadth and depth of the domain of measurement uncertainty in introductory physics labs. We conducted and analyzed interviews with introductory physics lab instructors across the US, identifying prevalent concepts and practices related to measurement uncertainty, and their level of emphasis in introductory physics labs. We find that instructors discuss a range of measurement uncertainty topics beyond basic statistical ideas like mean and standard deviation, including those connected to modeling, another lab learning goal. We describe how these findings will be used in the subsequent development of the assessment, called the Survey Of Physics Reasoning On Uncertainty Concepts In Experiments (SPRUCE).

One argument for keeping the physics Graduate Record Exam (GRE) is that it can help applicants who might otherwise be missed in the admissions process stand out. In this work, we evaluate whether this claim is supported by physics graduate school admissions decisions. We used admissions data from five Ph.D.-granting physics departments over a 2-year period (N = 2537) to see how the fraction of applicants admitted varied based on their physics GRE scores. We compared applicants with low GPAs to applicants with higher GPAs, applicants from large undergraduate universities to applicants from smaller undergraduate universities, and applicants from selective undergraduate institutions to applicants from less selective undergraduate institutions. We also performed a mediation and moderation analysis to provide statistical rigor and to better understand the previous relationships. We find that for applicants who might otherwise have been missed (e.g., have a low GPA or attended a small or less selective school), having a high physics GRE score did not seem to increase the applicant's chances of being admitted to the schools. However, having a low physics GRE score seemed to penalize otherwise competitive applicants (i.e., applicants with mid to high GPAs). Thus, our work suggests that the physics GRE does not, in fact, help applicants who might otherwise be missed stand out.

Across the field of education research there has been an increased focus on the development, critique, and evaluation of statistical methods and data usage due to recently created, very large datasets and machine learning techniques. In physics education research (PER), this increased focus has recently been shown through the 2019 Physical Review PER Focused Collection examining quantitative methods in PER. Quantitative PER has provided strong arguments for reforming courses by including interactive engagement, demonstrated that students often move away from scientist-like views due to science education, and has injected robust assessment into the physics classroom via concept inventories. The work presented here examines the impact that machine learning may have on physics education research, presents a framework for the entire process including data management, model evaluation, and results communication, and demonstrates the utility of this framework through the analysis of two types of survey data.

An increasing number of introductory physics courses are seeking to incorporate "authentic practices," and a large area of focus in this trend is the incorporation of computational problems into the curriculum. These problems offer students an opportunity to engage with the programming practices and numerical problem-solving methods used by physicists. Understanding how instructors approach teaching such problems is essential for improving instruction and problem design. We conducted a phenomenographic study using semistructured interviews with undergraduate learning assistants in a problem-based introductory mechanics course that incorporates several computational problems. The learning assistants' prior involvement as students, along with their relatively fewer experiences with programming and physics compared to the faculty instructors, give them a unique perspective on teaching in the course. We present here the results of our analysis: the identification of four approaches that learning assistants make take to teaching computational problems in this course. These approaches, "programming focus, learning physics via computation focus, computation as a tool focus, and shifting perceptions of learning focus," provide a lens for understanding the different ways learning assistants perceive computation, the degree to which they take up course-intended learning goals surrounding computation, the factors that may impact the approaches they take, and how we might affect their approaches through training and support.

We have used an unsupervised machine learning method called latent Dirichlet allocation (LDA) to thematically analyze all papers published in the Physics Education Research Conference Proceedings between 2001 and 2018. By looking at co-occurrences of words across the data corpus, this technique has allowed us to identify ten distinct themes or "topics" that have seen varying levels of prevalence in physics education research (PER) over time and to rate the distribution of these topics within each paper. Our analysis suggests that although all identified topics have seen sustained interest over time, PER has also seen several waves of increased interest in certain topics, beginning with initial interest in qualitative, theory-building studies of student understanding, which gave way to a focus on problem solving in the late 2000s. Since 2010 the field has seen a shift toward more sociocultural views of teaching and learning with a particular focus on communities of practice, student identities, and institutional change. Based on these results, we suggest that unsupervised text analysis techniques like LDA may hold promise for providing quantitative, independent, and replicable analyses of educational research literature.

[This paper is part of the Focused Collection on Curriculum Development: Theory into Design.] The communities of practice framework has become an essential framework for understanding identity development both in physics education research (PER) and in science, technology, engineering, and mathematics, more broadly. However, the use of communities of practice as a learning theory that informs curriculum design is significantly less prevalent within the PER community. One possible reason for this is that communities of practice as a theory originated in professional environments and it subsequently moved towards a framework that is centered around informing management practices. Some significant interpretations and negotiations need to be completed in order to apply and to design for communities of practice in the classroom environment. In this paper, we outline an introductory physics course called Projects and Practices in Physics (P-Cubed) that was designed using the communities of practice as a guiding framework. We present the curriculum decisions that focus specifically on the adaption process from professional practice to the classroom context along with the theoretical underpinnings of the curriculum design decisions that went into the development of the P-Cubed classroom.

Physics education research (PER) has used quantitative modeling techniques to explore learning, affect, and other aspects of physics education. However, these studies have rarely examined the predictive output of the models, instead focusing on the inferences or causal relationships observed in various data sets. This research introduces a modern predictive modeling approach to the PER community using transcript data for students declaring physics majors at Michigan State University. Using a machine learning model, this analysis demonstrates that students who switch from a physics degree program to an engineering degree program do not take the third semester course in thermodynamics and modern physics, and may take engineering courses while registered as a physics major. Performance in introductory physics and calculus courses, measured by grade as well as a students' declared gender and ethnicity play a much smaller role relative to the other features included in the model. These results are used to compare traditional statistical analysis to a more modern modeling approach.

Computation is a central aspect of 21st century physics practice; it is used to model complicated systems, to simulate impossible experiments, and to analyze mountains of data. Physics departments and their faculty are increasingly recognizing the importance of teaching computation to their students. We recently completed a national survey of faculty in physics departments to understand the state of computational instruction and the factors that underlie that instruction. The data collected from the faculty responding to the survey included a variety of scales, binary questions, and numerical responses. We then used random forest, a supervised learning technique, to explore the factors that are most predictive of whether a faculty member decides to include computation in their physics courses. We find that experience using computation with students in their research, or lack thereof and various personal beliefs to be most predictive of a faculty member having experience teaching computation. Interestingly, we find demographic and departmental factors to be less useful factors in our model. The results of this study inform future efforts to promote greater integration of computation into the physics curriculum as well as comment on the current state of computational instruction across the United States.

Computation is becoming an increasingly important part of physics education. However, there are currently few theories of learning that can be used to help explain and predict the unique challenges and affordances associated with computation in physics. In this study, we adapt the existing theory of computational literacy, which posits that computational learning can be divided into material, cognitive, and social aspects, to the context of undergraduate physics. Based on an exploratory study of undergraduate physics computational literacy, using a newly developed teaching tool known as a computational essay, we have identified a variety of student practices, knowledge, and beliefs across these three aspects of computational literacy. We illustrate these categories with data collected from students who engaged in an initial implementation of computational essays in a 3rd-semester electricity and magnetism class. We conclude by arguing that this framework can be used to theoretically diagnose student difficulties with computation, distinguish educational approaches that focus on material vs cognitive aspects of computational literacy, and highlight the benefits and limitations of open-ended projects like computational essays to student learning.

Assessing student learning is a cornerstone of educational practice. Standardized assessments have played a significant role in the development of instruction, curricula, and educational spaces in college physics. However, the use of these assessments to evaluate student learning is only productive if they continue to align with our learning goals. Recently, there have been calls to elevate the process of science ("scientific practices") to the same level of importance and emphasis as the concepts of physics ("core ideas" and "crosscutting concepts"). We use the recently developed Three-Dimensional Learning Assessment Protocol to investigate how well the most commonly used standardized assessments in introductory physics (i.e., concept inventories) align with this modern understanding of physics education's learning goals. We find that many of the questions on concept inventories do elicit evidence of student understanding of core ideas, but do not have the potential to elicit evidence of scientific practices or crosscutting concepts. Furthermore, we find that the individual scientific practices and crosscutting concepts that are assessed using these tools are limited to a select few. We discuss the implications that these findings have on designing and testing curricula and instruction both in the past and for the future.

Group work is becoming increasingly common in introductory physics classrooms. Understanding how students engage in these group learning environments is important for designing and facilitating productive learning opportunities for students. We conducted a study in which we collected video of groups of students working on conceptual electricity and magnetism problems in an introductory physics course. In this setting, students needed to negotiate a common understanding and coordinate group decisions in order to complete the activity successfully. We observed students interacting in several distinct ways while solving these problems. Analysis of these observations focused on identifying the different ways students interacted and articulating what defines and distinguishes them, resulting in the development of the modes of collaboration framework. The modes of collaboration framework defines student interactions along three dimensions: social, discursive, and disciplinary content. This multidimensional approach offers a unique lens through which to consider group work and provides a flexibility that could allow the framework to be adapted for a variety of contexts. We present the framework and several examples of its application here.

Reliable and validated assessments of introductory physics have been instrumental in driving curricular and pedagogical reforms that lead to improved student learning. As part of an effort to systematically improve our sophomore-level classical mechanics and math methods course (CM 1) at CU Boulder, we have developed a tool to assess student learning of CM 1 concepts in the upper division. The Colorado Classical Mechanics and Math Methods Instrument (CCMI) builds on faculty consensus learning goals and systematic observations of student difficulties. The result is a 9-question open-ended post test that probes student learning in the first half of a two-semester classical mechanics and math methods sequence. In this paper, we describe the design and development of this instrument, its validation, and measurements made in classes at CU Boulder and elsewhere.