Search

In introductory physics laboratory instruction, students often expect to confirm or demonstrate textbook physics concepts (Wilcox & Lewandowski, 2017; Hu & Zwickl, 2017; Hu & Zwickl, 2018). This expectation is largely undesirable: labs that emphasize confirmation of textbook physics concepts are unsuccessful at teaching those concepts (Wieman & Holmes, 2015; Holmes et al., 2017) and even in contexts that don't emphasize confirmation, such expectations can lead to students disregarding or manipulating their data in order to obtain the expected result (Smith et al., 2020). In other words, when students expect their lab activities to confirm a known result, they may relinquish epistemic agency and violate disciplinary practices. We claim that, in other cases, confirmatory expectations can actually support productive disciplinary engagement. In particular, when an expected result is not confirmed, students may enter a productive "troubleshooting" mode (Smith et al., 2020). We analyze the complex dynamics of students' epistemological framing in a lab where student's confirmatory expectations support and even generate epistemic agency and disciplinary practices, including developing original ideas, measures, and apparatuses to apply to the material world.

Gaining recognition as a physics person by peers is an important contributor to undergraduate students' physics identity and their success in physics courses. Previous research has separately demonstrated that women perceive less recognition from peers than men in their physics courses (perceived recognition) and that women receive fewer nominations from their peers as strong in their physics course than men (received recognition). The relationship between perceived and received peer recognition for men and women, however, is not well understood. Here we test three plausible models for this relationship. We conduct a large-scale, quantitative study of over 1,700 students enrolled in introductory physics courses at eight institutions in the United States. We directly compare student gender, perceived recognition, and received recognition, controlling for other student demographics and course-level variability. Results show with high precision that, for students receiving the same amount of recognition, and having the same race or ethnicity, academic year, and major, women report significantly lower perceived recognition than men. These findings offer important implications for testable instructional interventions.

Research suggests that interacting with more peers about physics course material is correlated with higher student performance. Some studies, however, have demonstrated that different topics of peer interactions may correlate with their performance in different ways, or possibly not at all. In this study, we probe both the peers with whom students interact about their physics course and the particular aspects of the course material about which they interacted in six different introductory physics courses: four lecture courses and two lab courses. Drawing on methods in social network analysis, we replicate prior work demonstrating that, on average, students who interact with more peers in their physics courses have higher final course grades. Expanding on this result, we find that students discuss a wide range of aspects of course material with their peers: concepts, small-group work, assessments, lecture, and homework. We observe that in the lecture courses, interacting with peers about concepts is most strongly correlated with final course grade, with smaller correlations also arising for small-group work and homework. In the lab courses, on the other hand, small-group work is the only interaction topic that significantly correlates with final course grade. We use these findings to discuss how course structures (e.g., grading schemes and weekly course schedules) may shape student interactions and add nuance to prior work by identifying how specific types of student interactions are associated (or not) with performance., Comment: Submitted to European Journal of Physics

Physics education researchers are interested in using the tools of machine learning and natural language processing to make quantitative claims from natural language and text data, such as open-ended responses to survey questions. The aspiration is that this form of machine coding may be more efficient and consistent than human coding, allowing much larger and broader data sets to be analyzed than is practical with human coders. Existing work that uses these tools, however, does not investigate norms that allow for trustworthy quantitative claims without full reliance on cross-checking with human coding, which defeats the purpose of using these automated tools. Here we propose a four-part method for making such claims with supervised natural language processing: evaluating a trained model, calculating statistical uncertainty, calculating systematic uncertainty from the trained algorithm, and calculating systematic uncertainty from novel data sources. We provide evidence for this method using data from two distinct short response survey questions with two distinct coding schemes. We also provide a real-world example of using these practices to machine code a data set unseen by human coders. We offer recommendations to guide physics education researchers who may use machine-coding methods in the future.

Uncertainty is an important and fundamental concept in physics education. Students are often first exposed to uncertainty in introductory labs, expand their knowledge across lab courses, and then are introduced to quantum mechanical uncertainty in upper-division courses. This study is part of a larger project evaluating student thinking about uncertainty across these contexts. In this research, we investigate advanced physics student thinking about uncertainty by asking them conceptual questions about how a hypothetical distribution of measurements would change if `more' or `better' data were collected in four different experimental scenarios. The scenarios include both classical and quantum experiments, as well as experiments that theoretically result in an expected single value or an expected distribution. This investigation is motivated by our goal of finding insights into students' potential point- and set-like thinking about uncertainty and of shining light on the limitations of those binary paradigms., Comment: 15 pages, 5 figures, accepted to Physical Review Physics Education Research

Uncertainty is an important concept in physics laboratory instruction. However, little work has examined how students reason about uncertainty beyond the introductory (intro) level. In this work we aimed to compare intro and beyond-intro students' ideas about uncertainty. We administered a survey to students at 10 different universities with questions probing procedural reasoning about measurement, student-identified sources of uncertainty, and predictive reasoning about data distributions. We found that intro and beyond-intro students answered similarly on questions where intro students already exhibited expert-level reasoning, such as in comparing two data sets with the same mean but different spreads, identifying limitations in an experimental setup, and predicting how a data distribution would change if more data were collected. For other questions, beyond-intro students generally exhibited more expert-like reasoning than intro students, such as when determining whether two sets of data agree, identifying principles of measurement that contribute to spread, and predicting how a data distribution would change if better data were collected. Neither differences in student populations, lab courses taken, nor research experience were able to fully explain the variability between intro and beyond-intro student responses. These results call for further research to better understand how students' ideas about uncertainty develop beyond the intro level., Comment: 19 pages, 12 figures

Measurement uncertainty is an important topic in the undergraduate laboratory curriculum. Previous research on student thinking about experimental measurement uncertainty has focused primarily on introductory-level students' procedural reasoning about data collection and interpretation. In this paper, we extended this prior work to study upper-level students' thinking about sources of measurement uncertainty across experimental contexts, with a particular focus on classical and quantum mechanics contexts. We developed a survey to probe students' thinking in the generic question "What comes to mind when you think about measurement uncertainty in [classical/quantum] mechanics?" as well as in a range of specific experimental scenarios and interpreted student responses through the lens of availability and accessibility of knowledge pieces. We found that limitations of the experimental setup were most accessible to students in classical mechanics while principles of the underlying physics theory were most accessible to students in quantum mechanics, even in a context in which this theory was not relevant. We recommend that future research probe which sources of uncertainty experts believe are relevant in which contexts and how instruction in both classical and quantum contexts can help students draw on appropriate sources of uncertainty in classical and quantum experiments., Comment: 15 pages, 8 figures

Previous work has identified that recognition from others is an important predictor of students' participation, persistence, and career intentions in physics. However, research has also found a gender bias in peer recognition in which student nominations of strong peers in their physics course disproportionately favor men over women. In this study, we draw on methods from social network analysis and find a consistent gender bias in which men disproportionately under-nominate women as strong in their physics course in two offerings of both a lecture course (for science and engineering, but not physics, majors) and a distinct lab course (for science, engineering, and physics majors). We also find in one offering of the lecture course that women disproportionately under-nominate men, contrary to what previous research would predict. We expand on prior work by also probing two data sources related to who and what gets recognized in peer recognition: students' interactions with their peers (who gets recognized) and students' written explanations of their nominations of strong peers (what gets recognized). Results suggest that the nature of the observed gender bias in peer recognition varies between the instructional contexts of lecture and lab. In the lecture course, the gender bias is related to who gets recognized: both men and women disproportionately over-nominate their interaction ties to students of their same gender as strong in the course. In the lab course, the gender bias is also related to what gets recognized: men nominate men more than women because of skills related to interactions, such as being helpful. These findings illuminate the different ways in which students form perceptions of their peers and add nuance to our understanding of the nature of gender bias in peer recognition., Comment: Submitted to Physical Review Physics Education Research

Research on nontraditional laboratory (lab) activities in physics shows that students often expect to verify predetermined results, as takes place in traditional activities. This understanding of what is taking place, or epistemic framing, may impact their behaviors in the lab, either productively or unproductively. In this paper, we present an analysis of student epistemic framing in a nontraditional lab to understand how instructional context, specifically instructor behaviors, may shape student framing. We present video data from a lab section taught by an experienced teaching assistant (TA), with 19 students working in seven groups. We argue that student framing in this lab is evidenced by whether or not students articulate experimental predictions and by the extent to which they take up opportunities to construct knowledge (epistemic agency). We show that the TA's attempts to shift student frames generally succeed with respect to experimental predictions but are less successful with respect to epistemic agency. In part, we suggest, the success of the TA's attempts reflects whether and how they are responsive to students' current framing. This work offers evidence that instructors can shift students' frames in nontraditional labs, while also illuminating the complexities of both student framing and the role of the instructor in shifting that framing in this context., Comment: Submitted to Physical Review Physics Education Research

Engaging in interactions with peers is important for student learning. Many studies have quantified patterns of student interactions in in-person physics courses using social network analysis, finding different network structures between instructional contexts (lecture and lab) and styles (active and traditional). Such studies also find inconsistent results as to whether and how student-level variables (e.g., grades and demographics) relate to the formation of interaction networks. In this cross-sectional research study, we investigate these relationships further by examining lecture and lab interaction networks in four different remote physics courses spanning various instructional styles and student populations. We apply statistical methods from social network analysis -- exponential random graph models -- to measure the relationship between network formation and multiple variables: students' discussion and lab section enrollment, final course grades, gender, and race/ethnicity. Similar to previous studies of in-person courses, we find that remote lecture interaction networks contain large clusters connecting many students, while remote lab interaction networks contain smaller clusters of a few students. Our statistical analysis suggests that these distinct network structures arise from a combination of both instruction-level and student-level variables, including the learning goals of each instructional context, whether assignments are completed in groups or individually, and the distribution of gender and major of students enrolled in a course. We further discuss how these and other variables help to understand the formation of interaction networks in both remote and in-person physics courses., Comment: To be published in Physical Review Physics Education Research

Researchers have pinpointed recognition from others as one of the most important dimensions of students' science and engineering identity. Studies, however, have found gender biases in students' recognition of their peers, with inconsistent patterns across introductory science and engineering courses. Toward finding the source of this variation, we examine whether a gender bias exists in students' nominations of strong peers across three different remote, introductory physics courses with varying student populations (varying demographics, majors, and course levels). We also uniquely evaluate possible racial/ethnic biases and probe the relationship between instructional context (whether lecture or laboratory) and recognition. Some of our results replicate previous findings (such as the the association of course grade and small class section enrollment with nominations), while others offer contradictions. Comparing across our three courses and the prior work, results suggest that course level (whether first-year students or beyond-first-year students) might be more associated with a gender bias in peer recognition than other variables. Surprisingly, we also find instances of racial/ethnic biases in favor of students from backgrounds historically underrepresented in science. Finally, we find that the nomination patterns differ when students nominate individuals strong in the lecture material versus laboratory material. This work serves as an important step in determining which courses and contexts exhibit biases in peer recognition, as well as how students' perceptions of one another form in remote teaching environments., Comment: To be published in Physical Review Physics Education Research

Here, a benchtop hybrid magnetic shield containing four mumetal cylinders and nine internal flexible printed circuit boards is designed, constructed, tested, and operated. The shield is designed specifically as a test-bed for building and operating ultra-sensitive quantum magnetometers. The geometry and spacing of the mumetal cylinders are optimized to maximize shielding efficiency while maintaining Johnson noise $<15$ fT/$\sqrt{}$Hz. Experimental measurements at the shield's center show passive shielding efficiency of $\left(1.0\pm0.1\right){\times}10^6$ for a $0.2$ Hz oscillating field applied along the shield's axis. The nine flexible printed circuit boards generate three uniform fields, which all deviate from perfect uniformity by ${\leq}0.5$% along $50$% of the inner shield axis, and five linear field gradients and one second-order gradient, which all deviate by ${\leq}4$% from perfect linearity and curvature, respectively, over measured target regions. Together, the target field amplitudes are adjusted to minimize the remnant static field along $40$% of the inner shield axis, as mapped using an atomic magnetometer. In this region, the active null reduces the norm of the magnitudes of the three uniform fields and six gradients by factors of $19.5$ and $19.8$, respectively, thereby reducing the total static field from $1.68$ nT to $0.23$ nT., Comment: 8 pages, 9 figures; This work has been submitted to the IEEE for possible publication

Quantum sensors based on cold atoms are being developed which produce measurements of unprecedented accuracy. Due to shifts in atomic energy levels, quantum sensors often have stringent requirements on their internal magnetic field environment. Typically, background magnetic fields are attenuated using high permeability magnetic shielding, with the cancelling of residual and introduction of quantisation fields implemented with coils inside the shield. The high permeability shield, however, distorts all magnetic fields, including those generated inside the sensor. Here, we demonstrate a solution by designing multiple coils overlaid on a 3D-printed former to generate three uniform and three constant linear gradient magnetic fields inside the capped cylindrical magnetic shield of a cold atom interferometer. The fields are characterised in-situ and match their desired forms to high accuracy. For example, the uniform transverse field, $B_x$, deviates by less than $0.2$% over more than $40$% of the length of the shield. We also map the field directly using the cold atoms and investigate the potential of the coil system to reduce bias from the quadratic Zeeman effect. This coil design technology enables targeted field compensation over large spatial volumes and has the potential to reduce systematic shifts and noise in numerous cold atom systems., Comment: 12 pages, 9 figures

Magnetically-sensitive experiments and newly-developed quantum technologies with integrated high-permeability magnetic shields require increasing control of their magnetic field environment and reductions in size, weight, power and cost. However, magnetic fields generated by active components are distorted by high-permeability magnetic shielding, particularly when they are close to the shield's surface. Here, we present an efficient design methodology for creating desired static magnetic field profiles by using discrete coils electromagnetically-coupled to a cylindrical passive magnetic shield. We utilize a modified Green's function solution that accounts for the interior boundary conditions on a closed finite-length high-permeability cylindrical magnetic shield, and determine simplified expressions when a cylindrical coil approaches the interior surface of the shield. We use an analytic formulation of simple discrete building blocks to provide a complete discrete coil basis to generate any physically-attainable magnetic field inside the shield. We then use a genetic algorithm to find optimized discrete coil structures composed of this basis. We use our methodology to generate an improved linear axial gradient field, $\mathrm{d}B_z/\mathrm{d}z$, and transverse bias field, $B_x$. These optimized structures increase, by a factor of seven and three compared to the standard configurations, the volume in which the desired and achieved fields agree within $1\%$ accuracy, respectively. This coil design method can be used to optimize active--passive magnetic field shaping systems that are compact and simple to manufacture, enabling accurate magnetic field control in spatially-confined experiments at low cost., Comment: The authors M. Packer and P. J. Hobson have contributed equally to this work. 24 pages, 16 figures

With the adoption of instructional laboratories (labs) that require students to make their own decisions, there is a need to better understand students' activities as they make sense of their data and decide how to proceed. In particular, understanding when students do not engage productively with unexpected data may provide insights into how to better support students in more open-ended labs. We examine video and audio data from groups within a lab session where students were expected to find data inconsistent with the predictions of two models. In prior work, we examined the actions of the four groups that productively grapple with this designed problem. Here, we analyze the engagement of the three groups that do not. We conducted three phases of analysis: 1) documenting large scale behaviors and time spent in on-topic discussion, 2) analyzing interactions with the teaching assistant, and 3) identifying students' framing--their expectations for what is taking place--when they were discussing their data. Our Phase 1 and 2 analysis show only minor differences between the groups that engaged with the problem and those that did not. Our Phase 3 analysis demonstrated that the groups that did not engage with the problem framed the lab activity as about confirming a known result or as a series of hoops to jump through to fulfill assignment requirements. Implications for instruction include supporting teaching assistants to attend to students' framing and agency within laboratory classrooms.

Hybrid magnetic shields with both active field generating components and high-permeability magnetic shielding are increasingly needed for a variety of technologies and experiments that require precision-controlled magnetic field environments. However, the fields generated by the active components interact with the passive magnetic shield, distorting the desired field profiles. Consequently, optimization of the active components needed to generate user-specified target fields must include coupling to the high-permeability passive components. Here, we consider the optimization of planar active systems, on which an arbitrary static current flows, coupled to a closed high-permeability cylindrical shield. We modify the Green's function for the magnetic vector potential to match boundary conditions on the shield's interior surface, enabling us to construct an inverse optimization problem to design planar coils that generate user-specified magnetic fields inside high-permeability shields. We validate our methodology by designing two bi-planar hybrid active--passive systems, which generate a constant transverse field, $\mathbf{B}=\mathbf{\hat{x}}$, and a linear field gradient, $\mathbf{B}=(-x~\mathbf{\hat{x}}-y~\mathbf{\hat{y}}+2z~\mathbf{\hat{z}})$, respectively. For both systems, the inverse-optimized magnetic field profiles agree well with forward numerical simulations. Our design methodology is accurate and flexible, facilitating the miniaturization of high-performance hybrid magnetic field generating technologies with strict design constraints and spatial limitations., Comment: 24 pages, 10 figures

The number and use of research-based assessments (RBAs) has grown significantly over the last several decades. Data from RBAs can be compared against national datasets to provide instructors with empirical evidence on the efficacy of their teaching practices. Many physics instructors, however, opt not to use RBAs due to barriers such as having to use class time to administer them. In this article we examine how these barriers can be mitigated through online administrations of RBAs, particularly through the use of free online RBA platforms that automate administering, scoring, and analyzing RBAs (e.g., the Learning About STEM Student Outcomes [LASSO], Colorado Learning Attitudes About Science Survey for Experimental Physics [E-CLASS], Physics Lab Inventory of Critical thinking [PLIC], and PhysPort DataExplorer platforms). We also explore the research into common concerns of administering RBAs online and conclude with a practical how-to guide for instructors., Comment: 4 pages, 0 figures

Group work during lab instruction can be a source of inequity between male and female students. In this preliminary study, we explored the activities male and female students take on during a lab session at a university in Denmark. Different from many studies, the class was majority-female, so three of the seven groups were all female and the rest were mixed-gender. We found that students in mixed-gender groups divide tasks in similar ways to mixed-gender groups at North American institutions, with men handling the equipment and women handling the computer more often. We also found that women in single-gender groups took on each of the available roles with approximately equal frequency, but women in single-gender groups spent more time on the equipment than students in mixed-gender groups. We interpret the results through poststructual gender theory and the notion of `doing physics' and `doing gender' in physics labs., Comment: 6 pages, 3 figures

In physics labs, students experience a wide range of equitable and inequitable interactions. We developed a methodology to characterize different lab groups in terms of their bid exchanges and inchargeness. An equitable group is one in which every student's bids are heard and acknowledged. Our analysis of equitable and inequitable groups raises questions about how inchargeness and gender interact to affect the functionality of a lab group., Comment: Submitted to ICLS 2020

Instructional physics labs are critical junctures for many STEM majors to develop an understanding of experimentation in the sciences. Students can acquire useful experimental skills and grow their identities as scientists. However, many traditionally-instructed labs do not necessarily involve authentic physics experimentation features in their curricula. Recent research calls for a reformation in undergraduate labs to incorporate more student agency and choice in the learning processes. In our institution, we have adopted open-ended lab teaching in the introductory physics courses. By using reformed curricula that provide higher student agency, we analyzed approximately 100 students in the introductory-level lab courses to examine their views towards the open-ended physics labs. Between the start and the end of the semester, we found a statistically significant shift in students' perceptions about the agency afforded in lab activities. We also examined students' responses to "Which lab unit was your favorite and why?". The analysis showed that majority of the students preferred Project Lab, which had the highest student agency and coding analysis showed that "freedom" was the most frequent response for students' reason for picking Project Lab. Finally, we also examined student views across gender and found no significant gender effect on students' sense of agency.

Prior research has shown that physics students often think about experimental procedures and data analysis very differently from experts. One key framework for analyzing student thinking has found that student thinking is more point-like, putting emphasis on the results of a single experimental trial, whereas set-like thinking relies on the results of many trials. Recent work, however, has found that students rarely fall into one of these two extremes, which may be a limitation of how student thinking is evaluated. Measurements of student thinking have focused on probing students' procedural knowledge by asking them, for example, what steps they might take next in an experiment. Two common refrains are to collect more data, or to improve the experiment and collect better data. In both of these cases, the underlying reasons behind student responses could be based in point-like or set-like thinking. In this study we use individual student interviews to investigate how advanced physics students believe the collection of more and better data will affect the results of a classical and a quantum mechanical experiment. The results inform future frameworks and assessments for characterizing students thinking between the extremes of point and set reasoning in both classical and quantum regimes., Comment: Accepted for publication in 2020 PERC Proceedings

Measurement uncertainty and experimental error are important concepts taught in undergraduate physics laboratories. Although student ideas about error and uncertainty in introductory classical mechanics lab experiments have been studied extensively, there is relatively limited research on student thinking about experimental measurement uncertainty in quantum mechanics. In this work, we used semi-structured interviews to study advanced physics students' interpretations of fictitious data distributions from two common undergraduate laboratory experiments in quantum mechanics and one in classical mechanics. To analyze these interpretations, we developed a coding scheme that classifies student responses based on what factors they believe create uncertainty and differentiates between different types of uncertainty (e.g. imprecision, inaccuracy). We found that participants in our study expressed a variety of ideas about measurement uncertainty that varied with the context (classical/quantum) and the type of uncertainty.

Magnetic shields that use both active and passive components to enable the generation of a tailored low-field environment are required for many applications in science, engineering, and medical imaging. Until now, accurate field nulling, or field generation, has only been possible over a small fraction of the overall volume of the shield. This is due to the interaction between the active field-generating components and the surrounding high-permeability passive shielding material. In this paper, we formulate the interaction between an arbitrary static current flow on a cylinder and an exterior closed high-permeability cylinder. We modify the Green's function for the magnetic vector potential and match boundary conditions on the shield's interior surface to calculate the total magnetic field generated by the system. We cast this formulation into an inverse optimization problem to design active--passive magnetic field shaping systems that accurately generate any physical static magnetic field in the interior of a closed cylindrical passive shield. We illustrate this method by designing hybrid systems that generate a range of magnetic field profiles to high accuracy over large interior volumes, and simulate them in real-world shields whose passive components have finite permeability, thickness, and axial entry holes. Our optimization procedure can be adapted to design active--passive magnetic field shaping systems that accurately generate any physical user-specified static magnetic field in the interior of a closed cylindrical shield of any length, enabling the development and miniaturization of systems that require accurate magnetic shielding and control., Comment: 22 pages, 9 figures, 1 table

Instructional labs are being transformed to better reflect authentic scientific practice, often by removing aspects of pedagogical structure to support student agency and decision-making. We explored how these changes impact men's and women's participation in group work associated with labs through clustering methods on the quantified behavior of students. We compared the group roles students take on in two different types of instructional settings; (1) highly structured traditional labs, and (2) less structured inquiry-based labs. Students working in groups in the inquiry-based (less structured) labs assumed different roles within their groups, however men and women systematically took on different roles and men behaved differently when in single- versus mixed-gender groups. We found no such systematic differences in role division among male and female students in the traditional (highly-structured) labs. Students in the inquiry-based labs were not overtly assigned these roles, indicating that the inequitable division of roles was not a result of explicit assignment. Our results highlight the importance of structuring equitable group dynamics in educational settings, as a gendered division of roles can emerge without active intervention. As the culture in physics evolves to remove systematic gender biases in the field, instructors in educational settings must not only remove explicitly biased aspects of curricula but also take active steps to ensure that potentially discriminatory aspects are not inadvertently reinforced.

Immersive virtual reality (VR) has enormous potential for education, but classroom resources are limited. Thus, it is important to identify whether and when VR provides sufficient advantages over other modes of learning to justify its deployment. In a between-subjects experiment, we compared three methods of teaching Moon phases (a hands-on activity, VR, and a desktop simulation) and measured student improvement on existing learning and attitudinal measures. While a substantial majority of students preferred the VR experience, we found no significant differences in learning between conditions. However, we found differences between conditions based on gender, which was highly correlated with experience with video games. These differences may indicate certain groups have an advantage in the VR setting., Comment: 28 pages, 7 figures, 4 tables. Published in PLOS ONE March 25, 2020

It is a well-studied notion that women are under-represented in the physical sciences, with a "leaky pipeline" metaphor describing how the number of women decreases at higher levels in academia[1,2]. It is unclear, however, where the major leaks exist and what factors are responsible for this[2]. Our focus here is on women in physics with an emphasis on practical laboratory work.

Introductory physics lab instruction is undergoing a transformation, with increasing emphasis on developing experimentation and critical thinking skills. These changes present a need for standardized assessment instruments to determine the degree to which students develop these skills through instructional labs. In this article, we present the development and validation of the Physics Lab Inventory of Critical thinking (PLIC). We define critical thinking as the ability to use data and evidence to decide what to trust and what to do. The PLIC is a 10-question, closed-response assessment that probes student critical thinking skills in the context of physics experimentation. Using interviews and data from 5584 students at 29 institutions, we demonstrate, through qualitative and quantitative means, the validity and reliability of the instrument at measuring student critical thinking skills. This establishes a valuable new assessment instrument for instructional labs., Comment: 16 pages, 4 figures

Calls for reform to instructional labs means many instructors and departments are facing the daunting task of identifying goals for their introductory lab courses. Fortunately, the American Association of Physics Teachers (AAPT) released a set of recommendations for learning goals for the lab to support lab redevelopment [1]. Here we outline the process we have undergone to identify a set of learning goals for the labs that operationalize those provided by the AAPT. We also provide two examples of newly developed lab activities that aim to meet those goals to demonstrate this operationalization. We aim to provide departments and instructors with a few ideas of a procedure that they can follow or a set of goals that they can use to align lab instruction with the AAPT learning goals., Comment: 6 pages, 4 figures, accepted to The Physics Teacher

Despite the significant amount of time undergraduate students spend in introductory physics labs, there is little consensus on instructional goals and accepted diagnostic assessments for these labs. In response to these issues, we have developed the Physics Lab Inventory of Critical thinking (PLIC) to assess students' proficiency with critical thinking in a physics lab context. Specifically, the PLIC aims to evaluate students' skills in making sense of data, variability, models, and experimental methods and to assess the effectiveness of lab courses at developing these skills. Here, we discuss the statistical and expert validation of this instrument using 2681 valid student responses collected from 12 institutions during the 2017-2018 academic year. As a part of our validation, we address the impact of lab design on student performance on the instrument.

A ball on a stick is a common and simple activity for teaching the phases of the Moon. This activity, like many others in physics and astronomy, gives students a perspective they otherwise could only imagine. For Moon phases, a third person view and control over time allows students to rapidly build a mental model that connects all the moving parts. Computer simulations of many traditional physics and astronomy activities provide new features, controls, or vantage points to enhance learning beyond a hands-on activity. Virtual reality provides the capabilities of computer simulations and embodied cognition experiences through a hands-on activity making it a natural step to improve learning. We recreated the traditional ball-and-stick moon phases activity in virtual reality and compared participant learning using this simulation with using traditional methods. We found a strong participant preference for VR relative to the traditional methods. However, we observed no difference across conditions in average levels of performance on a pre/post knowledge test., Comment: 4 pages - Submitted to the 2018 Physics Education Research Conference

While laboratory instruction is a cornerstone of physics education, the impact of student behaviours in labs on retention, persistence in the field, and the formation of students' physics identity remains an open question. In this study, we performed in-lab observations of student actions over two semesters in two pedagogically different sections of the same introductory physics course. We used a cluster analysis to identify different categories of student behaviour and analyzed how they correlate with lab structure and gender. We find that, in lab structures which fostered collaborative group work and promoted decision making, there was a task division along gender lines with respect to laptop and equipment usage (and found no such divide among students in guided verification labs)., Comment: 4 pages, 3 figures, 3 tables

Many institutions are changing the focus of their introductory physics labs from verifying physics content towards teaching students about the skills and nature of science. As instruction shifts, so too will the ways students approach and behave in the labs. In this study, we evaluated students' lab notes from an early activity in an experimentation-focused lab course. We found that about 30% of student groups (out of 107 groups at three institutions) recorded questionable research practices in their lab notes, such as subjective interpretations of results or manipulating equipment and data. The large majority of these practices were associated with confirmatory goals, which we suspect stem from students' prior exposure to verification labs. We propose ways for experimentation-focused labs to better engage students in the responsible conduct of research and authentic scientific practice., Comment: 4 pages, 4 figures

Framing affects how students interpret, approach, and accomplish tasks. Little is known, however, about how students frame tasks in physics labs. During the first lab of a sequence designed to teach students about modeling and critical thinking with data, students test a simple model of a pendulum that breaks down with improved measurements. Using in-lab video and follow-up interviews, we identified students' frequent use of a model-verifying frame that substantially interferes with the instructional goals. We present a case study analysis of two students who approach the lab with a model-verifying frame, engage in problematic behaviors including questionable research practices, but later shift their frames to accommodate goals aligned with instructional intention. As instructors transition their instructional labs to open-inquiry experiences, an activity that directly challenges the model-verifying frame may be productive for shifting students away from this problematic frame to one that supports their engagement in authentic experimentation.

Although an important goal of introductory physics labs is to train students in scientific reasoning and critical thinking, currently there are no standard tests in physics designed to assess such skills. We are in the process of developing and validating the Physics Lab Inventory of Critical thinking (PLIC), an assessment to probe students' critical thinking abilities in physics lab courses. The instrument asks students to critique a set of experimental methods and data and use them to evaluate a particular physical model (the period of a mass on a spring). Currently, we are validating the closed-response survey through interviews with students and present the results of 12 such interviews here. We describe a trend that has emerged from these interviews, with students' reasoning falling into three main patterns of behavior: selecting all options, cuing on keywords, and critically analyzing. We have found ways to shift students to the last and more desirable behavior. We discuss ways in which these findings are likely relevant to the design of other concept inventories., Comment: 4 pages

Instructors in introductory physics courses often use labs and demonstrations to reinforce that the physics equations introduced in lectures and textbooks describe what actually happens in the real world. The surface features and instructional learning goals of such labs and demonstrations may seem quite similar (predict-observe-explain). Nonetheless, physics education research has found very different impacts of these instructional methods on students' physics content knowledge that depend critically on several instructional characteristics. In this paper, we disentangle the research by comparing details of the learning methods in verification labs, traditional lecture demonstrations, and enhanced lecture demonstrations. We discuss possible mechanisms for the measurably different learning outcomes by dissecting the activities according to the role of prediction, cognitive load, and engagement. We use this characterization to motivate rethinking the goals of labs towards experimentation skills and beliefs, rather than for verifying physics content.

Instructional labs are widely seen as a unique, albeit expensive, way to teach scientific content. We measured the effectiveness of introductory lab courses at achieving this educational goal across nine different lab courses at three very different institutions. These institutions and courses encompassed a broad range of student populations and instructional styles. The nine courses studied had two key things in common: the labs aimed to reinforce the content presented in lectures, and the labs were optional. By comparing the performance of students who did and did not take the labs (with careful normalization for selection effects), we found universally and precisely no added value to learning from taking the labs as measured by course exam performance. This work should motivate institutions and departments to reexamine the goals and conduct of their lab courses, given their resource-intensive nature. We show why these results make sense when looking at the comparative mental processes of students involved in research and instructional labs, and offer alternative goals and instructional approaches that would make lab courses more educationally valuable., Comment: Accepted to Phys Rev PER

Developing critical thinking skills is a common goal of an undergraduate physics curriculum. How do students make sense of evidence and what do they do with it? In this study, we evaluated students' critical thinking behaviors through their written notebooks in an introductory physics laboratory course. We compared student behaviors in the Structured Quantitative Inquiry Labs (SQILabs) curriculum to a control group and evaluated the fragility of these behaviors through procedural cueing. We found that the SQILabs were generally effective at improving the quality of students' reasoning about data and making decisions from data. These improvements in reasoning and sensemaking were thwarted, however, by a procedural cue. We describe these changes in behavior through the lens of epistemological frames and task orientation, invoked by the instructional moves.