Search

Abstract Contemporary science competitions particularly have two intentions: (1) identifying the students demonstrating the highest levels of domain-specific cognitive abilities and (2) recognizing and valuing the efforts of engaged and motivated students, even those without exceptional abilities. This study aimed to examine the relative influence of affective and cognitive variables on predicting success among 136 participants of the first two rounds of the German Physics Olympiad, and based on that, evaluate the extent to which the Physics Olympiad meets the outlined intentions. Our findings indicate that the competition’s initial round erects a hurdle for engaged and motivated students who lack sufficient cognitive abilities, which goes against the above mentioned second intention. Conversely, the Physics Olympiad appears to effectively align with its first intention by successfully identifying students with high developed physics-specific abilities. Building on our findings, we discuss ways for better aligning the competition with its intentions, thus contributing to the ongoing further development of science competitions.

Addressing 21st century challenges, professionals competent in science, technology, engineering, and mathematics (STEM) will be indispensable. A stronger individualisation of STEM learning environments is commonly considered a means to help more students develop the envisioned level of competence. However, research suggests that career aspirations are not only dependent on competence but also on STEM identity development. STEM identity development is relevant for all students, but particularly relevant for already under-served students. Focusing solely on the development of competence in the individualisation of STEM learning environments is not only harming the goal of educating enough professionals competent in STEM, but may also create further discrimination against those students already under-served in STEM education. One contemporary approach for individualisation of learning environments is learning analytics. Learning analytics are known to come with the threat of the reproduction of historically grown inequalities. In the research field, responsible learning analytics were introduced to navigate between potentials and threats. In this paper, we propose a theoretical framework that expands responsible learning analytics by the context of STEM identity development with a focus on under-served students. We discuss two major issues and deduce six suppositions aimed at guiding the use of as well as future research on the use of learning analytics in STEM education. Our work can inform political decision making on how to regulate learning analytics in STEM education to help providing a fair chance for the development of STEM identities for all students.

[This paper is part of the Focused Collection on Qualitative Methods in PER: A Critical Examination.] Qualitative research methods have provided key insights in physics education research (PER) by drawing on non-numerical data such as text or video data. While different methods towards qualitative research exist, they share two essential steps: recognizing patterns in the data and interpreting these patterns. Although these methods have led to the development of rigorous theory, there are challenges: As such methods require a series of judgments by the analyst, they are difficult to validate and reproduce. Further, they are hard to scale so that they are unavailable to the analysis of large-scale data. In this way, important phenomena may remain inaccessible to qualitative analysis. Reacting to these challenges and leveraging the potential of emerging methods of artificial intelligence (AI) such as machine learning and natural language processing, sociologist Nelson has proposed the concept of computational grounded theory (CGT). CGT proceeds in a process of three consecutive steps: In the first step, one leverages the power of computational techniques, especially natural language processing and unsupervised machine learning techniques, for pattern detection in large datasets—those of a size and scope that may prohibit human-driven analysis from the outset. In the second step, one relies on the integrative and interpretative capabilities of human researchers to add quality and depth to the quantity and breadth of the first step. In the last step, one again uses computational techniques to test the extent to which the detected and refined patterns from the first and second step hold throughout the whole dataset under investigation. Interestingly, CGT does not aim at simply automating parts of the qualitative process by using AI, but rather aims at integrating AI into the human analyst’s workflow within a qualitative analysis. This leads to an analytical system that can do something that is quantitatively and qualitatively different from what a human or machine can do alone. In this way, CGT aims at addressing questions about validity, reproducibility, and scalability in qualitative research while preserving the theoretical sensitivity and unique inferencing capabilities of the human analyst. In this paper, we provide a primer on CGT, present how it can be used to investigate the physics problem-solving approaches of N=417 students based on textual data, and discuss CGT’s potentials and challenges in PER. In consequence, this paper can provide critical input to the discussion of how emerging AI technologies can provide new avenues in qualitative PER.

National educational standards stress the importance of science and mathematics learning for today’s students. However, across disciplines, students frequently struggle to meet learning goals about core concepts like energy. Digital learning environments enhanced with artificial intelligence hold the promise to address this issue by providing individualized instruction and support for students at scale. Scaffolding and feedback, for example, are both most effective when tailored to students’ needs. Providing individualized instruction requires continuous assessment of students’ individual knowledge, abilities, and skills in a way that is meaningful for providing tailored support and planning further instruction. While continuously assessing individual students’ science and mathematics learning is challenging, intelligent tutoring systems show that it is feasible in principle. However, the learning environments in intelligent tutoring systems are typically not compatible with the vision of how effective K-12 science and mathematics learning looks like. This leads to the challenge of designing digital learning environments that allow for both – meaningful science and mathematics learning and the reliable and valid assessment of individual students’ learning. Today, digital devices such as tablets, laptops, or digital measurement systems increasingly enter science and mathematics classrooms. In consequence, students’ learning increasingly produces rich product and process data. Learning Analytics techniques can help to automatically analyze this data in order to obtain insights about individual students’ learning, drawing on general theories of learning and relative to established domain specific models of learning, i.e., learning progressions. We call this approach Learning Progression Analytics (LPA). In this manuscript, building of evidence-centered design (ECD), we develop a framework to guide the development of learning environments that provide meaningful learning activities and data for the automated analysis of individual students’ learning – the basis for LPA and scaling individualized instruction with artificial intelligence.

Abstract Energy conservation is a fundamental concept in physics and across the sciences as it provides a lens for investigating a wide range of phenomena. Research into energy learning progressions has shown that a majority of students across K-12 struggle with energy conservation. These studies characterize students’ learning progressions as starting from energy being manifest in different forms. Research suggests that learning progressions that begin with the idea of forms only lead to an understanding of conservation for a minority of students. Thus, the question arises whether there are alternative, more productive pathways towards conservation than going through forms. We investigated to what extent students progress towards conservation if they are taught in a transfer-only approach to teaching energy that does not require forms. Drawing on interviews from N = 30 students across different time points in a 10 week transfer-only unit, we found that at the end of the unit, most students tracked energy successfully across systems, and did not violate energy conservations when explaining phenomena, that is, progressed towards a qualitative understanding of conservation. Our results imply that energy learning progressions do not have to go through forms and in fact a more productive pathway towards conservation may exist in the transfer-only approach.

This paper presents a teacher dashboard intervention study in secondary school practice involving teachers (n = 16) with their classes (n = 22) and students (n = 403). A quasi-experimental treatment-control group design was implemented to compare student learning outcomes between classrooms where teachers did not have access to the dashboard and classrooms where teachers had access to the dashboard. We examined different points in the impact chain of the "LA Cockpit," a teacher dashboard with a feedback system through which teachers can send feedback to their students on student learning. To investigate this impact chain from teacher use of dashboards to student learning, we analyzed 1) teachers' perceived technology acceptance of the LA Cockpit, 2) teacher feedback practices using the LA Cockpit, and 3) student knowledge gains as measured by pre- and post-tests. The analysis of n = 355 feedback messages sent by teachers through the LA Cockpit revealed that the dashboard assists teachers in identifying students facing difficulties and that teachers mostly provided process feedback, which is known to be effective for student learning. For student learning, significantly higher knowledge gains were found in the teacher dashboard condition compared to the control condition.

Science education research has repeatedly revealed students’ problems with understanding the energy concept, in particular, their understanding of potential energy. Newer research suggests that teaching students fields as a means to store potential energy may promote a more consistent understanding of the energy concept. Yet, it is presently unknown if the helpfulness of fields is limited to particular manifestations of potential energy or if the conceptualization of potential energy as energy stored in fields may constitute a coherent approach that supports students’ continued learning on energy. To address this issue, we carried out a quasiexperimental pre-post-test study with N=64 students from grade 6, with students being assigned to one of two conditions (fields-based vs nonfields-based approach to energy instruction). We also compared students continued learning in a subsequent unit on electric energy. Our findings suggest that students in the fields approach not only outperformed students in the nonfields approach, they were, in particular, able to use their understanding of potential energy to make better sense of electric energy, a form of energy previously unknown to them. The results of our study imply that fields can help students develop a deeper understanding of energy, in particular potential energy, and support continued learning about energy; that is, the learning about additional forms of energy.

Background: Teacher dashboards can help secondary school teachers manage online learning activities and inform instructional decisions by visualising information about class learning. However, when designing teacher dashboards, it is not trivial to choose which information to display, because not all of the vast amount of information retrieved from digital learning environments is useful for teaching. Information elicited from formative assessment (FA), though, is a strong predictor for student performance and can be a useful data source for effective teacher dashboards. Especially in the secondary education context, FA and feedback on FA, have been extensively studied and shown to positively affect student learning outcomes. Moreover, secondary teachers struggle to make sense of the information displayed in dashboards and decide on pedagogical actions, such as providing feedback to students. Objectives: To facilitate the provision of feedback for secondary school teachers via a teacher dashboard, this study identifies requirements for designing a Learning Analytics Cockpit (LA Cockpit), that is, (1) a teacher dashboard that provides teachers with visualisations of results from formative assessment (FA) and (2) a feedback system that supports teachers in providing feedback to students. Methods: This study was conducted in the context of STEM classes and is based on semi-structured co-design interviews with German secondary school teachers. In these interviews, we first explored challenges teachers encountered in monitoring students' learning and providing feedback. Second, in the ideation phase, teachers were asked to define features an LA Cockpit for FA should have. Finally, in the evaluation phase, we provided teachers with a design template for an LA Cockpit, the LAC_Template, which was built upon our previous work and feedback theory, and asked them to evaluate and improve it. Further design requirements were derived based on the evaluation of the LAC_Template and teachers' suggestions for improvement. Results: We derived 16 requirements for designing an LA Cockpit for FA in secondary schools. Findings from the interviews indicated that the feedback system of an LA Cockpit should address teachers' time limitations in giving students individualised feedback. It should therefore be designed to minimise the steps required to deliver feedback. To reduce workload, teachers requested an automated reminder to send feedback, but with the ability to adjust feedback to the learning context. Such a semi-automated feedback system can help teachers support students individually but also underline the importance of actively involving teachers in the feedback loop and giving them control when using such technologies in secondary school practice. A challenge for future teacher dashboard designs could be to find a balance between technology and teacher control that utilises the strengths of both in a beneficial combination.

One reason for the widespread use of the energy concept across the sciences is that energy analysis can be used to interpret the behavior of systems even if one does not know the particular mechanisms that underlie the observed behavior. By providing an approach to interpreting unfamiliar phenomena, energy provides a lens on phenomena that can set the stage for deeper learning about how and why phenomena occur. However, not all energy ideas are equally productive in setting the stage for new learning. In particular, researchers have debated the value of teaching students to interpret phenomena in terms of energy forms and transformations. In this study, we investigated how two different approaches to middle school energy instruction--one emphasizing energy transformations between forms and one emphasizing energy transfers between systems--prepared students to use their existing energy knowledge to engage in new learning about a novel energy-related phenomenon. To do this, we designed a new assessment instrument to elicit student initial ideas about the phenomenon and to compare how effectively students from each approach learned from authentic learning resources. Our results indicate that students who learned to interpret phenomenon in terms of energy transfers between systems learned more effectively from available learning resources than did students who learned to interpret phenomena in terms of energy forms and transformations. This study informs the design of introductory energy instruction and approaches for assessing how students existing knowledge guides new learning about phenomena.

This study pursued two major objectives. The first was to use bibliometric techniques to examine bidirectionality in the relationship between teachers and researchers, as indicated by collaborative authorship among these communities. The second was to explore more deeply knowledge mobilization to classrooms by documenting the extent to which research is cited in science education practitioner journals (SEPJ). Specifically, we examined: (a) the frequency of collaboration between researchers and practitioners in the writing of journal articles for both practitioner-focused and academic journals in science education, and (b) the extent to which authors of articles in practitioner-focused journals drew on academic research to support their advocacy for and/or description of science education programs, policies, or practices. Findings indicate that writing collaborations among academic researchers and practitioners are relatively infrequent, even on practitioner-focused articles. Also, articles in SEPJs more often cite books and other resources over academic journals, even those academic journals focused on informing science education teaching and learning. Recommendations include providing open access to published research, development of research summaries for lay audiences, and incentivizing practitioners to engage in research and writing. This study explores only one mechanism by which knowledge can be mobilized to classrooms and only one type of dissemination product (i.e., journal articles) upon which researchers and practitioners can collaborate. Additional limitations are noted including the applicability of the findings only to the specific journals and timeframes analyzed.

Energy is a central concept across the sciences and an important goal of science education is to support all students so that they develop a full understanding of the energy concept. However, given the abstract and complex nature of the energy concept, only a few students develop an understanding so that they can use energy ideas to make sense of phenomena. Research into energy learning progressions aims at developing models of learning about energy to guide instruction so that students can be best supported in developing competence and has provided a rich model of how students' understanding of energy develops over time. Being largely based on cross-section data, however, the extent to which this model can guide instruction is limited, especially concerning the continued learning of students about energy. To address this gap - the limited evidence regarding what supports students' continued learning about energy - it was investigated how holding non-normative ideas and the integratedness of students' energy knowledge affect students' continued learning about energy. Drawing on data from a 4-year longitudinal study covering Grades 6-9 on students' learning about energy, diagnostic classification models were used to characterize students' non-normative idea profiles and the integratedness of their knowledge and then related both to their continued learning. The results suggest no detrimental effects of holding non-normative ideas and strong positive effects of holding integrated knowledge for students' continued learning about energy. Implications for teaching and future research are discussed.

Knowledge-in-Use, i.e., the ability to apply what one has learned, is a major goal of education and involves the ability to transfer one’s knowledge. While some general principles of knowledge transfer have been revealed, the literature is full of inconclusive results and it remains hard to predict successful transfer. However, research into expertise suggests that how one organizes one’s knowledge is critical for successful transfer. Drawing on data from a larger study on the learning of energy, we employed network analysis to investigate how the organization of students’ knowledge about energy influenced their ability to transfer and what role achievement goal orientation may have played in this. We found that students that had more coherently organized knowledge networks were more successful in transfer. Furthermore, we also found a connection between mastery goal orientation and the organization of students’ knowledge networks. Our results extend the literature by providing evidence for a direct connection between the organization of students’ knowledge networks, their success in transfer, and their goal orientation and hint at the complexities in the relationship between mastery approach goal orientation and successful transfer beyond what is reported in the literature.

Science competitions target students interested in science with the aim to support them in developing science competence and career aspirations. Contrary to the common belief that science competition participants are exceptionally competent and strongly motivated to pursue a science career, there is growing evidence that the entirety of participants is rather heterogeneous in terms of their cognitive and affective characteristics. For science competitions to better support all participants in developing competence and career aspirations, a better understanding of the cognitive and affective characteristics of the entirety of participants is required. This study examined the Physics Olympiad as a specific type of science competitions, leading to a nuanced characterisation of N = 155 Physics Olympiad participants. Latent profile analyses revealed four participant profiles distinguished by specific patterns in cognitive abilities, physics interest, and physics self-efficacy. Profiles differed in their performance in the competition and their physics career aspiration. Grade level, gender, previous participation in the competition, and teacher support explained differences in profile membership. Our findings emphasise that Physics Olympiad participants are a heterogeneous group with varying needs and offer implications for more individualised support activities to better support the entirety of participants in developing science competence and career aspirations.