Search

Previous research and experiences have indicated the potential that games have in educational settings. One of the possible uses of games in education is as game-based assessments (GBA), using game tasks to generate evidence about skills and content knowledge that can be valuable. There are different approaches in the literature to implement the assessment machinery of these GBA, all of them having strengths and drawbacks. In this article, we propose using multivariate Elo-based learner modeling, as we believe it has a strong potential in the context of GBA for three aims: first, to simultaneously measure students competence across several knowledge components in a game; second, to predict task performance; and finally, to estimate task difficulty within the game. To do so, we present our GBA Shadowspect, which is focused on solving geometry puzzles, and we depict our implementation using data collected from several high schools across the USA. We obtain high-performing results (AUC of 0.87) and demonstrate that the model enables analysis of how each student's competency evolves after each puzzle attempt. Moreover, the model provides accurate estimations of each task's difficulty, enabling iterative improvement of the game design. This study highlights the potential that multivariate Elo-based learner modeling has within the context of GBA, sharing lessons learned, and encouraging future researchers in the field to consider this algorithm to build their assessment machinery.

While school educators are increasingly interested in adopting maker pedagogy, many schools struggle to integrate making with their existing core curriculum because of the difficulty in assessing the learning that occurs in maker classrooms. To address this issue, we collaborated with educators on design-based research focused on assessment in maker classrooms. We investigated assessment approaches that support the process of making. This paper reports on four design principles of assessment in school-based making that emerged from a literature review and the collaborative development process with educators. We discuss the challenges and opportunities for assessment in maker classrooms, as well as strategies to seamlessly embed assessment within a classroom's culture, norms, and activities.

Game-based assessment (GBA), a specific application of games for learning, has been recognized as an alternative form of assessment. While there is a substantive body of literature that supports the educational benefits of GBA, limited work investigates the validity and generalizability of such systems. In this paper, we describe applications of learning analytics methods to provide evidence for psychometric qualities of a digital GBA called Shadowspect, particularly to what extent Shadowspect is a robust assessment tool for middle school students' spatial reasoning skills. Our findings indicate that Shadowspect is a valid assessment for spatial reasoning skills, and it has comparable precision for both male and female students. In addition, students' enjoyment of the game is positively related to their overall competency as measured by the game regardless of the level of their existing spatial reasoning skills.

Purpose: When using embedded, student-centered assessment tools for maker education, understanding the characteristics of a body of evidence can help teachers guide the assessment process. This study aims to examine assessment artifacts from a makerspace program and present a set of qualities that emerged, which researchers and maker educators can use to evaluate the quality of evidence before interpreting it and making claims about student learning. Design/methodology/approach: This study used the interpretive analysis approach to identify salient qualities in a body of evidence of maker learning. Data sources included student assessment artifacts, researchers' analytic memos, notes on the coding and analysis process, background stories and field observations. Findings: The study found that the assessment artifacts generated by students aligned with the maker-related target skills. A set of qualities was produced that can be used to describe the strength of a body of evidence and help determine whether it is appropriate to be used in the meaning making phase. Practical implications: The qualities identified in this study can be directly incorporated into the embedded assessment toolkit to provide feedback on the strength of evidence for learning in makerspaces. Originality/value: Assessment methods for maker education are nascent, and ways to describe the quality of a student-generated body of evidence have not yet been established. This study applies existing knowledge of embedded assessment and reflective practice toward the creation of a new way of assessing skills that are difficult to measure.

Purpose: As maker-centered learning grows rapidly in school environments, there is an urgent need for new forms of assessment. The purpose of this paper is to report on the development and implementation of tools to support embedded assessment of maker competencies within school-based maker programs and describes alternative assessment approaches to rubrics and portfolios. Design/methodology/approach: This study used a design-based research (DBR) method, with researchers collaborating with US middle school teachers to iteratively design a set of tools that support implementation of embedded assessment. Based on teacher and student interviews, classroom observations, journal notes and post-implementation interviews, the authors report on the final phase of DBR, highlighting how teachers can implement embedded assessment in maker classrooms as well as the challenges that teachers face with assessment. Findings: This study showed that embedded assessment can be implemented in a variety of ways, and that flexible and adaptable assessment tools can play a crucial role in supporting teachers in this process. Additionally, though teachers expressed a strong desire for student involvement in the assessment process, we observed minimal student agency during implementation. Further study is needed to investigate how establishing classroom culture and norms around assessment may enable students to fully participate in assessment processes. Originality/value: Due to the dynamic and collaborative nature of maker-centered learning, teachers may find it difficult to provide on-the-fly feedback. By employing an embedded assessment approach, this study explored a new form of assessment that is flexible and adaptable, allowing teachers to formally plan ahead while also adjusting in the moment.

Learning games have great potential to become an integral part of new classrooms of the future. One of the key reported benefits is the capacity to keep students deeply engaged during their learning process. Therefore, it is necessary to develop models that can measure quantitatively how learners are engaging with learning games to inform game designers and educators, and to find ways to maximize learner engagement. In this article, we present our proposal to multidimensionally measure engagement in a learning game over four dimensions: general activity, social, exploration, and quests. We apply metrics from these dimensions to data from The Radix Endeavor, an inquiry-based online game for STEM learning that has been tested in K-12 classrooms as part of a pilot study across numerous schools. Based on these dimensions, we apply clustering and report four different engagement profiles that we define as "integrally engaged," "lone achiever," "social explorer," and "nonengaged." We also use three variables (account type, class grade, and gender) to perform a cross-sectional analysis finding interesting, statistically significant differences in engagement. For example, in-school students and accounts registered to males engaged socially much more than out-of-school learners or accounts registered to females, and that older students have better performance metrics than younger ones.

Interactive simulations allow preservice teachers to connect education theory and pedagogy in scaffolded environments. We created digital simulations with scenarios from in-person simulations and used them to prepare novice teachers for conversations with parents. Using a design-based approach we implemented the simulations in an education class, gathered data through surveys and observations, and incorporated feedback into subsequent designs. Novice teachers perceived the simulation as authentic and practiced maintaining composure and articulating pedagogical approaches. Recordings of novice teachers' responses produced by the simulation enabled self-reflection and peer and instructor feedback. Results suggest that these digital simulations hold promise as low-cost, flexible tools for novice teachers to engage in targeted practice in a low-stakes setting.

Game-based assessment (GBA) is a specific use of educational games that employs game activities to elicit evidence for educationally valuable skills and knowledge. While this approach can provide individualized and diagnostic information about students, the design and development of assessment mechanics for a GBA is a nontrivial task. In this article, we describe the 10-step procedure that the design team of "Physics Playground" (formerly known as "Newton's Playground") has established by adapting evidence-centered design to address unique challenges of GBA. The scaling method used for Physics Playground was Bayesian networks; thus this article describes specific actions taken for the iterative process of constructing and revising Bayesian networks in the context of the game Physics Playground.