178,207 results on '"measurement"'
Search Results
2. Professional Collaboration among Elementary School Teachers in Lesson Study
- Author
-
Adriana Richit, João Pedro da Ponte, and Mauri Luís Tomkelski
- Abstract
Professional teacher collaboration, a fundamental characteristic of lesson study, has been pointed out as a principle of teacher personal and professional growth. This article aims to describe and analyze the development of professional collaboration in a lesson study, highlighting the main stages of this process. The investigation involved six elementary school teachers of Brazilian public schools, in planning, observing, and reflecting on a lesson on the curriculum topic of length measure at grade 4. The participants worked collaboratively planning a lesson about this topic. The methodology is qualitative, with data collected by records and field notes of sessions, conversations, and interviews. The analyses pointed out three remarkable stages of the development of professional collaboration, which constituted the categories of analysis: recognition of collaboration, development of collaboration, and valuing collaboration. The results show that the professional collaboration that developed during the lesson study was facilitated by the dynamics of the different steps of this process and, especially, by the relationship between participants and teacher educators. Collaboration emerged from the activities of planning, teaching, observing, and reflecting on the lesson, and was valued given the encouragement and strengthening of the group insofar as teachers felt confident, encouraged, and welcomed in the group.
- Published
- 2024
3. Making Assessment Reports More Meaningful for Students & Families
- Author
-
Education Trust, Shayna Levitan, and Nicholas Munyan-Penney
- Abstract
Statewide annual assessments are designed to help guide educational decisions by evaluating student growth and learning so resources can be equitably distributed. They also provide valuable individual results that summarize how each student is progressing against grade-level academic standards, identify additional ways their school system can support their academic success, and allow families to demand appropriate actions from their education leaders. This report offers guidance on how schools can better communicate the results of students' federally required annual tests so parents can be better informed about how their students are doing in school and work with educators to get appropriate support. Specifically, this brief: (1) Calls for individual assessment reporting results to be integrated into existing communications with families; (2) Highlights the role that trusted voices play in communicating this information; (3) Identifies best practices for designing individual score reports for students and families; and (4) Offers promising strategies and policy recommendations that advocates can push for to improve assessment reporting at the state level.
- Published
- 2024
4. Universal Design and K-12 Academic Assessments: A Scoping Review of the Literature. NCEO Report 442
- Author
-
National Center on Educational Outcomes (NCEO), Council of Chief State School Officers (CCSSO), Applied Engineering Management Corporation (AEM), Center for Parent Information & Resources (CPIR), National Association of State Directors of Special Education (NASDSE), WestEd, Kristin K. Liu, Martha L. Thurlow, Mari Quanbeck, Jessica A. Bowman, and Amy Riegelman
- Abstract
This report summarizes an investigation of literature published in 1985-2023 on universal design (UD) of large-scale assessments. State education agencies, K-12 assessment vendors, teacher trainers, and classroom teachers are increasingly applying the principles of UD to the instruction and assessment of students from special populations. In theory, UD ensures that instruction and assessment are created from the beginning to be accessible to the widest population of students possible. Yet, it is unclear how these UD frameworks and their associated principles and elements are being applied, or even which UD framework is used. This obscurity and lack of defined UD principles in the research literature creates obstacles in research replication and in identification of evidence-based practices. This scoping review identifies how the concept of UD has been applied to the broad range of U.S. district and state large-scale academic assessments. We included literature on the application of UD to the broad range of K-12 district and state large-scale academic assessments in the U.S.
- Published
- 2024
5. Preferred Problem-Solving Methods Employed by Grade 4 Learners for Measurement Word Problems
- Author
-
Rajendran Govender, Stanley A. Adendorf, and Shabbeer Rawoot
- Abstract
Background: Problem-solving as a vehicle to develop independent thinking skills is mostly underestimated and is often either overlooked or not given adequate attention within the existing South African mathematics curriculum. Consequently, numerous learners often display limited skills or lack skills to adequately crack Mathematics problems by applying methods put forward in class. This generally results in under-achievement. Aim: This study aims to explore and emphasise the problem-solving methods applied by Grade 4 learners involved in solving measurement word problems, and to reveal what transpires when the selected learners apply these methods to arrive at meaningful solutions. Setting: Data were collected from a class of 42 Grade 4 learners at a primary school in Cape Town South Africa. Learners were conveniently selected. Methods: A qualitative case study research design was adopted. Data gathering instruments of the study included observing learners solving, measurement word problem activities and focus group interviews. Results: The study revealed that singular methods were applied by Grade 4 learners, such as, adding, multiplying, creating a sketch or diagram, grouping, dividing, subtracting, logical reasoning, guessing and tabulating values. Conclusion: Grade 4 learners are prone to applying methods such as clustering or organising into groups, tabulating numerical values and logical reasoning were all applying mathematically sound methods. Such learners, however, needed a degree of supervision and instruction to indicate the way in which such methods were applied successfully as these methods were not necessarily dealt with in classroom context or in textbooks. Contribution: The findings emphasise the need for tackling learners' limited problem-solving competencies and accentuate the necessity for greater attention to develop and grow methods for optimal and successful solving of problems in context.
- Published
- 2024
6. Developing Complex Unfamiliar Mathematics Questions: A Perspective
- Author
-
Mathematics Education Research Group of Australasia (MERGA), David Chinofunga, and Philemon Chigeza
- Abstract
This paper reports on the development of complex unfamiliar mathematics questions using conventional mathematics questions. The autoethnographic research data included retrospective reflections of two educators and review of literature. Data analysis resulted in a three-phase development process: identifying conventional questions and subject matter to be modified to enhance levels of understandings and skills, modifying conventional questions and subject matter to complex unfamiliar questions, and identifying enhanced understandings and skills in modified questions. The paper then discusses the educators' insights developing complex unfamiliar mathematics questions for the practice of teaching and learning.
- Published
- 2024
7. PIRLS 2026 Assessment Frameworks
- Author
-
International Association for the Evaluation of Educational Achievement (IEA) (Netherlands), Boston College, TIMSS & PIRLS International Study Center, Matthias von Davier, and Ann Kennedy
- Abstract
The Progress in International Reading Literacy Study (PIRLS) has been monitoring international trends in reading achievement among fourth-grade students for 25 years. As a critical point in a student's education, the fourth year of schooling establishes the foundations of literacy, with reading becoming increasingly central to learning across all subjects. PIRLS 2026 represents a significant milestone in the assessment's history by completing the transition to a fully digital reading assessment. This move reflects PIRLS' commitment to leading the field and staying relevant in an era where digital literacy is essential for every child's education. By embracing the digital reading assessment format, PIRLS 2026 aims to create an engaging, authentic assessment experience that aligns with young learners' daily digital reading practices. Additionally, technology enhances assessment practices, including more efficient data collection, innovative test design, and automated scoring. The PIRLS 2026 Assessment Frameworks consist of three chapters covering major assessment components: (1) Reading Assessment Framework (Chapter 1): Describes the aspects of reading literacy assessed in PIRLS; (2) Contextual Framework (Chapter 2): Addresses topics covered in the PIRLS 2026 context questionnaires and Encyclopedia; and (3) Assessment Design (Chapter 3): Presents the rationale and procedures for the groupadaptive assessment design. Updates to the PIRLS 2026 framework chapters involved contributions from National Research Coordinators (NRCs) in participating countries and from experts at partner institutions who assisted in developing item drafts and framework chapters. Their input informed new constructs of interest for the questionnaires, ensuring the frameworks remain relevant for measuring and reporting on reading achievement over time. Expert committees--the Reading Development Group (RDG) and Questionnaire Development Group (QDG)--guided revisions based on evolving theories and policy-focused research.
- Published
- 2024
8. Discovering Educational Data Mining: An Introduction
- Author
-
Zachary K. Collier, Joshua Sukumar, and Roghayeh Barmaki
- Abstract
This article introduces researchers in the science concerned with developing and studying research methods, measurement, and evaluation (RMME) to the educational data mining (EDM) community. It assumes that the audience is familiar with traditional priorities of statistical analyses, such as accurately estimating model parameters and inferences from those models. Instead, this article focuses on data mining's adoption of statistics and machine learning to produce cutting-edge methods in educational contexts. It answers three questions: (1) What are the primary interests of EDM and RMME researchers?; (2) What is their discipline-specific vocabulary?; and (3) What are the similarities and differences in how the EDM and RMME communities analyze similar types of data?
- Published
- 2024
9. Performance-Based Accountability: Examining Turkish Teachers' Perceptions Regarding the Implementation of Large-Scale Assessment
- Author
-
Fatma Cobanoglu
- Abstract
This study, designed as a basic qualitative research, aims to evaluate the perspectives of Turkish teachers regarding the nationally implemented large-scale assessments in specific subjects. By employing purposive sampling methods, particularly maximum variation sampling, 14 teachers with different seniorities and branches from various high schools were included in the research. Data were collected using a semi-structured interview form prepared by the researcher. Through the application of descriptive analysis, it was found that teachers viewed these assessments more as a preparation for changes in the higher education transition system and as a tool for achieving standardization across the country rather than as an accountability policy instrument. Despite the low-stakes nature of this assessment, it was observed that teachers exhibited responses like those documented in the literature for high-stakes accountability. Additionally, it was found that teachers prepared students for the exams primarily through practice exercises and experienced anxiety before the exams due to the limitation of their autonomy and the comparison of their students and themselves. Furthermore, most participants believed that the assessment results were not effective in determining students' learning, primarily held the students accountable for the results, and considered such assessments as a source of stress. Based on the findings, recommendations were made for practitioners and researchers.
- Published
- 2024
10. Large-Scale Assessments for Learning: A Human-Centred AI Approach to Contextualizing Test Performance
- Author
-
Hongwen Guo, Matthew S. Johnson, Kadriye Ercikan, Luis Saldivia, and Michelle Worthington
- Abstract
Large-scale assessments play a key role in education: educators and stakeholders need to know what students know and can do, so that they can be prepared for education policies and interventions in teaching and learning. However, a score from the assessment may not be enough--educators need to know why students got low scores, how students engaged with the tasks and the assessment, and how students with different levels of skills worked through the assessment. Process data, combined with response data, reflect students' test-taking processes and can provide educators such rich information, but manually labelling the complex data is hard to scale for large-scale assessments. From scratch, we leveraged machine learning techniques (including supervised, unsupervised, and active learning) and experimented with a general human-centred AI approach to help subject matter experts efficiently and effectively make sense of big data (including students' interaction sequences with the digital assessment platform, such as response, timing, and tool use sequences) to provide process profiles, that is, a holistic view of students' entire test-taking processes on the assessment, so that performance can be viewed in context. Process profiles may help identify different sources for low performance and help generate rich feedback to educators and policy makers. The released National Assessment of Educational Progress (NAEP) Grade 8 mathematics data were used to illustrate our proposed approach.
- Published
- 2024
11. Metrics but Little Meaning: Department of Education, Skills and Employment Reporting about Australian Sociology PhDs Completions
- Author
-
Adam Rajcan and Edgar A. Burns
- Abstract
As part of a study investigating research productivity of sociology PhD students in Australia, an application to the federal government's Department of Education, Skills and Employment (DESE) aimed to establish a baseline count of completed sociology doctorates by university. It was anticipated that university totals might be different from PhD completions within the main sociology teaching units. The government data furnished by this inquiry have proved to be problematic in several ways. The gaps in the data may be explained by the fact that universities report sociology PhDs to DESE in more generic rather than discipline-specific Field of Education (FoE) codes or as PhDs in other disciplines. DESE has either not required or been able to enforce consistent reporting. This raises questions about the time and cost value of these annual compliance efforts. It also limits the value of these data to researchers in charting current and potential future trends.
- Published
- 2024
12. 2024-2025 Accessibility & Accommodations Information for Statewide Assessments
- Author
-
Indiana Department of Education (IDOE), Office of Student Assessment
- Abstract
The Indiana Department of Education's (IDOE's) Accessibility and Accommodations Information for Statewide Assessments is a document intended for school-level personnel and decision-making teams as they prepare for and implement Indiana statewide assessments. Information is provided for school personnel as a reference to inform guidance on accessibility considerations including universal features, designated features, and accommodations for students. Federal and state laws require that all students, including students with disabilities and English learners (ELs), participate in statewide assessments to maintain school accountability for student academic performance. The validity of assessment results depends on the utilization of the full array of accessibility features and accommodations appropriately for each student. These options include appropriate universal features, designated features, and accommodations, when needed, based on the constructs being measured by the assessment. Indiana statewide assessments are developed using principles of universal design to support the fair and valid measurement of all students' knowledge and skills. This means that the assessment systems were designed to have the broadest use possible for the full student population. IDOE implements systematic steps through item development and content presentation to ensure accessibility is interwoven throughout all stages of assessment delivery and scoring outcomes. This document serves to describe and clarify the appropriate uses of accessibility features and accommodations for Indiana assessments ILEARN, IREAD-3, I AM, WIDA Assessments, Digital SAT School Day, ISPROUT, and Indiana's Kindergarten Readiness Assessment (IKRA).
- Published
- 2024
13. SPART -- An Affordable Mobile Augmented Reality Alternative to Interactive Tabletops in Education
- Author
-
Sebastian Simon, Iza Marfisi-Schottman, and Sébastien George
- Abstract
Technology to support collaborative learning has come a long way. Interactive tabletops support collaboration when correctly integrated in activity design. While these devices can now, in principle, be purchased by the general public, wide adoption in schools is hindered by their high cost and lack of mobility. In this paper, we analyze the potential technologies that could replace such devices. After developing a dozen prototypes to further test the potential of the most promising technologies, we found one robust, affordable and accurate solution: SPART (on-Surface Positioning for Augmented RealiTy) allows augmenting any flat surface (images etc.) with tablets or smartphones, addressing both affordability and mobility. This paper focuses on the technological innovations that were designed for SPART. [For the full proceedings, see ED659933.]
- Published
- 2024
14. A Refined and Concise Model of Indices for Quantitatively Measuring Lexical Richness of Chinese University Students' EFL Writing
- Author
-
Yang Yang and Ze Zheng
- Abstract
In the existing literature, scholars have proposed various indices to measure the lexical richness (LR) of English as a foreign language (EFL) writing. However, there are currently issues of redundant indices and inconsistent usage. Attempting to address the research question of which indices are the most sensitive and effective ones to distinguish between different grade levels of Chinese university students' EFL writing, this study aims to put forward a refined and concise model of indices that can truthfully reflect LR in EFL writing. A total of 180 compositions were selected from a Chinese EFL learner corpus: "Spoken and written English corpus of Chinese learners." Scores of 28 LR indices of these compositions were computed using the software "Lexical Complexity Analyzer," "MATTR," and "Coh-Metrix." One-way ANOVA or Welch's ANOVA, depending on the variable's homogeneity of variances, was conducted for each index. Two criteria were applied to determine which index of a measure should be included in the refined model: whether the difference of an index is significant among different grade levels and the effect size of ANOVA. Based on the quantitative results of ANOVAs and qualitative human judgment based on literature, six indices of the six LR measures were included in the refined model: lexical density, lexical sophistication-I, verb sophistication-II, number of different words-expected sequence 50, corrected TTR, and squared verb variation-I. This refined model addresses the issues of redundancy and inconsistency in previous studies, providing a more accurate and efficient tool for assessing LR in EFL writing.
- Published
- 2024
15. Impact Story: Implementing Statewide Systems -- Formative Assessment in Missouri
- Author
-
Region 12 Comprehensive Center (R12CC)
- Abstract
Research suggests that students who participate in formative assessment perform better on measures of academic achievement than students who do not. To raise school and systematic performance, teachers need access to high-quality, rigorous tools designed to provide timely and individualized feedback for student learning. According to a report published in 2023 by the Missouri Department of Elementary and Secondary Education (DESE), 58% of third grade students are reading at only a basic or below basic level, based on 2022-2023 state assessment results. The Region 12 Comprehensive Center (R12CC), in collaboration with DESE, is working to implement a statewide common formative assessment (CFA) system, referred to as the Formative Assessment Builder. The CFA system will provide Missouri educators with equitable access to common formative assessments for student learning in the area of Grade 2 literacy as they prepare for the transition to 3rd grade instruction and high stakes assessments. The CFA system is an online portal hosted by DESE that provides access to professional learning modules and supporting materials. The portal will store assessment items created, vetted, and uploaded by DESE. All Missouri teachers and administrators will have free, unlimited, and direct access to the system. This report examines how the CFA system is an integral tool that will benefit both teachers in their instruction and students in their learning.
- Published
- 2024
16. Finite Complement Clauses in Disciplinary Research Articles Authored by Filipino Academic Writers
- Author
-
Hjalmar Punla Hernandez
- Abstract
Finite complement clauses (FCCs) are an understudied syntactic structure of L2 advanced academic writing. The present study cross-investigated FCCs in qualitative (QUALI) and quantitative (QUANTI) research articles written by Filipino academic writers (FAWs) in Applied Linguistics (APPLING), Communication (COMM), and Measurement and Evaluation (MEEV). Specifically, it determined the FCCs with the most occurrences across five disciplinary research article sub-registers and identified whether these FCCs differ significantly in terms of frequencies of use. With Hernandez's (2021) framework adapted from Biber and Gray's (2016) study, this research analyzed five FCCs in 42 disciplinary research articles. Major results revealed that verb-controlled that-clauses (with 33.69 as normalized frequency count) outnumbered other FCCs in research article sub-registers across disciplines. They were more recurrent in QUALI and QUANTI APPLING research articles (7.91 and 7.54, respectively) and QUANTI and QUALI COMM research articles (7.34 and 6.96, respectively) than in QUANTI MEEV research articles (3.94). In addition, a significant difference exists between them and other FCCs at the p<0.05 level in terms of frequencies of use. In view of these findings, it can be concluded that verb-controlled that-clauses are the most useful FCCs in APPLING, COMM, and MEEV research articles. Likewise, FAWs across the three disciplines write more informally as signposted by their repeated use of that complementizer with these FCCs controlled by verbs. The study draws its implications for academic writing instruction.
- Published
- 2024
17. Unravelling the Nexus between Human Values and Competencies to Religion among Educational Stakeholders in Mauritius Using Structural Equation Modelling
- Author
-
Shakeel M. C. Atchia
- Abstract
Religion has been associated with human values and competencies needed to shape young people into responsible citizens able to make a positive difference in the lives of others. However, this correlation remains dependent on the context. As the correlation has not yet been investigated in Mauritius, which is a multicultural and multi-ethnic country, this study aims to bridge this knowledge gap using a purely positivist epistemological approach. It will analyze the Mauritius-specific data derived from a public data set which was part of the "Mapping the Terrain" project to examine the correlation between religion and a broad set of human values and competencies, namely forgiveness, individualistic orientation, collectivistic orientation, self-efficacy, problem-solving, meaning-making, sense of belongingness, religiosity/spirituality, hope, life satisfaction, gratitude, emotion, regulation, empathy, and self-regulation. The descriptive analysis and the Structural Equation Model (SEM), developed using the SPSS and AMOS software, revealed that (i) independent of ethnic group, educational stakeholders in Mauritius have a high affinity to most of the human values and competences, and (ii) as compared with others, the Muslim community has a slightly lower affinity to forgiveness, life satisfaction, individualistic orientation, self-efficacy, and problem solving. These data, which represent important baseline information needed to inform policymakers and other educational stakeholders, were used to derive a set of recommendations that will further the development of the key human values and competencies needed to promote harmony in a multicultural and multi-ethnic country.
- Published
- 2024
18. A Systematic Literature Review on it Governance Mechanisms and Frameworks
- Author
-
Merve Hatice Karatas and Hüseyin Çakir
- Abstract
Nowadays, it is not common to come across sectors that can work and succeed without using information technology (IT). IT has now become a part of organizations' management and strategic decision-making mechanisms. Therefore, organizations make serious investments to develop software and system infrastructures and transfer business processes to digital environments such as cloud technologies. So, they get business resilient while they gain opportunities of spreading their business activities to wide stakeholders and customers. Here it is essential to provide assurance that an organization evaluates, develops, implements, maintains, disposes of its IT systems and manages life cycle of all information and information technology systems properly on all physical and logical environments. However, the most important matter is that an organization's objectives are met by the management practices of its information systems. While organizations carry out their digital transformation at a dizzying speed, organizations should be protected from material and moral damages such as loss of money, loss of commercial confidential information, loss of reputation by protecting the confidentiality, integrity and accessibility of the information assets they own and also comply with legal legislations such as the protection of personal data and intellectual property rights. In order to achieve this, organizations benefit from one or more frameworks, standards or directives that are suitable for their own structures and needs, separately or by integrating them. However, the most important point in this transformation process is the transition of organizations from IT management understanding to IT governance. In other words, IT has now become a part of the strategic decision-making mechanism by rising from the level of support tool to carry out business processes in organizations. When we search the academic literature, frameworks and standards, it is seen that the key elements of IT governance are structures, processes and relational mechanisms. If these three components are set up correctly and fit the organization, it can be assured that IT aligns with and supports the objectives of the organizations and that effective IT governance is achieved in every kind of organizations. In this research, databases such as Web of Science, IEEE, and SCOPUS are scanned and found articles are reviewed in order to make a literature review on IT Governance Mechanisms and Frameworks. Results are evaluated and discussed. By the way, suggestions for further studies were made and advanced researches were shed light on.
- Published
- 2024
19. A Comprehensive Study on Evaluating and Mitigating Algorithmic Unfairness with the MADD Metric
- Author
-
Melina Verger, Chunyang Fan, Sébastien Lallé, François Bouchet, and Vanda Luengo
- Abstract
Predictive student models are increasingly used in learning environments due to their ability to enhance educational outcomes and support stakeholders in making informed decisions. However, predictive models can be biased and produce unfair outcomes, leading to potential discrimination against certain individuals and harmful long-term implications. This has prompted research on fairness metrics meant to capture and quantify such biases. Nonetheless, current metrics primarily focus on predictive performance comparisons between groups, without considering the behavior of the models or the severity of the biases in the outcomes. To address this gap, we proposed a novel metric in a previous work (Verger et al., 2023) named "Model Absolute Density Distance" (MADD), measuring algorithmic unfairness as the difference of the probability distributions of the model's outcomes. In this paper, we extended our previous work with two major additions. Firstly, we provided theoretical and practical considerations on a hyperparameter of MADD, named "bandwidth," useful for optimal measurement of fairness with this metric. Secondly, we demonstrated how MADD can be used not only to measure unfairness but also to mitigate it through postprocessing of the model's outcomes while preserving its accuracy. We experimented with our approach on the same task of predicting student success in online courses as our previous work, and obtained successful results. To facilitate replication and future usages of MADD in different contexts, we developed an open-source Python package called maddlib (https://pypi.org/project/maddlib/). Altogether, our work contributes to advancing the research on fair student models in education.
- Published
- 2024
20. Development and Validation of a Reliable and Valid Assessment Tool for Measuring Innovative Thinking in Vocational Students
- Author
-
Peeraya Sukkeewan, Noawanit Songkram, and Jaitip Nasongkhla
- Abstract
The objective of this study was to develop a measure that possesses both reliability and validity in order to evaluate innovative thinking within the realm of education. To achieve this, the instrument's validity and reliability were evaluated through quantitative methods in two distinct phases. A team of educational experts conducted the process of establishing content validity and ensuring that the items on the instrument accurately reflected the intended constructs of creative thinking. Following that, the assessment of concept validity was conducted using confirmatory factor analyses. The aforementioned investigations resulted in the discovery of a five-factor solution consisting of 25 elements, all of which demonstrated scores beyond the crucial threshold. This successful outcome confirmed the presence of distinct factors representing different dimensions of innovative thinking. The study enrolled 1250 students from vocational education institutions as participants. The data obtained from the participants was subjected to principal component analysis and confirmatory factor analysis, leading to the development of a model that exhibited a good match with the empirical data. This indicated the effectiveness of the developed instrument in measuring innovative thinking capacity. In conclusion, the research effectively developed an accurate and dependable tool for evaluating innovative thinking in the realm of education. The gathering of positive data from the participants and meticulous quantitative analyses were responsible for this.
- Published
- 2024
21. Examining 5th Grade Students' Learning on Surface Area Calculations with Realistic Mathematics Education Approach
- Author
-
Elif Ertem Akbas and Lütfiye Yildirm
- Abstract
The fact that the mathematics course is abstract, that it is not possible to associate it with daily life, and that it is impossible to concretize abstract expressions causes a prejudice against the this course and leads to a decrease in the academic achievements of students. It is seen that throughout history, various studies have been carried out taking this situation into consideration. A group of these studies is on realistic mathematics education. Realistic mathematics education is an approach that aims to concretize the abstract concepts of mathematics by relating them to real-life situations. The purpose is to make mathematics more understandable by concretizing it. In the study, the subject of calculating the surface area of prisms with the use of realistic mathematics education was taught, and the purpose of the study was to examine the effects of this application. The study group consisted of 20 5th grade students. While determining this group, maximum variety sampling strategy was preferred. In the study, action research, one of the qualitative research methods, was used, and the data were interpreted with the help of descriptive analysis and content analysis. The data collection tools in the study included semi-structured interview forms, video recordings, photos and activity sheets prepared by the students under the guidance of the researcher in accordance with the realistic mathematics education approach. When the findings were examined, it was seen that the students showed a positive development in the meaning and visualization processes; that they could reach mathematical inferences on their own; that they enjoyed the mathematics lesson; and that the realistic mathematics education approach gave positive results on learning. Based on these results, it was suggested that not only the application of the realistic mathematics education approach for different acquisitions but also the inclusion of these activities in the textbooks in accordance with the curriculum will result in improvements in favor of learning.
- Published
- 2024
22. Looking Back to Move Forward
- Author
-
Digital Promise, Empirical Education Inc., Jeremy Roschelle, and Adam Schellinger
- Abstract
SEERNet digital learning platforms (DLPs) are developing new infrastructure to support research in authentic contexts where student learning is happening. In order to contextualize this work within the larger field, we trace historical precedents along four main categories: data repositories, data collection services, research design interfaces, and research communities. By situating this innovative movement alongside its predecessors, we can identify the opportunities for SEERNet and others to progress and sustain the mission of making research more scalable, equitable, and rigorous.
- Published
- 2024
23. Investigation of Preschoolers' Mathematical Skills: A Systematic Literature Review
- Author
-
Antonia Petropoulou, Konstantinos Lavidas, and Stamatis Papadakis
- Abstract
Background/purpose: Awareness of the mathematical skills and knowledge children possess in their early years is widely accepted. This includes various common positive aspects, not only for educators but also for researchers and policymakers. This study presents a systematic review conducted to meticulously identify empirical studies published in the Scopus-Index Journal database about the mathematical skills children aged 3 to 8 years old have mastered. Materials/methods: This review followed the PRISMA guidelines and the research database comprised of Scopus-indexed journals. The technique followed used "keywords" and Boolean operators. The screening processes included reviewing abstracts, scanning complete texts of published articles, and rejecting those not meeting preset inclusion criteria. Moreover, systematic reviews, meta-analyses, and papers not written in English were also excluded. Of the 801 studies initially identified, a total of 15 empirical studies were included in the systematic review. Results: Children master various math skills from a very young age, mainly in "numbers and operations", but face difficulties in skills related to "algebra" as well as "geometry and measurement". Additionally, several preschoolers' characteristics help to explain the acquisition of these skills, with "age" being the primary factor. Researchers use various research instruments and mainly conduct individual semi-structured interviews. Children's geometry skills and knowledge appear to have been studied to a small extent. The areas of "measurement" and "data analysis and probability" were found to be under active investigation. Conclusion: It is worth noting that not only does a noticeable research gap exist for the math domains of "measurement," "geometry," and "data analysis and probability". Factors that seem to affect young children's math skills, such as "gender", "parents' educational level", and "attendance to a preparatory preschool" need further investigation. The implications of the current study's results extend beyond academia, providing valuable insights that educators and policymakers can leverage to enhance the quality of mathematics education during the early years period.
- Published
- 2024
24. The Neurodiversity Attitudes Questionnaire: Development and Initial Validation
- Author
-
Rachel K. Schuck, Sunghee Choi, Kaitlynn M. P. Baiden, Patrick Dwyer, and Mirko Uljarevic
- Abstract
Neurodiversity refers to the idea that all brains--no matter their differences--are valuable and should be accepted. Attitudes toward the neurodiversity perspective can have real-life impacts on the lives of neurodivergent people, from effects on daily interactions to how professionals deliver services for neurodivergent individuals. In order to identify negative attitudes toward neurodiversity and potentially intervene to improve them, we first need to measure these attitudes. This article describes the development and initial validation of the Neurodiversity Attitudes Questionnaire (NDAQ), including item revision based on expert review, cognitive interviews, systematic evaluation of participants' response process, and analysis of the instrument's internal factor structure using exploratory structural equation modeling. Pilot analysis with 351 individuals mostly living in the United States who were currently working in or intending to pursue helping professions indicates that the NDAQ has construct validity, is well understood by participants, and fits a five-factor structure. While the NDAQ represents the first instrument designed to specifically assess attitudes toward the neurodiversity perspective, further validation work is still needed.
- Published
- 2024
- Full Text
- View/download PDF
25. The Impact of Survey Mode Design and Questionnaire Length on Measurement Quality
- Author
-
Alexandru Cernat, Joseph Sakshaug, Pablo Christmann, and Tobias Gummer
- Abstract
Mixed-mode surveys are popular as they can save costs and maintain (or improve) response rates relative to single-mode surveys. Nevertheless, it is not yet clear how design decisions like survey mode or questionnaire length impact measurement quality. In this study, we compare measurement quality in an experiment of three distinct survey designs implemented in the German sample of the European Values Study: a single-mode face-to-face design, a mixed-mode mail/web design, and a shorter (matrix) questionnaire in the mixed-mode design. We compare measurement quality in different ways, including differences in distributions across several data quality indicators as well as equivalence testing over 140 items in 25 attitudinal scales. We find similar data quality across the survey designs, although the mixed-mode survey shows more item nonresponse compared to the single-mode survey. Using equivalence testing we find that most scales achieve metric equivalence and, to a lesser extent, scalar equivalence across the designs.
- Published
- 2024
- Full Text
- View/download PDF
26. Measuring the Success of Information Systems in Higher Education -- A Systematic Review
- Author
-
Tea Mijac, Mario Jadric, and Maja Cukušic
- Abstract
Higher education institutions (HEIs) are encountering numerous challenges in today's era of globalization and internationalization. Among these challenges, digital transformation stands out as one of the most demanding, as it involves using ICT to transform business processes. The successful implementation of such systems and related services requires careful management and measurement. To gain deeper insights into the present state of information systems (IS) and digital services in higher education and relevant theory-based models, a systematic literature review and bibliometric analysis were conducted. After the search and screening, this paper systematically reviewed 63 publications to identify various ISs and classify the research conducted to date. This paper highlights a critical gap in the investigation of ISs beyond learning activities, emphasizing the under-researched area of administrative digital services. Furthermore, the paper contributes by highlighting the prevalent use of DeLone and McLean's success model, particularly in combination with TAM/UTAUT, for empirically measuring IS success in higher education. Results also identify a need for greater consistency and rigor in the use of indicators across studies, since there is no evident agreement on indicators used for measuring the success dimensions. Bibliometric analysis on 1791 records underscores the evolving research landscape, with a shift towards evaluating the quality dimension emphasizing the significance of success evaluation. By achieving its primary objective of examining and organizing the relevant scientific literature on IS in higher education, this study contributes to the existing body of knowledge and offers foundational resources researchers and HEIs seeking to implement new ISs or improve their current systems.
- Published
- 2024
- Full Text
- View/download PDF
27. Measurement Invariance between Subjects: What Can We Learn about Subject-Related Differences in Teaching Quality?
- Author
-
Wida Wemmer-Rogh, Urs Grob, Charalambos Y. Charalambous, and Anna-Katharina Praetorius
- Abstract
Recent publications emphasize the need to take greater account of differences in teaching quality between subjects. The empirical analysis of this topic requires a comparison of teaching quality in different subjects to distinguish generic aspects of teaching quality from subject-specific ones. In this paper, we compare teaching quality in mathematics and German lessons using observational data from primary schools in Switzerland (N[subscript Math] = 319; N[subscript German] = 237). Data were collected using an observation instrument reflecting the teaching dimensions of the MAIN-TEACH model, which was developed based on a synthesis of established observation frameworks. The dimensions of classroom management, motivational-emotional support, selection and implementation of content, cognitive activation, support for consolidation, assessment and feedback, and adaptation were tested for subject-related measurement invariance. With a two-fold measurement invariance approach, differences between the subjects were investigated at both a global and an indicator level. When applying alpha accumulation correction, no significant subject-related differences in factor loadings or intercepts were found. The factorial structure of our data was basically identical for the two subjects. The comparison of latent factor means revealed no mean differences between the subjects. We discuss the implications for both the MAIN-TEACH model and research into subject-related differences of teaching in general.
- Published
- 2024
- Full Text
- View/download PDF
28. Multi-Group Regularized Gaussian Variational Estimation: Fast Detection of DIF
- Author
-
Weicong Lyu, Chun Wang, and Gongjun Xu
- Abstract
Data harmonization is an emerging approach to strategically combining data from multiple independent studies, enabling addressing new research questions that are not answerable by a single contributing study. A fundamental psychometric challenge for data harmonization is to create commensurate measures for the constructs of interest across studies. In this study, we focus on a regularized explanatory multidimensional item response theory model (re-MIRT) for establishing measurement equivalence across instruments and studies, where regularization enables the detection of items that violate measurement invariance, also known as differential item functioning (DIF). Because the MIRT model is computationally demanding, we leverage the recently developed Gaussian Variational Expectation-Maximization (GVEM) algorithm to speed up the computation. In particular, the GVEM algorithm is extended to a more complicated and improved multi-group version with categorical covariates and Lasso penalty for re-MIRT, namely, the importance weighted GVEM with one additional maximization step (IW-GVEMM). This study aims to provide empirical evidence to support feasible uses of IW-GVEMM for re-MIRT DIF detection, providing a useful tool for integrative data analysis. Our results show that IW-GVEMM accurately estimates the model, detects DIF items, and finds a more reasonable number of DIF items in a real world dataset. The proposed method has been integrated into R package VEMIRT (\url{https://map-lab-uw.github.io/VEMIRT}). [This paper will be published in "Psychometrika."]
- Published
- 2024
29. Measurement Invariance of the Parent-Reported Strengths and Difficulties Questionnaire in Autistic Adolescents
- Author
-
Chloe Turcan, Henry Delamain, Asher Loke, Richard Pender, Will Mandy, and Rob Saunders
- Abstract
Mental health conditions are more prevalent in autistic people than non-autistic people. The Strengths and Difficulties Questionnaire is a commonly used screening tool for emotional and behavioural difficulties in autistic children and adolescents. However, few studies have assessed the psychometric properties of the Strengths and Difficulties Questionnaire in populations of autistic people, especially measurement invariance, to examine whether the underlying latent structure is consistent across time or groups. Measurement invariance of the parent-reported Strengths and Difficulties Questionnaire in a nationally representative cohort study was examined: longitudinal invariance (autistic 11-, 14- and 17-year-olds) and group invariance (autistic and non-autistic 17-year-olds). Differential item functioning was explored in the autistic/non-autistic group analysis. The five-factor structure showed inadequate fit, especially in the autistic group; by extension, measurement invariance could not be established. Differential item functioning was observed for several conduct problems, prosocial behaviour and peer problems subscale items, as well as reverse-coded items. Poor structural validity and item-level invariance may confound findings from research within and between populations of autistic and non-autistic adolescents, as well as screening of mental health conditions in autistic adolescents. This research calls for the development and improvement of screening tools for emotional and behavioural difficulties in populations of autistic people while accounting for their heterogeneity.
- Published
- 2024
- Full Text
- View/download PDF
30. Artificial Intelligence and Educational Measurement: Opportunities and Threats
- Author
-
Andrew D. Ho
- Abstract
I review opportunities and threats that widely accessible Artificial Intelligence (AI)-powered services present for educational statistics and measurement. Algorithmic and computational advances continue to improve approaches to item generation, scale maintenance, test security, test scoring, and score reporting. Predictable misuses of AI for these purposes will result in biased scores, construct underrepresentation, and differential impact over time. Recent efforts to develop standards for AI use in testing like those of Burstein are promising. I argue that similar efforts to develop AI standards for educational measurement will benefit from increased attention to the context of test use and explicit commitment to ongoing monitoring of bias and scale drift over time.
- Published
- 2024
- Full Text
- View/download PDF
31. Strive for Measurement, Set New Standards, and Try Not to Be Evil
- Author
-
Derek C. Briggs
- Abstract
I consider recent attempts to establish standards, principles, and goals for artificial intelligence (AI) through the lens of educational measurement. Distinctions are made between generative AI and AI-adjacent methods and applications of AI in formative versus summative assessment contexts. While expressing optimism about its possibilities, I caution that the examples of truly generative AI in educational testing have the potential to be overexaggerated, that efforts to establish standards for AI should complement the Standards for Educational and Psychological Testing and focus attention on the issues of fairness and social responsibility, and that scientific advance and transparency in the development and application of AI in educational assessment may be incompatible with the competitive marketplace that is funding this development.
- Published
- 2024
- Full Text
- View/download PDF
32. Assessing the Psychometric Qualities of the Data-Informed School Leadership Survey
- Author
-
Jingping Sun, Jiangang Xia, Cheng Hua, Kaiwen Man, and Bob L. Johnson
- Abstract
Purpose: There is little consensus in the literature regarding a) what it means for a school leader to lead with data, and b) how to measure data-informed leadership in a reliable and valid way. This study examines the psychometric properties of an operational measure intended to assess the extent to which a school leader is a data-informed school leader. The measurement invariance, reliabilities and construct and predictive validities of the "Data-Informed School Leadership Survey" (DISL Survey) are assessed using various psychometric statistical techniques. Methods: Using data collected from 155 teachers from 7 public schools in a southern state, the following psychometric statistics used to address our purpose: the Many-Facet Rasch (MFR) Model, Bayesian second-order Confirmatory Factor Analysis (CFA), Bayesian Structural Equation Modeling--Multiple Indicators, Multiple Causes analysis (Bayesian SEM-MIMIC), and reliability analysis. Findings: Results show an adequate fit from all MFR, Bayesian CFA, and MIMIC models and a high reliability (Cronbach [alpha] = 0.98). The DISL Survey instrument exhibits sound psychometric properties. Results likewise confirm the value of using MFR modeling and Bayesian methods to examine the psychometric properties of DISL Survey as a means of improving educational leadership measures. Implications for Research and Practice: Data from this study confirm the validity and reliability of the "Data-Informed School Leadership Survey" (DISL Survey) as an instrument to assess the strengths and weaknesses of Data-Informed School Leadership (DISL) and as a means for providing feedback for improving such leadership. Heretofore a measure for assessing this leadership was non-existent.
- Published
- 2024
- Full Text
- View/download PDF
33. Effects of Prenatal Breastfeeding Education on Breastfeeding Duration beyond 12 Weeks: A Systematic Review
- Author
-
Megan K. Oggero, Cathy L. Rozmus, and Geri LoBiondo-Wood
- Abstract
The proportion of infants in the United States who are breastfed at 1 year remains well below the Healthy People 2030 target. The health implications of suboptimal breastfeeding durations are significant, including increased risk of childhood leukemia and maternal Type 2 diabetes. Prenatal breastfeeding education provides an opportunity to improve breastfeeding self-efficacy among pregnant individuals and to establish their coping skills in case future breastfeeding problems arise. Although prenatal breastfeeding education is known to improve breastfeeding self-efficacy, characteristics of prenatal breastfeeding education interventions that are successful at increasing breastfeeding duration have not been well defined. Using Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and the Health Action Process Approach, we conducted a systematic review of the literature examining the impact of prenatal breastfeeding education interventions on breastfeeding duration measured at least 12 weeks postpartum. Twenty-one studies were identified. Prenatal breastfeeding education was most likely to increase breastfeeding duration when education interventions integrated psychological components (Health Action Process Approach coping planning) or were paired with in-person postpartum breastfeeding support. Additional research is needed to examine the role of psychological components in breastfeeding education interventions in diverse populations and to determine the specific psychological intervention components with the greatest impact on breastfeeding duration.
- Published
- 2024
- Full Text
- View/download PDF
34. Why Don't We Count Zero?: Understanding Number Lines
- Author
-
Seanyelle Yagi and Linda C. Venenciano
- Abstract
On the surface, the number line may seem like a basic tool with obvious applications. However, using a number line is not always intuitive for students. Students may not recognize significant features such as the size of the unit, how units are represented by iterated equal lengths, or that the accumulation of iterated units is a magnitude of length. In addition, transitioning from counting a set of objects to quantifying iterated lengths on a number line can be confusing for students. In the authors' work with students, they draw students' attention to recognizing what units mean and how units are used on a number line. The authors found that after first defining a unit of length (e.g., from 0 to 1) when measuring or creating a length, students are able to make sense of number and quantity on a number line This article describes the authors' experiences with developing students' understanding of concepts underlying the number line within a measurement context. Using carefully sequenced lessons, the students described how the number line showed the iteration of the same unit. This provided a basis for understanding more sophisticated concepts, such as how a smaller unit will need to be iterated more frequently than a larger unit to measure the same quantity. If students are struggling, reflect on how to introduce the number line and take the time to facilitate a discussion that focuses student attention on the units and lengths, rather than on the tick marks. Use measurement vocabulary, such as unit and length, when discussing the number line. This will help students build a robust and flexible understanding of the mathematics concepts underlying this tool.
- Published
- 2024
- Full Text
- View/download PDF
35. Layering and Institutional Change: Framework Plan Revision in Norwegian Early Childhood Education and Care
- Author
-
Anne Homme, Kari Ludvigsen, and Hilde Danielsen
- Abstract
Central among a range of reforms and policy measures aimed at enhancing the quality and social cohesion of Norwegian early childhood education and care (ECEC) services is the 2017 "Framework Plan for Kindergartens." In this article, we investigate the policy formation process and how framework plan reform has been carried out by public and private stakeholders at different levels. The reform is explored as institutional layering, referring to theoretical contributions that stress the role of incremental development and gradual institutional change. We ask: What are the policies added, and to what extent do added policies instigate institutional change? This article relies on qualitative document analyses, semistructured interviews, and quantitative survey data. We find high legitimacy for the framework plan in the ECEC sector, and informants broadly regard it as a continuation of earlier versions of the plan. Our analysis further shows that the implementation of the 2017 framework plan reform cannot be viewed in isolation, as it interacts with other reforms, trends, and instruments in the sector. We argue that blending reform design features with existing values and practices adds layers to Norwegian ECEC, instigating change. The analysis points to the need for further research on how national educational reforms are implemented in complex settings and to investigate the relationships between policy design and measures and between policy outcomes and ambitions. The relationships between different measures can be unclear to local implementers. Thus, we stress the importance of considering how reform outcomes depend on intertwined reforms and varying local conditions regarding capacity, norms, and values.
- Published
- 2024
- Full Text
- View/download PDF
36. Carbon Dioxide Clathrate Hydrate Formation and Mass Balance Calculation: A Laboratory Experiment
- Author
-
Keita Yasuda and Izuru Senaha
- Abstract
Clathrate hydrates are attractive materials for education because they form from water and familiar compounds, such as carbon dioxide, and are relevant to novel technologies. In this study, an attempt to propose a laboratory experiment was summarized, which used carbon dioxide clathrate hydrate as an educational material with carbon capture technologies as the background of the experiment. The experiment was provided from October 2020 to August 2022 in the "Laboratory Experiments in Energy and Environment Engineering" class for second- and third-year undergraduate students. A total of 90 students, divided into 14 groups and 28 subgroups, performed the experiment. The carbon dioxide clathrate hydrate was formed under low-temperature and high-pressure conditions. Based on the measured temperature and pressure conditions, the amount of formed carbon dioxide clathrate hydrate was calculated as exercise problems. In addition to the exercise problems, the students submitted a report describing the experiments and related discussions. 92% of the students passed the required score to earn the credit for the class. The average score for those who passed the class was 84%. The educational effect was discussed based on the content of the exercise problems and reports. A stirring device to enhance the clathrate hydrate formation was designed and constructed by two students enrolled in "Project Management Practice" for fourth-year undergraduate students. They obtained a 100% score because they successfully constructed the device based on a project management method. The device was used in a laboratory experiment.
- Published
- 2024
- Full Text
- View/download PDF
37. Measurement Tools to Assess Usual Dietary Intake and Physical Activity in Individuals with Autism Spectrum Disorder: A Scoping Review
- Author
-
Jamie M. Holloway, Heewon L. Gray, Acadia W. Buro, Jodi Thomas, Rachel Sauls, and Allison M. Howard
- Abstract
This review aimed to identify measurement tools to assess dietary intake and physical activity (PA) among individuals with autism spectrum disorder (ASD) and describe the evidence of validity and availability of each tool. Searches were conducted in PubMed, Embase, CINAHL, and PsycINFO using keywords for ASD, PA, diet, and assessment/measurement (n = 2364 articles). After removing duplicates and title/abstract screening, 212 full-text articles were reviewed, and 113 articles were included for data extraction. The most common dietary and PA assessment tools were identified, and each reviewer independently extracted the data on target population of the assessment tool, type of instrument, evidence on psychometrics validation, constructs measured, and instrument availability. Our findings highlight the need for instruments developed for and/or validated in this population.
- Published
- 2024
- Full Text
- View/download PDF
38. Quantifying Quality: The Impact of Measures of School Quality on Children's Academic Achievement across Diverse Societies
- Author
-
Bruce S. Rawlings, Helen Elizabeth Davis, Adote Anum, Oskar Burger, Lydia Chen, Juliet Carolina Castro Morales, Natalia Dutra, Ardain Dzabatou, Vivian Dzokoto, Alejandro Erut, Frankie T. K. Fong, Sabrina Ghelardi, Micah Goldwater, Gordon Ingram, Emily Messer, Jessica Kingsford, Sheina Lew-Levy, Kimberley Mendez, Morgan Newhouse, Mark Nielsen, Gairan Pamei, Sarah Pope-Caldwell, Karlos Ramos, Luis Emilio Echeverria Rojas, Renan A. C. dos Santos, Lara G. S. Silveira, Julia Watzek, Ciara Wirth, and Cristine H. Legare
- Abstract
Recent decades have seen a rapid acceleration in global participation in formal education, due to worldwide initiatives aimed to provide school access to all children. Research in high income countries has shown that school quality indicators have a significant, positive impact on numeracy and literacy--skills required to participate in the increasingly globalized economy. Schools vary enormously in kind, resources, and teacher training around the world, however, and the validity of using diverse school quality measures in populations with diverse educational profiles remains unclear. First, we assessed whether children's numeracy and literacy performance across populations improves with age, as evidence of general school-related learning effects. Next, we examined whether several school quality measures related to classroom experience and composition, and to educational resources, were correlated with one another. Finally, we examined whether they were associated with children's (4-12-year-olds, N = 889) numeracy and literacy performance in 10 culturally and geographically diverse populations which vary in historical engagement with formal schooling. Across populations, age was a strong positive predictor of academic achievement. Measures related to classroom experience and composition were correlated with one another, as were measures of access to educational resources and classroom experience and composition. The number of teachers per class and access to writing materials were key predictors of numeracy and literacy, while the number of students per classroom, often linked to academic achievement, was not. We discuss these results in the context of maximising children's learning environments and highlight study limitations to motivate future research.
- Published
- 2024
- Full Text
- View/download PDF
39. Control-Value Theory: From Achievement Emotion to a General Theory of Human Emotions
- Author
-
Reinhard Pekrun
- Abstract
In its original version, control-value theory describes and explains achievement emotions. More recently, the theory has been expanded to also explain epistemic, social, and existential emotions. In this article, I outline the development of the theory, from preliminary work in the 1980s to early versions of the theory and the recent generalized control-value theory. I provide summaries of the theory's evidence-based propositions on antecedents, outcomes, and regulation of emotions, including the fundamentally important role of control and value appraisals across different types of human emotions that are relevant to education (and beyond). The theory includes descriptive taxonomies of emotions as well as propositions explaining (a) the influence of individual factors, social environments, and socio-cultural contexts on emotions; (b) the effects of emotions on learning, performance, and health; (c) reciprocal causation linking emotions, outcomes, and antecedents; (d) ways to regulate emotions; and (e) strategies for intervention. Subsequently, I outline the relevance of the theory for educational practice, including individual and large-scale assessments of emotions; students', teachers', and parents' understanding of emotions; and change of educational practices. In conclusion, I discuss strengths of the theory, open questions, and future directions.
- Published
- 2024
- Full Text
- View/download PDF
40. A Tutorial on Cross Wave Measurement Invariance Testing with Item Factor Analysis
- Author
-
R. Noah Padgett
- Abstract
The consistency of psychometric properties across waves of data collection provides valuable evidence that scores can be interpreted consistently. Evidence supporting the consistency of psychometric properties can come from using a longitudinal extension of item factor analysis to account for the lack of independence of observation when evaluating the cross-wave properties. In this study, we demonstrate how to conduct and interpret a longitudinal examination of psychometric properties. This demonstration uses data on the Comprehensive Measure of Meaning across two waves of data collection. A simplified, structured set of R syntax for analyses is provided, and all remaining code is freely available in the accompanying Open Science Framework repository.
- Published
- 2023
41. Culturally Responsive Assessment: Provisional Principles. ETS RR--23-11
- Author
-
Michael E. Walker, Margarita Olivera-Aguilar, Blair Lehman, Cara Laitusis, Danielle Guzman-Orth, and Melissa Gholson
- Abstract
Recent criticisms of large-scale summative assessments have claimed that the assessments are biased against historically excluded groups because of the assessments' lack of cultural representation. Accompanying these criticisms is a call for more culturally responsive assessments--assessments that take into account the background characteristics of the students; their beliefs, values, and ethics; their lived experiences; and everything that affects how they learn and behave and communicate. In this paper, we present provisional principles, based on a review of research, that we deem necessary for fostering cultural responsiveness in assessment. We believe the application of these principles can address the criticisms of current assessments.
- Published
- 2023
42. 7 Recommendations for Using Education Data to Support Equitable Learning Outcomes
- Author
-
WestEd, Jessica Arnold, and Julie Webb
- Abstract
While there are many different types of education data, policymakers and education leaders often place heavy emphasis on data from large-scale quantitative measures, such as annual state assessments. But data from these sources alone do not provide a complete picture of learning and are often not well suited to informing improvements at the local level. This brief provides seven recommendations to help educators, administrators, policymakers, and others use a wider range of data to more effectively support equitable learning outcomes.
- Published
- 2024
43. Measures of Career Adaptability and Employability in Bahasa Indonesia
- Author
-
Henriette van Rensburg, Harsha N. Perera, Peter McIlveen, Fitriah Asad, and P. Nancey Hoare
- Abstract
There is emergent evidence of the Career Adapt-Abilities Scale's utility in the Global South. The present research provides evidence about the measurement properties of a Bahasa Indonesia language version of the Career Adapt-Abilities Scale and comparator measures, the Career Futures Inventory-Revised and the Self-perceived Employability Scale, using a sample of students (N = 277) enrolled in Indonesian college degrees. Furthermore, for the purpose of cross-cultural comparisons, the measurement properties of the Indonesian version of the Career Adapt-Abilities Scale were compared with an independent dataset (N = 387) using an Australian English language version. The Indonesian version had acceptable measurement properties.
- Published
- 2024
- Full Text
- View/download PDF
44. Development of Metacognitive Monitoring and Control Skills in Elementary School: A Latent Profile Approach
- Author
-
Mariëtte van Loon and Claudia M. Roebers
- Abstract
This study aims to understand individual differences between children in metacognitive monitoring and control processes and the developmental trajectories of metacognition over one year. Three indicators of procedural metacognition were used: monitoring accuracy (discrimination of confidence judgments between correct and incorrect test responses), effective restudy selections, and accuracy of response maintenance/withdrawal decisions. These indicators were measured for two tasks (text comprehension and Kanji memory) at two measurement points one year apart. Participants were 151 second graders (M age 7.61 years) and 176 fourth graders (M age 9.62 years). With latent profile analyses, distinct metacognition profiles were found for both grade levels at both measurement points. Children showed heterogeneity in the proficiency of metacognition but also in the extent to which metacognitive skills were generalizable across the two tasks. For second-grade children, being low at metacognition at the first measurement point was not associated with extra risks for low metacognition one year later. However, for fourth graders, children with low metacognitive skills appeared likely to stay low in metacognition over time and particularly showed ineffective restudy decisions. This indicates that they seemed at risk for a longer-term metacognitive deficiency. Findings may improve understanding of the heterogeneity of metacognition and support distinguishing typical from at-risk metacognitive development.
- Published
- 2024
- Full Text
- View/download PDF
45. Differential Performance of Computerized Adaptive Testing in Students with and without Disabilities -- A Simulation Study
- Author
-
Nikola Ebenbeck and Markus Gebhardt
- Abstract
Technologies that enable individualization for students have significant potential in special education. Computerized Adaptive Testing (CAT) refers to digital assessments that automatically adjust their difficulty level based on students' abilities, allowing for personalized, efficient, and accurate measurement. This article examines whether CAT performs differently for students with and without special educational needs (SEN). Two simulation studies were conducted using a sample of 709 third-grade students from general and special schools in Germany, who took a reading test. The results indicate that students with SEN were assessed with fewer items, reduced bias, and higher accuracy compared to students without SEN. However, measurement accuracy decreased, and test length increased for students whose abilities deviated more than two SD from the norm. We discuss potential adaptations of CAT for students with SEN in the classroom, as well as the integration of CAT with AI-supported feedback and tailored exercises within a digital learning environment.
- Published
- 2024
- Full Text
- View/download PDF
46. Online Instructional Materials in Mathematics VI for Selected Public Elementary Schools in District 4 Division of City Schools Manila: In Response to Department of Education's (DepEd's) Learning Continuity Plan
- Author
-
Julie Gravoso Maranan
- Abstract
This study is an attempt to design and develop as well as validate sets of Online Instructional Materials in Mathematics VI at selected elementary schools in DCS, Manila, specifically, focused on the study of Geometry and Measurement. Using both documentary analysis and descriptive method of study, two (2) major groups of respondents were chosen to participate in the study. They were the 6 school administrators who were the School Principals, Supervisor, Head Teacher, Master Teacher and the 35 Mathematics teachers who represented the seven (7) selected elementary schools in the District 4 of the DCS, Manila. Aside from these experts, she also made use of two (2) sections of Grade VI pupils of a public elementary school in the District, who are the subjects of the study. The following are the salient findings of the investigations on: 1. Topics to be included in the Online Instructional Materials in Mathematics VI. Using documentary and historical data, the following indicators were bases in how the researcher decided in how she chose Geometry and Measurement as the core subject areas specifically covered in the modules: (1.1) Content-based for Grade VI Mathematics--Five (5) units were covered, of these areas of concern Geometry was the least covered with only 2 or 9.09 topics and Measurement and Whole Numbers equally had more with 5 or 22.73 percent topics. Rational Numbers have the most substantial topics covered in the MLC. (1.2) DepEd's Prescribed Textbooks and Subject Contents--When the three (3) DepEd's prescribed textbooks were analyzed in terms of the topics and pages covered as prescribed in the prescribed content-based curriculum, again, Geometry and Measurement, which are considered difficult areas in Mathematics VI were those that were least covered. They obtained an overall (Measurement with 47:86 topic and page covered ratio) and Geometry with 15:25 topic and page covered ratio and were ranked 3 and 4, respectively among the 5 major units of the subject under study. (1.3) Budget of Work for Mathematics VI--Not much emphasis was also given to both Geometry and Measurement as proofs of the first registering only 11 suggested lessons and 17 sessions; and the second, with 11 suggested lessons with 23 sessions; very limited topics and pages were not only covered, but also not evenly and appropriately distributed considering the significance of the topics in preparing the pupils for higher mathematical procedures and processes. (1.4) Results of the Inventory and Achievement Tests in Mathematics VI in the District 4 of DCS, Manila--The Grade VI pupils totaling 1474 for the schoolyear 2022-2023 enrolled in Mathematics VI were able to perform which resulted to get only 29.27 mean raw or 72.21 transmitted scores in the inventory test; and their performance in the achievement test revealed 44.07 or 86.95 mean raw and transmitted scores which generated a gain of 14.8 mean scores/14.74 transmitted scores. (1.5) Table of Specification Covered in the Achievement Test for the Academic Year 2022-2023--In terms of item placement of the topics testing the mathematical skills of the pupils in knowledge, comprehension, application, and analysis on the distribution of test questions, only one (1) item was noted in Geometry and nil in Measurement under application skills. In knowledge and comprehension only one (1) each was recorded, but none in analysis. (1.6) Report on the DepEd's District 4 Organizational Structure on the Status of Basic Education Resource Textbooks, District of DCS, Manila, 2022-2023--The Annual Report of the Division significantly revealed the dearth of textbooks and other supplementary materials for the need of the pupils. In schools population, the pupil-textbooks ratio only of 3:7 for Grade VI pupils which revealed the inadequacy of basic textbooks requirement for the grade level. These data are revealing of the need to come out with supplementary instructional materials to augment the inadequacy and limited textbooks to replace the traditional and obsolete materials to revolutionize the teaching of mathematics, using alternative approach in no in-class interactive classroom activities. In response to these needs, the researcher, a Math teacher who is concern more to improve instruction to effect quality education, endeavors to design and develop sets of content-based modules for the use of Grade VI pupils in Mathematics in the School District of Manila. The Online Instructional Materials in Mathematics VI concentrated only on the topics of great concern -- Geometry and Measurement. Each major area is covered in 7 lessons or a total of 14. Her objective is to provide and equip the learners with needed numerical skills and competencies in the study of the areas focusing in attaining knowledge, developing the skills in comprehension, and applying concepts and analysis. She also saw to it that the content-based modules were designed and developed with a systems approach in mind, like following and observing each step of the modular process, such as: 1. Entry (starting) behavior of the learner was analyzed; 2. Specific objectives stated; 3. Content selected and sequenced; 4. Learning activities well-planned, designed and developed; 5. Continuous assessments in the form of pre and post and built-in tests; and 6. Effectiveness of the module evaluated. The content of the online tools is so prepared to allow the pupils to work independently by themselves, and if there is a need for teacher assistance, such help will be at the minimum, and on online instruction. Another key components of the online tools are that they are highly supportive of each other. Parts like the specific objectives, learner's activities, and evaluation are interrelated and suggested practice exercises or activities are the means used to achieve the predetermined objectives. Likewise, the evaluation serves as the feedback to find out the extent of how much the objectives are realized. 2. Validity of the Contents of the Online Instructional Tools in Mathematics VI. Based on the survey conducted on the assessment of the school administrators and experts, they were of the professional views that these sets of tools were not only valid as they were very acceptable, as proofs of obtained composite means of X = 4.67 for relevance of objectives; X = 4.78 for relevance of content; X = 4.76 for clarity; X = 4.72 for adequacy; X = 4.82 for suitability; and X = 4.67 for usefulness or resulted to an overall composite mean of X = 4.74. 3. Performance of the Pupils in the Pre and Posttests Using the Sets of Online Instructional Tools in Mathematics VI; and Significant Difference, if any, in the Results of the Pre and Posttests. The results of the pre- and posttest in Mathematics VI was satisfactory as evidenced by mean gain of 11.15 and further affirmed by the overall result of t = 20.871 which is much > than its tubular t = 2.576 that at 0.01 alpha level and 85 dfs, that the null Ho posited at the onset of the study is rejected. These findings suggest that pupils were able to enhance their knowledge and skills in the use of the online instructional tools in Mathematics, as an alternative tools in the teaching of Mathematics. Conclusions: In view of the cited findings, the following conclusions were drawn: 1. Geometry and Measurement subject areas were least covered areas in the DepEd's prescribed textbooks, budget of work, in the inventory ad achievement test and table of specification, results of the yearly test, and annual district report that the researcher focused on the content of her proposal on these two (2) not well-tread and tried topics in Mathematics VI in Geometry and Measurement. 2. In terms of judgmental evaluation, the proposed sets of online instructional materials in Mathematics VI were valid as they were very acceptable to the school administrators / experts (X = 4.71); and Mathematics teacher (X = 4.75); or an overall composite mean of X = 4.73. 3. In terms of concurrent validity, the results of the achievement test and the average grade of the pupils in Mathematics VI were highly and positively significant. 4. Significant difference is found to exist between the assessments of the school administrators and Mathematics teachers as to the acceptability of the proposed online instructional materials in Mathematics VI. Recommendations: Based from the conclusions drawn in the study, the researcher offers the following: 1. The Proposed Online Instructional Materials in Mathematics VI should be endorsed and adopted for the use of the Grade VI pupils not only in the District 4, but also on the whole Division of City Schools (DCS) of Manila, to serve as online supplementary materials in the teaching of Mathematics. 2. These sets of online instructional tools in Mathematics VI should further undergo testing and re-testing to improve the design and enrich the topics as well as the classroom activities that suit particular setting. These can also serve as alternative source in the absence in the inadequacy of teachers and basic textbooks and other reference materials, as well as serve as online instructional tool for pupils as no in-class interactive class is still partially suspended because of the pandemic situations. 3. More online instructional tools in Mathematics should be developed in other areas of the subject to ease the problem of big classes, inadequate classrooms and facilities, as well as and ease the burden of the teachers as pupils start to assume responsibility for learning, especially, now that partial no in-class is ordered by the DepEd because of the fear of safety and health satisfaction. 4. Because of the limitation of the study, the researcher further recommends a follow-up research to firmly establish the logical and statistical validity of other sets of online instructional materials in other areas of Mathematics subject.
- Published
- 2023
47. Different Methods for Assessing Pre-Service Teachers' Instruction: Why Measures Matter. EdWorkingPaper No. 23-862
- Author
-
Annenberg Institute for School Reform at Brown University, Arielle Boguslav, and Julie Cohen
- Abstract
Teacher preparation programs are increasingly expected to use data on pre-service teacher (PST) skills to drive program improvement and provide targeted supports. Observational ratings are especially vital, but also prone to measurement issues. Scores may be influenced by factors unrelated to PSTs' instructional skills, including rater standards and mentor teachers' skills. Yet we know little about how these measurement challenges play out in the PST context. Here we investigate the reliability and sensitivity of two observational measures. We find measures collected during student teaching are especially prone to measurement issues; only 3-4% of variation in scores reflects consistent differences between PSTs, while 9-17% of variation can be attributed to the mentors with whom they work. When high scores stem not from strong instructional skills, but instead from external circumstances, we cannot use them to make consequential decisions about PSTs' individual needs or readiness for independent teaching.
- Published
- 2023
48. Fixed Is Not the Opposite of Growth: Item Keying Matters for Measuring Mindsets
- Author
-
David J. Grüning, Beatrice Rammstedt, and Clemens M. Lechner
- Abstract
Research on growth mindset, the belief that one's cognitive abilities are malleable and can be developed through dedication and practice, has received considerable media attention and influenced educational policy and practice. However, mindset theory and measurement have also drawn criticism. In the present paper, we add a cautionary note pertaining to the conceptualization and measurement of growth mindset. Through a critical reanalysis of a large-scale representative study of adolescents from the US (N = 15,362), we show that a growth (i.e., forward-keyed) and a fixed (i.e., reverse keyed) mindset item from a widely used scale are only moderately correlated (r = -0.31). Further, we demonstrate that the two items are very differently related with a range of educationally relevant criteria such as learning engagement and self-efficacy, and sociodemographic characteristics such as sex. This leads us to conclude that the growth and fixed mindset items are not mutually interchangeable (apart from keying) indicators of a unidimensional construct that has fixed and growth mindset at its opposing poles. Which items researchers choose to measure mindset (fixed, growth, or a blend thereof) may therefore have a significant impact on the findings they obtain. Our insights highlight the need for greater attention to the conceptual foundations and measurement of mindset in future studies.
- Published
- 2024
- Full Text
- View/download PDF
49. Bias Risks in ILSA Related to Non-Participation: Evidence from a Longitudinal Large-Scale Survey in Germany (PISA Plus)
- Author
-
Sabine Meinck, Jörg-Henrik Heine, Julia Mang, and Gabriel Nagy
- Abstract
This study uses evidence from a longitudinal survey (PISA Plus, Germany) to examine the potential of bias in international large-scale assessments (ILSAs). In PISA Plus, participation was mandatory at the first measurement point, but voluntary at the second measurement point. The study provides evidence for relevant selection bias regarding student competencies and background variables when participation is voluntary. Sample dropout at the second measurement point was related to characteristics such as family background, achievement in mathematics, reading and science, and other student and school demographic variables at both the student and school levels, with lower performing students and those with less favorable background characteristics having higher dropout frequencies, from which higher dropout probabilities of such students can be inferred. We further contrast the possibilities for addressing non-response through weight adjustments in longitudinal surveys with those in cross-sectional surveys. Considering our results, we evaluate and confirm the validity and appropriateness of strict participation rate requirements in ILSAs. Likely magnitudes of bias in cross-sectional studies in varying scenarios are illustrated. Accordingly, if combined participation rates drop below 70%, a difference of at least one-fifth of a standard deviation in an achievement score between non-respondents and participants leads to relevant bias. When participation drops below 50%, even a very small difference (one-tenth of a standard deviation) will cause non-negligible bias. Finally, we conclude that the stringent participation rate requirements established in most ILSAs are fully valid, reasonable, and important since they ensure a relatively low risk of biased results.
- Published
- 2024
- Full Text
- View/download PDF
50. Development and Evaluation of Predictive Models for Predicting Students Performance in MOOCs
- Author
-
Anagha Ani and Ean Teng Khor
- Abstract
Predictive modelling in the education domain can be utilised to significantly improve teaching and learning experiences. Massive Open Online Courses (MOOCs) generate a large volume of data that can be exploited to predict and evaluate student performance based on various factors. This paper has two broad aims. Firstly, to develop and tune several Machine Learning (ML) models to perform classification tasks on the dataset to predict student performance, including Linear Regression, Logistic Regression, Random Forests, K-Nearest Neighbours, and more. Secondly, to evaluate the efficacy of these ML models and identify those which are best suited to this task. The categories of data utilised in achieving these aims include (i) demographic information, (ii) academic background, and (iii) interaction with MOOC course materials. The research procedure comprises five phases: data exploration to analyse the dataset, feature engineering which involves discerning the most important features and converting them into a format decipherable by the ML models, model building, model evaluation by measurement of accuracy, and subsequent comparative evaluation between the different models. The results achieved in this study are expected to have implications on how MOOC platforms utilise data to improve user experience. As indicated by the findings of this study, the data collected by these platforms may be used to predict performance with accuracy of over 77%; this extracted information can be exploited to enhance educational theory or practices in the context of MOOCs, for instance by implementing varying teaching methodologies or providing different types of resources based on predicted performance.
- Published
- 2024
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.