Search

It is imperative to use new technologies in a supportive manner to meet the learners' and teachers' demanding needs as educational environments change in the digital age. The continuous expansion of online learning and distance education opportunities responds to the demands of learners and teachers while pioneering the use of technology in education. One advancement in English language teaching and learning in online environments, which assists teachers in reducing their workload and providing students with instant digital feedback, is the automated writing evaluation (AWE) tools. To gain a deeper understanding of the potential and limitations of these digital tools, this study aims to investigate the effectiveness of AWE feedback in error reduction in writing in English and the explore views of students regarding the utility of AWE tools. For this purpose, a total of 38 students at a university in Turkiye participated in the study, and three of their essays were evaluated. Within a concurrent triangulation mixed-method design, the changes in errors of the experimental group (n=18) receiving AWE feedback, and the control group (n=20) receiving teacher feedback were analyzed quantitatively, and the written reflection reports and semi-structured interviews conducted with the students were analyzed qualitatively. The results indicated that teacher feedback and AWE feedback were both effective in reducing errors in 11 categories. AWE feedback appeared to minimize errors in mechanics and usage more efficiently and teacher feedback was required more in content and organization issues. As a result, AWE was found as a complementary and effective tool supporting the improvement of target language writing skills saving time and energy for teachers. Furthermore, students expressed positive views regarding the use of AWE despite minor limitations. The findings of this study in general sheds light on using online digital tools of ubiquitous nature such as AWE to assist language improvement outside the class.

Intelligent systems of essay grading constitute important tools for educational technologies. They can significantly replace the manual scoring efforts and provide instructional feedback as well. These systems typically include two main parts: a feature extractor and an automatic grading model. The latter is generally based on computational and artificially intelligent methods. In this work, we focus on the feature extraction part. More precisely, we focus on argumentation and discourse-related features, which constitute high-level features. We discuss some state-of-the-art systems and analyze how argumentation and discourse analysis are used for extracting features and providing feedback.

This review explores how artificial intelligence (AI henceforth) can reshape education through insights from situated learning literature. The objective was to critically examine opportunities and challenges of situated learning, and how AI could augment strengths while overcoming obstacles. A systematic review using the PRISMA method analyzed 60 articles from peer-reviewed journals over three decades. Key concepts associated with situated learning were extracted and analyzed qualitatively and quantitatively. Findings identified major obstacles: the traditional school system's one-way passive learning; the predominant educational approach fixated on predefined outcomes; and teachers' lack of contextual knowledge. AI presents solutions including adaptive systems tailored to students' evolving needs; intelligent tutoring situated in authentic scenarios; automation of administrative tasks; and data-driven teacher support. When implemented thoughtfully, AI has the potential to enhance situated learning through increased personalization, interactivity, and real-world connections. This promises a better effective, adaptive education - but human guidance remains essential for ethical grounding. This review offers teachers, researchers, and policymakers valuable insights on integrating both AI and situated learning to keep education relevant in an interconnected world.

This study examined the integration of artificial intelligence-powered speech recognition technology within early reading assessments in Morocco's Teaching at the Right Level (TaRL) program. The purpose was to evaluate the effectiveness of an automated speech recognition tool compared to traditional paper-based assessments in improving reading skills among 100 Moroccan first to third-graders. The mixed-method approach combined pre-post standardized reading tests with qualitative feedback. Results showed students receiving the AI-enabled speech recognition assessments demonstrated significant gains in reading achievement compared to peers assessed via traditional methods. Qualitative findings revealed benefits of instant feedback and enhanced engagement provided by the speech recognition tool. This study contributes timely empirical evidence on adopting learning technologies, specifically AI-driven automated speech assessment instruments, to enhance foundational literacy development within under-resourced education systems implementing student-centered pedagogical techniques like TaRL. It provides valuable insights and guidance for integrating innovative speech analysis tools within localized teaching and learning frameworks to strengthen early reading instruction and monitoring.

The purpose of this multiple intrinsic case study was to describe how Northeastern United States middle school teachers and students engaged with a new automated writing evaluation tool used to score and provide feedback on extended essay assignments to improve teaching and learning writing. Richard Elmore's (1993) instructional core framework is the theory guiding this study. The study's central research question is: How do public and private middle school students and teachers engage with the automated writing evaluation program WisdomK12? The study leveraged an intrinsic case study design and triangulated data from educational artifacts, individual interviews, and questionnaires. Results indicated that students and teachers found WisdomK12 to save time, provide relevant, encouraging, and authentic feedback, and inspire them to write more. Implications and future research are discussed.

No records of systematic reviews focused on deep learning in open learning have been found, although there has been some focus on other areas of machine learning. Through a systematic review, this study aimed to determine the trends, applied computational techniques, and areas of educational use of deep learning in open learning. The PRISMA protocol was used, and the Web of Science Core Collection (2019-2023) was searched. VOSviewer was used for networking and clustering, and in-depth analysis was employed to answer the research questions. Among the main results, it is worth noting that the scientific literature has focused on the following areas: (a) predicting student dropout, (b) automatic grading of short answers, and (c) recommending MOOC courses. It was concluded that pedagogical challenges have included the effective personalization of content for different learning styles and the need to address possible inherent biases in the datasets (e.g., socio-demographics, traces, competencies, learning objectives) used for training. Regarding deep learning, we observed an increase in the use of pre-trained models, the development of more efficient architectures, and the growing use of interpretability techniques. Technological challenges related to the use of large datasets, intensive computation, interpretability, knowledge transfer, ethics and bias, security, and cost of implementation were also evident.

The release of ChatGPT has marked the dawn of a new information revolution that will transform how people communicate and make meaning. However, to date, little is known about the implications of ChatGPT for L2 composition instruction. To address this gap, the present study uses a systematic review design to synthesize available research on the educational potentials of ChatGPT as an instructional assistant, outline the implications of these potentials for L2 writing instruction, and discuss their practical applications. The findings, based on a meta-analysis of 42 research articles, demonstrate that ChatGPT can enhance L2 writing instruction by boosting learners' motivation, automating instructional tasks, and offering instantaneous, personalized feedback to learners. These findings have important implications for harnessing the instructional potential of generative AI in L2 writing classes.

Multimodal learning analytics (MMLA) research has shown the feasibility of building automated models of collaboration quality using artificial intelligence (AI) techniques (e.g., supervised machine learning (ML)), thus enabling the development of monitoring and guiding tools for computer-supported collaborative learning (CSCL). However, the practical applicability and performance of these automated models in authentic settings remains largely an under-researched area. In such settings, the quality of data features or attributes is often affected by noise, which is referred to as attribute noise. This paper undertakes a systematic exploration of the impact of attribute noise on the performance of different collaboration-quality estimation models. Moreover, we also perform a comparative analysis of different ML algorithms in terms of their capability of dealing with attribute noise. We employ four ML algorithms that have often been used for collaboration-quality estimation tasks due to their high performance: random forest, naive Bayes, decision tree, and AdaBoost. Our results show that random forest and decision tree outperformed other algorithms for collaboration-quality estimation tasks in the presence of attribute noise. The study contributes to the MMLA (and learning analytics (LA) in general) and CSCL fields by illustrating how attribute noise impacts collaboration-quality model performance and which ML algorithms seem to be more robust to noise and thus more likely to perform well in authentic settings. Our research outcomes offer guidance to fellow researchers and developers of (MM)LA systems employing AI techniques with multimodal data to model collaboration-related constructs in authentic classroom settings.

The implementation of computer-assisted pronunciation training (CAPT) has been proven to be successful in improving learners' pronunciation abilities. Automatic speech recognition (ASR) software was used to provide mediated support to 103 pre-intermediate level students (62 males and 41 females). After experiencing a two-semester of CAPT instruction in their Freshman English course, students completed a questionnaire to assess their perceptions of and attitudes towards technology. This paper reports on the findings that examine the structural relationships using the Technology Acceptance Model (TAM). The findings indicate that students, generally, were in favor of using ASR-based pronunciation training, and although no statistically significant gender difference was found, female students appeared to view its use more favorably than were their male counterparts. The perceived effectiveness of the system, and the attitudes of students towards using it, were shown to be significantly correlated, which encourages the ongoing use of ASR-based CAPT. Based on these responses, it was established that the ASR function enhanced students' awareness of their pronunciation errors. Furthermore, they willingly engaged in individual, repetitive pronunciation exercises, allowing them to build confidence in speaking practices without fearing embarrassment in front of their peers. Recommendations were provided for EFL educators interested in implementing CAPT in EFL settings.

The purpose of this study was to evaluate the factors that influence behavioural intention (BI) to use the Online Summary-with Automated Feedback (OSAF) in a MOOCs platform. Task-Technology Fit (TTF) was the main framework used to analyse the match between task requirements and technology characteristics, predictng the utilisation of the technology. The relationships between TTF and BI was moderated by students' performance. This TTF provides an illustration of the extent to which the suitability of technology support for tasks will affect the performance and utilization of technology. There were 9 hypotheses examined in this study. The participants consisted of 151 students at a public university in East Java, Indonesia. In order to analyse the collected data, PLS-SEM (partial least squares -- structural equation modeling) was employed, using SmartPLS 3.0. In this study, several points can be concluded, namely: (1) task characteristics and technology characteristics were not positively and significantly effected by TTF, while students' characteristics had a positive and significant effect on TTF; (2) TTF and utilization which are influenced by social influence, have a positive effect on performance impact. In this case the performance impact is constructed from 3 dimensions, namely: learning performance, personal integrity, self-confidence, except TTF were not postitive and were significantly affected by self-confidence. (3) TTF and performance impact positively influence behavioural intention, except in the dimension of performance impact, personal integrity was not postively and significantly effected by behavioural intention.

Evaluation is an essential component of the learning process when discerning learning situations. Assessing natural language responses, like short answers, takes time and effort. Artificial intelligence and natural language processing advancements have led to more studies on automatically grading short answers. In this review, we systematically analyze short-answer evaluation studies. We present the development of the field in terms of scientific production features, datasets, and automatic evaluation features. The field has developed with pioneering studies in the US. Researchers generally conduct applications with English datasets. There has been a significant increase in research in recent years with large language models that support many different languages. These models have applications that achieve accuracy close to that of human evaluators. In addition, deep learning models do not require traditional approaches' detailed preprocessing and feature engineering processes. The dataset size trend is 1000 and above regarding the number of responses. It was observed that metrics such as accuracy, precision, and F1 score were used in performance determination. It is seen that the majority of the studies focus on scoring or rating. In this context, there needs to be more literature on the context of evaluation system applications that can provide descriptive and formative feedback. In addition, the developed assessment systems must be actively used in learning environments.

This study aimed at examining the performance of generative artificial intelligence to extract argumentation elements from text. Thus, the researchers developed a web-based framework to provide automated assessment and feedback relying on a large language model, ChatGPT. The results produced by ChatGPT were compared to human experts across scientific and non-scientific contexts. The findings revealed marked discrepancies in the performance of AI for extracting argument components, with a significant variance between issues of a scientific nature and those that are not. Higher accuracy was noted in identifying claims, data, and qualifiers, as opposed to rebuttals, backing, and warrants. The study illuminated AI's promise for educational applications but also its shortcomings, such as the increased frequency of erroneous element identification when accuracy was low. This highlights the essential need for more in-depth comparative research on models and the further development of AI to enhance its role in supporting argumentation training.

With the fast development of artificial intelligence and emerging technologies, automatic recognition of students' facial expressions has received increased attention. Facial expressions are a kind of external manifestation of emotional states. It is important for teachers to assess students' emotional states and adjust teaching activities accordingly. However, existing methods for automatic facial expression recognition have the limitations of low accuracy of recognition and poor feature extraction. To address the problem, this study proposed a novel deep learning model called DenseNetX-CBAM to improve facial expression recognition by utilizing a variant of densely connected convolutional networks (DenseNet) to reduce unnecessary parameters and strengthen the reuse of expression features between networks; moreover, convolutional block attention module (CBAM) was integrated to allow the networks to focus on important special regions and important channels when representing features. The proposed model was tested using 217 video clips of 33 students in an online course. The results demonstrated promising effects of the method in improving the accuracy of facial expression recognition, which can help teachers to accurately recognize students' emotions and provide real-time adjustment in online learning environments.

Ensuring equity in education is a goal for sustainable development. Among the factors that hinder equity, socioeconomic status (SES) has the highest impact on learning Mathematics. This paper addresses the issue of equity at the secondary school level by proposing an approach based on adopting automatic formative assessment (AFA). Carefully designed mathematical activities with interactive feedback were experimented with a sample of 299 students of grade 8 for a school year. A control group of 257 students learned the same topics using traditional methodologies. Part of the sample belonged to low SES. The learning achievement was assessed through pre-and post-tests to understand if the adoption of AFA impacted learning and whether the results depended on the students' SES. The results show a positive effect of the experimentation (effect size: 0.42). Moreover, the effect size of the experimentation restricted to the low-SES group is high (0.77). In the treatment group, the results do not depend on SES, while in the control group, they do, suggesting that AFA is an equitable approach while traditional instruction risks perpetuating inequalities.

Purpose: Chatbots and artificial intelligence (AI) have the potential to alleviate some of the challenges faced by humans. Faculties frequently swamped with teaching and research may find it difficult to act in a parental role for students by offering them individualized advice. Hence, the primary purpose of this study is to review the literature on chatbots and AI in light of their role in auto-advising systems. The authors aimed to gain insights into the most pertinent topics and concerns related to robo academic advisor and identify any gaps in the literature that could serve as potential avenues for further research. Design/methodology/approach: The research employs a systematic literature review and bibliometric techniques to find 67 primary papers that have been published between 1984 and 2023. Using the Scopus database, the researchers built a summary of the literature on chatbots and AI in academic advice. Findings: Chatbot applications can be a promising approach to address the challenges of balancing personalized student advising with automation. More empirical research is required, especially on chatbots and other AI-based advising systems, to understand their effectiveness and how they can be integrated into educational settings. Research limitations/implications: This research's sample size may restrict its findings' generalizability. Furthermore, the study's focus on chatbots may overlook the potential benefits of other AI technologies in enhancing robo academic advising systems. Future research could explore the impact of robo academic advisors in diverse societal backgrounds to gain a more comprehensive understanding of their implications. Practical implications: Higher educational institutions (HEIs) should establish a robo academic advising system that serves various stakeholders. The system's chatbots and AI features must be user-friendly, considering the customers' familiarity with robots. Originality/value: This study contributes to a better understanding of HEIs' perceptions of the adoption of chatbots and AI in academic advising by providing insightful information about the main forces behind robo academic advising, illuminating the most frequently studied uses of chatbots and AI in academic advising.

Artificial intelligence (AI) and automation are newly impacting the governance of international students, a temporary resident category significant for both direct economic contributions and the formation of a "pool" of potential future immigrants in many immigrant-dependent countries. This paper focuses on tensions within Canada's education-migration ("edugration") system as new technologies intersect with migration regimes, which in turn relate to broader issues of security, administrative burdens, migration governance, and border imperialism. Using an Accidental Ethnography (AccE) approach drawing from practitioner-based legal research, we discuss three themes: (1) "bots at the gate" and the guise of AI's objectivity; (2) a murky international edu-tech industry; and (3) the administrative burdens of digitalized application systems. We suggest that researchers, particularly in education, can benefit from the insights of immigration practitioners who often become aware of potential trends before those less embedded in the everyday negotiation of migration governance.

Data transfer between isolated clusters is imperative for cybersecurity education, research, and testing. Such techniques facilitate hands-on cybersecurity learning in isolated clusters, allow cybersecurity students to practice with various hacking tools, and develop professional cybersecurity technical skills. Educators often use these remote learning environments for research as well. Researchers and students use these isolated environments to test sophisticated hardware, software, and procedures using full-fledged operating systems, networks, and applications. Virus and malware researchers may wish to release suspected malicious software in a controlled environment to observe their behavior better or gain the information needed to assist their reverse engineering processes. The isolation prevents harm to networked systems. However, there are times when the data is required to move in such quantities or speeds that it makes downloading onto an intermediate device untenable. This study proposes a novel turnstile model, a mechanism for one-way file transfer from one enterprise system to another without allowing data leakage. This system protects data integrity and security by connecting the isolated environment to the external network via a locked-down interconnection. Using medium-security isolated clusters, the researchers successfully developed a unidirectional file transfer system that acts as a one-way "turnstile" for secure file transfer between systems not connected to the internet or other external networks. The Turnstile system (source code available at github.com/monnin/turnstile) provides unidirectional file transfer between two computer systems. The solution enabled data to be transferred from a source system to a destination system without allowing any data to be transferred back in the opposite direction. The researchers found an automated process of transferring external files to isolated clusters optimized the transfer speed of external files to isolated clusters using Linux distributions and commands.

Background: Rapid automatised naming (RAN) has been found to predict children's reading and arithmetic abilities. However, the underlying mechanisms for its involvement in the two abilities are not clear. This study examines how RAN shared variances with domain-general and domain-specific abilities in predicting reading and arithmetic in Chinese children. Methods: One hundred and sixty-four children (mean age = 8 years 0 months, SD = 4 months) were administered with RAN tasks, word reading and arithmetic tasks and measures of working memory, processing speed, morphological awareness, phonological awareness and number line estimation. Results: RAN mainly shared variance with morphological awareness in predicting word reading, while it shared variance with processing speed and number line estimation in predicting arithmetic calculation. Conclusions: The findings indicated that RAN was related to reading and arithmetic for different reasons. The RAN-reading relationship partly reflected the semantic facilitation of the orthography-phonology links for both RAN stimuli and Chinese characters, while the RAN-arithmetic relationship partly reflected the shared process of retrieving semantic information from long-term memory embedded in the two tasks.

Selecting undergraduate majors or specializations is a crucial decision for students since it considerably impacts their educational and career paths. Moreover, their decisions should match their academic background, interests, and goals to pursue their passions and discover various career paths with motivation. However, such a decision remains challenging are unfamiliar with the job market, the demand for the required skills, and being in the proper placement in a major is not straightforward. Thus, an automatic recommendation system can be helpful for students to assist and guide them in the right decision. In this context, we developed a machine learning model to predict and recommend suitable specializations for undergraduate students according to the job market and student's academic history. Two hundred twenty-five records of students are considered to establish this work. The proposed approach encompasses four major steps, including data preprocessing to clean, scale, and prepare the data for training to avoid obtaining suboptimal results, accompanied by an oversampling process to equal the samples' distribution to prevent the model from being biased or poorly generalized. Furthermore, we conducted a feature selection step using Sequential Backward Selection (SBS) to extract the relevant features to improve the outcomes and reduce the risk of noise. The selected subset is used to train the model using the AdaBoost classifier. We deployed the Genetic algorithm to optimize the classifier's hyperparameters to maximize results. As a result, the findings of this study exhibit noticeable results compared to existing models, with an accuracy of 98.1%. The proposed model can be reliable in guiding undergraduate students through proper decisions regarding selecting their major.

Automated scoring is a current hot topic in creativity research. However, most research has focused on the English language and popular verbal creative thinking tasks, such as the alternate uses task. Therefore, in this study, we present a large language model approach for automated scoring of a scientific creative thinking task that assesses divergent ideation in experimental tasks in the German language. Participants are required to generate alternative explanations for an empirical observation. This work analyzed a total of 13,423 unique responses. To predict human ratings of originality, we used XLM-RoBERTa (Cross-lingual Language Model-RoBERTa), a large, multilingual model. The prediction model was trained on 9,400 responses. Results showed a strong correlation between model predictions and human ratings in a held-out test set (n = 2,682; r = 0.80; CI-95% [0.79, 0.81]). These promising findings underscore the potential of large language models for automated scoring of scientific creative thinking in the German language. We encourage researchers to further investigate automated scoring of other domain-specific creative thinking tasks.

Automatic generation of math word problems (MWPs) is a challenging task in Natural Language Processing (NLP), particularly connecting it to real-life problems because it can benefit students in developing a higher level of mathematical thinking. However, most of the MWPs are presented within a scholastic setting in a decontextualized way. This paper describes a prototype system that generates authentic math word problems (i.e., real-life problems) using authentic contextual recognition with controlled difficulty levels. Our innovative approach includes the acquisition of authentic contextual information through recognition technology, an instructional-based prompt generator for three different difficulty levels, and question generation through the Generative Pre-trained Transformer (GPT) model. We evaluated the performance of the system in terms of the quality of generated questions using automatic evaluation and human evaluation. Further, we assessed the usability of the system using heuristic evaluation. The automatic evaluation showed our generated MWPs were relevant for geometry topics and varied in sentence generation. The human evaluation found our system generated more realistic problems with satisfactory language quality and mathematical validity. Our system also produced more questions with higher difficulty levels based on human evaluation. Heuristic evaluation captured the usability of the system and highlighted the potential to be applied in a pedagogical context for mathematics learning.

University students often learn statistics in large classes, and in such learning environments, students face an exceptionally high risk of failure. One reason for this is students' frequent statistics anxiety. This study shows how students can be supported using e-learning exercises with automated knowledge of correct response feedback, supplementing a face-to-face lecture. To this end, we surveyed 67 undergraduate social science students at a German university and observed their weekly e-learning exercises. We aggregated students' exercise behavior throughout the semester to explain their exam performance. To control for participation bias, we included essential predictors of educational success, such as prior achievement, motivation, personality traits, time preferences, and goals. We applied a double selection procedure based on the machine learning method Elastic Net to include an optimal but sparse set of control variables. The e-learning exercises indirectly promoted the self-regulated learning techniques of retrieval practice and spacing and provided corrective feedback. Working on the e-learning exercises increased students' performance on the final exam, even after controlling for the rich set of control variables. Two-thirds of students used our designed e-learning exercises; however, only a fraction of students spaced out the exercises, although students who completed the exercises during the semester and were not cramming at the end benefited additionally. Finally, we discuss how the results of our study inform the literature on retrieval practice, spacing, feedback, and e-learning in higher education.

Providing consistent, individualized feedback to teachers is essential for improving instruction but can be prohibitively resource-intensive in most educational contexts. We develop M-Powering Teachers, an automated tool based on natural language processing to give teachers feedback on their uptake of student contributions, a high-leverage dialogic teaching practice that makes students feel heard. We conduct a randomized controlled trial in an online computer science course (N = 1,136 instructors), to evaluate the effectiveness of our tool. We find that M-Powering Teachers improves instructors' uptake of student contributions by 13% and present suggestive evidence that it also improves students' satisfaction with the course and assignment completion. These results demonstrate the promise of M-Powering Teachers to complement existing efforts in teachers' professional development.

Research on improving engineering skills in students advocates for high-quality teaching practices as well as the implementation of digitally enhanced management systems, such as e-Learning. Furthermore, COVID-19 led to several changes in education, such as switching drastically from face to face to emergency remote and later hybrid teaching. This study has focused on the proposal, the application and the evaluation of a new e-Learning teaching method. "FastTest PlugIn" has been used for the creation of large sets of questions, to develop parameterised and individualised exercises in Moodle. The method has been applied since COVID-19, considering different types of teaching: completely remote, hybrid and face to face. It has been implemented in some courses of the bachelor's and master's degree in engineering at the University of Cadiz (Spain). The academic performance and students' feedback on the method have been obtained. During remote teaching, students presented lower scores than before the pandemic. However, as students became familiar with the method and at least face-to-face exams were carried out, the scores and students' perception of the difficulty of the courses improved. The main novelty of this method is the reliability of creating different exercises for a consistent level of difficulty; the ability to avoid dishonest actions of students; and the time saved by instructors, as exercises are automatically corrected. This study provides the advantages and benefits of digitally enhancing university teaching. Furthermore, it is the first to investigate the integration of "FastTest PlugIn" to enhance teaching quality and engineering skills.

To explore the differential impact of job automation for different groups of primary and secondary school students, an analysis of variance was conducted using survey data on the occupational aspirations of British school students (aged 7-18) and probability statistics derived from a model of job automation. Results indicated that students aged 13 years old and above were more than twice as likely to express an occupational aspiration associated with a high risk of automation, along with a higher proportion of male students, lower socio-economic groups, and respondents knowing someone (particularly a parent) holding their desired occupation (P < 0.05).

Handwritten tasks are better suited than digital ones to assess higher-order mathematics skills, as students can express themselves more freely. However, maintaining reliability and providing feedback can be challenging when assessing high-stakes, handwritten mathematics exams involving multiple assessors. This paper discusses a new semi-automated grading approach called 'checkbox grading'. Checkbox grading gives each assessor a list of checkboxes consisting of feedback items for each task. The assessor then ticks those feedback items which apply to the student's solution. Dependencies between the checkboxes can be set to ensure all assessors take the same route on the grading scheme. The system then automatically calculates the grade and provides atomic feedback to the student, giving a detailed insight into what went wrong and how the grade was obtained. Atomic feedback consists of a set of format requirements for mathematical feedback items, which has been shown to increase feedback's reusability. Checkbox grading was tested during the final high school mathematics exam (grade 12) organised by the Flemish Exam Commission, with 60 students and 10 assessors. This paper focuses on students' perceptions of the received checkbox grading feedback and how easily they interpreted it. After the exam was graded, all students were sent an online questionnaire, including their personalised exam feedback. The questionnaire was filled in by 36 students, and 4 of them participated in semi-structured interviews. Findings suggest that students could interpret the feedback from checkbox grading well, with no correlation between students' exam scores and feedback understanding. Therefore, we suggest that checkbox grading is an effective way to provide feedback, also for students with shaky subject matter knowledge.

With the burgeoning popularity and swift advancements of automated writing evaluation (AWE) systems in language classrooms, scholarly and practical interest in this area has noticeably increased. This systematic review aims to comprehensively investigate current research on three prominent AWE systems: Grammarly, Pigai, and Criterion. Objectives include assessing each system's characteristics, advantages, and drawbacks, analyzing prior studies' frameworks, methodologies, findings, and implications, and identifying research gaps and future directions. The analysis of 39 articles underscored an escalating interest in scrutinizing AWE systems, predominantly focusing on their efficacy and learners' viewpoints. The findings demonstrated the positive impact of AWE systems on enhancing students' writing proficiency, with both learners and educators conveying positive attitudes towards these digital tools. However, several noteworthy research gaps endure, including the need to further investigate the usage patterns of AWE tools, expanding the participants to wider language proficiency and research comparing AWE feedback with peer feedback. The majority of the studies focused on non-native English-speaking university students over a single academic semester, using quantitative and mixed research methods. The review concludes by offering insights and recommendations for educators and researchers in the field, stressing the importance of tackling the identified research gaps and further delving into the potential of AWE systems in the age of generative artificial intelligence.

Purpose: This study examines the accuracy of Interaction Detection in Early Childhood Settings (IDEAS), a program that automatically transcribes audio files and estimates linguistic units relevant to speech-language therapy, including part-of-speech units that represent features of language complexity, such as adjectives and coordinating conjunctions. Method: Forty-five video-recorded speech-language therapy sessions involving 27 speech-language pathologists (SLPs) and 56 children were used. The F measure determines the accuracy of IDEAS diarization (i.e., speech segmentation and speaker classification). Two additional evaluation metrics, namely, median absolute relative error and correlation, indicate the accuracy of IDEAS for the estimation of linguistic units as compared with two conditions, namely, Oracle (manual diarization) and Voice Type Classifier (existing diarizer with acceptable accuracy). Results: The high F measure for SLP talk data suggests high accuracy of IDEAS diarization for SLP talk but less so for child talk. These differences are reflected in the accuracy of IDEAS linguistic unit estimates. IDEAS median absolute relative error and correlation values for nine of the 10 SLP linguistic unit estimates meet the accuracy criteria, but none of the child linguistic unit estimates meet these criteria. The type of linguistic units also affects IDEAS accuracy. Conclusions: IDEAS was tailored to educational settings to automatically convert audio recordings into text and to provide linguistic unit estimates in speech-language therapy sessions and classroom settings. Although not perfect, IDEAS is reliable in automatically capturing and returning linguistic units, especially in SLP talk, that are relevant in research and practice. The tool offers a way to automatically measure SLP talk in clinical settings, which will support research seeking to understand how SLP talk influences children's language growth.

Contribution: This research explores the effectiveness of a proposed teaching strategy in blockchain education, finding that it enhances learning outcomes, cognitive well-being, and student engagement in tertiary education, ultimately resulting in a shallower learning curve for STEM knowledge. Background: In the context of Industry 4.0, blockchain technology has emerged as a key driver of transformation in data management and system automation across a range of industrial applications. Despite its significance, the intricate theories and concepts associated with blockchain often serve as a deterrent for novice learners, inhibiting their ability to appreciate the value of industrial blockchain. Consequently, there is a pressing need to develop interactive teaching content that alleviates the steep learning curve. Intended Outcomes: The teaching strategy for the gamification in blockchain education is proposed, which positively influence students' cognitive well-being in terms of knowledge retention, cognitive curiosity, and heightened enjoyment. Application Design: Based on the experimental learning theory, the gamification of blockchain education, namely "BlockTrainHK", is implemented in the experimental learning cycle. Therefore, the gamified learning in experimental learning (GEL) strategy is proposed to examine the effectiveness of concrete experience, reflective observation, abstract conceptualization and active experimentation by two case studies. Findings: The results of the two-year study on the gamified blockchain education are encouraging: test groups using the GEL strategy were better in the cognitive well-being, and students' cognitive well-being is positively proportional to the level of individual technical knowledge and skills.

Background: Peer assessment has played an important role in large-scale online learning, as it helps promote the effectiveness of learners' online learning. However, with the emergence of numerical grades and textual feedback generated by peers, it is necessary to detect the reliability of the large amount of peer assessment data, and then develop an effective automated grading model to analyse the data and predict learners' learning results. Objectives: The present study aimed to propose an automated grading model with reliability detection. Methods: A total of 109,327 instances of peer assessment from a large-scale teacher online learning program were tested in the experiments. The reliability detection approach included three steps: recurrent convolutional neural networks (RCNN) was used to detect grade consistency, bidirectional encoder representations from transformers (BERT) was used to detect text originality, and long short-term memory (LSTM) was used to detect grade-text consistency. Furthermore, the automated grading was designed with the BERT-RCNN model. Results and Conclusions: The effectiveness of the automated grading model with reliability detection was shown. For reliability detection, RCNN performed best in detecting grade consistency with an accuracy rate of 0.889, BERT performed best in detecting text originality with an improvement of 4.47% compared to the benchmark model, and LSTM performed best with an accuracy rate of 0.883. Moreover, the automated grading model with reliability detection achieved good performance, with an accuracy rate of 0.89. Compared to the absence of reliability detection, it increased by 12.1%. Implications: The results strongly suggest that the automated grading model with reliability detection for large-scale peer assessment is effective, with the following implications: (1) The introduction of reliability detection is necessary to help filter out low reliability data in peer assessment, thus promoting effective automated grading results. (2) This solution could assist assessors in adjusting the exclusion threshold of peer assessment reliability, providing a controllable automated grading tool to reducing manual workload with high quality. (3) This solution could shift educational institutions from labour-intensive grading procedures to a more efficient educational assessment pattern, allowing for more investment in supporting instructors and learners to improve the quality of peer feedback.

While studies on teacher written feedback and automated writing evaluation (AWE) feedback have proliferated in recent decades, little attention has been paid to how AWE-teacher integrated feedback would influence students' engagement and their writing performance in second language (L2) writing. Against this backdrop, a quasi-experimental design was adopted to address this important gap. In our study, an intervention was implemented in two classes of English major sophomores in China, with a treatment group receiving AWE-teacher integrated feedback and a comparison group receiving teacher feedback. Data were collected from multiple sources over a 13-week semester to explore the Chinese EFL learners' engagement with the integrated feedback and their writing performance. Results showed that the students in the treatment group engaged with feedback more profoundly in behavior and cognition than those in the comparison group while both groups demonstrated similar affective engagement. Furthermore, compared with the comparison group, the treatment group improved their writing performance in content, organization, vocabulary, and language use significantly. Important implications are also discussed.

With the rapid advancement of information technologies, automated writing evaluation technologies have developed so fast that they can be applied to writing assessments. However, scanty studies have pooled the effects of automated writing evaluation on writing performance. Through a PRISMA protocol-based meta-analysis, this study concludes that automated writing evaluation technologies can significantly reduce anxiety such as writing anxiety and computer anxiety compared with those without the assistance of automated writing evaluation. Automated writing evaluation can also significantly improve writing motivation and second language (L2) writing skills. Educators and designers should highlight how to combine automated writing evaluation with human rating methods and maximize the improvements in L2 writing skills. In the future, artificial intelligence may be integrated into automated writing evaluation to develop advanced automated writing evaluation to address higher-level issues in writing practice.

To estimate the distribution of the number of acquaintances of the members of a society, the network scale-up method asks survey respondents about the number of people they know with features for which national statistics are available. While many features have been used for this purpose, first names have been suggested to produce particularly low levels of transmission error and recall bias. For this method to be precise, a set of names needs to be selected for the survey that jointly represents the population in relevant variables such as gender or age. This article provides a solution approach to finding the optimal set of names. This can be applied to any population for which a joint distribution of first names and relevant variables is available. We show that our approach successfully provides sets of names closely mirroring the population distributions for six countries with different name statistics.

In the context of formative assessment in classrooms, the incorporation of automated evaluation (AE) systems and teachers' interactions with them hold significant importance. This study aimed to investigate the cognitive processes of pre-service teachers as they engaged with an AE system. We developed an unsupervised learning-based AE system, the Scoring Assistant using Artificial Intelligence (SAAI). SAAI calculates scores without relying on predefined labels and generates scientific keywords from student responses to provide constructive feedback. We collected a substantial number of constructed responses from students, and four pre-service teachers evaluated these responses initially without any external assistance and then re-evaluated them using SAAI scores as a reference point. Employing a mixed-methods approach, this study demonstrated a strong level of consistency between human raters and SAAI scores. Pre-service teachers also reflectively recalibrated their assessments and adjusted their rubrics to identify students' learning more accurately. This study highlights the practical application of AE in real classroom settings and demonstrates how AE can enhance efficiency and accuracy in K-12 science assessments, thus supporting teachers.

Purpose: Oral language skills provide a critical foundation for formal education and especially for the development of children's literacy (reading and spelling) skills. It is therefore important for teachers to be able to assess children's language skills, especially if they are concerned about their learning. We report the development and standardization of a mobile app--LanguageScreen--that can be used by education professionals to assess children's language ability. Method: The standardization sample included data from approximately 350,000 children aged 3;06 (years;months) to 8;11 who were screened for receptive and expressive language skills using LanguageScreen. Rasch scaling was used to select items of appropriate difficulty on a single unidimensional scale. Results: LanguageScreen has excellent psychometric properties, including high reliability, good fit to the Rasch model, and minimal differential item functioning across key student groups. Girls outperformed boys, and children with English as an additional language scored less well compared to monolingual English speakers. Conclusions: LanguageScreen provides an easy-to-use, reliable, child-friendly means of identifying children with language difficulties. Its use in schools may serve to raise teachers' awareness of variations in language skills and their importance for educational practice.

Advances in computational language models increasingly enable adaptive support for self-regulated learning (SRL) in digital learning environments (DLEs; eg, via automated feedback). However, the accuracy of those models is a common concern for educational stakeholders (eg, policymakers, researchers, teachers and learners themselves). We compared the accuracy of four Dutch language models (ie, spaCy medium, spaCy large, FastText and ConceptNet NumberBatch) in the context of secondary school students' learning of causal relations from expository texts, scaffolded by causal diagram completion. Since machine learning relies on human-labelled data for the best results, we used a dataset with 10,193 students' causal diagram answers, compiled over a decade of research using a diagram completion intervention to enhance students' monitoring of their text comprehension. The language models were used in combination with four popular machine learning classifiers (ie, logistic regression, random forests, support vector machine and neural networks) to evaluate their performance on automatically scoring students' causal diagrams in terms of the correctness of events and their sequence (ie, the causal structure). Five performance metrics were studied, namely accuracy, precision, recall, F1 and the area under the curve of the receiver operating characteristic (ROC-AUC). The spaCy medium model combined with the neural network classifier achieved the best performance for the correctness of causal events in four of the five metrics, while the ConceptNet NumberBatch model worked best for the correctness of the causal sequence. These evaluation results provide a criterion for model adoption to adaptively support SRL of causal relations in DLEs.

Effective collaboration and teamwork skills are critical in high-risk sectors, as deficiencies in these areas can result in injuries and risk of death. To foster the growth of these vital skills, immersive learning spaces have been created to simulate real-world scenarios, enabling students to safely improve their teamwork abilities. In such learning environments, multiple dialogue segments can occur concurrently as students independently organise themselves to tackle tasks in parallel across diverse spatial locations. This complex situation creates challenges for educators in assessing teamwork and for students in reflecting on their performance, especially considering the importance of effective communication in embodied teamwork. To address this, we propose an automated approach for generating teamwork analytics based on spatial and speech data. We illustrate this approach within a dynamic, immersive healthcare learning environment centred on embodied teamwork. Moreover, we evaluated whether the automated approach can produce transcriptions and epistemic networks of spatially distributed dialogue segments with a quality comparable to those generated manually for research objectives. This paper makes two key contributions: (1) it proposes an approach that integrates automated speech recognition and natural language processing techniques to automate the transcription and coding of team communication and generate analytics; and (2) it provides analyses of the errors in outputs generated by those techniques, offering insights for researchers and practitioners involved in the design of similar systems.

Multimodal learning analytics (MMLA) research has made significant progress in modelling collaboration quality for the purpose of understanding collaboration behaviour and building automated collaboration estimation models. Deploying these automated models in authentic classroom scenarios, however, remains a challenge. This paper presents findings from an evaluation of collaboration quality estimation models. We collected audio, video and log data from two different Estonian schools. These data were used in different combinations to build collaboration estimation models and then assessed across different subjects, different types of activities (collaborative-writing, group-discussion) and different schools. Our results suggest that the automated collaboration model can generalize to the context of different schools but with a 25% degradation in balanced accuracy (from 82% to 57%). Moreover, the results also indicate that multimodality brings more performance improvement in the case of group-discussion-based activities than collaborative-writing-based activities. Further, our results suggest that the video data could be an alternative for understanding collaboration in authentic settings where higher-quality audio data cannot be collected due to contextual factors. The findings have implications for building automated collaboration estimation systems to assist teachers with monitoring their collaborative classrooms.

Background: Perceptual measures such as speech intelligibility are known to be biased, variant and subjective, to which an automatic approach has been seen as a more reliable alternative. On the other hand, automatic approaches tend to lack explainability, an aspect that can prevent the widespread usage of these technologies clinically. Aims: In the present work, we aim to study the relationship between four perceptual parameters and speech intelligibility by automatically modelling the behaviour of six perceptual judges, in the context of head and neck cancer. From this evaluation we want to assess the different levels of relevance of each parameter as well as the different judge profiles that arise, both perceptually and automatically. Methods and Procedures: Based on a passage reading task from the Carcinologic Speech Severity Index (C2SI) corpus, six expert listeners assessed the voice quality, resonance, prosody and phonemic distortions, as well as the speech intelligibility of patients treated for oral or oropharyngeal cancer. A statistical analysis and an ensemble of automatic systems, one per judge, were devised, where speech intelligibility is predicted as a function of the four aforementioned perceptual parameters of voice quality, resonance, prosody and phonemic distortions. Outcomes and Results: The results suggest that we can automatically predict speech intelligibility as a function of the four aforementioned perceptual parameters, achieving a high correlation of 0.775 (Spearman's [rho]). Furthermore, different judge profiles were found perceptually that were successfully modelled automatically. Conclusions and Implications: The four investigated perceptual parameters influence the global rating of speech intelligibility, showing that different judge profiles emerge. The proposed automatic approach displayed a more uniform profile across all judges, displaying a more reliable, unbiased and objective prediction. The system also adds an extra layer of interpretability, since speech intelligibility is regressed as a direct function of the individual prediction of the four perceptual parameters, an improvement over more black box approaches.

Cognitive presence is a core construct of the Community of Inquiry (CoI) framework. It is considered crucial for deep and meaningful online-based learning. CoI-based real-time dashboards visualizing students' cognitive presence may help instructors to monitor and support students' learning progress. Such real-time classifiers are often based on the linguistic analysis of the content of posts made by students. It is unclear whether these classifiers could be improved by considering other learning traces, such as files attached to students' posts. We aimed to develop a German-language cognitive presence classifier that includes linguistic analysis using the Linguistic Inquiry and Word Count (LIWC) tool and other learning traces based on 1,521 manually coded meaningful units from an online-based university course. As learning traces, we included not only the linguistic features from the LIWC tool, but also features such as attaching files to a post, tagging, or using terms from the course glossary. We used the k-nearest neighbor method, a random forest model, and a multilayer perceptron as classifiers. The results showed an accuracy of up to 82% and a Cohen's K of 0.76 for the cognitive presence classifier for German posts. Including learning traces did not improve the predictive ability. In conclusion, we developed an automatic classifier for German-language courses based on a linguistic analysis of students' posts. This classifier is a step toward a teacher dashboard. Our work also provides the first fully CoI-coded German dataset for future research on cognitive presence.

The template-based automated item-generation (TAIG) approach that involves template creation, item generation, item selection, field-testing, and evaluation has more steps than the traditional item development method. Consequentially, there is more margin for error in this process, and any template errors can be cascaded to the generated items. Therefore, it is essential to eliminate the source of errors and ensure the quality of the template so items can be problem-free. The article introduces a process to reduce template errors at the early stage of template development, minimize the impact of template errors on generated items, and increase the survival rates of generated items. The article also discusses a statistical method to establish confidence in the quality of the template by systematically examining the quality of the generated items. The proposed method can reduce the review process for some items generated from a template.

The use of automated feedback (AF) has been increasing in English writing courses for emergency remote education (ERE) due to the escalating COVID-19 pandemic crisis in Vietnam. The ERE English writing courses for in-service bachelor programs demanded an effective, fast, free, and user-friendly AF tool that does not require a login ID, which can help students to develop English writing skills and increase their motivation and self-learning ability. The main objectives of Vietnamese in-service students were to improve their English writing skills and update teaching methodologies that integrate technology. Although many studies emphasized the impact of AF tools in normal teaching contexts, relatively less research was conducted the use of AF tools among in-service students. This study examined the contribution of the Marking Mate AF to the improvement of the English writing skills of in-service learners in ERE writing classes. The participants were 82 in-service students in a Vietnamese public university. The dataset included English writing pre and posttest results collected during the first semester of academic year 2020-2021. The mean scores (standard deviation) of these tests were used to benchmark the writing performance of students. A self-report survey was also conducted to investigate the attitude of students toward Marking Mate AF. The results demonstrated that the posttest score (8.739; SD = 0.8495) was higher than the pretest score (7.439; SD = 0.976). Students expressed favorable opinions toward the usefulness of Marking Mate. Based on the findings, the study discussed the pedagogical implications regarding the incorporation of AF in face-to-face and online classes.

Learning engagement is an essential indication to define students' learning pacification in the class, and its automated identification technique is the foundation for exploring how to effectively explain the motive of learning impact modifications and making intelligent teaching choices. Current research have demonstrated that there is a direct link between learning engagement and emotional investment and behavioural investment, and it is appropriate and required to apply artificial intelligence to perform autonomous assessment. Unfortunately, the number of relevant research is limited, and the features of learning engagement in certain contexts have not been thoroughly examined. In this research, we highlight the features of a particular application scenario of learning engagement: the application scenario of learning engagement has to incorporate both the coarse-grained information of human body position and the fine-grained information of facial expressions. On the basis of this analysis, a fine-grained learning participation recognition model that suppresses background clutter information is presented. This model can effectively extract coarse and fine-grained information to improve the recognition of learning participation in real-world teaching situations. Particularly, the CNN-Transformer model suggested in this study employs CNN to extract fine-grained information of facial expressions and Transformer to recover coarse-grained information of human body position. Simultaneously, we gathered and categorised real teaching data based on the features of learning engagement situations and enhanced the data quality via crowdsourcing and expert verification. The experimental findings indicate that the CNN-Transformer model can accurately predict the learning engagement of unknown participants with a 92.9% rate of accuracy. Comparative trials reveal that the model's recognition impact is much greater than that of other sophisticated deep learning approaches. Our research offers a framework for future work on deep learning approaches in learning engagement settings.

Differentiated instruction is much demanded yet quite challenging in face of the growing student diversity in today's K-12 classrooms. One major challenge is the provision of differentiated materials to students. Automated text simplification (ATS) tools fueled by natural language processing may serve as a useful assistant for teachers. However, little is known about teachers' contextualized use of ATS over time. This case study traced two teachers' use of ATS systems over a semester. Drawing upon three semi-structured interviews and teacher-generated materials with ATS, we identified an evolving pattern of teachers' engagement with ATS systems, a progression from a blind reliance on the tool to a more critical and coordinated use of the tool over time. We further revealed that teachers' evolving understanding of DI, positioning of the role of ATS systems and human instructors, and interpretation of DI need in specific teaching situations interplayed to shape their particular ways of engagement. Overall, this study contributes to the understanding of teachers' contextualized use of ATS technology for DI. By revealing the influencing factors, the findings hold significant pedagogical implications to inform the design of ATS tools and the creation of favorable conditions to maximize the potential of ATS tools for DI and language teaching and learning in general.

Technological tools that provide automated feedback on classroom teaching afford a unique opportunity for educators to engage in self-reflection and work towards improvement goals, in particular to ensure that their instructional environment is equitable and productive for students. More information is needed about how teachers experience automated professional learning tools, including what they perceive as relevant and impactful for their everyday teaching. This mixed-methods study explored the perceptions and engagement of 21 math teachers who used an AI-based tool that generates information about their discourse practices from classroom recordings. Findings indicate that teachers perceived the tool to have a high utility value, especially those who elected to use it over two school years. These teachers increased their use of talk moves over time, suggesting that they were making intentional changes due to their review and uptake of the personalized feedback. These results from this study speak to promising directions for developing AI-based professional learning tools that can support teacher learning and instructional improvement, particularly tools with robust perceived utility.

Computer-supported learning technologies are essential for conducting hands-on cybersecurity training. These technologies create environments that emulate a realistic IT infrastructure for the training. Within the environment, training participants use various software tools to perform offensive or defensive actions. Usage of these tools generates data that can be employed to support learning. This paper investigates innovative methods for leveraging the trainee data to provide automated feedback about the performed actions. We proposed and implemented feedback software with four modules that are based on analyzing command-line data captured during the training. The modules feature progress graphs, conformance analysis, activity timeline, and error analysis. Then, we performed field studies with 58 trainees who completed cybersecurity training, used the feedback modules, and rated them in a survey. Quantitative evaluation of responses from 45 trainees showed that the feedback is valuable and supports the training process, even though some features are not fine-tuned yet. The graph visualizations were perceived as the most understandable and useful. Qualitative evaluation of trainees' comments revealed specific aspects of feedback that can be improved. We publish the software as an open-source component of the KYPO Cyber Range Platform. Moreover, the principles of the automated feedback generalize to different learning contexts, such as operating systems, networking, databases, and other areas of computing. Our results contribute to applied research, the development of learning technologies, and the current teaching practice.

The Internet of Things (IoT) is a network of objects that can exchange data with other devices also connected to the Internet. One of the most common consumer examples of IoT is home automation, as a variety of smart devices, including doorbells, lightbulbs, thermostats, and refrigerators are now available which users can control remotely using mobile apps or smart speakers. In this hands-on activity, students will apply their basic skills in accessing wireless networks and using mobile devices to connect an Amazon Echo smart speaker to a home network, configure smart plugs to communicate with the Echo, and develop routines to interact with the smart plugs, smartphones, and other smart devices.