Search

While studies on teacher written feedback and automated writing evaluation (AWE) feedback have proliferated in recent decades, little attention has been paid to how AWE-teacher integrated feedback would influence students' engagement and their writing performance in second language (L2) writing. Against this backdrop, a quasi-experimental design was adopted to address this important gap. In our study, an intervention was implemented in two classes of English major sophomores in China, with a treatment group receiving AWE-teacher integrated feedback and a comparison group receiving teacher feedback. Data were collected from multiple sources over a 13-week semester to explore the Chinese EFL learners' engagement with the integrated feedback and their writing performance. Results showed that the students in the treatment group engaged with feedback more profoundly in behavior and cognition than those in the comparison group while both groups demonstrated similar affective engagement. Furthermore, compared with the comparison group, the treatment group improved their writing performance in content, organization, vocabulary, and language use significantly. Important implications are also discussed.

Differentiated instruction is much demanded yet quite challenging in face of the growing student diversity in today's K-12 classrooms. One major challenge is the provision of differentiated materials to students. Automated text simplification (ATS) tools fueled by natural language processing may serve as a useful assistant for teachers. However, little is known about teachers' contextualized use of ATS over time. This case study traced two teachers' use of ATS systems over a semester. Drawing upon three semi-structured interviews and teacher-generated materials with ATS, we identified an evolving pattern of teachers' engagement with ATS systems, a progression from a blind reliance on the tool to a more critical and coordinated use of the tool over time. We further revealed that teachers' evolving understanding of DI, positioning of the role of ATS systems and human instructors, and interpretation of DI need in specific teaching situations interplayed to shape their particular ways of engagement. Overall, this study contributes to the understanding of teachers' contextualized use of ATS technology for DI. By revealing the influencing factors, the findings hold significant pedagogical implications to inform the design of ATS tools and the creation of favorable conditions to maximize the potential of ATS tools for DI and language teaching and learning in general.

Aided by big-data technology and artificial intelligence, automated writing evaluation (AWE) systems aim to help students engage in self-regulated learning and improve their academic writing in the digital era. While much research on student engagement with AWE systems has been conducted in mainstream classrooms, little attention has been paid to ethnic minority students who have different educational experiences and sociocultural backgrounds. This study investigates how a group of ethnic minority students from Xinjiang Uyghur Autonomous Region engaged with an AWE system in academic writing during an 18-week semester at a comprehensive Chinese university. Data were collected from multiple sources including student multiple-draft assignments, AWE feedback, and retrospective interviews with the students. The participants were found to engage with the AWE system behaviourally, cognitively, and affectively in their writing and revision, but the extent to which they engaged in the process is influenced by their multilingual proficiency, familiarity with digital technologies, learning beliefs and sociocultural identities. Those who were more actively engaged and obtained more learning gains tended to capitalise on their linguistic and cultural capital in their engagement with AWE feedback in their writing. The article concludes with educational implications for ethnic minority students in higher education.

Online asynchronous interaction is considered a core part of online teacher training, which has an important impact on learners' learning experience and learning outcomes. How to provide immediate and effective feedback through technical support based on the learners' interactive content and enhance interactive connection has become a key issue in massive online teacher training. This study designed an automated feedback framework from feedback strategy, feedback way, and feedback type. Then, we designed an emotional-cognitive feedback approach and an emotional-cognitive-metacognitive feedback approach based on automated analysis of online asynchronous interactions. A massive online teacher training course was conducted to provide automated feedback to 1438 learners in the online asynchronous interaction. Results showed that the two approaches enabled learners to interact more positively, slow down the growth of the dropout rate, and help learners adjust their emotional state and cognitive level. Particularly, the emotional-cognitive-metacognitive feedback approach could facilitate learners' self-regulation and improve feedback quality. Through questionnaires and semi-structured interviews, we found that learners were obsessed by automated emotional-cognitive-metacognitive feedback approach and they believed that it was helpful for their learning. This study is of great significance and application value to widely carry out high-quality, massive and personalized online learning.

Automatic Speech Scoring (ASS) has increasingly become a useful tool in oral proficiency testing for Second Language (L2) learners. However, limited studies investigate the alignment of ASS indices with the Complexity, Accuracy, and Fluency (CAF)--the three dimensions in evaluating L2 speakers' oral proficiency, and the subsequent impact indices on the oral performance of Chinese college students. To bridge this gap, this study used comparative analysis, Pearson analysis, and linear regression analysis to delve into the relationship and correlations between paired ASS indicators of "pronunciation", "fluency", "integrity", "speed", "duration", and "overall", while also analyzing the relationships between "overall" and other variables. These analyses were conducted using 956 audio clips of freshmen who took the College English Test-Spoken English Test Band 4 (CET-SET-4) in May 2022 in China. The findings reveal that (1) the ASS indicators and evaluation methods are similar but not identical to those employed in prior studies; (2) "pronunciation" encapsulates both the accuracy and fluency dimensions of CAF; (3) "pronunciation" and "integrity" have significant impacts on Chinese college students' oral English performance in read-aloud tasks. The study suggests that future research should further investigate the specific pronunciation challenges faced by Chinese college students, such as phonetics, stress, and intonation. Additionally, it highlights the need to comprehend teachers' attitudes and preferences towards ASS to enhance its effectiveness in assessing second language (L2) learners' oral proficiency. The study would provide some references to teachers for oral English teaching design and students for their self-assessment of oral English proficiencies.

Chinese as a second language (CSL) learning has attracted more attention and supporting learners with adaptive resources becomes difficult. Some online systems recommended pre-designed resources from existing databases while the resources could not match learners' context. Designing resources dynamically according to learners' needs could be a solution while it's time-consuming. Targeting this problem, we proposed a "content-structure" loosely coupled model. Based on the model, we developed an automatic resource generation system and used it in a university. One class was chosen and the students were randomly assigned to the experimental (22 students) and control group (21 students). They participated in the course all the same except the resources generation method. During the learning process, the online behaviors were recorded for behavioral analysis and the learners' learning outcome and perceptions were measured by tests and questionnaires. Results showed that the system played positive roles in improving learners' learning outcome and perceptions. Moreover, we found that learners in experimental group participated more actively and there's evidence that the system could help learners better reflect on their needs. The results revealed the effectiveness of the system in supporting CSL learners' contextualized learning. This design will provide inspiration for future context-aware CSL learning research.

The extensive use of automated speech scoring in large-scale speaking assessment can be revolutionary not only to test design and rating, but also to the learning and instruction of speaking based on how students and teachers perceive and react to this technology. However, its washback remained underexplored. This mixed-method study aimed to investigate the washback of TOEFL iBT Speaking's "SpeechRater" on Chinese EFL learners through questionnaire and interviews, and explore its associations with test performance and its multi-levelled influential factors. The participants received a mixture of positive and negative washback, such as their motivated individual learning through personal devices, decreasing real-life communicative practices, and increasing exam-driven behaviours. Test takers' personal understandings of automated speech scoring were found directly influential to the washback of "SpeechRater" that they experienced. Furthermore, their test scores of TOEFL iBT Speaking were positively correlated with the implicit washback of "SpeechRater" on their learning but uncorrelated with its explicit washback on their test preparation. The findings have been drawn on to propose a washback model of automated speech scoring and make suggestions to test designers, teachers and learners on how to boost its positive washback and mitigate its negative washback. This research has concluded the importance of test takers' awareness of the integrated dimensions to evaluate spoken English in real-life use. Accordingly, instructional implications are discussed on how teachers can guide the students to utilize automated speech scoring in learning and set up comprehensive learning goals for spoken English.

Online collaborative learning is implemented extensively in higher education. Nevertheless, it remains challenging to help learners achieve high-level group performance, knowledge elaboration, and socially shared regulation in online collaborative learning. To cope with these challenges, this study proposes and evaluates a novel automated assessment and feedback approach that is based on knowledge graph and artificial intelligence technologies. Following a quasi-experimental design, we assigned a total of 108 college students into two conditions: an experimental group that participated in online collaborative learning and received automated assessment and feedback from the tool, and a control group that participated in the same collaborative learning activities without automated assessment and feedback. Analyses of quantitative and qualitative data indicated that the introduced automated assessment and feedback significantly promoted group performance, knowledge elaboration, and socially shared regulation of collaborative learning. The proposed knowledge graph-based automated assessment and feedback approach shows promise in providing a valuable tool for researchers and practitioners to support online collaborative learning.

With the rapid advance in educational technology, electronic feedback (e-feedback) has found its way to EFL writing process. The aim of this study is to investigate the effects of three e-feedback modes, that is, automated written corrective feedback (AWCF), asynchronous computer-mediated communication (ACMC), and their combination on EFL learner's writing performance and to examine learner's perceptions towards these three feedback modes. A total of 122 EFL learners from a Chinese university participated in this study. They were randomly assigned to four groups, each of which received a different type of feedback (i.e. self-feedback, AWCF, ACMC, AWCF plus ACMC) in their writing process. Findings revealed that the AWCF plus ACMC group outperformed the others in terms of complexity, accuracy, and fluency (CAF). In addition, the results of the questionnaire and interviews indicated that higher evaluations and more positive feelings went to the AWCF plus ACMC mode. These results suggest that this combination mode may meet individual needs in revising, maximize the writing effectiveness, and help the learners to become more autonomous and motivated in the writing process. Finally, implications for English writing pedagogy in the post-pandemic era are provided.

The development of artificial intelligence (AI) technology has enhanced the use of automated speech evaluation systems for language learners to practice speaking skills. This study investigated whether various automatic feedback offered by AI speech evaluation programs can help English as a foreign language (EFL) learners develop speaking skills. Forty EFL learners in China participated in this study. Data collection included qualitative and quantitative data. The results showed that the majority of participants believed they improved their speaking skills with the feedback offered by the AI speaking evaluation program. The findings also revealed that there were significant improvements in their mean scores of speaking skills in pre- and post-tests. Therefore, it is suggested that AI speaking evaluation systems could provide more varied textual feedback and practical suggestions to assist EFL learners in developing speaking skills.

Despite a large number of studies on the adoption of automated writing evaluation (AWE) systems, the effects of automated written corrective feedback (AWCF) on English as a Foreign Language (EFL) students' writing has been insufficiently documented. This study employed a mixed-method approach to examine such effects because of the significance of AWCF in EFL writing. Using a quasi-experimental design, this study explored how AWCF through Grammarly affected EFL students' writing quality. A total of 67 EFL students from two intact university English classes participated in this study, with a treatment group receiving two rounds of Grammarly feedback and teacher feedback while a comparison group receiving teacher feedback only. The results of the posttest writing task revealed that the students from the treatment group did not significantly outperform the students from the comparison group in syntactic and lexical complexity, accuracy, and fluency. A follow-up questionnaire consisting of fixed-response and open-ended questions was administered to the students from the treatment group after the posttest to elicit the students' perceptions of Grammarly feedback effects on their writing. The qualitative findings supported and provided deeper insights into the quantitative results. This study was concluded with a discussion of its limitations and implications.

Group formation is a critical factor which influences collaborative processes and performances in computer-supported collaborative learning (CSCL). Automatic grouping has been widely used to generate groups with heterogeneous attributes and to maximize the diversity of students' characteristics within a group. But there are two dominant challenges that automatic grouping methods need to address, namely the barriers of uneven group size problem, and the inaccessibility of student characteristics. This research proposes an optimized, genetic algorithm-based grouping method that includes a conceptual model and an algorithm module to address these challenges. Through a quasi-experiment research, we compare collaborative groups' performance, processes, and perceptions in China's higher education. The results indicate that the experimental groups outperform the traditional grouping methods (i.e., random groups and student-formed groups) in terms of final performances, collaborative processes, and student perceptions. Based on the results, we propose implications for implementation of automatic grouping methods, and the use of collaborative analytics methods in CSCL.

Notable mean score differences for the "e-rater"® automated scoring engine and for humans for essays from certain demographic groups were observed for the "GRE"® General Test in use before the major revision of 2012, called rGRE. The use of e-rater as a check-score model with discrepancy thresholds prevented an adverse impact on the examinee score at the item or test level. Despite this control, there remains a need to understand the root causes of these demographically based score differences and to identify potential mechanisms for avoiding future instances of discrepancy. In this study, we used a combination of statistical methods and human review to propose hypotheses about the root cause of score differences and whether such discrepancies reflect inadequacies of e-rater, human scoring, or both. The human rating process was found to be influenced strongly by the scale structure and did not fully correspond to the e-rater scoring mechanism. The human raters appeared to be using conditional logic and a rule-based approach to their scoring, while e-rater uses linear weighting of all the features. These analyses have implications for future research and operational policies for the scoring of the rGRE.

The role of internet technology in higher education and particularly in teaching English as a Foreign language is increasingly prominent because of the interest in the ways in which technology can be applied to support students. The automated evaluation scoring system is a typical demonstration of the application of network technology in the teaching of English writing. Many writing scoring platforms have been developed and used in China, which can provide online instant and corrective feedback on students' writing. However, the validity of "Aim Writing," a product developed by Microsoft Research Asia, which claims to be the best tool to facilitate Chinese EFL learners, has not been tested in previous studies. In this mixed methods study, the feedback and effect of "Aim Writing" on college students' writing will be investigated and compared to the instructor's feedback. The results indicate that "Aim Writing"'s performance is insufficient to support all students' needs for writing and that colleges should encourage a hybrid model that contains both AES and instructor's feedback in writing.

While automated feedback is becoming readily accessible to student writers, how students employ resources and strategies to use such feedback remains largely unexplored. Informed by activity theory and the construct of appropriation, this study conceptualizes students' use of automated feedback as social appropriation mediated by resources and internalization strategies. Based on multiple sources of data including students' drafts, automated feedback and submission information, semi-structured interviews and documents, this study explored a cohort of English as a Foreign Language (EFL) students' experiences of appropriating automated feedback in their writing activities. The findings revealed three forms of appropriation (i.e., regular, partial, and rare) among the students, who had used various artifacts (e.g., dictionary) and rules (e.g., teacher requirement), taken on different roles (e.g., student writer, spotlight-avoider), and resorted to different community members (e.g., peers) to mediate their appropriation. The findings also showed that the students further differed in their internalization of the resources through three sub-processes: selecting, emotion-regulating, and goal-setting. These findings pose a critical need to revisit the idea that submission for immediate automated feedback is motivating and to watch the potential washback effect of setting a threshold score when pedagogically using automated feedback. The study argues that students should develop their awareness of the mediating resources and the internationalization strategies for more self-regulated use of automated feedback.

Automated writing evaluation (AWE) has been used increasingly to provide feedback on student writing. Previous research typically focused on its inter-rater reliability with human graders and validation frameworks. The limited body of research has only discussed students' attitudes or perceptions in general. A systematic investigation of the driving factors contributing to students' acceptance is still lacking. This study proposes an extended technology acceptance model (TAM) to identify the environmental, individual, educational, and systemic factors that influence college students' acceptance of AWE feedback and examine how they affect college students' usage intention. Structural equation modeling (SEM) was used to analyze the quantitative survey data from 448 Chinese college students who had used AWE feedback for at least one semester. Results revealed that students' behavioral intention to use AWE feedback was affected by the subjective norm, facilitating conditions, perceived trust, AWE self-efficacy, cognitive feedback, and system characteristics. Among them, subjective norm, perceived trust, and cognitive feedback positively influenced perceived usefulness; facilitating conditions, AWE self-efficacy, and system characteristics were significant determinants of perceived ease of use; anxiety played no role for experienced users. Implications from these findings to AWE developers and practitioners are further elaborated.

Despite the growing attention being paid to the use of Automated Writing Evaluation (AWE) in China, it is still uncertain what factors lie behind EFL (English-as-a-foreign-language) learners' continuance intention to use it. To this end, by adding two external factors (i.e., computer self-efficacy and perceived ease of use) to the expectation confirmation model (ECM), we surveyed 345 Chinese EFL learners and tested a number of proposed hypotheses using their response data. Data were analyzed using descriptive statistics and structural equation modeling (SEM). Results demonstrated that four factors directly influenced EFL learners' continuance intention to use AWE, of which perceived ease of use was the most significant factor. Furthermore, confirmation was the most important factor impacting on EFL learners' satisfaction and perceived ease of use of AWE. Perceived ease of use of AWE played an important role in influencing EFL learners' perceived usefulness of AWE. Implications regarding the findings were also discussed.

Collaborative reflection (co-reflection) plays a vital role in collaborative knowledge construction and behavior shared regulation. Although the mixed effect of online co-reflection was reported in the literature, few studies have comprehensively examined both individual and group factors and their relationships that affect the co-reflection level. Therefore, this study explored the structural relationships between achievement goals (task-based, self-based, and other-based goals), online community identification, and co-reflection, which can consequently assist instructors in improving the related pedagogical strategies. To this end, 26813 posts on MOOC and college online learning platforms were gathered. Specifically, deep learning techniques were first used to train a classifier that classifies the large-scale co-reflection text automatically. The Bayesian method was then applied to disclose the structural relationships among achievement goals, community identification, and co-reflection. The results showed that the proposed classification algorithm achieved the best performance. Two best-fit models for characterizing the respective relationships between co-reflection and community identification as well as achievement goals were obtained using the Bayesian method. The results of the experiments on these two models demonstrated that both task-avoidance and other-avoidance goals were related directly to co-reflection, all task-approach, self-approach and other-approach goals were related indirectly to co-reflection, but self-avoidance goals had both a direct and an indirect relationship with co-reflection. The relationship between community identification and co-reflection was mediated by other-based goals. Some theoretical and practical implications were discussed for instructors and practitioners to build an online community.

This study aims to explore to what extent syntactic complexity predicts holistic scores generated by China's two prominent automated writing evaluation systems--"pigai" and "iWrite," the results of which will have some implications for the validity of these two systems. In the meanwhile, this study targets how syntactic complexity differs across writing tasks--between- and within- discourse mode as well as the effects of topic on the use of syntactic measures. To these ends, we examined a corpus of 445 writing samples with 4 prompts on two discourse modes (argumentative and expository) written by Chinese freshmen. These samples were analyzed for the sensitivity of China's AES-generated scores to syntactic complexity, effects of topic and discourse-mode. Results indicated that there were no relationships between the variations of syntactic measures and the computer-generated scores. Strong topic effects were found within the same discourse mode, but no significant changes were observed in the use of syntactic measures across discourse modes.

With the advancement of automatic speech recognition (ASR) technology, ASR-based pronunciation assessment can diagnose learners' pronunciation problems. Meanwhile, ASR-based pronunciation training allows more opportunities for pronunciation practice. This study aims to investigate the effectiveness of ASR technology in diagnosing English pronunciation errors and to explore teachers' and learners' attitudes towards using ASR technology as a pronunciation assessment tool and as a learning tool. Five Chinese EFL learners participated in read-aloud tests, including a human-assessed test and an ASR-assessed test. Pronunciation error types diagnosed by the two tests were compared to determine the extent of overlapping areas. The findings demonstrate that there were overlaps between human rating and machine rating at the segmental level. Moreover, it was found that learners' varied pronunciation learning needs were met by using the ASR technology. Implications of the study will provide insights relevant to using ASR technology to facilitate English pronunciation assessment and learning.

This study implemented a data-driven approach to identify Chinese high school students' common errors in a Java-based introductory programming course using the data in an automated assessment tool called the Mulberry. Students' error-related behaviors were also analyzed, and their relationships to success in introductory programming were investigated. This study identified 15 common compilation errors and 6 common test errors. The results showed that these common errors accounted for a large proportion of all errors, so identifying the common errors is important to help students succeed in introductory programming courses. Based on these common errors, five underlying student difficulties were identified and are discussed. In addition, after analyzing existing measures of students' error-related behaviors, we developed a measure called improvement rate to quantify students' success in fixing errors. The results of our study suggest that students' competence of improving code is important to their success in introductory programming. We recommend researchers design and develop automated assessment tools that provide feedback messages for common student errors and instructors who explicitly teach knowledge and skills of improving code in class.

Automated writing scoring can not only provide holistic scores but also instant and corrective feedback on L2 learners' writing quality. It has been increasing in use throughout China and internationally. Given the advantages, the past several years has witnessed the emergence and growth of writing evaluation products in China. To the best of our knowledge, no previous studies have touched upon the validity of China's automated essay scoring systems. By drawing on the four major categories of argument for validity framework proposed by Kane--scoring, generalization, extrapolation, and implication, this article aims to evaluate the performance of one of the China's automated essay scoring systems--iWrite against human scores. The results show that iWrite fails to be a valid tool to assess L2 writings and predict human scores. Therefore, iWrite currently should be restricted to nonconsequential uses and cannot be employed as an alternative to or a substitute for human raters.

This article aims to find out the validity of rhythm measurements to capture the rhythmic features of Chinese English. Besides, the reliability of the valid rhythm measurements applied in automatically scoring the English rhythm proficiency of Chinese EFL learners is also explored. Thus, two experiments were carried out. First, thirty students of English major and five native English speakers were selected to read ten English sentences. The participants were divided into four proficiency groups according to human scoring. Then seven previously proposed rhythm measurements were investigated in four proficiency groups. One-way ANOVA results showed that five rhythm measurements were valid to distinguish different English rhythm patterns among four proficiency groups. Based on the valid measurements, an experiment of automatic scoring for English rhythm proficiency was also conducted through statistical technique Multiple Regression. The correlation coefficient between the autoscores and the scores made by experienced teachers reached 0.866. The result showed a high reliability of the objective evaluation for English rhythm proficiency of Chinese EFL learners.

Automated Writing Evaluation (AWE) has been widely applied in computer-assisted language learning (CALL) in China. However, little is known about factors that influence learners' intention to use AWE. To this end, by adding two external factors (i.e. computer self-efficacy and computer anxiety) to the technology acceptance model (TAM), we surveyed 245 Chinese college students and tested a number of proposed hypotheses using their response data. Results revealed that learners' behavioural intention to use AWE is directly determined by perceived usefulness, attitude towards using, and computer self-efficacy; and it is indirectly influenced by learners' perceived ease of use; learners' computer self-efficacy and computer anxiety play no role to perceived usefulness. Instead, they significantly affect perceived ease of use. Implications of these results are discussed as well.

This study investigates the effects of using an online Automated Essay Assessing (AEA) system on EFL graduate students' writing. Eighty four EFL graduate students divided into the treatment group and the control group participated in this study. The treatment group was asked to use an AEA system to assist their essay writing. Both groups were pretested before the treatment. After the sixteen-week treatment period, the writing posttest was given to both groups to measure their progress in their writing. The results indicate that, although both groups have no significant difference in the pretest, the difference between their mean scores on the posttest is statistically significant. The mean score of the treatment group is higher than that of the control group, and students' use frequency of the AEA system is closely related to their writing improvement. Therefore, it is concluded that, students' using the AEA system contributes significantly to their writing improvement. Thus a linear regression is modeled to predict the weight of the AEA system use frequency to proficiency improvement in writing. Finally, students' perceptions based on their learning journals and interviews show the changes in the process of writing.

Currently, it has become a fundamental goal for the engineering major to cultivate high-quality engineering technicians with innovation ability in scientific research which is an important academic ability necessary for them. This paper mainly explores the development of comprehensive and designing experiments in automation based on scientific research projects and elaborates on the significance and indispensable role of these experiments in cultivating students' research innovation ability.

Automated writing evaluation (AWE) systems can provide immediate computer-generated quantitative assessments and qualitative diagnostic feedback on an enormous number of submitted essays. However, limited research attention has been paid to locally designed AWE systems used in English as a foreign language (EFL) classroom contexts. This study explored the precision of feedback provided by a Chinese AWE programme "Pigai" and students' uptake of such feedback. Data comprising "Pigai" feedback, students' revised submissions, incorporated "Pigai" suggestions and proportions of "good", "neutral" and "bad" revisions were collected from 210 drafts by 30 undergraduate students in an EFL writing class. Results show that (1) the precision rates of AWE feedback differed across categories of feedback (i.e. grammar, collocations and mechanics); (2) students were selective in their utilisation of AWE feedback and able to adjust their uptake of AWE suggestions according to the accuracy of the feedback. Survey and interview data demonstrated that the students were cognisant of the strengths and limitations of the AWE system as a source of feedback for improving their writing. These results indicate that AWE feedback can usefully supplement peer and instructor feedback in the EFL writing classroom but cannot replace the latter, given the low precision and accuracy rates of the former in identifying a range of errors.

Writing an essay is a very important skill for students to master, but a difficult task for them to overcome. It is particularly true for English as Second Language (ESL) students in China. It would be very useful if students could receive timely and effective feedback about their writing. Automatic essay feedback generation is a challenging task, which requires understanding the relationship between the text features of the essay and feedback. In this study, we first analyzed 1,290 teacher comments on their 327 English-major students and annotated the feedback on seven aspects of writing, including the "grammar, spelling, sentence diversity, structure, organization, supporting ideas, coherence, and conclusion", for each paper. Then, an automatic feedback classification experiment was conducted with the machine learning approach. Finally, we investigated the impact of the system generated-indirect corrective feedback (ICF) and human teachers' direct corrective feedback (DCF) in two English writing classes (in ICF class; in DCF class) at a key Chinese university through a web-based assignment management system. The study results indicated the feasibility of this approach that system generated ICF can be as useful as direct comments made by the teachers in terms of improving the quality of the content regarding to the "structure, organization, supporting ideas, coherence, and conclusion", and encouraging students to spend more time on self-correction.

In this special issue on high-stakes English language testing in China, the two articles on computer-based testing (Jin & Yan; He & Min) highlight a number of consistent, ongoing challenges and concerns in the development and implementation of the nationwide IB-CET (Internet Based College English Test) and institutional computer-adaptive English tests, respectively: conceptualizing the construct of computer-based language testing, ensuring fairness for test takers with differing levels of computer literacy, and achieving comparability between tests or tasks of different delivery modes. In this article, we provide an overview of the research studies on computer-based English language testing conducted by Chinese scholars and published in major Chinese academic journals in recent decades, aiming to identify the research topics, gaps, and agendas that could have implications beyond Chinese contexts in promoting better use of computer technologies "in" and "for" English language testing.

This article presents a novel method, the Complex Dynamics Essay Scorer (CDES), for automated essay scoring using complex network features. Texts produced by college students in China were represented as scale-free networks (e.g., a word adjacency model) from which typical network features, such as the in-/out-degrees, clustering coefficient (CC), and dynamic networks, were obtained. The CDES integrates the classical concepts of network feature representation and essay score series variation. Several experiments indicated that the network measures different essay qualities and can be clearly demonstrated to develop complex networks for autoscoring tasks. The average agreement of the CDES and human rater scores was 86.5%, and the average Pearson correlation was .77. The results indicate that the CDES produced functional complex systems and autoscored Chinese essays in a method consistent with human raters. Our research suggests potential applications in other areas of educational assessment.

This study investigates the consistency between human raters and an automated essay scoring system in grading high school students' English compositions. A total of 923 essays from 23 classes of 12 senior high schools in Taiwan (Republic of China) were obtained and scored manually and electronically. The results show that the consistency between human raters is significantly higher than the consistency between human raters and the automated essay scoring system. To discover whether the students were confident with the automated essay grading system, a questionnaire was also distributed. The results indicate the participants realize their vocabulary level was inadequate, and they wanted to know the scores the automated essay grading system gave them along with the generated comments regarding their compositions. (Contains 6 tables.)

Describes the rapid progress of computer technologies in China. Details the increased role of computers in business. Presents the problem of Chinese characters in computer usage especially in business. Enumerates computer usage in China at this time. Cites the quantity and quality of computer software as a major limitation. (CW)

OBJECTIVE: We reviewed recent reports of recalled children's products contaminated with lead to learn more about what could be done to inform public health partners about the recalls in a timely manner to assist with broader consumer notification for proactive lead poisoning prevention efforts. METHODS: Centers for Disease Control and Prevention staff reviewed lead-related recall notices for children's products issued during June 2022 through April 2024. Recall notices were extracted from the Consumer Product Safety Commission using Really Simple Syndication feed technology to identify and capture the most current recall information. Specific product details in the database were extracted and analyzed descriptively to identify response trends and best practices. RESULTS: Centers for Disease Control and Prevention's automated tool identified 30 recalls from Consumer Product Safety Commission. Lead-contaminated toys were the most frequent type of product recalled (n = 11, 37%). Most products were manufactured in China (n = 24, 86%). Products were on the market for an average of 25 months before they were recalled. No injuries were reported. The 30 recalls resulted in a combined number of 914 598 recalled units sold. CONCLUSIONS: The current approach to protecting children from lead hazards in consumer products could be augmented by timely notifying the public health community about recalls so they can broadly disseminate information through their channels to reduce lead exposure in children. Additional steps to reduce lead contamination in children'sproductswhen sourcing raw materials and components may help to decrease the number of recalls. [ABSTRACT FROM AUTHOR]

Purpose: Studying perivascular spaces (PVSs) is important for understanding the pathogenesis and pathological changes of neurological disorders. Although some methods for automated segmentation of PVSs have been proposed, most of them were based on 7T MR images that were majorly acquired in healthy young people. Notably, 7T MR imaging is rarely used in clinical practice. Herein, we propose a deep-learning-based method that enables automatic segmentation of PVSs on T2-weighted 3T MR images. Method: Twenty patients with Parkinson’s disease (age range, 42–79 years) participated in this study. Specifically, we introduced a multi-scale supervised dense nested attention network designed to segment the PVSs. This model fosters progressive interactions between high-level and low-level features. Simultaneously, it utilizes multi-scale foreground content for deep supervision, aiding in refining segmentation results at various levels. Result: Our method achieved the best segmentation results compared with the four other deep-learning-based methods, achieving a dice similarity coefficient (DSC) of 0.702. The results of the visual count of the PVSs in our model correlated extremely well with the expert scoring results on the T2-weighted images (basal ganglia: rs = 0.845, P < 0.001; rs = 0.868, P < 0.001; centrum semiovale: rs = 0.845, P < 0.001; rs = 0.823, P < 0.001 for raters 1 and 2, respectively). Experimental results show that the proposed method performs well in the segmentation of PVSs. Conclusion: The proposed method can accurately segment PVSs; it will facilitate practical clinical applications and is expected to replace the method of visual counting directly on T1-weighted images or T2-weighted images. [ABSTRACT FROM AUTHOR]

Background: Ameloblastoma, a common benign tumor found in the jaw bone, necessitates accurate localization and segmentation for effective diagnosis and treatment. However, the traditional manual segmentation method is plagued with inefficiencies and drawbacks. Hence, the implementation of an AI-based automatic segmentation approach is crucial to enhance clinical diagnosis and treatment procedures. Methods: We collected CT images from 79 patients diagnosed with ameloblastoma and employed a deep learning neural network model for training and testing purposes. Specifically, we utilized the Mask R-CNN neural network structure and implemented image preprocessing and enhancement techniques. During the testing phase, cross-validation methods were employed for evaluation, and the experimental results were verified using an external validation set. Finally, we obtained an additional dataset comprising 200 CT images of ameloblastoma from a different dental center to evaluate the model's generalization performance. Results: During extensive testing and evaluation, our model successfully demonstrated the capability to automatically segment ameloblastoma. The DICE index achieved an impressive value of 0.874. Moreover, when the IoU threshold ranged from 0.5 to 0.95, the model's AP was 0.741. For a specific IoU threshold of 0.5, the model achieved an AP of 0.914, and for another IoU threshold of 0.75, the AP was 0.826. Our validation using external data confirms the model's strong generalization performance. Conclusion: In this study, we successfully applied a neural network model based on deep learning that effectively performs automatic segmentation of ameloblastoma. The proposed method offers notable advantages in terms of efficiency, accuracy, and speed, rendering it a promising tool for clinical diagnosis and treatment. [ABSTRACT FROM AUTHOR]

Background: Many studies have investigated the transfer of skills between laparoscopic and robot-assisted surgery (RAS). These studies have considered time, error, and clinical outcomes in the assessment of skill transfer. However, little is known about the specific operations of the surgeon. Clutch control use is an important skill in RAS. Therefore, the present study aimed to propose a novel objective algorithm based on computer vision that can automatically evaluate a surgeon's clutch use. Additionally, the study aimed to evaluate the correlation between clutch metrics and surgical skill on different surgical robot platforms. Methods: The robotic surgery training center of Wuhan University trained 30 laparoscopic surgeons as the study group between 2023 and 2024. Laparoscopic surgeons were trained by combining robotic simulator exercises and RAS animal experiments. During the training, video and hand movement data were collected. Hand movements identified by a skin-color model were combined with labeling information to classify clutch use. The metrics were validated on different robotic platforms (dv-Trainer, EDGE MP1000, Toumai™ MT1000, and DaVinci Xi system) and among surgeons with different surgical skill levels. Results: On the robotic simulator, clutch accuracy in the expert group was significantly higher than in the study group for all tasks. No significant differences were observed in the number of clutches between the expert and study groups. In the RAS experiment, the number of clutches decreased significantly for both study and expert groups. The accuracy was maintained at a high level in the expert group but decreased rapidly in the study group. Conclusions: We proposed a new objective assessment of surgical skills, clutch use metrics, in cross-platform RAS. Additionally, we verified that the metrics significantly correlated with the surgical skill levels of the surgeons. [ABSTRACT FROM AUTHOR]

Background: Peer assessment has played an important role in large‐scale online learning, as it helps promote the effectiveness of learners' online learning. However, with the emergence of numerical grades and textual feedback generated by peers, it is necessary to detect the reliability of the large amount of peer assessment data, and then develop an effective automated grading model to analyse the data and predict learners' learning results. Objectives: The present study aimed to propose an automated grading model with reliability detection. Methods: A total of 109,327 instances of peer assessment from a large‐scale teacher online learning program were tested in the experiments. The reliability detection approach included three steps: recurrent convolutional neural networks (RCNN) was used to detect grade consistency, bidirectional encoder representations from transformers (BERT) was used to detect text originality, and long short‐term memory (LSTM) was used to detect grade‐text consistency. Furthermore, the automated grading was designed with the BERT‐RCNN model. Results and Conclusions: The effectiveness of the automated grading model with reliability detection was shown. For reliability detection, RCNN performed best in detecting grade consistency with an accuracy rate of 0.889, BERT performed best in detecting text originality with an improvement of 4.47% compared to the benchmark model, and LSTM performed best with an accuracy rate of 0.883. Moreover, the automated grading model with reliability detection achieved good performance, with an accuracy rate of 0.89. Compared to the absence of reliability detection, it increased by 12.1%. Implications: The results strongly suggest that the automated grading model with reliability detection for large‐scale peer assessment is effective, with the following implications: (1) The introduction of reliability detection is necessary to help filter out low reliability data in peer assessment, thus promoting effective automated grading results. (2) This solution could assist assessors in adjusting the exclusion threshold of peer assessment reliability, providing a controllable automated grading tool to reducing manual workload with high quality. (3) This solution could shift educational institutions from labour‐intensive grading procedures to a more efficient educational assessment pattern, allowing for more investment in supporting instructors and learners to improve the quality of peer feedback. Lay Description: What is already known about this topic: Peer assessment has played an important role in large‐scale online learning, as it helps promote the effectiveness of learners' online learning.Issues such as disagreement between peer assessors, rough assessment, and plagiarism in large‐scale online learning can decrease peer assessment reliabilityIncorporating extensive data into a training model may result in grading uncertainties. What this paper adds: Detecting the peer assessment reliability before grading is essential in the context of large‐scale online learning.This study aimed to propose and validate an automated grading model with reliability detection for the large‐scale online peer assessment, which will help improve the effectiveness of automated grading, combining the advantages of computer technology and human expertise. Implications for practice and/or policy: The introduction of reliability detection is necessary to help filter out low reliability data in peer assessment, thus promoting effective automated grading results.This solution could assist assessors in adjusting the exclusion threshold of peer assessment reliability, providing a controllable automated grading tool to reducing manual workload with high quality. [ABSTRACT FROM AUTHOR]

Objective: Depth camera-based measurement has demonstrated efficacy in automated assessment of upper limb Fugl-Meyer Assessment for paralysis rehabilitation. However, there is a lack of adequately sized studies to provide clinical support. Thus, we developed an automated system utilizing depth camera and machine learning, and assessed its feasibility and validity in a clinical setting. Design: Validation and feasibility study of a measurement instrument based on single cross-sectional data. Setting: Rehabilitation unit in a general hospital Participants: Ninety-five patients with hemiparesis admitted for inpatient rehabilitation unit (2021–2023). Main measures: Scores for each item, excluding those related to reflexes, were computed utilizing machine learning models trained on participant videos and readouts from force test devices, while the remaining reflex scores were derived through regression algorithms. Concurrent criterion validity was evaluated using sensitivity, specificity, percent agreement and Cohen's Kappa coefficient for ordinal scores of individual items, as well as correlations and intraclass correlation coefficients for total scores. Video-based manual assessment was also conducted and compared to the automated tools. Result: The majority of patients completed the assessment without therapist intervention. The automated scoring models demonstrated superior validity compared to video-based manual assessment across most items. The total scores derived from the automated assessment exhibited a high coefficient of 0.960. However, the validity of force test items utilizing force sensing resistors was relatively low. Conclusion: The integration of depth camera technology and machine learning models for automated Fugl-Meyer Assessment demonstrated acceptable validity and feasibility, suggesting its potential as a valuable tool in rehabilitation assessment. [ABSTRACT FROM AUTHOR]

Background: Dietary assessment methods have limitations in capturing real-time eating behaviour accurately. Equipped with automated dietary-data-collection capabilities, the "intelligent ordering system" (IOS) has potential applicability in obtaining long-term consecutive, relatively detailed on-campus dietary records among university students with little resource consumption. We investigated (1) the relative validity of IOS-derived nutrient/food intakes compared to those from the 7-day food diary (7DFD); (2) whether including a supplemental food frequency questionnaire (SFFQ) improves IOS accuracy; and (3) sex differences in IOS dietary intake estimation. Methods: Medical students (n = 221; age = 22.2 ± 2.4 years; 38.5% male and 61.5% female) completed the 7DFD and SFFQ. During the consecutive 7-day survey period, students weighed and photographed each meal before and after consumption. Then, students reviewed their 3-month diet and completed the SFFQ, which includes eight underprovided school-canteen food items (e.g., dairy, fruits, nuts). Meanwhile, 9385 IOS dietary data entries were collected. We used Spearman coefficients and linear regression models to estimate the associations among the different dietary intake assessment methods. Individual- and group-level agreement was assessed using the Wilcoxon signed-rank test, cross-classification, and Bland‒Altman analysis. Results: IOS mean daily energy, protein, fat, and carbohydrate intake estimations were significantly lower (-15-20%) than those of the 7DFD. The correlation coefficients varied from 0.52 (for added sugar) to 0.88 (for soybeans and nuts), with fruits (0.37) and dairy products (0.29) showing weaker correlations. Sixty-two (milk and dairy products) to 97% (soybeans and nuts) of participants were classified into the same or adjacent dietary intake distribution quartile using both methods. The energy and macronutrient intake differences between the IOS + SFFQ and 7DFD groups decreased substantially. The separate fruit intake measurements from each assessment method did not significantly differ from each other (p > 0.05). IOS and IOS + SFFQ regression models generally yielded higher R2 values for males than for females. Conclusion: Despite estimation differences, the IOS can be reliable for medical student dietary habit assessment. The SFFQ is useful for measuring consumption of foods that are typically unavailable in school cafeterias, improving the overall dietary evaluation accuracy. The IOS assessment was more accurate for males than for females. [ABSTRACT FROM AUTHOR]

Fundus fluorescein angiography (FFA) is a crucial diagnostic tool for chorioretinal diseases, but its interpretation requires significant expertise and time. Prior studies have used Artificial Intelligence (AI)-based systems to assist FFA interpretation, but these systems lack user interaction and comprehensive evaluation by ophthalmologists. Here, we used large language models (LLMs) to develop an automated interpretation pipeline for both report generation and medical question-answering (QA) for FFA images. The pipeline comprises two parts: an image-text alignment module (Bootstrapping Language-Image Pre-training) for report generation and an LLM (Llama 2) for interactive QA. The model was developed using 654,343 FFA images with 9392 reports. It was evaluated both automatically, using language-based and classification-based metrics, and manually by three experienced ophthalmologists. The automatic evaluation of the generated reports demonstrated that the system can generate coherent and comprehensible free-text reports, achieving a BERTScore of 0.70 and F1 scores ranging from 0.64 to 0.82 for detecting top-5 retinal conditions. The manual evaluation revealed acceptable accuracy (68.3%, Kappa 0.746) and completeness (62.3%, Kappa 0.739) of the generated reports. The generated free-form answers were evaluated manually, with the majority meeting the ophthalmologists' criteria (error-free: 70.7%, complete: 84.0%, harmless: 93.7%, satisfied: 65.3%, Kappa: 0.762–0.834). This study introduces an innovative framework that combines multi-modal transformers and LLMs, enhancing ophthalmic image interpretation, and facilitating interactive communications during medical consultation. [ABSTRACT FROM AUTHOR]

We study how technology adoption and changes in global value chain (GVC) integration jointly affect labor shares and business function specialization in a sample of 14 manufacturing industries in 14 European countries in 1999–2011. Increases in upstream, forward GVC integration directly reduce labor shares, mostly through reductions in fabrication, but also via other business functions. We do not find any direct effects of robot adoption; robotization affects labor only indirectly, by increasing upstream, forward GVC integration. In this sense robotization is "upstream‐biased". Rapid robotization in China shaped robotization in Europe and, therefore, relative demand for labor there. [ABSTRACT FROM AUTHOR]

This article selects state‐owned enterprises listed on the A‐share market of China from 2010 to 2021 as the research sample, and adopts the Ordered Logistic Model to analyze the relationship between digital transformation (DX) and environment, social and governance (ESG) performance of enterprises, and studies the moderating effect of mixed‐ownership reform. The study found that: the DX of state‐owned enterprises can indeed improve ESG performance, and its time lag effect is significant; for state‐owned enterprises located in high‐tech industries, with higher level of marketization, or in the maturity stage, their DX has a significant promoting effect on ESG performance; In addition, mixed‐ownership reform can amplify the positive impact of DX of state‐owned enterprises on ESG performance. [ABSTRACT FROM AUTHOR]

Vision and aesthetics are inseparable dimensions of national image building. Based on 106,562 China-related images from Twitter (renamed as X), this paper introduced a computational aesthetic approach to investigate the visual communication activities of social bots on Twitter and compared the similarities and differences between human and bot accounts' posted images so as to explore the influence of social bots' aesthetic strategies. The results show that social bots have displayed different aesthetic strategies in the construction of the China-related visual frame, and formed obvious stylistic differences with humans in brightness, saturation, color, etc. Negative binomial regression indicates that the aesthetic strategies of social bots contribute to more likes and shares. The automation of visual communication and aesthetic construction not only makes the global building and communication of national image face new situations and challenges, but also pushes the whole human visual aesthetic, creation, and communication activities under the potential subjectivity crisis. [ABSTRACT FROM AUTHOR]

In safety-critical automatic systems, safety can be compromised if operators lack engagement. Effective detection of undesirable engagement states can inform the design of interventions for enhancing engagement. However, the existing engagement measurement methods suffer from several limitations which damage their effectiveness in the work environment. A novel engagement evaluation methodology, which adopts Artificial Intelligence (AI) technologies, has been proposed. It was developed using motorway control room operators as subjects. Openpose and Open Source Computer Vision Library (OpenCV) were used to estimate the body postures of operators, then a Support Vector Machine (SVM) was utilised to build the engagement evaluation model based on discrete states of operator engagement. The average accuracy of the evaluation results reached 0.89 and the weighted average precision, recall, and F1-score were all above 0.84. This study emphasises the importance of specific data labelling when measuring typical engagement states, forming the basis for potential control room improvements. Practitioner summary: This study demonstrates an automatic, real-time, objective, and relatively unobtrusive method for measuring dynamic operator engagement states. Computer vision technologies were used to estimate body posture, then machine learning (ML) was utilised to build the engagement evaluation model. The overall evaluation shows the effectiveness of this framework. Abbreviations: AI: Artificial Intelligence; OpenCV: Open Source Computer Vision Library; SVM: Support Vector Machine; UWES: Utrecht Work Engagement Scale; ISA Engagement Scale: Intellectual, Social, Affective Engagement Scale; DSSQ: Dundee Stress State Questionnaire; SSSQ: Short Stress State Questionnaire; EEG: electroencephalography; ECG: Electrocardiography; VMOE: Video-based Measurement for Operator Engagement; CMU: Carnegie Mellon University; CNN: Convolutional Neural Network; 2D: two dimensional; ML: Machine learning. [ABSTRACT FROM AUTHOR]

• The process of automation could hinder political trust in government. • The impact results from unemployment, higher down-side risk and lower group participation. • The performance-based legitimacy structure in a developmental state may hinder its political legitimacy when promoting development. • The government should focus more on citizens' trust when pursuing technological development. • Social security is crucial for governance faced with fast technological advancement. The side effects of automation on the economy have been discussed frequently, but little is known regarding its political consequences. This paper examines the causal effect that automation induces on political costs for the local government. By combining the national individual-level panel data of political trust with the prefecture-level robot exposure rate in China, we find that the development of automation would reduce individuals' political trust in the Chinese local government. Furthermore, we explore the channels through which the automation process could affect political trust, namely the risk of unemployment, intensified pessimism about local government, higher downside risk, and declining organization participation. This paper provides empirical evidence for the impact of automation and the source of political legitimacy, emphasizing the crucial role of governments in coping with technological progress and making good use of endogenous creative destruction. [Display omitted] [ABSTRACT FROM AUTHOR]