Search

The communication gap between deaf and non-deaf communities arises due to the use of distinct mother languages. A deaf student, who used to communicate in sign language, cannot read fluently materials written in spoken language. This fact causes serious difficulties to deaf students since most didactic materials in higher education are available exclusively in spoken languages. In this paper, we propose a pedagogical model to deliver educational materials in sign language aiming to provide deaf students the same conditions to succeed as the others, i.e., didactical materials available in one's mother language. Our approach involves the integration of automatic translation technology between spoken and sign language pairs into MOOCs. As a proof of concept, we used this methodology to design a course addressing digital literacy for schoolteachers. The evaluation of the inclusive MOOC and its underlying model reveals its potential, even though further improvements are required, especially regarding evaluation and usability features.

We examined the extent to which automated written expression curriculum-based measurement (aWE-CBM) can be accurately used to computer score student writing samples for screening and progress monitoring. Students (n = 174) with learning difficulties in Grades 1 to 12 who received 1:1 academic tutoring through a community-based organization completed narrative writing samples in the fall and spring across two academic years. The samples were evaluated using four automated and hand-calculated WE-CBM scoring metrics. Results indicated automated and hand-calculated scores were highly correlated at all four timepoints for counts of total words written (rs = 1.00), words spelled correctly (rs = 0.99-1.00), correct word sequences (CWS; rs = 0.96-0.97), and correct minus incorrect word sequences (CIWS; rs = 0.86-0.92). For CWS and CIWS, however, automated scores systematically overestimated hand-calculated scores, with an unacceptable amount of error for CIWS for some types of decisions. These findings provide preliminary evidence that aWE-CBM can be used to efficiently score narrative writing samples, potentially improving the feasibility of implementing multi-tiered systems of support in which the written expression skills of large numbers of students are screened and monitored.

Currently, special attention is paid to the development of digital technologies. In the context of the pandemic, the remote, distance, digital format of work has become top requested. Those events coincided with the general trend towards total digitalization of all spheres of life, including education. New concepts have emerged and taken hold - digital education and digital university. However, a mechanism for the development of a digital university has not yet been proposed. In the present study, an institutional approach was used to analyze the problems of the development of a digital university. The research scheme was used: institutions - motivation - control. The results of the empirical study (questionnaire and survey of university entries and its partners) made it possible to identify the existing contradictions between the participants in university activities and the external environment in terms of assessing aspects of the digital development of the university, to highlight the most effective institutions, conditions of positive motivation and control mechanisms for successful digital transformation of a university. The authors propose their own version of the concept for the development of a digital university, introducing as the core of it a new scientific concept - infrastructure logic. They lay a foundation for the principles of infrastructure logic, formation mechanisms and approaches to the use of indicators of digital development based on infrastructure logic, directions for improving the strategy of digital development of the university 4.0. The conclusions and recommendations formulated in this study can be used by other educational systems and universities.

Agreement statistics and measures of prediction accuracy are often used to assess the quality of two measures of a construct. Agreement statistics are appropriate for measures that are supposed to be interchangeable, whereas prediction accuracy statistics are appropriate for situations where one variable is the target and the other variables are predictors. Using bivariate normality assumptions, we analytically examine the impact of categorization of a continuous variable and mean/sigma scaling on different measures of agreement and different measures of prediction accuracy. We vary the degree of relationship (squared correlation) between two continuous measures of a construct and the degree to which these measures are reduced to fewer and fewer categories (categorization). The main findings include that (a) categorization influences all the statistics investigated, (b) the correlation between the continuous variables affects the values of the statistics, and (c) scaling a prediction of a target variable to have the same mean and variability as the target increases agreement (according to Cohen's kappa and quadratic weighted kappa) but does so at the expense of prediction accuracy. The implications of these results for scoring of essays by humans or machines are also discussed.

Understanding students' perceptions, aptitudes, and interest in using technology, and in developing technological solutions to problems, is important for effectively incorporating it into course pedagogy. This paper presents an effort to understand student technological self-conception, especially regarding the use-of-technology versus the development-of-technology self-perception schema. Concern exists that because students are so comfortable with using technology, they are being lulled into a false sense of security and may be at risk of being replaced by it if they lack the ability to be creators of technological processes and tools themselves. We seek to broaden student conception of themselves to consider becoming creators, thereby beginning the journey towards becoming citizen developers. A survey designed with consideration of the Theory of Planned Behavior is used to understand student predisposition and self-conception pertaining to technology use versus development. Inspiration is also drawn from design thinking, an approach that promotes innovative development as a means of combining technology with the human element to create viable, effective solutions. The intended outcome of the overall project is to initiate the creation of citizen developers, within existing curriculum and course requirements we are bound to honor, thereby serving as a bridge towards a more holistic design thinking in students, sparking a creative process of solution and problem-solving generation and development. This first stage of the research project is focused on developing an understanding of student technology comprehension and their prior experience in not just utilizing technology but also in leveraging it to develop solutions to meaningful problems.

Research on bias in artificial intelligence has grown exponentially in recent years, especially around racial bias. Many modern technologies which impact people's lives have been shown to have significant racial biases, including automatic speech recognition (ASR) systems. Emerging studies have found that widely-used ASR systems function much more poorly on the speech of Black people. Yet, this work is limited because it lacks a deeper consideration of the sociolinguistic literature on African American Language (AAL). In this paper, then, we seek to integrate AAL research into these endeavors to analyze ways in which ASRs might be biased against the linguistic features of AAL and how the use of biased ASRs could prove harmful to speakers of AAL. Specifically, we (1) provide an overview of the ways in which AAL has been discriminated against in the workforce and healthcare in the past, and (2) explore how introducing biased ASRs in these areas could perpetuate or even deepen linguistic discrimination. We conclude with a number of questions for reflection and future work, offering this document as a resource for cross-disciplinary collaboration.

Digital automation is on the rise in a diverse range of industries. The technologies employed here often make use of artificial intelligence (AI) and its common form, machine learning (ML) to augment or replace the work completed by human agents. The recent emergence of a variety of design automation platforms inspired the authors to undertake a review of the research literature on the impact of Automation, AI and ML on visual communication, and its subset practice of graphic design, with a view to understanding the implications for the education of practitioners entering that specific field. This review discovered that there was relatively little research published on the topic but what did exist noted that graphic design as we have known it has an uncertain future. Furthermore, the scant literature argued for a shift in educational and professional focus away from the aesthetic and technical skills required to design visual modes of communication and towards a deeper engagement with the softer, more human skills associated with negotiation, facilitation and judgement. The paucity of literature on this topic suggests to the authors that visual communication design education and the industry are poorly prepared for the impact of automation, AI and ML on them.

STEM fields, such as physics, increasingly rely on complex programs to analyse large datasets, thus teaching students the required programming skills is an important component of all STEM curricula. Since undergraduate students often have no prior coding experience, they are reliant on error messages as the primary diagnostic tool to identify and correct coding mistakes. However, such messages are notoriously cryptic and often undecipherable for novice programmers, presenting a significant learning hurdle that leads to frustration, discouragement, and ultimately a loss of confidence. Addressing this, we developed a tool to enhance error messages for the popular Python language, translating them into plain English to empower students to resolve the underlying error independently. We used a mixed methods approach to study the tool's effect on first-year physics students undertaking an introductory programming course. We find a broadly similar distribution of the most common error types to previous studies in other contexts. Our results show a statistically significant reduction in negative student emotions, such as frustration and anxiety, with the mean self-reported intensity of these emotions reducing by (73 ± 12)% and (55 ± 18)%, respectively. This led to a corresponding decrease in discouragement and an increase in student confidence. We conclude that enhanced error messages provide an effective way to alleviate negative student emotions and promote confidence. However, further longer-term investigations are necessary to confirm if this translates into improved learning outcomes. To our knowledge, this is the first physics-specific investigation of the effect of Python error message enhancement on student learning.

Background: Deep learning methods, where models do not use explicit features and instead rely on implicit features estimated during model training, suffer from an explainability problem. In text classification, saliency maps that reflect the importance of words in prediction are one approach toward explainability. However, little is known about whether the salient words agree with those identified by humans as important. Objectives: The current study examines in-line annotations from human annotators and saliency map annotations from a deep learning model (ELECTRA transformer) to understand how well both humans and machines provide evidence for their assigned label. Methods: Data were responses to test items across a mix of United States subjects, states, and grades. Humans were trained to annotate responses to justify a crisis alert label and two model interpretability methods (LIME, Integrated Gradients) were used to obtain engine annotations. Human inter-annotator agreement and engine agreement with the human annotators were computed and compared. Results and Conclusions: Human annotators agreed with one another at similar rates to those observed in the literature on similar tasks. The annotations derived using the integrated gradients (IG) agreed with human annotators at higher rates than LIME on most metrics; however, both methods underperformed relative to the human annotators. Implications: Saliency map-based engine annotations show promise as a form of explanation, but do not reach human annotation agreement levels. Future work should examine the appropriate unit for annotation (e.g., word, sentence), other gradient based methods, and approaches for mapping the continuous saliency values to Boolean annotations.

Automated Writing Evaluation (AWE) is one of the machine techniques used for assessing learners' writing. Recently, this technique has been widely implemented for improving learners' editing strategies. Several studies have been conducted to compare self-editing with peer editing. However, only a few studies have compared automated peer and self-editing. To fill this research gap, the present study implements AWE software, WRITER, for peer and self-editing. For this purpose, a pre-post quasi-experimental research design with convenience sampling is done for automated and non-automated editing of cause-effect essay writing. Arab, EFL learners of English, 44 in number, have been assigned to four groups: two peer and self-editing control groups and two automated peer and self-editing experimental groups. There is a triangulation of the quasi-experimental design with qualitative data from retrospective notes and questionnaire responses of the participants during and after automated editing. The quantitative data have been analyzed using non-parametric tests. The qualitative data have undergone thematic and content analysis. The results reveal that the AWE software has positively affected both the peer and self-editing experimental groups. However, no significant difference is detected between them. The analysis of the qualitative data reflects participants' positive evaluation of both the software and the automated peer and self-editing experience.

Grading assignments is inherently subjective and time-consuming; automatic scoring tools can greatly reduce teacher workload and shorten the time needed for providing feedback to learners. The purpose of this paper is to propose a novel method for automatically scoring student responses to picture-cued writing tasks. As a popular paradigm for language instruction and assessment, a picture-cued writing task typically requires students to describe a picture or pictures. Correspondingly, the automatic scoring methods must measure the link(s) between visual pictures and their textual descriptions. For this purpose, we first designed a picture-cued writing test and collected nearly 4 k responses from 279 K12 students. Based on these responses, we then developed an AI scoring model by incorporating the emerging cross-modal matching technology and some NLP algorithms. The performance of the model was evaluated carefully with six popular measures and was found to demonstrate accurate scoring results with a small mean absolute error of 0.479 and a high adjacent-agreement rate of 90.64%. We believe this method could reduce the subjective elements inherent in human grading and save teachers' time from the mundane task of grading to other valuable endeavors such as designing teaching plans based on AI-generated diagnosis of student progress.

Hate speech may be the research focus of the interdisciplinary field of hate studies, but it is also a difficult phenomenon to define. Internationally, there are several detection studies on automatically detecting hate speech. They can be grouped according to two approaches: the first includes searching using only machine learning methods, while the second includes studies that combine automatic searching with human classification. The case study on anti-Gypsy hate in Italian on Twitter in the second half of 2020 falls into the second category, and its methods are outlined here. Based on the results (annotation as 'hate'/'non-hate', identification of forms of rhetoric and anti-Gypsyism), the researchers propose classifying online content according to seven indicators called the 'spectrum of online hate'.

This paper aims to illustrate an automated system developed to give formative and personalized feedback to teachers in training. It is an expert system (Paviotti, Rossi & Zarka, 2012) that uses concrete examples, cases and scenarios to guide the engaged learners (Leake, 1996). In this regard, this system is able to create questionnaires, deliver them, collect and analyze data, send feedback to the participants to provide information about their beliefs and behaviors about teaching and learning processes. Far from constituting an assessment of teaching practices, the automated feedback demonstrates its usefulness in identifying teachers' mindframes at an early stage, so as to be able to implement more specific and personalized training. This allows its application to be extended to further training areas as well as constituting an effective approach for need analysis and a preparatory action for numerous training activities (guided discussion with experts, observation on practice, modeling, etc.).

In online courses, discussion forums play a key role in enhancing student interaction with peers and instructors. Due to large enrolment sizes, instructors often struggle to respond to students in a timely manner. To address this problem, both traditional machine learning (ML) (e.g., Random Forest) and deep learning (DL) approaches have been applied to classify educational forum posts (e.g., those that required urgent responses versus those that did not). However, there lacks an in-depth comparison between these two kinds of approaches. To better guide people to select an appropriate model, we aimed at providing a comparative study on the effectiveness of six frequently-used traditional ML and DL models across a total of seven different classification tasks centering around two datasets of educational forum posts. Through extensive evaluation, we showed that (1) the up-to-date DL approaches did not necessarily outperform traditional ML approaches; (2) the performance gap between the two kinds of approaches can be up to 3.68% (measured in F1 score); (3) the traditional ML approaches should be equipped with carefully-designed features, especially those of common importance across different classification tasks. Based on the derived findings, we further provided insights to help instructors and educators construct effective classifiers for characterizing educational forum discussions, which, ultimately, would enable them to provide students with timely and personalized learning support.

Peer grading has diverse applications in many fields, including the peer grading of open assignments in online courses. The major challenge in peer grading is improving the seriousness (reviewing carefully) of reviewers. Previous studies have proposed several incentive reward mechanisms intended to reward or punish reviewers. Although these mechanisms are effective in massive open online courses environments with a large number of students, they are not suitable for small private online courses (SPOCs) environments with a small number of students. This article analyzes the characteristics of reviewers in a small private online course environment and designs an incentive and reward mechanism based on a hybrid human-machine peer grading framework, including an autograder (reviewed by the machine). The main component of the proposed mechanism is the autograder whose function is to evaluate the reviewers' reports (reviewers refer to students) and provide feedback information on reports to reviewers. The framework can incentivize reviewers to review carefully and give their reports truthfully when the reports between reviewers' and the autograder' are consistent. The proposed framework is verified experimentally. The experimental results demonstrate that the hybrid human-machine peer grading framework can effectively incentivize reviewers to review submissions carefully and improve the overall effect of peer grading.

Globalizations across organizations are impacted by economic, political, legal, security, social, cultural, ecological, and technological dimensions among others. This paper presents the readings from relevant articles and studies pertaining to the relationship between technology and its dimensions with globalization. Globalization and technological advancement are indeed interrelated where the success or failure of one is associated with the other. With this, Technology Education and Globalization, as intertwined disciplines, must be inculcated across curriculums and program offerings to address the demand of the changing and challenging times. Besides, research on technology and globalization are encouraged among educators.

Digitization across the healthcare industry has witnessed the advent of emerging Cognitive Computing (CC) healthcare technologies that improve diagnostic accuracy and efficiency, predict illnesses, automate routine healthcare tasks, and refine processes and care beyond human capabilities. Increased adoption of this technology can be attributed to its ability of processing enormous amounts of data promptly in addressing specific queries and producing customized intelligent recommendations. While CC's transformative technologies offer profound benefits to the healthcare industry, it also carries an unpredictable burden of risk and mistakes with damaging consequences to patients. At this juncture, CC's legal place in healthcare is largely undefined as the applicable liability framework is ambiguous. CC fits into the traditional liability rules in a piecemeal manner; however a single theory of recovery sufficiently addressing the potential liability questions arising from a computer system capable of practicing medicine and possessing the ability of parsing through enormous data for better patient outcomes is absent. The present research therefore sets out to chart the analysis of cases involving emerging medical technologies comparable to CC, in hope of examining ways in which the traditional theories of liability is projected to develop in adapting to this novel contrivance. A doctrinal and case study methods formed an integrated qualitative approach adopted by this research in opting the deployment of emerging medical technologies akin to CC and the bearing it has on the imposition of liability in the United States. CC's potential contributions to healthcare are revolutionary, however its legal repercussions are just as alarming and therefore demands for more discussion in addressing the concerns.

Recent advances in automated writing evaluation have enabled educators to use automated writing quality scores to improve assessment feasibility. However, there has been limited investigation of bias for automated writing quality scores with students from diverse racial or ethnic backgrounds. The use of biased scores could contribute to implementing unfair practices with negative consequences on student learning. The goal of this study was to investigate score bias of writeAlizer, a free and open-source automated writing evaluation program. For 421 students in Grades 4 and 7 who completed a state writing exam that included composition and multiple choice revising and editing questions, writeAlizer was used to generate automated writing quality scores for the composition section. Then, we used multiple regression models to investigate whether writeAlizer scores demonstrated differential predictions of the composition and overall scores on the state-mandated writing exam for students from different racial or ethnic groups. No evidence of bias for automated scores was observed. However, after controlling for automated scores in Grade 4, we found statistically significant group differences in regression models predicting overall state test scores 3 years later but not the essay composition scores. We hypothesize that the multiple choice revising and editing sections, rather than the scoring approach used for the essay portion, introduced construct-irrelevant variance and might lead to differential performance among groups. Implications for assessment development and score use are discussed. [For the corresponding grantee submission, see ED628830.]

Despite the popularity of automated writing evaluation (AWE) that has provoked an increased scholarly interest, synthesized research to comprehensively understand its pedagogical effects is still in paucity. To fill the gap, this study aims to meta-analyse the overall effect of AWE on learners' writing skill development and whether the effect differs from how it has been used. From activity theory (AT) perspective, this study reported results based on a meta-analysis of 33 valid effect sizes involving 2995 participants from 25 experimental and quasi-experimental studies published during 2000-22. The results of the study showed that the overall effect was g = 0.429, 95% CI [0.266, 0.592], suggesting that AWE for writing skill development is more effective than traditional methods, such as traditional classroom interaction, teacher-led interaction and peer interaction. Regarding AT-related moderators for the overall effect, different moderator effects of educational levels, proficiency levels, sample size types, target language types, intervention durations, text genre types, human-computer interaction types, feedback types and program types were reported. Informed by the results obtained, this study also provided some insights into the pedagogical implications for practice.

Contribution: This research designs an experimental education project of an automatic material magnetism properties measurement system. It explores how the do-it-yourself (DIY), hands-on establishment, and hardware-software integration experiment system could be leveraged to enhance the understanding of the electromagnetism theory and nonelectrical quantity measurement application. Background: Designing and establishing a prototype of an automatic measurement system are essential, inspiring and beneficial to comprehending and consolidating the theoretical principles and simulations in electromagnetism and nonelectrical quantity measurement technology courses. Furthermore, in the vision of batch deployment and flexible group teaching, a cost-efficiency, hard/soft integration system for the electromagnetism property measurement definitely fits the aim of innovation laboratory education. Intended Outcomes: This project is intended to guide the students to finish a comprehensive research-oriented project loop for multiple skills training. It is also intended to encourage student motivation and confidence through theory investigation, hands-on group collaboration, and tunning for performance calibration. Application Design: The design of the proposed project includes the literature review, hardware selection and assembly, system integration, software programming, interface visualization, and system performance calibration. The participants are assessed with both objective exam questions and subjective open survey for training objectives to reveal their increased knowledge, interests, skill confidence, and creative thinking. Findings: The proposed project has been used in 4-year laboratory open experiments with students at different levels from freshmen undergraduates to 1st-year postgraduates. A use case and survey assessment demonstrate tremendous knowledge, skills, and confidence improvement among students in electromagnetism and measurement technology courses.

Automatic grading of short answers is an important task in computer-assisted assessment (CAA). Recently, embeddings, as semantic-rich textual representations, have been increasingly used to represent short answers and predict the grade. Despite the recent trend of applying embeddings in automatic short answer grading (ASAG), there are no systematic reviews of literature reporting on their usage. Therefore, following the PRISMA-ScR guidelines, this scoping review summarizes relevant literature on the use of embeddings in ASAG, and reports on the current state of the art in that research area and on the identified knowledge gaps. We searched seven research databases for the relevant journal, conference, and workshop papers published from 2016 to 2021. The inclusion criteria were based on the type of publication, its venue ranking, study focus, and evaluation methods. Upon the full-text screening, 17 articles were included in the scoping review. Among these, most of the articles used word embeddings, mainly to estimate the similarity of student and model answers using the cosine similarity measure or to initialize a neural network-based classification model. The contribution of embeddings to the performance of ASAG models compared to nonembedding features is inconclusive. Models employing embeddings were mostly evaluated on four public ASAG datasets using earlier ASAG methods as baselines. We summarize the reported evaluation results and draw conclusions on the performance of the state-of-the-art ASAG models.

With the increased popularity of electronic textbooks, there is a growing interest in developing a new generation of "intelligent textbooks," which have the ability to guide readers according to their learning goals and current knowledge. Intelligent textbooks extend regular textbooks by integrating machine-manipulable knowledge, and the most popular type of integrated knowledge is a list of relevant concepts mentioned in the textbooks. With these concepts, multiple intelligent operations, such as content linking, content recommendation, or student modeling, can be performed. However, existing automatic keyphrase extraction methods, even supervised ones, cannot deliver sufficient accuracy to be practically useful in this task. Manual annotation by experts has been demonstrated to be a preferred approach for producing high-quality labeled data for training supervised models. However, most researchers in the education domain still consider the concept annotation process as an ad-hoc activity rather than a carefully executed task, which can result in low-quality annotated data. Using the annotation of concepts for the "Introduction to Information Retrieval" textbook as a case study, this paper presents a knowledge engineering method to obtain reliable concept annotations. As demonstrated by the data we collected, the inter-annotator agreement gradually increased along with our procedure, and the concept annotations we produced led to better results in document linking and student modeling tasks. The contributions of our work include a validated knowledge engineering procedure, a codebook for technical concept annotation, and a set of concept annotations for the target textbook, which could be used as a gold standard in further intelligent textbook research.

This study aimed to develop an automated computer scoring system (ACSS) incorporating a Korean spell checker to assess students' constructed responses and to check the efficacy of this system. To accomplish this, we examined the performance of automatic spelling correction in reporting and correcting spelling errors, the interaction of gender and grade level in making spelling errors, the relationship between spelling errors and academic achievement, and the scoring efficacy of an ACSS that incorporated a spell checker. The analysis of percentage, two-way ANOVA, t-test, Pearson's correlation, and human-computer correspondence were conducted. The results revealed that an automatic spelling correction system could report 66.44% and correct 26.78% of all total misspelled words. We also found gender and grade-level differences in misspelling words. Students misspelled fewer words as they advanced in grade level, and male students misspelled more words than females. In terms of the relationship between spelling errors and concepts, we found that the number of concepts included in student's responses had a significant relationship with the total number of written words and misspelled words. This indicates that students who made more spelling errors had discussed more concepts in their responses. Based on these results, we discuss practical implications for preventing students' responses being scored lower due to spelling errors caused by being less attentive using an ACSS with a spelling correction system.

Traditionally, researchers employ human raters for scoring responses to creative thinking tasks. Apart from the associated costs this approach entails two potential risks. First, human raters can be subjective in their scoring behavior (inter-rater-variance). Second, individual raters are prone to inconsistent scoring patterns (intra-rater-variance). In light of these issues, we present an approach for automated scoring of Divergent Thinking (DT) Tasks. We implemented a pipeline aiming to generate accurate rating predictions for DT responses using text mining and machine learning methods. Based on two existing data sets from two different laboratories, we constructed several prediction models incorporating features representing meta information of the response or features engineered from the response's word embeddings that were obtained using pre-trained GloVe and Word2Vec word vector spaces. Out of these features, word embeddings and features derived from them proved to be particularly effective. Overall, longer responses tended to achieve higher ratings as well as responses that were semantically distant from the stimulus object. In our comparison of three state-of-the-art machine learning algorithms, Random Forest and XGBoost tended to slightly outperform the Support Vector Regression.

Learning engagement has gained increasing attention in the field of education. Previous studies have adopted conventional methods to analyze learning engagement, but these methods cannot provide timely feedback for learners. This study analyzed automated group learning engagement via deep neural network models in a computer-supported collaborative learning (CSCL) context. A quasi-experimental research design was implemented to examine the effects of the automated group learning engagement analysis and feedback approach on collaborative knowledge building, group performance, socially shared regulation, and cognitive load. In total, 120 college students participated in this study; they were assigned to 20 experimental groups and 20 control groups of three students each. The students in the experimental groups adopted the automated group learning engagement analysis and feedback approach, whereas those in the control groups used the traditional online collaborative learning approach. Both quantitative and qualitative data were collected and analyzed in depth. The results indicated significant differences in group learning engagement, group performance, collaborative knowledge building, and socially shared regulation between the experimental and control groups. The proposed approach did not increase the cognitive load for the experimental groups. The implications of the findings can potentially contribute to improving group learning engagement and group performance in CSCL.

We aimed to evaluate the performance of supervised machine learning algorithms in predicting articles relevant for full-text review in a systematic review. Overall, 16,430 manually screened titles/abstracts, including 861 references identified relevant for full-text review were used for the analysis. Of these, 40% (n = 6573) were sub-divided for training (70%) and testing (30%) the algorithms. The remaining 60% (n = 9857) were used as a validation set. We evaluated down- and up-sampling methods and compared unigram, bigram, and singular value decomposition (SVD) approaches. For each approach, Naïve Bayes, Support Vector Machines (SVM), regularized logistic regressions, neural networks, random forest, Logit boost, and XGBoost were implemented using simple term frequency or Tf-Idf feature representations. Performance was evaluated using sensitivity, specificity, precision and area under the Curve. We combined predictions of the best-performing algorithms (Youden Index [greater than or equal to]0.3 with sensitivity/specificity [greater than or equal to]70/60%). In a down-sample unigram approach, Naïve Bayes, SVM/quanteda text models with Tf-Idf, and linear SVM e1071 package with Tf-Idf achieved >90% sensitivity at specificity >65%. Combining the predictions of the 10 best-performing algorithms improved the performance to reach 95% sensitivity and 64% specificity in the validation set. Crude screening burden was reduced by 61% (5979) (adjusted: 80.3%) with 5% (27) false negativity rate. All the other approaches yielded relatively poorer performances. The down-sampling unigram approach achieved good performance in our data. Combining the predictions of algorithms improved sensitivity while screening burden was reduced by almost two-third. Implementing machine learning approaches in title/abstract screening should be investigated further toward refining these tools and automating their implementation.

Systematic reviews are vital instruments for researchers to understand broad trends in a field and synthesize evidence on the effectiveness of interventions in addressing specific issues. The quality of a systematic review depends critically on having comprehensively surveyed all relevant literature on the review topic. In addition to database searching, handsearching is an important supplementary technique that helps increase the likelihood of identifying all relevant studies in a literature search. Traditional handsearching requires reviewers to manually browse through a curated list of field-specific journals and conference proceedings to find articles relevant to the review topic. This manual process is not only time-consuming, laborious, costly, and error-prone due to human fatigue, but it also lacks replicability due to its cumbersome manual nature. To address these issues, this paper presents a free and open-source Python package and an accompanying web-app, "Paperfetcher," to automate the retrieval of article metadata for handsearching. With "Paperfetcher's" assistance, researchers can retrieve article metadata from designated journals within a specified time frame in just a few clicks. In addition to handsearching, it also incorporates a beta version of citation searching in both forward and backward directions. "Paperfetcher" has an easy-to-use interface, which allows researchers to download the metadata of retrieved studies as a list of DOIs or as an RIS file to facilitate seamless import into systematic review screening software. To the best of our knowledge, "Paperfetcher" is the first tool to automate handsearching with high usability and a multi-disciplinary focus.

Machine learning algorithms that automatically score scientific explanations can be used to measure students' conceptual understanding, identify gaps in their reasoning, and provide them with timely and individualized feedback. This paper presents the results of a study that uses Hebrew NLP to automatically score student explanations in Biology according to fine-grained analytic grading rubrics that were developed for formative assessment. The experimental results show that our algorithms achieve a high-level of agreement with human experts, on par with previous work on automated assessment of scientific explanations in English, and that [approximately]500 examples are typically enough to build reliable scoring models. The main contribution is twofold. First, we present a conceptual framework for constructing analytic grading rubrics for scientific explanations, which are composed of dichotomous categories that generalize across items. These categories are designed to support automated guidance, but can also be used to provide a composite score. Second, we apply this approach in a new context -- Hebrew, which belongs to a group of languages known as Morphologically-Rich. In languages of this group, among them also Arabic and Turkish, each input token may consist of multiple lexical and functional units, making them particularly challenging for NLP. This is the first study on automatic assessment of scientific explanations (and more generally, of open-ended questions) in Hebrew, and among the firsts to do so in Morphologically-Rich Languages.

English proficiency assessments have become a necessary metric for filtering and selecting prospective candidates for both academia and industry. With the rise in demand for such assessments, it has become increasingly necessary to have the automated human-interpretable results to prevent inconsistencies and ensure meaningful feedback to the second language learners. Feature-based classical approaches have been more interpretable in understanding what the scoring model learns. Therefore, in this work, we utilize classical machine learning models to formulate a speech scoring task as both a classification and a regression problem, followed by a thorough study to interpret and study the relation between the linguistic cues and the English proficiency level of the speaker. First, we extract linguist features under five categories (fluency, pronunciation, content, grammar and vocabulary, and acoustic) and train models to grade responses. In comparison, we find that the regression-based models perform equivalent to or better than the classification approach. Second, we perform ablation studies to understand the impact of each of the feature and feature categories on the performance of proficiency grading. Further, to understand individual feature contributions, we present the importance of top features on the best performing algorithm for the grading task. Third, we make use of Partial Dependence Plots and Shapley values to explore feature importance and conclude that the best performing trained model learns the underlying rubrics used for grading the dataset used in this study.

As intelligent systems demand for human-automation interaction increases, the need for learners' cognitive traits adaptation in adaptive educational hypermedia systems (AEHS) has dramatically increased. AEHS utilize learners' cognitive processes to attain fair human-automation interaction for their adaptive processes. However, obtaining accurate cognitive trait for the AEHS adaptation process has been a challenge due to the fact that it is difficult to determine what extent such traits can comprehend system functionalities. Hence, this study has explored correlation among learners' pupil size dilation, learners' reading time and endogenous blinking rate when using AEHS so as to enable cognitive load estimation in support of AEHS adaptive process. An eye-tracking sensor was used and the study found correlation among learners' pupil size dilation, reading time and learners' endogenous blinking rate. Thus, the results show that endogenous blinking rate, pupil size and reading time are not only AEHS reliable parameters for cognitive load measurement but can also support human-automation interaction at large.

Over its short disciplinary history, computing has seen a stunning number of descriptions of the field's characteristic ways of thinking and practicing, under a large number of different labels. One of the more recent variants, notably in the context of K-12 education, is "computational thinking", which became popular in the early 2000s, and which has given rise to many competing views of the essential character of CT. This article analyzes CT from the perspective of computing's disciplinary ways of thinking and practicing, as expressed in writings of computing's pioneers. The article describes six windows into CT from a computing perspective: its intellectual origins and justification, its aims, and the central concepts, techniques, and ways of thinking in CT that arise from those different origins. The article also presents a way of analyzing CT over different dimensions, such as in terms of breadth vs. depth, specialization vs. generalization, and in terms of skill progression from beginner to expert. Those different views have different aims, theoretical references, conceptual frameworks, and origin stories, and they justify their intellectual essence in different ways. [Note: The issue number (1) shown on the PDF is incorrect. The correct citation for this article is v20 n3.]

Automated Writing Evaluation (AWE) has been considered a potential pedagogical technique that exploits technology to assist the students' writing. However, little attention has been devoted to examining students' perceptions of Grammarly use in higher education context. This paper aims to obtain information regarding the writing process and the merits and drawbacks of Grammarly use among Indonesian undergraduate EFL students. A hundred (n=100) students majoring in English education from a public university in Banten Province were involved in this research. They were divided into three groups of users, i.e., first-, second-, and third-year student cohorts, to test whether the frequency of using Grammarly can affect their perceptions of Grammarly use in a writing class. A questionnaire and an interview guide were used to obtain the data. While the questionnaire results were analyzed using SPSS version 20, the interview results were coded and categorized based on common themes. The findings showed that there is no difference among student cohorts in their perceiving that the use of Grammarly was considered necessary to compose and revise their writing because they still dealt with several writing constraints. They thought they got immediate and comprehensive feedback, notifications of errors, and suggestions to revise the errors. Furthermore, the frequency of using Grammarly, as viewed from the student cohorts, may affect their perceptions of usefulness and drawbacks of Grammarly use, especially whether Grammarly's feedback is always helpful or not. Examining the opportunities of using Grammarly in sparking a constructive learning atmosphere in EFL writing class is worth-researching further.