189 results on '"Automated grading"'
Search Results
2. An Automated Hybrid Exam Evaluation Framework for Textual Courses Using AI
- Author
-
Alpdemir, M. Nedim, Alpdemir, Yusuf, Doğan, Sakup, Xhafa, Fatos, Series Editor, Schlippe, Tim, editor, Cheng, Eric C. K., editor, and Wang, Tianchong, editor
- Published
- 2025
- Full Text
- View/download PDF
Catalog
3. Automated magnetic resonance imaging‐based grading of the lumbar intervertebral disc and facet joints.
- Author
-
Nikpasand, Maryam, Middendorf, Jill M., Ella, Vincent A., Jones, Kristen E., Ladd, Bryan, Takahashi, Takashi, Barocas, Victor H., and Ellingson, Arin M.
- Subjects
ZYGAPOPHYSEAL joint ,CONVOLUTIONAL neural networks ,JOINTS (Anatomy) ,PEARSON correlation (Statistics) ,INTERVERTEBRAL disk - Abstract
Background: Degeneration of both intervertebral discs (IVDs) and facet joints in the lumbar spine has been associated with low back pain, but whether and how IVD/joint degeneration contributes to pain remains an open question. Joint degeneration can be identified by pairing T1 and T2 magnetic resonance imaging (MRI) with analysis techniques such as Pfirrmann grades (IVD degeneration) and Fujiwara scores (facet degeneration). However, these grades are subjective, prompting the need to develop an automated technique to enhance inter‐rater reliability. This study introduces an automated convolutional neural network (CNN) technique trained on clinical MRI images of IVD and facet joints obtained from public‐access Lumbar Spine MRI Dataset. The primary goal of the automated system is to classify health of lumbar discs and facet joints according to Pfirrmann and Fujiwara grading systems and to enhance inter‐rater reliability associated with these grading systems. Methods: Performance of the CNN on both the Pfirrmann and Fujiwara scales was measured by comparing the percent agreement, Pearson's correlation and Fleiss kappa value for results from the classifier to the grades assigned by an expert grader. Results: The CNN demonstrates comparable performance to human graders for both Pfirrmann and Fujiwara grading systems, but with larger errors in Fujiwara grading. The CNN improves the reliability of the Pfirrmann system, aligning with previous findings for IVD assessment. Conclusion: The study highlights the potential of using deep learning in classifying the IVD and facet joint health, and due to the high variability in the Fujiwara scoring system, highlights the need for improved imaging and scoring techniques to evaluate facet joint health. All codes required to use the automatic grading routines described herein are available in the Data Repository for University of Minnesota (DRUM). [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
- Full Text
- View/download PDF
4. Exploring an effective automated grading model with reliability detection for large‐scale online peer assessment.
- Author
-
Lin, Zirou, Yan, Hanbing, and Zhao, Li
- Subjects
- *
HIGH schools , *RESEARCH funding , *AFFINITY groups , *EDUCATIONAL outcomes , *HIGH school students , *EDUCATIONAL tests & measurements , *DESCRIPTIVE statistics , *TEACHERS , *MIDDLE school students , *DEEP learning , *ONLINE education , *ARTIFICIAL neural networks , *WEB development , *AUTOMATION , *COMPUTER assisted instruction , *SHORT-term memory , *MIDDLE schools , *COMPUTER assisted testing (Education) - Abstract
Background: Peer assessment has played an important role in large‐scale online learning, as it helps promote the effectiveness of learners' online learning. However, with the emergence of numerical grades and textual feedback generated by peers, it is necessary to detect the reliability of the large amount of peer assessment data, and then develop an effective automated grading model to analyse the data and predict learners' learning results. Objectives: The present study aimed to propose an automated grading model with reliability detection. Methods: A total of 109,327 instances of peer assessment from a large‐scale teacher online learning program were tested in the experiments. The reliability detection approach included three steps: recurrent convolutional neural networks (RCNN) was used to detect grade consistency, bidirectional encoder representations from transformers (BERT) was used to detect text originality, and long short‐term memory (LSTM) was used to detect grade‐text consistency. Furthermore, the automated grading was designed with the BERT‐RCNN model. Results and Conclusions: The effectiveness of the automated grading model with reliability detection was shown. For reliability detection, RCNN performed best in detecting grade consistency with an accuracy rate of 0.889, BERT performed best in detecting text originality with an improvement of 4.47% compared to the benchmark model, and LSTM performed best with an accuracy rate of 0.883. Moreover, the automated grading model with reliability detection achieved good performance, with an accuracy rate of 0.89. Compared to the absence of reliability detection, it increased by 12.1%. Implications: The results strongly suggest that the automated grading model with reliability detection for large‐scale peer assessment is effective, with the following implications: (1) The introduction of reliability detection is necessary to help filter out low reliability data in peer assessment, thus promoting effective automated grading results. (2) This solution could assist assessors in adjusting the exclusion threshold of peer assessment reliability, providing a controllable automated grading tool to reducing manual workload with high quality. (3) This solution could shift educational institutions from labour‐intensive grading procedures to a more efficient educational assessment pattern, allowing for more investment in supporting instructors and learners to improve the quality of peer feedback. Lay Description: What is already known about this topic: Peer assessment has played an important role in large‐scale online learning, as it helps promote the effectiveness of learners' online learning.Issues such as disagreement between peer assessors, rough assessment, and plagiarism in large‐scale online learning can decrease peer assessment reliabilityIncorporating extensive data into a training model may result in grading uncertainties. What this paper adds: Detecting the peer assessment reliability before grading is essential in the context of large‐scale online learning.This study aimed to propose and validate an automated grading model with reliability detection for the large‐scale online peer assessment, which will help improve the effectiveness of automated grading, combining the advantages of computer technology and human expertise. Implications for practice and/or policy: The introduction of reliability detection is necessary to help filter out low reliability data in peer assessment, thus promoting effective automated grading results.This solution could assist assessors in adjusting the exclusion threshold of peer assessment reliability, providing a controllable automated grading tool to reducing manual workload with high quality. [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
- Full Text
- View/download PDF
5. THE IMPACT OF AI ON TEACHERS: SUPPORT OR REPLACEMENT?
- Author
-
Nikitina, Iryna and Ishchenko, Tetyana
- Subjects
INTELLIGENT tutoring systems ,CAREER changes ,ARTIFICIAL intelligence ,EMOTIONAL intelligence ,LESSON planning - Abstract
The article explores the growing role of AI in education, analyzing whether it serves as a supportive tool for teachers or poses a threat to their jobs. AI can automate routine tasks such as grading and administrative work, freeing up teachers' time for more meaningful activities like personalized student interaction and lesson planning. Additionally, AI-powered tools enhance data-driven teaching, providing teachers with valuable insights to improve student performance and offer individualized support. While AI offers significant advantages, the article emphasizes that it cannot replace the human qualities essential to teaching, such as emotional intelligence, empathy, and creativity. These are areas where AI falls short, making it unlikely to fully replace teachers. Instead, AI allows teachers to evolve their roles from traditional instruction to mentorship and guidance, focusing on fostering critical thinking and creativity in students. However, there are concerns that AI could reduce teaching jobs or fundamentally change the role of educators. Teachers may also resist AI due to fears of job displacement or lack of sufficient training to use these technologies effectively. The article concludes that AI, when used responsibly, acts as a valuable partner rather than a replacement. By taking over repetitive tasks, AI enables teachers to concentrate on more impactful educational activities, ensuring that the human element remains at the heart of learning. [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
- Full Text
- View/download PDF
6. Automated grading of anatomical objective structured practical examinations using decision trees: An artificial intelligence approach.
- Author
-
Bernard, Jason, Sonnadara, Ranil, Saraco, Anthony N., Mitchell, Josh P., Bak, Alex B., Bayer, Ilana, and Wainman, Bruce C.
- Abstract
An Objective Structured Practical Examination (OSPE) is an effective and robust, but resource‐intensive, means of evaluating anatomical knowledge. Since most OSPEs employ short answer or fill‐in‐the‐blank style questions, the format requires many people familiar with the content to mark the examinations. However, the increasing prevalence of online delivery for anatomy and physiology courses could result in students losing the OSPE practice that they would receive in face‐to‐face learning sessions. The purpose of this study was to test the accuracy of Decision Trees (DTs) in marking OSPE questions as a first step to creating an intelligent, online OSPE tutoring system. The study used the results of the winter 2020 semester final OSPE from McMaster University's anatomy and physiology course in the Faculty of Health Sciences (HTHSCI 2FF3/2LL3/1D06) as the data set. Ninety percent of the data set was used in a 10‐fold validation algorithm to train a DT for each of the 54 questions. Each DT was comprised of unique words that appeared in correct, student‐written answers. The remaining 10% of the data set was marked by the generated DTs. When the answers marked by the DT were compared to the answers marked by staff and faculty, the DT achieved an average accuracy of 94.49% across all 54 questions. This suggests that machine learning algorithms such as DTs are a highly effective option for OSPE grading and are suitable for the development of an intelligent, online OSPE tutoring system. [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
- Full Text
- View/download PDF
7. Sensitivity of Automated SQL Grading in Computer Science Courses
- Author
-
Wanjiru, Benard, van Bommel, Patrick, Hiemstra, Djoerd, Kacprzyk, Janusz, Series Editor, Gomide, Fernando, Advisory Editor, Kaynak, Okyay, Advisory Editor, Liu, Derong, Advisory Editor, Pedrycz, Witold, Advisory Editor, Polycarpou, Marios M., Advisory Editor, Rudas, Imre J., Advisory Editor, Wang, Jun, Advisory Editor, Daimi, Kevin, editor, and Al Sadoon, Abeer, editor more...
- Published
- 2024
- Full Text
- View/download PDF
8. Grading Documentation with Machine Learning
- Author
-
Messer, Marcus, Shi, Miaojing, Brown, Neil C. C., Kölling, Michael, Goos, Gerhard, Series Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Olney, Andrew M., editor, Chounta, Irene-Angelica, editor, Liu, Zitao, editor, Santos, Olga C., editor, and Bittencourt, Ig Ibert, editor more...
- Published
- 2024
- Full Text
- View/download PDF
9. Marking: Visual Grading with Highlighting Errors and Annotating Missing Bits
- Author
-
Sonkar, Shashank, Liu, Naiming, Mallick, Debshila B., Baraniuk, Richard G., Goos, Gerhard, Series Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Olney, Andrew M., editor, Chounta, Irene-Angelica, editor, Liu, Zitao, editor, Santos, Olga C., editor, and Bittencourt, Ig Ibert, editor more...
- Published
- 2024
- Full Text
- View/download PDF
10. EvaAI: A Multi-agent Framework Leveraging Large Language Models for Enhanced Automated Grading
- Author
-
Lagakis, Paraskevas, Demetriadis, Stavros, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Sifaleras, Angelo, editor, and Lin, Fuhua, editor more...
- Published
- 2024
- Full Text
- View/download PDF
11. Automated magnetic resonance imaging‐based grading of the lumbar intervertebral disc and facet joints
- Author
-
Maryam Nikpasand, Jill M. Middendorf, Vincent A. Ella, Kristen E. Jones, Bryan Ladd, Takashi Takahashi, Victor H. Barocas, and Arin M. Ellingson
- Subjects
automated grading ,deep learning ,facet joint ,Fujiwara ,intervertebral disc ,machine learning ,Orthopedic surgery ,RD701-811 - Abstract
Abstract Background Degeneration of both intervertebral discs (IVDs) and facet joints in the lumbar spine has been associated with low back pain, but whether and how IVD/joint degeneration contributes to pain remains an open question. Joint degeneration can be identified by pairing T1 and T2 magnetic resonance imaging (MRI) with analysis techniques such as Pfirrmann grades (IVD degeneration) and Fujiwara scores (facet degeneration). However, these grades are subjective, prompting the need to develop an automated technique to enhance inter‐rater reliability. This study introduces an automated convolutional neural network (CNN) technique trained on clinical MRI images of IVD and facet joints obtained from public‐access Lumbar Spine MRI Dataset. The primary goal of the automated system is to classify health of lumbar discs and facet joints according to Pfirrmann and Fujiwara grading systems and to enhance inter‐rater reliability associated with these grading systems. Methods Performance of the CNN on both the Pfirrmann and Fujiwara scales was measured by comparing the percent agreement, Pearson's correlation and Fleiss kappa value for results from the classifier to the grades assigned by an expert grader. Results The CNN demonstrates comparable performance to human graders for both Pfirrmann and Fujiwara grading systems, but with larger errors in Fujiwara grading. The CNN improves the reliability of the Pfirrmann system, aligning with previous findings for IVD assessment. Conclusion The study highlights the potential of using deep learning in classifying the IVD and facet joint health, and due to the high variability in the Fujiwara scoring system, highlights the need for improved imaging and scoring techniques to evaluate facet joint health. All codes required to use the automatic grading routines described herein are available in the Data Repository for University of Minnesota (DRUM). more...
- Published
- 2024
- Full Text
- View/download PDF
12. A pilot cost-analysis study comparing AI-based EyeArt® and ophthalmologist assessment of diabetic retinopathy in minority women in Oslo, Norway
- Author
-
Mia Karabeg, Goran Petrovski, Silvia NW Hertzberg, Maja Gran Erke, Dag Sigurd Fosmark, Greg Russell, Morten C. Moe, Vallo Volke, Vidas Raudonis, Rasa Verkauskiene, Jelizaveta Sokolovska, Inga-Britt Kjellevold Haugen, and Beata Eva Petrovski more...
- Subjects
Screening ,Diabetic retinopathy ,Minority women ,Norway ,Manual grading ,Automated grading ,Ophthalmology ,RE1-994 - Abstract
Abstract Background Diabetic retinopathy (DR) is the leading cause of adult blindness in the working age population worldwide, which can be prevented by early detection. Regular eye examinations are recommended and crucial for detecting sight-threatening DR. Use of artificial intelligence (AI) to lessen the burden on the healthcare system is needed. Purpose To perform a pilot cost-analysis study for detecting DR in a cohort of minority women with DM in Oslo, Norway, that have the highest prevalence of diabetes mellitus (DM) in the country, using both manual (ophthalmologist) and autonomous (AI) grading. This is the first study in Norway, as far as we know, that uses AI in DR- grading of retinal images. Methods On Minority Women’s Day, November 1, 2017, in Oslo, Norway, 33 patients (66 eyes) over 18 years of age diagnosed with DM (T1D and T2D) were screened. The Eidon - True Color Confocal Scanner (CenterVue, United States) was used for retinal imaging and graded for DR after screening had been completed, by an ophthalmologist and automatically, using EyeArt Automated DR Detection System, version 2.1.0 (EyeArt, EyeNuk, CA, USA). The gradings were based on the International Clinical Diabetic Retinopathy (ICDR) severity scale [1] detecting the presence or absence of referable DR. Cost-minimization analyses were performed for both grading methods. Results 33 women (64 eyes) were eligible for the analysis. A very good inter-rater agreement was found: 0.98 (P more...
- Published
- 2024
- Full Text
- View/download PDF
13. CodeBuddy: A Programming Assignment Management System for Short-Form Exercises
- Author
-
Stephen R. Piccolo, Emme Tuft, P. J. Tatlow, Zach Eliason, and Ashlie Stephenson
- Subjects
programming education ,automated grading ,pair programming ,intelligent tutor ,web application ,automated assessment ,Computer software ,QA76.75-76.765 - Abstract
CodeBuddy is a software system for delivering computer-programming assignments to students. It is primarily used for short-form exercises, such as those delivered in introductory-programming courses and informal-learning settings. It provides a Web-based interface, the ability to execute code in a secure environment, support for custom testing logic, near-immediate feedback to students, and support for many programming languages. Other features include support for graphics-based programming exercises, pair programming, the ability for students to review the instructor’s solution after solving an exercise, and an intelligent tutor. Upon creating an account, each student is randomly assigned to an “A” or “B” cohort, thus enabling researchers to perform pedagogical research via online controlled experiments. These and other features offer opportunities for instructors to customize the learning experience, in diverse ways, for students learning to program. more...
- Published
- 2025
- Full Text
- View/download PDF
14. A pilot cost-analysis study comparing AI-based EyeArt® and ophthalmologist assessment of diabetic retinopathy in minority women in Oslo, Norway.
- Author
-
Karabeg, Mia, Petrovski, Goran, Hertzberg, Silvia NW, Erke, Maja Gran, Fosmark, Dag Sigurd, Russell, Greg, Moe, Morten C., Volke, Vallo, Raudonis, Vidas, Verkauskiene, Rasa, Sokolovska, Jelizaveta, Haugen, Inga-Britt Kjellevold, and Petrovski, Beata Eva more...
- Subjects
DIABETIC retinopathy ,MINORITY women ,ARTIFICIAL intelligence ,OPHTHALMOLOGISTS ,EYE examination - Abstract
Background: Diabetic retinopathy (DR) is the leading cause of adult blindness in the working age population worldwide, which can be prevented by early detection. Regular eye examinations are recommended and crucial for detecting sight-threatening DR. Use of artificial intelligence (AI) to lessen the burden on the healthcare system is needed. Purpose: To perform a pilot cost-analysis study for detecting DR in a cohort of minority women with DM in Oslo, Norway, that have the highest prevalence of diabetes mellitus (DM) in the country, using both manual (ophthalmologist) and autonomous (AI) grading. This is the first study in Norway, as far as we know, that uses AI in DR- grading of retinal images. Methods: On Minority Women's Day, November 1, 2017, in Oslo, Norway, 33 patients (66 eyes) over 18 years of age diagnosed with DM (T1D and T2D) were screened. The Eidon - True Color Confocal Scanner (CenterVue, United States) was used for retinal imaging and graded for DR after screening had been completed, by an ophthalmologist and automatically, using EyeArt Automated DR Detection System, version 2.1.0 (EyeArt, EyeNuk, CA, USA). The gradings were based on the International Clinical Diabetic Retinopathy (ICDR) severity scale [1] detecting the presence or absence of referable DR. Cost-minimization analyses were performed for both grading methods. Results: 33 women (64 eyes) were eligible for the analysis. A very good inter-rater agreement was found: 0.98 (P < 0.01), between the human and AI-based EyeArt grading system for detecting DR. The prevalence of DR was 18.6% (95% CI: 11.4–25.8%), and the sensitivity and specificity were 100% (95% CI: 100–100% and 95% CI: 100–100%), respectively. The cost difference for AI screening compared to human screening was $143 lower per patient (cost-saving) in favour of AI. Conclusion: Our results indicate that The EyeArt AI system is both a reliable, cost-saving, and useful tool for DR grading in clinical practice. [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
- Full Text
- View/download PDF
15. Smart grading: A generative AI-based tool for knowledge-grounded answer evaluation in educational assessments
- Author
-
Samuel Tobler
- Subjects
Artificial intelligence ,Test evaluation ,Educational assessment ,Automated grading ,GPT ,Large language model ,Science - Abstract
Evaluating text-based answers obtained in educational settings or behavioral studies is time-consuming and resource-intensive. Applying novel artificial intelligence tools such as ChatGPT might support the process. Still, currently available implementations do not allow for automated and case-specific evaluations of large numbers of student answers. To counter this limitation, we developed a flexible software and user-friendly web application that enables researchers and educators to use cutting-edge artificial intelligence technologies by providing an interface that combines large language models with options to specify questions of interest, sample solutions, and evaluation instructions for automated answer scoring. We validated the method in an empirical study and found the software with expert ratings to have high reliability. Hence, the present software constitutes a valuable tool to facilitate and enhance text-based answer evaluation. • Generative AI-enhanced software for customizable, case-specific, and automized grading of large amounts of text-based answers. • Open-source software and web application for direct implementation and adaptation. more...
- Published
- 2024
- Full Text
- View/download PDF
16. Automated Analytic Dataset Generation and Assessment for Engineering Analytics Education.
- Author
-
Wilcox, Bruce, Yufan Fei, Jihao LI, Junqiang Wang, and Junmeng Xu
- Abstract
In recent years, there has been significant growth in analytics programs at the undergraduate and masters' levels in Industrial and Systems Engineering (ISE) departments at universities across the country. When teaching analytics techniques, especially predictive analytics, instructors are always looking for datasets that contain statistical characteristics that we want to discuss including multicollinearity, interaction effects between variables, skewed distributions, and nonlinear relationships between predictor and response variables. Instructors generally must either search for existing datasets that have these attributes or create them "manually" using programmatic techniques. An academic toolset to permit instructors to specify the statistical properties desired in an analytic, to generate multiple, randomized versions of this dataset (using a newly developed Python library), to provide automation for creating individualized datasets for each student (to avoid inappropriate collaboration on assignments and take-home exams among students), and to provide for automated grading support for assignments and examinations. This work is supported by a gift from the USC-Meta Center for Research and Education in AI and Learning. [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
17. Making Electric Machinery Labs Easier to Grade.
- Author
-
Wrate, Glenn T.
- Abstract
The best way to teach electric machinery is with hands-on labs. At the beginning of the Fall 2020 semester, due to COVID-19 concerns, it was unclear whether in-person laboratories were going to be allowed. As the semester drew closer, it was determined that we could have inperson labs, but we could only have one person per lab bench. Due to high voltages and rotating machinery on the benches, this would be a safety hazard. A compromise was reached and we were allowed to have two students per bench, but the students were required to wear face shields in addition to the required face mask and gloves. When running the lab in the past, all lab data, calculations, and discussions about the lab assignment were written on engineering data sheets (similar to industry). With the possibility of touch transmission of COVID-19, we switched to electronic forms. To do this we added input boxes into the existing lab handouts using Adobe Acrobat. The students then submitted their assignments via our Moodle-based learning management system (LMS). With forms created with Adobe Acrobat each of the input blocks can be assigned field names. You can then transfer the form data into an Excel spreadsheet. Once in spreadsheet form, you can create equations in Excel to quickly check the accuracy and appropriateness of the students' data. Color coding was used to indicate when values were off by more than 0.5% to 5% of the expected value. The instructor could then determine if points should be deducted. If the error was small, it was simply noted with a comment on the student's submittal. For larger errors, points were deducted using a rubric in the LMS created for each lab assignment. For the discussions, the text was also imported into the spreadsheet and the rubric was used to assign a grade. Both the lab handouts and the Excel spreadsheets have been used in subsequent semesters. This paper looks at the process to create forms from the lab handouts, creating the Excel spreadsheet to check the students' data, the responses of the students to this type of lab assignment, and problems encounter with the implementation. [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
18. Automated Grading of LabVIEW Files.
- Author
-
Hekman, Keith
- Abstract
Instructors frequently use automated grading in programming classes. Institutions have developed graders for C++, Java, MATLAB, and many other programming languages. LabVIEW is a graphical programming language that people frequently use for data acquisition. Since there were no automated grading programs for LabVIEW, a computerized grading system has been developed. With the grading program, students email the LabVIEW files they have written, and the program provides their assignment score and feedback concerning missing program functions or wires. Students then can resubmit their work until the due date. The grading program was implemented in a LabVIEW programming course at California Baptist University using NI's LabVIEW Core 1 and Core 2 curricula. When using the grading system, students appreciated the immediate feedback from the program, and the instructor/teaching assistant's grading time was reduced. [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
19. The development of an artificial intelligence classifier to automate assessment in large class settings: preliminary results.
- Author
-
Lindsay, Euan and Sabet Jahromi, Mohammad Naser
- Abstract
This evidence based practice paper presents preliminary results in using an artificial intelligence classifier to mark student assignments in a large class setting. The assessment task consists of an approximately 2000 word reflective essay that is produced under examination conditions and submitted electronically. The marking is a simple pass/fail determination, and no explicit feedback beyond the pass/fail grade is provided to the students. Each year around 1500 students complete this assignment, which places a significant and time-constrained marking load upon the teaching faculty. This paper presents a Natural Language Process (NLP) framework/tool for developing a machine learning based binary classifier for automated assessment of these assignments. The classifier allocates each assignment a score representing the probability that the assignment would receive a passing grade from a human marker. The effectiveness and performance of the classifier is measured by investigating the accuracy of those predictions. Several iterations and statistical analyses were carried out to determine operational thresholds that balance the risks of false positives and false negatives with the required quantity of human marking to assess the assignment. The resulting classifier was able to provide accuracy levels that are potentially feasible in an operational context, and the potential for significant overall reductions in the human marking load for this assignment. [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
20. Examining the Efficacy of ChatGPT in Marking Short-Answer Assessments in an Undergraduate Medical Program
- Author
-
Leo Morjaria, Levi Burns, Keyna Bracken, Anthony J. Levinson, Quang N. Ngo, Mark Lee, and Matthew Sibbald
- Subjects
ChatGPT ,artificial intelligence ,short-answer assessment ,automated grading ,generative AI ,undergraduate medical education ,Special aspects of education ,LC8-6691 ,Medicine - Abstract
Traditional approaches to marking short-answer questions face limitations in timeliness, scalability, inter-rater reliability, and faculty time costs. Harnessing generative artificial intelligence (AI) to address some of these shortcomings is attractive. This study aims to validate the use of ChatGPT for evaluating short-answer assessments in an undergraduate medical program. Ten questions from the pre-clerkship medical curriculum were randomly chosen, and for each, six previously marked student answers were collected. These sixty answers were evaluated by ChatGPT in July 2023 under four conditions: with both a rubric and standard, with only a standard, with only a rubric, and with neither. ChatGPT displayed good Spearman correlations with a single human assessor (r = 0.6–0.7, p < 0.001) across all conditions, with the absence of a standard or rubric yielding the best correlation. Scoring differences were common (65–80%), but score adjustments of more than one point were less frequent (20–38%). Notably, the absence of a rubric resulted in systematically higher scores (p < 0.001, partial η2 = 0.33). Our findings demonstrate that ChatGPT is a viable, though imperfect, assistant to human assessment, performing comparably to a single expert assessor. This study serves as a foundation for future research on AI-based assessment techniques with potential for further optimization and increased reliability. more...
- Published
- 2024
- Full Text
- View/download PDF
21. Research on performance evaluation of higher vocational education informatization based on data envelopment analysis
- Author
-
Sergii Khrapatyi, Kseniia Tokarieva, Olena Hlushchenko, Oleksandra Paramonova, and Ielyzaveta Lvova
- Subjects
personalized learning ,educational chatbots ,ai-driven assessments ,automated grading ,educational technology ,Theory and practice of education ,LB5-3640 ,Science - Abstract
This article highlights the multifaceted role of AI in modern education and offers insights into innovative ways to revolutionize educational practices through AI technologies. Since this article provides comprehension of the scope and depth of AI's impact on the education sphere, it appeals to a diverse readership, encompassing educators, policymakers, researchers, and the general public. This article explores key issues within the domain of AI in education, including personalized learning, AI-driven assessments, data analytics, and the integration of AI into learning management systems. The article highlights promises, potentials, and challenges accompanying this technological advancement. The authors emphasize the need for a balanced and informed approach to using AI to enhance the education system. more...
- Published
- 2024
- Full Text
- View/download PDF
22. Research on performance evaluation of higher vocational education informatization based on data envelopment analysis.
- Author
-
Khrapatyi, Sergii, Tokarieva, Kseniia, Hlushchenko, Olena, Paramonova, Oleksandra, and Lvova, Ielyzaveta
- Subjects
VOCATIONAL education ,HIGHER education ,DATA envelopment analysis ,ARTIFICIAL intelligence in education ,PERFORMANCE evaluation ,LEARNING management system - Abstract
This article highlights the multifaceted role of AI in modern education and offers insights into innovative ways to revolutionize educational practices through AI technologies. Since this article provides comprehension of the scope and depth of AI's impact on the education sphere, it appeals to a diverse readership, encompassing educators, policymakers, researchers, and the general public. This article explores key issues within the domain of AI in education, including personalized learning, AI-driven assessments, data analytics, and the integration of AI into learning management systems. The article highlights promises, potentials, and challenges accompanying this technological advancement. The authors emphasize the need for a balanced and informed approach to using AI to enhance the education system. [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
- Full Text
- View/download PDF
23. Automated grading software tool with feedback process to support learning of hardware description languages.
- Author
-
Corso Pinzón, Andrés Francisco, Ramírez-Echeverry, Jhon J., and Restrepo-Calle, Felipe
- Subjects
LEARNING ,SOFTWARE development tools ,DIGITAL electronics ,GRAPHICAL user interfaces ,EDUCATIONAL intervention - Abstract
Hardware Description Languages (HDL) have gained popularity in the field of digital electronics design, driven by the increasing complexity of modern electronic circuits. Consequently, supporting students in their learning of these languages is crucial. This work aims to address this need by developing an automated assessment software tool with feedback process to support the learning of HDL and making an educational intervention to support the learning process of students. The tool's features were selected based on similar developments, and a prototype was designed and implemented. Additionally, an educational intervention was conducted over a five-week period in a Digital Electronics course at the National University of Colombia. Through analyzing students' interactions with the tool and their perceptions of its usage, the study examined their learning experiences. Among the features highlighted by students as most beneficial for their HDL learning process were the online availability of the tool, the feedback system that helped them identify and correct errors in their code, the provision of immediate feedback, the online editor with syntax highlighting, and the graphical user interface. This work makes two significant contributions to the field of HDL teaching in engineering. Firstly, a publicly accessible HDL grading tool has been developed, offering students immediate formative and summative feedback through an automated grader. Secondly, empirical evidence has been provided regarding the benefits of using such a tool in enhancing students' learning process. [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
- Full Text
- View/download PDF
24. Experiences and Critical Reflection on Online-Assessment with Excel Case Studies – Review on a Successful Online-Assessment Practice as Well as the Adaptation to a Remote Setting Due to the COVID-19 Pandemic
- Author
-
Braun, Johanna, Schläfli, Roland, Schmocker, David, Wilding, Benjamin, Egger, Rudolf, Series Editor, Brinker, Tobina, Series Editor, Eugster, Balthasar, Series Editor, Frederiksen, Jan, Series Editor, Hummel, Sandra, editor, and Donner, Mana-Teresa, editor more...
- Published
- 2023
- Full Text
- View/download PDF
25. Evaluating the Effectiveness of a New Programming Teaching Methodology Using CodeRunner
- Author
-
Haidar, Siba, Yaacoub, Antoun, Ionascu, Felicia, Kacprzyk, Janusz, Series Editor, Gomide, Fernando, Advisory Editor, Kaynak, Okyay, Advisory Editor, Liu, Derong, Advisory Editor, Pedrycz, Witold, Advisory Editor, Polycarpou, Marios M., Advisory Editor, Rudas, Imre J., Advisory Editor, Wang, Jun, Advisory Editor, Guralnick, David, editor, Auer, Michael E., editor, and Poce, Antonella, editor more...
- Published
- 2023
- Full Text
- View/download PDF
26. Evaluating ChatGPT’s Decimal Skills and Feedback Generation in a Digital Learning Game
- Author
-
Nguyen, Huy A., Stec, Hayden, Hou, Xinying, Di, Sarah, McLaren, Bruce M., Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Viberg, Olga, editor, Jivet, Ioana, editor, Muñoz-Merino, Pedro J., editor, Perifanou, Maria, editor, and Papathoma, Tina, editor more...
- Published
- 2023
- Full Text
- View/download PDF
27. Using Machine Learning to Identify Patterns in Learner-Submitted Code for the Purpose of Assessment
- Author
-
Tarcsay, Botond, Perez-Tellez, Fernando, Vasic, Jelena, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Rodríguez-González, Ansel Yoan, editor, Pérez-Espinosa, Humberto, editor, Martínez-Trinidad, José Francisco, editor, Carrasco-Ochoa, Jesús Ariel, editor, and Olvera-López, José Arturo, editor more...
- Published
- 2023
- Full Text
- View/download PDF
28. GPT-4 in Education: Evaluating Aptness, Reliability, and Loss of Coherence in Solving Calculus Problems and Grading Submissions
- Author
-
Gandolfi, Alberto
- Published
- 2024
- Full Text
- View/download PDF
29. Improving the network architecture of YOLOv7 to achieve real-time grading of canola based on kernel health
- Author
-
Angshuman Thakuria and Chyngyz Erkinbaev
- Subjects
YOLOv7 ,Convolutional Block Attention Mechanism ,Multi-Object Tracking ,Canola ,Quality Control ,Automated Grading ,Agriculture (General) ,S1-972 ,Agricultural industries ,HD9000-9495 - Abstract
The occurrence of heated and immature canola kernels caused by excessive drying and frost damage is undesired by grain buyers due to lower oil yield and diminished market value. The current grading process is visually examining each kernel's endosperm color and counting the damaged seeds. As this process is time-consuming, laborious, and prone to errors, this study proposes an automated grading technique based on object detection, multi-object tracking, and counting. The detection task was achieved via an improved YOLOv7 network (YOLOv7_ours) that was modified to increase its performance in accurately identifying small objects by adding two convolutional block attention modules in the neck region and decrease its computational complexity (cost) and size by substituting convolutional layers with ghost layers in all the Efficient Layer Aggregation Networks modules, and in the Spatial Pyramid Pooling Cross Stage Partial module present in YOLOv7. The weights of the trained network were fed to the ByteTrack multiple object tracker to track the detections frame-by-frame in a video feed. The unique identities generated by the tracker for each detected object of interest were then used to count the number of defects using a line cross algorithm. The mean average precision (mAP@0.5) obtained after training the YOLOv7_ours model was 1.02% better and its cost and size were 32.1% and 37.1% lower than the baseline YOLOv7 model. In a test video, the overall model achieved a multi-object tracking accuracy and counting accuracy of 84.8% and 93.9%, respectively. This three-stage model can be readily deployed in an edge device for accurate and real-time grading of canola kernels by grain buyers. more...
- Published
- 2023
- Full Text
- View/download PDF
30. CAN AI-ASSISTED ESSAY ASSESSMENT SUPPORT TEACHERS? A CROSS-SECTIONAL MIXED-METHODS RESEARCH CONDUCTED AT THE UNIVERSITY OF MONTENEGRO.
- Author
-
IVANOVIČ, Igor
- Subjects
- *
ARTIFICIAL intelligence , *LANGUAGE models , *CROSS-sectional method , *NATURAL language processing , *CHATGPT - Abstract
In this study, we will try to answer the question if an AI language model can provide teachers with essay assessment solutions that are on a par with the solutions provided by experienced professors. We designed a study with the aim of comparing the essay assessment outputs of the AI language model and three of our colleagues working at the University of Montenegro. The main aim of this paper is to investigate if this AI language model can be a viable teachers' assistance tool that provides immediate and meaningful feedback to teachers and students. Our hypothesis is, with some caveats, that the AI language model is more than a viable and useful tool, capable of providing meaningful and immediate feedback, greatly reducing the assessment time, and thus helping the teachers become more efficient and consistent. We will compare the results of 78 essays assessed by three teachers with the results provided by ChatGPT and see where the two sets of results converge or diverge in terms of their individual and overall scores. [ABSTRACT FROM AUTHOR] more...
- Published
- 2023
- Full Text
- View/download PDF
31. Use of Machine Learning Methods in the Assessment of Programming Assignments
- Author
-
Tarcsay, Botond, Vasić, Jelena, Perez-Tellez, Fernando, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Sojka, Petr, editor, Horák, Aleš, editor, Kopeček, Ivan, editor, and Pala, Karel, editor more...
- Published
- 2022
- Full Text
- View/download PDF
32. Grading Programming Assignments with an Automated Grading and Feedback Assistant
- Author
-
Messer, Marcus, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Rodrigo, Maria Mercedes, editor, Matsuda, Noburu, editor, Cristea, Alexandra I., editor, and Dimitrova, Vania, editor more...
- Published
- 2022
- Full Text
- View/download PDF
33. Machine Learning Techniques for Grading of PowerPoint Slides
- Author
-
Borade, Jyoti G., Netak, Laxman D., Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Woeginger, Gerhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Kim, Jong-Hoon, editor, Singh, Madhusudan, editor, Khan, Javed, editor, Tiwary, Uma Shanker, editor, Sur, Marigankar, editor, and Singh, Dhananjay, editor more...
- Published
- 2022
- Full Text
- View/download PDF
34. Semiautomatic Grading of Short Texts for Open Answers in Higher Education
- Author
-
de-la-Fuente-Valentín, Luis, Verdú, Elena, Padilla-Zea, Natalia, Villalonga, Claudia, Blanco Valencia, Xiomara Patricia, Baldiris Navarro, Silvia Margarita, Filipe, Joaquim, Editorial Board Member, Ghosh, Ashish, Editorial Board Member, Prates, Raquel Oliveira, Editorial Board Member, Zhou, Lizhu, Editorial Board Member, Casalino, Gabriella, editor, Cimitile, Marta, editor, Ducange, Pietro, editor, Padilla Zea, Natalia, editor, Pecori, Riccardo, editor, Picerno, Pietro, editor, and Raviolo, Paolo, editor more...
- Published
- 2022
- Full Text
- View/download PDF
35. Feature Extraction for Automatic Grading of Students’ Presentations
- Author
-
Borade, Jyoti G., Kiwelekar, Arvind W., Netak, Laxman D., Kacprzyk, Janusz, Series Editor, Gomide, Fernando, Advisory Editor, Kaynak, Okyay, Advisory Editor, Liu, Derong, Advisory Editor, Pedrycz, Witold, Advisory Editor, Polycarpou, Marios M., Advisory Editor, Rudas, Imre J., Advisory Editor, Wang, Jun, Advisory Editor, Tuba, Milan, editor, Akashe, Shyam, editor, and Joshi, Amit, editor more...
- Published
- 2022
- Full Text
- View/download PDF
36. Improved Automated Classification of Sentences in Data Science Exercises
- Author
-
Angelone, Anna Maria, Galassi, Alessandra, Vittorini, Pierpaolo, Kacprzyk, Janusz, Series Editor, Gomide, Fernando, Advisory Editor, Kaynak, Okyay, Advisory Editor, Liu, Derong, Advisory Editor, Pedrycz, Witold, Advisory Editor, Polycarpou, Marios M., Advisory Editor, Rudas, Imre J., Advisory Editor, Wang, Jun, Advisory Editor, De la Prieta, Fernando, editor, Gennari, Rosella, editor, Temperini, Marco, editor, Di Mascio, Tania, editor, Vittorini, Pierpaolo, editor, Kubincova, Zuzana, editor, Popescu, Elvira, editor, Rua Carneiro, Davide, editor, Lancia, Loreto, editor, and Addone, Agnese, editor more...
- Published
- 2022
- Full Text
- View/download PDF
37. Automated grading system for quantifying KOH microscopic images in dermatophytosis.
- Author
-
KV, Rajitha, Govindan, Sreejith, PY, Prakash, Kamath, Asha, Rao, Raghavendra, and Prasad, Keerthana
- Subjects
- *
RECEIVER operating characteristic curves , *IMAGE segmentation , *SKIN imaging , *RINGWORM , *DEEP learning - Abstract
• Quantifying fungal load supports efficient clinical management of tinea infections. • Automated dermatophyte quantification is, hitherto unexplored. • Rising tinea infections and drug resistance support automated quantification. • U-Net applied segmentation and pixel-based grading were the main steps followed. [Display omitted] Concerning the progression of dermatophytosis and its prognosis, quantification studies play a significant role. Present work aims to develop an automated grading system for quantifying fungal loads in KOH microscopic images of skin scrapings collected from dermatophytosis patients. Fungal filaments in the images were segmented using a U-Net model to obtain the pixel counts. In the absence of any threshold value for pixel counts to grade these images as low, moderate, or high, experts were assigned the task of manual grading. Grades and corresponding pixel counts were subjected to statistical procedures involving cumulative receiver operating characteristic curve analysis for developing an automated grading system. The model's specificity, accuracy, precision, and sensitivity metrics crossed 92%, 86%, 82%, and 76%, respectively. 'Almost perfect agreement' with Fleiss kappa of 0.847 was obtained between automated and manual gradings. This pixel count-based grading of KOH images offers a novel, cost-effective solution for quantifying fungal load. [ABSTRACT FROM AUTHOR] more...
- Published
- 2025
- Full Text
- View/download PDF
38. Mass and volume estimation of diverse kimchi cabbage forms using RGB-D vision and machine learning.
- Author
-
Yang, Hae-Il, Min, Sung-Gi, Yang, Ji-Hee, Eun, Jong-Bang, and Chung, Young-Bae
- Subjects
- *
MACHINE learning , *COMPUTER vision , *WESTERN diet , *FEATURE extraction , *CABBAGE - Abstract
This study introduces a custom-built RGB-D-based machine vision system designed to accurately estimate the mass and volume of whole kimchi cabbage (WC) and longitudinally cut kimchi cabbage (LCC). Given the pivotal role of kimchi cabbage (KC) in both Asian and Western diets, accurate post-harvest assessment of its mass and volume is critical for quality control, sorting, and pricing. Conventional manual measurements and visual estimations are laborious and inaccurate. Our research leveraged RGB-D data to refine machine learning models and enhance the extraction and analysis of 2D, 3D, and colorimetric features for a more reliable estimation approach. The results demonstrate that integrating 3D and colorimetric features markedly improves the estimation accuracy, with notable success in mass estimation for LCC (R² = 0.913, ratio of performance to deviation (RPD) = 3.38) and robust volume predictions for both cabbage types (R² > 0.90, RPD > 3). However, challenges such as potential over-exclusion of outer leaves in LCC and the need for more advanced WC mass estimation techniques have been identified. Future work will focus on refining the feature extraction methods and assessing various imaging environments to enhance the precision of mass and volume predictions across different forms of KC. • Novel RGB-D-based system accurately estimates mass and volume of kimchi cabbage. • Integration of 3D and colorimetric features enhances estimation accuracy. • Significant success in mass estimation for longitudinally cut cabbage (R² = 0.913). • Robust volume predictions achieved for both whole and cut cabbage (R² > 0.90). • Study addresses critical need for reliable post-harvest assessment methods. [ABSTRACT FROM AUTHOR] more...
- Published
- 2024
- Full Text
- View/download PDF
39. Automatic Short Answer Grading on High School's E-Learning Using Semantic Similarity Methods.
- Author
-
Wilianto, Daniel and Girsang, Abba Suganda
- Subjects
- *
STANDARD deviations , *ELEMENTARY schools , *HIGH schools , *DIGITAL learning - Abstract
Grading students' answers has always been a daunting task which takes a lot of teachers' time. The aim of this study is to grade students' answers automatically in a high school's e-learning system. The grading process must be fast, and the result must be as close as possible to the teacher assigned grades. We collected a total of 840 answers from 40 students for this study, each already graded by their teachers. We used Python library sentencetransformers and three of its latest pre-trained machine learning models (all-mpnet-base-v2, alldistilroberta-v1, all-MiniLM-L6-v2) for sentence embeddings. Computer grades were calculated using Cosine Similarity. These grades were then compared with teacher assigned grades using both Mean Absolute Error and Root Mean Square Error. Our results showed that all-MiniLM-L6-v2 gave the most similar grades to teacher assigned grades and had the fastest processing time. Further study may include testing these models on more answers from more students, also fine tune these models using more school materials. [ABSTRACT FROM AUTHOR] more...
- Published
- 2023
- Full Text
- View/download PDF
40. Automated Grading of Exam Responses: An Extensive Classification Benchmark
- Author
-
Ljungman, Jimmy, Lislevand, Vanessa, Pavlopoulos, John, Farazouli, Alexandra, Lee, Zed, Papapetrou, Panagiotis, Fors, Uno, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Woeginger, Gerhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Soares, Carlos, editor, and Torgo, Luis, editor more...
- Published
- 2021
- Full Text
- View/download PDF
41. Automated SQL Grading System
- Author
-
Kanchan, Shohna, Kalsekar, Samruddhi, Dubey, Nishita, Fernandes, Chelsea, Hamdare, Safa, Kacprzyk, Janusz, Series Editor, Gomide, Fernando, Advisory Editor, Kaynak, Okyay, Advisory Editor, Liu, Derong, Advisory Editor, Pedrycz, Witold, Advisory Editor, Polycarpou, Marios M., Advisory Editor, Rudas, Imre J., Advisory Editor, Wang, Jun, Advisory Editor, Saini, H. S., editor, Sayal, Rishi, editor, Govardhan, A., editor, and Buyya, Rajkumar, editor more...
- Published
- 2021
- Full Text
- View/download PDF
42. Improved Feedback in Automated Grading of Data Science Assignments
- Author
-
Galassi, Alessandra, Vittorini, Pierpaolo, Kacprzyk, Janusz, Series Editor, Pal, Nikhil R., Advisory Editor, Bello Perez, Rafael, Advisory Editor, Corchado, Emilio S., Advisory Editor, Hagras, Hani, Advisory Editor, Kóczy, László T., Advisory Editor, Kreinovich, Vladik, Advisory Editor, Lin, Chin-Teng, Advisory Editor, Lu, Jie, Advisory Editor, Melin, Patricia, Advisory Editor, Nedjah, Nadia, Advisory Editor, Nguyen, Ngoc Thanh, Advisory Editor, Wang, Jun, Advisory Editor, Kubincová, Zuzana, editor, Lancia, Loreto, editor, Popescu, Elvira, editor, Nakayama, Minoru, editor, Scarano, Vittorio, editor, and Gil, Ana B., editor more...
- Published
- 2021
- Full Text
- View/download PDF
43. Towards a Reliable and Rapid Automated Grading System in Facial Palsy Patients: Facial Palsy Surgery Meets Computer Science.
- Author
-
Knoedler, Leonard, Baecher, Helena, Kauke-Navarro, Martin, Prantl, Lukas, Machens, Hans-Günther, Scheuermann, Philipp, Palm, Christoph, Baumann, Raphael, Kehrer, Andreas, Panayi, Adriana C., and Knoedler, Samuel more...
- Subjects
- *
FACIAL paralysis , *COMPUTER science , *BELL'S palsy , *MODULAR forms , *PLASTIC surgery , *TUMOR grading - Abstract
Background: Reliable, time- and cost-effective, and clinician-friendly diagnostic tools are cornerstones in facial palsy (FP) patient management. Different automated FP grading systems have been developed but revealed persisting downsides such as insufficient accuracy and cost-intensive hardware. We aimed to overcome these barriers and programmed an automated grading system for FP patients utilizing the House and Brackmann scale (HBS). Methods: Image datasets of 86 patients seen at the Department of Plastic, Hand, and Reconstructive Surgery at the University Hospital Regensburg, Germany, between June 2017 and May 2021, were used to train the neural network and evaluate its accuracy. Nine facial poses per patient were analyzed by the algorithm. Results: The algorithm showed an accuracy of 100%. Oversampling did not result in altered outcomes, while the direct form displayed superior accuracy levels when compared to the modular classification form (n = 86; 100% vs. 99%). The Early Fusion technique was linked to improved accuracy outcomes in comparison to the Late Fusion and sequential method (n = 86; 100% vs. 96% vs. 97%). Conclusions: Our automated FP grading system combines high-level accuracy with cost- and time-effectiveness. Our algorithm may accelerate the grading process in FP patients and facilitate the FP surgeon's workflow. [ABSTRACT FROM AUTHOR] more...
- Published
- 2022
- Full Text
- View/download PDF
44. Building a Corpus of Task-Based Grading and Feedback Systems for Learning and Teaching Programming.
- Author
-
Strickroth, Sven and Striewe, Michael
- Subjects
INSTRUCTIONAL systems ,SCIENTIFIC community ,CORPORA ,COMMUNITIES ,EDUCATIONAL support ,VIRTUAL communities - Abstract
Using grading and feedback systems in the context of learning and teaching programming is quite common. During the last 20 to 40 years research results on several hundred systems and approaches have been published. Existing papers may tell researchers what works well in terms of educational support and how to make a grading and feedback system stable, extensible, secure, or sustainable. However, finding a solid basis for such kind of research is hard due to the vast amount of publications from a very diverse community. Hardly any recent systematic review includes data from more than 100 systems (most include less than 30). Hence, the authors started an endeavor to build a corpus of all task-based grading and feedback systems for learning and teaching programming that deal with source code and have been published in recent years. The intention is to provide the community with a solid basis for their research. The corpus is also designed to be updated and extended by the community with future systems. This paper describes the process of building the corpus and presents some meta-analysis that shed light on the involved research communities. [ABSTRACT FROM AUTHOR] more...
- Published
- 2022
- Full Text
- View/download PDF
45. Automated Short-Answer Grading using Semantic Similarity based on Word Embedding
- Author
-
Fetty Fitriyanti Lubis, Mutaqin, Atina Putri, Dana Waskita, Tri Sulistyaningtyas, Arry Akhmad Arman, and Yusep Rosmansyah
- Subjects
automated grading ,short answer ,semantic similarity ,syntax analysis ,word embeddings ,Technology ,Technology (General) ,T1-995 - Abstract
Automatic short-answer grading (ASAG) is a system that aims to help speed up the assessment process without an instructor’s intervention. Previous research had successfully built an ASAG system whose performance had a correlation of 0.66 and mean absolute error (MAE) starting from 0.94 with a conventionally graded set. However, this study had a weakness in the need for more than one reference answer for each question. It used a string-based equation method and keyword matching process to measure the sentences’ similarity in order to produce an assessment rubric. Thus, our study aimed to build a more concise short-answer automatic scoring system using a single reference answer. The mechanism used a semantic similarity measurement approach through word embedding techniques and syntactic analysis to assess the learner’s accuracy. Based on the experiment results, the semantic similarity approach showed a correlation value of 0.70 and an MAE of 0.70 when compared with the grading reference. more...
- Published
- 2021
- Full Text
- View/download PDF
46. A Protocol for Simulated Experimentation of Automated Grading Systems
- Author
-
Sterbini, Andrea, Temperini, Marco, Vittorini, Pierpaolo, Kacprzyk, Janusz, Series Editor, Pal, Nikhil R., Advisory Editor, Bello Perez, Rafael, Advisory Editor, Corchado, Emilio S., Advisory Editor, Hagras, Hani, Advisory Editor, Kóczy, László T., Advisory Editor, Kreinovich, Vladik, Advisory Editor, Lin, Chin-Teng, Advisory Editor, Lu, Jie, Advisory Editor, Melin, Patricia, Advisory Editor, Nedjah, Nadia, Advisory Editor, Nguyen, Ngoc Thanh, Advisory Editor, Wang, Jun, Advisory Editor, Vittorini, Pierpaolo, editor, Di Mascio, Tania, editor, Tarantino, Laura, editor, Temperini, Marco, editor, Gennari, Rosella, editor, and De la Prieta, Fernando, editor more...
- Published
- 2020
- Full Text
- View/download PDF
47. The Automated Grading of R Code Snippets: Preliminary Results in a Course of Health Informatics
- Author
-
Angelone, Anna Maria, Vittorini, Pierpaolo, Kacprzyk, Janusz, Series Editor, Pal, Nikhil R., Advisory Editor, Bello Perez, Rafael, Advisory Editor, Corchado, Emilio S., Advisory Editor, Hagras, Hani, Advisory Editor, Kóczy, László T., Advisory Editor, Kreinovich, Vladik, Advisory Editor, Lin, Chin-Teng, Advisory Editor, Lu, Jie, Advisory Editor, Melin, Patricia, Advisory Editor, Nedjah, Nadia, Advisory Editor, Nguyen, Ngoc Thanh, Advisory Editor, Wang, Jun, Advisory Editor, Gennari, Rosella, editor, Vittorini, Pierpaolo, editor, De la Prieta, Fernando, editor, Di Mascio, Tania, editor, Temperini, Marco, editor, Azambuja Silveira, Ricardo, editor, and Ovalle Carranza, Demetrio Arturo, editor more...
- Published
- 2020
- Full Text
- View/download PDF
48. Automata Tutor v3
- Author
-
D’Antoni, Loris, Helfrich, Martin, Kretinsky, Jan, Ramneantu, Emanuel, Weininger, Maximilian, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Woeginger, Gerhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Lahiri, Shuvendu K., editor, and Wang, Chao, editor more...
- Published
- 2020
- Full Text
- View/download PDF
49. Measuring the effectiveness of online problem solving for improving academic performance in a probability course.
- Author
-
González, José Antonio, Giuliano, Mónica, and Pérez, Silvia N.
- Subjects
PROBLEM solving ,ACADEMIC achievement ,PROBABILITY theory ,STATISTICS education ,INSTRUCTIONAL systems ,EDUCATIONAL outcomes - Abstract
Research on impact in student achievement of online homework systems compared to traditional methods is ambivalent. Methodological issues in the study design, besides of technological diversity, can account for this uncertainty. Hypothesis This study aims to estimate the effect size of homework practice with exercises automatically provided by the 'e-status' platform, in students from five Engineering programs. Instead of comparing students using the platform with others not using it, we distributed the subject topics into two blocks, and created nine probability problems for each block. After that, the students were randomly assigned to one block and could solve the related exercises through e-status. Teachers and evaluators were masked to the assignation. Five weeks after the assignment, all students answered a written test with questions regarding all topics. The study outcome was the difference between both blocks' scores obtained from the test. The two groups comprised 163 and 166 students. Of these, 103 and 107 respectively attended the test, while the remainder were imputed with 0. Those assigned to the first block obtained an average outcome of −1.85, while the average in the second block was −3.29 (95% confidence interval of difference, −2.46 to −0.43). During the period in which they had access to the platform before the test, the average total time spent solving problems was less than three hours. Our findings provide evidence that a small amount of active online work can positively impact on student performance. [ABSTRACT FROM AUTHOR] more...
- Published
- 2022
- Full Text
- View/download PDF
50. Validity of a graph-based automatic assessment system for programming assignments: human versus automatic grading.
- Author
-
Zougari, Soundous, Tanana, Mariam, and Lyhyaoui, Abdelouahid
- Subjects
COMPUTERS in education ,TEACHERS' workload ,COMPUTER programming ,LEARNING goals ,COVID-19 ,TEACHERS - Abstract
Programming is a very complex and challenging subject to teach and learn. A strategy guaranteed to deliver proven results has been intensive and continual training. However, this strategy holds an extra workload for the teachers with huge numbers of programming assignments to evaluate in a fair and timely manner. Furthermore, under the current coronavirus (COVID-19) distance teaching circumstances, regular assessment is a fundamental feedback mechanism. It ensures that students engage in learning as well as determines the extent to which they reached the expected learning goals, in this new learning reality. In sum, automating the assessment process will be particularly appreciated by the instructors and highly beneficial to the students. The purpose of this paper is to investigate the feasibility of automatic assessment in the context of computer programming courses. Thus, a prototype based on merging static and dynamic analysis was developed. Empirical evaluation of the proposed grading tool within an introductory C-language course has been presented and compared to manually assigned marks. The outcomes of the comparative analysis have shown the reliability of the proposed automatic assessment prototype. [ABSTRACT FROM AUTHOR] more...
- Published
- 2022
- Full Text
- View/download PDF
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.