Search

While open-ended self-explanations have been shown to promote robust learning in multiple studies, they pose significant challenges to automated grading and feedback in technology-enhanced learning, due to the unconstrained nature of the students' input. Our work investigates whether recent advances in Large Language Models, and in particular ChatGPT, can address this issue. Using decimal exercises and student data from a prior study of the learning game Decimal Point, with more than 5,000 open-ended self-explanation responses, we investigate ChatGPT's capability in (1) solving the in-game exercises, (2) determining the correctness of students' answers, and (3) providing meaningful feedback to incorrect answers. Our results showed that ChatGPT can respond well to conceptual questions, but struggled with decimal place values and number line problems. In addition, it was able to accurately assess the correctness of 75% of the students' answers and generated generally high-quality feedback, similar to human instructors. We conclude with a discussion of ChatGPT's strengths and weaknesses and suggest several venues for extending its use cases in digital teaching and learning., Comment: Be accepted as a Research Paper in 18th European Conference on Technology Enhanced Learning

ITS authoring tools make creating intelligent tutoring systems more cost effective, but few authoring tools make it easy to flexibly incorporate an open-ended range of student modeling methods and learning analytics tools. To support a cumulative science of student modeling and enhance the impact of real-world tutoring systems, it is critical to extend ITS authoring tools so they easily accommodate novel student modeling methods. We report on extensions to the CTAT/Tutorshop architecture to support a plug-in approach to extensible student modeling, which gives an author full control over the content of the student model. The extensions enhance the range of adaptive tutoring behaviors that can be authored and support building external, student- or teacher-facing real-time analytics tools. The contributions of this work are: (1) an open architecture to support the plugging in, sharing, re-mixing, and use of advanced student modeling techniques, ITSs, and dashboards; and (2) case studies illustrating diverse ways authors have used the architecture.

ITS authoring tools make creating intelligent tutoring systems more cost effective, but few authoring tools make it easy to flexibly incorporate an open-ended range of student modeling methods and learning analytics tools. To support a cumulative science of student modeling and enhance the impact of real-world tutoring systems, it is critical to extend ITS authoring tools so they easily accommodate novel student modeling methods. We report on extensions to the CTAT/Tutorshop architecture to support a plug-in approach to extensible student modeling, which gives an author full control over the content of the student model. The extensions enhance the range of adaptive tutoring behaviors that can be authored and support building external, student- or teacher-facing real-time analytics tools. The contributions of this work are: (1) an open architecture to support the plugging in, sharing, re-mixing, and use of advanced student modeling techniques, ITSs, and dashboards; and (2) case studies illustrating diverse ways authors have used the architecture.

Recent student knowledge modeling algorithms such as Deep Knowledge Tracing (DKT) and Dynamic Key-Value Memory Networks (DKVMN) have been shown to produce accurate predictions of problem correctness within the same learning system. However, these algorithms do not attempt to directly infer student knowledge. In this paper we present an extension to these algorithms to also infer knowledge. We apply this extension to DKT and DKVMN, resulting in knowledge estimates that correlate better with a posttest than knowledge estimates from Bayesian Knowledge Tracing (BKT), an algorithm designed to infer knowledge, and another classic algorithm, Performance Factors Analysis (PFA). We also apply our extension to correctness predictions from BKT and PFA, finding that knowledge estimates produced with it correlate better with the posttest than BKT and PFA's standard knowledge estimates. These findings are significant since the primary aim of education is to prepare students for later experiences outside of the immediate learning activity., Comment: 8 pages, accepted to the International Conference on Computers in Education

When teaching at scale in the physical classroom or online classroom of a MOOC, the scarce resource of personal instructor communication becomes a differentiating factor between the quality of learning experience available in smaller classrooms. In this paper, through real-time predictive modeling of engagement analytics, we augment a MOOC platform with personalized communication affordances, allowing the instructional staff to direct communication to learners based on individual predictions of three engagement analytics. The three model analytics are the current probability of earning a certificate, of submitting enough materials to pass the class, and of leaving the class and not returning. We engineer an interactive analytics interface in edX which is populated with real-time predictive analytics from a backend API service. The instructor can target messages to, for example, all learners who are predicted to complete all materials but not pass the class. Our approach utilizes the state-of-the-art in recurrent neural network classification, evaluated on a MOOC dataset of 20 courses and deployed in one. We provide evaluation of these courses, comparing a manual feature engineering approach to an automatic feature learning approach using neural networks. Our provided code for the front-end and back-end allows any instructional team to add this personalized communication dashboard to their edX course granted they have access to the historical clickstream data from a previous offering of the course, their course’s daily provided log data, and an external machine to run the model service API.

Accurately interpreting student responses is a critical requirement of dialog-based intelligent tutoring systems. The accuracy of supervised learning methods, used for interpreting or analyzing student responses, is strongly dependent on the availability of annotated training data. Collecting and grading student responses is tedious, time-consuming, and expensive. This work proposes an iterative data collection and grading approach. We show that data collection efforts can be significantly reduced by predicting question difficulty and by collecting answers from a focused set of students. Further, grading efforts can be reduced by filtering student answers that may not be helpful in training Student Response Analyzer (SRA). To ensure the quality of grades, we analyze the grader characteristics, and show improvement when a biased grader is removed. An experimental evaluation on a large scale dataset shows a reduction of up to 28% in the data collection cost, and up to 10% in grading cost while improving the response analysis macro-average F1.

Automatic short answer grading for Intelligent Tutoring Systems has attracted much attention of the researchers over the years. While the traditional techniques for short answer grading are rooted in statistical learning and hand-crafted features, recent research has explored sentence embedding based techniques. We observe that sentence embedding techniques, while being effective for grading in-domain student answers, may not be best suited for out-of-domain answers. Further, sentence embeddings can be affected by non-sentential answers (answers given in the context of the question). On the other hand, token level hand-crafted features can be fairly domain independent and are less affected by non-sentential forms. We propose a novel feature encoding based on partial similarities of tokens (Histogram of Partial Similarities or HoPS), its extension to part-of-speech tags (HoPSTags) and question type information. On combining the proposed features with sentence embedding based features, we are able to further improve the grading performance. Our final model achieves better or competitive results in experimental evaluation on multiple benchmarking datasets and a large scale industry dataset.

When teaching at scale in the physical classroom or online classroom of a MOOC, the scarce resource of personal instructor communication becomes a differentiating factor between the quality of learning experience available in smaller classrooms. In this paper, through real-time predictive modeling of engagement analytics, we augment a MOOC platform with personalized communication affordances, allowing the instructional staff to direct communication to learners based on individual predictions of three engagement analytics. The three model analytics are the current probability of earning a certificate, of submitting enough materials to pass the class, and of leaving the class and not returning. We engineer an interactive analytics interface in edX which is populated with real-time predictive analytics from a backend API service. The instructor can target messages to, for example, all learners who are predicted to complete all materials but not pass the class. Our approach utilizes the state-of-the-art in recurrent neural network classification, evaluated on a MOOC dataset of 20 courses and deployed in one. We provide evaluation of these courses, comparing a manual feature engineering approach to an automatic feature learning approach using neural networks. Our provided code for the front-end and back-end allows any instructional team to add this personalized communication dashboard to their edX course granted they have access to the historical clickstream data from a previous offering of the course, their course’s daily provided log data, and an external machine to run the model service API.

Teacher education providers have an important role to play in preparing the next generation of teachers to be able to use ICT in their future teaching practice. Government reports and research have consistently argued that graduates are ill-prepared in their use of ICT [1]. In Australia an increasing regulatory environment means that teacher education providers need to design programs that target the attainment of specific ICT professional standards. To contribute to the discussion of ICT challenges in teacher education, this small-scale study investigated where 69 pre-service teachers acquired the ICT skills/resources commonly used in primary and secondary schools. Findings suggest that many of the generalized skills/resources were learned in their everyday life prior to undertaking their teaching program, and that they acquired a lesser number through their university coursework and/or professional experience placements. A number of implications for teacher education conclude this paper.

For some time, Teacher Education policy and research has consistently argued that graduates are ill prepared to use ICT in their practice (TEMAG, 2015). In Australia, an increasing regulatory environment means that Teacher Education providers need to meet national accreditation demands as well as design programs that address professional standards for graduates where an effective ICT use is a requirement. In an effort to have greater understanding of how to design Teacher Education programs that meet these challenges, this small scale study investigated where 69 pre-service teachers learned how to use a number of ICT resources commonly used in primary and secondary schools. Findings suggest that they learned how to use many resources (particularly general-type resources) in their everyday life prior to undertaking their teaching qualification and that they learned how to use a lesser number in university coursework or practicum in schools. A number of implications for Teacher Education conclude this paper

This article, created by Bruce J. McLaren and Constance H. McLaren from Indiana State University, describes a dataset containing monthly household electric billing charges for ten years. The data can be used to illustrate graphing, descriptive statistics, correlation, seasonal decomposition, a variety of smoothing methods, ARIMA models, forecasting, and multiple regression. Additionally, the authors have created seven different assignments testing students knowledge of these subjects. This is a great lesson in applied statistics.

Online teamwork is an instructional strategy widely used in education courses to ensure active knowledge construction and deeper learning. There is a challenge for online course designers and technology designers to create group environments that encourage participation, and have the ability to enhance positive attitudes toward group work. It is hypothesised that incorporating cultural factors into the design of teamwork technology has the potential to encourage participation and increase students’ positive attitudes towards group work. This paper looks to do exactly that, although the definition of culture in this paper is limited to the individualism–collectivism dimension. The paper summarises our findings from interviews conducted with lecturers and students who have experience with teamwork. It then presents culturally-related design strategies which are identified from cross-cultural psychology literature and our interviews finding. Finally, it demonstrates how culturally-related design strategies are incorporated into the IdeasRoom prototype design.

Recommender systems in e-learning platforms, can utilise various data about learners in order to provide them with the next best material to study. We build on our previous work, which defines the recommendations in terms of two measures (i.e. relevance and effort) calculated from data of successful students in the previous runs of the courses. In this paper we investigate the impact of students’ socio-demographic factors and analyse how these factors improved the recommendation. It has been shown that education and age were found to have a significant impact on engagement with materials.

In this paper we describe work to investigate the creation of engaging programming learning experiences. Background research informed the design of four fieldwork studies to explore how programming tasks could be framed to motivate learners. Our empirical findings from these four field studies are summarized here, with a particular focus upon one – Whack a Mole – which compared the use of a physical interface with the use of a screen-based equivalent interface to obtain insights into what made for an engaging learning experience. Emotions reported by two sets of participant undergraduate students were analyzed, identifying the links between the emotions experienced during programming and their origin. Evidence was collected of the very positive emotions experienced by learners programming with a physical interface (Arduino) in comparison with a similar program developed using a screen-based equivalent interface. A follow-up study provided further evidence of the motivation of personalized design of programming tangible physical artefacts. Collating all the evidence led to the design of a set of ‘Learning Dimensions’ which may provide educators with insights to support key design decisions for the creation of engaging programming learning experiences.

This paper presents a study developed within the international project JuxtaLearn. This project aims to improve student understanding of threshold concepts by promoting student curiosity and creativity through video creation. The math concept of 'Division', widely referred in the literature as problematic for students, was recognised as a 'Tricky Topic' by teachers with the support of the Tricky Topic Tool and the Problem Distiller tool, two apps developed under the JuxtaLearn project. The methodology was based on qualitative data collected through Think Aloud protocol from a group of teachers of a public Elementary school as they used these tools. Results show that the Problem Distiller tool fostered the teachers to reflect more deeply on the causes of the students’ misunderstandings of that complex math concept. This process enabled them to develop appropriate strategies to help the students overcome these misunderstandings. The results also suggest that the stumbling blocks associated to the Tricky Topic ‘Division’ are similar to the difficulties reported in the literature describing Threshold Concepts. This conclusion is the key issue discussed in this paper and a contribution to the state of the art.

In this paper we describe work to investigate the creation of engaging programming learning experiences. Background research informed the design of four fieldwork studies to explore how programming tasks could be framed to motivate learners. Our empirical findings from these four field studies are summarized here, with a particular focus upon one – Whack a Mole – which compared the use of a physical interface with the use of a screen-based equivalent interface to obtain insights into what made for an engaging learning experience. Emotions reported by two sets of participant undergraduate students were analyzed, identifying the links between the emotions experienced during programming and their origin. Evidence was collected of the very positive emotions experienced by learners programming with a physical interface (Arduino) in comparison with a similar program developed using a screen-based equivalent interface. A follow-up study provided further evidence of the motivation of personalized design of programming tangible physical artefacts. Collating all the evidence led to the design of a set of ‘Learning Dimensions’ which may provide educators with insights to support key design decisions for the creation of engaging programming learning experiences.

Recommender systems in e-learning platforms, can utilise various data about learners in order to provide them with the next best material to study. We build on our previous work, which defines the recommendations in terms of two measures (i.e. relevance and effort) calculated from data of successful students in the previous runs of the courses. In this paper we investigate the impact of students’ socio-demographic factors and analyse how these factors improved the recommendation. It has been shown that education and age were found to have a significant impact on engagement with materials.

This paper presents a study developed within the international project JuxtaLearn. This project aims to improve student understanding of threshold concepts by promoting student curiosity and creativity through video creation. The math concept of 'Division', widely referred in the literature as problematic for students, was recognised as a 'Tricky Topic' by teachers with the support of the Tricky Topic Tool and the Problem Distiller tool, two apps developed under the JuxtaLearn project. The methodology was based on qualitative data collected through Think Aloud protocol from a group of teachers of a public Elementary school as they used these tools. Results show that the Problem Distiller tool fostered the teachers to reflect more deeply on the causes of the students’ misunderstandings of that complex math concept. This process enabled them to develop appropriate strategies to help the students overcome these misunderstandings. The results also suggest that the stumbling blocks associated to the Tricky Topic ‘Division’ are similar to the difficulties reported in the literature describing Threshold Concepts. This conclusion is the key issue discussed in this paper and a contribution to the state of the art.

Online teamwork is an instructional strategy widely used in education courses to ensure active knowledge construction and deeper learning. There is a challenge for online course designers and technology designers to create group environments that encourage participation, and have the ability to enhance positive attitudes toward group work. It is hypothesised that incorporating cultural factors into the design of teamwork technology has the potential to encourage participation and increase students’ positive attitudes towards group work. This paper looks to do exactly that, although the definition of culture in this paper is limited to the individualism–collectivism dimension. The paper summarises our findings from interviews conducted with lecturers and students who have experience with teamwork. It then presents culturally-related design strategies which are identified from cross-cultural psychology literature and our interviews finding. Finally, it demonstrates how culturally-related design strategies are incorporated into the IdeasRoom prototype design.

Reflective writing is widely acknowledged to be one of the most effective learning activities for promoting students’ self-reflection and critical thinking. However, manually assessing and giving feedback on reflective writing is time consuming, and known to be challenging for educators. There is little work investigating the potential of automated analysis of reflective writing, and even less on machine learning approaches which offer potential advantages over rule-based approaches. This study reports progress in developing a machine learning approach for the binary classification of pharmacy students’ reflective statements about their work placements. Four common statistical classifiers were trained on a corpus of 301 statements, using emotional, cognitive and linguistic features from the Linguistic Inquiry and Word Count (LIWC) analysis, in combination with affective and rhetorical features from the Academic Writing Analytics (AWA) platform. The results showed that the Random-forest algorithm performed well (F-score = 0.799) and that AWA features, such as emotional and reflective rhetorical moves, improved performance.