Search

Background: Skill integration is vital in students' mastery development and is especially prominent in developing code tracing skills which are foundational to programming, an increasingly important area in the current STEM education. However, instructional design to support skill integration in learning technologies has been limited. Objectives: The current work presents the development and empirical evaluation of instructional design targeting students' difficulties in code tracing particularly in integrating component skills in the Trace Table Tutor (T3), an intelligent tutoring system. Methods: Beyond the instructional features of active learning, step-level support, and individualized problem selection of intelligent tutoring systems (ITS), the instructional design of T3 (e.g., hints, problem types, problem selection) was optimized to target skill integration based on a domain model where integrative skills were represented as combinations of component skills. We conducted an experimental study in a university-level introductory Python programming course and obtained three findings. Results and Conclusions: First, the instructional features of the ITS technology support effective learning of code tracing, as evidenced by significant learning gains (medium-to-large effect sizes). Second, performance data supports the existence of integrative skills beyond component skills. Third, an instructional design focused on integrative skills yields learning benefits beyond a design without such focus, such as improving performance efficiency (medium-to-large effect sizes). Major Takeaways: Our work demonstrates the value of designing for skill integration in learning technologies and the effectiveness of the ITS technology for computing education, as well as provides general implications for designing learning technologies to foster robust learning.

With the increased popularity of electronic textbooks, there is a growing interest in developing a new generation of "intelligent textbooks," which have the ability to guide readers according to their learning goals and current knowledge. Intelligent textbooks extend regular textbooks by integrating machine-manipulable knowledge, and the most popular type of integrated knowledge is a list of relevant concepts mentioned in the textbooks. With these concepts, multiple intelligent operations, such as content linking, content recommendation, or student modeling, can be performed. However, existing automatic keyphrase extraction methods, even supervised ones, cannot deliver sufficient accuracy to be practically useful in this task. Manual annotation by experts has been demonstrated to be a preferred approach for producing high-quality labeled data for training supervised models. However, most researchers in the education domain still consider the concept annotation process as an ad-hoc activity rather than a carefully executed task, which can result in low-quality annotated data. Using the annotation of concepts for the "Introduction to Information Retrieval" textbook as a case study, this paper presents a knowledge engineering method to obtain reliable concept annotations. As demonstrated by the data we collected, the inter-annotator agreement gradually increased along with our procedure, and the concept annotations we produced led to better results in document linking and student modeling tasks. The contributions of our work include a validated knowledge engineering procedure, a codebook for technical concept annotation, and a set of concept annotations for the target textbook, which could be used as a gold standard in further intelligent textbook research.

Online learning systems allow learners to freely access learning contents and record their interactions throughout their engagement with the content. By using data mining techniques on the student log data of those systems, it is possible to examine learning behavior and reveal navigation patterns through learning contents. This study was aimed at investigating how learners' characteristics affect their online learning behavior by using sequential pattern mining and clustering techniques. Participants were 74 undergraduate students from a public university, whose interactions with four different kinds of learning contents in an online learning system were collected. Results showed significant differences in sequential patterns of learners with different cognitive characteristics.

Over the last 10 years, learning analytics have provided educators with both dashboards and tools to understand student behaviors within specific technological environments. However, there is a lack of work to support educators in making data-informed design decisions when designing a blended course and planning appropriate learning activities. In this paper, we introduce knowledge-based design analytics that uncover facets of the learning activities that are being created. A knowledge-based visualization is integrated into edCrumble, a (blended) learning design authoring tool. This new approach is explored in the context of a higher education programming course, where instructors design labs and home practice sessions with online smart learning content on a weekly basis. We performed a within-subjects user study to compare the use of the design tool both with and without visualization. We studied the differences in terms of cognitive load, controllability, confidence and ease of choice, design outcomes, and user actions within the system to compare both conditions with the objective of evaluating the impact of using design analytics during the decision-making phase of course design. Our results indicate that the use of a knowledge-based visualization allows the teachers to reduce the cognitive load (especially in terms of mental demand) and that it facilitates the choice of the most appropriate activities without affecting the overall design time. In conclusion, the use of knowledge-based design analytics improves the overall learning design quality and helps teachers avoid committing design errors.

The increasing popularity of digital textbooks as a new learning media has resulted in a growing interest in developing a new generation of "adaptive textbooks" that can help readers to learn better through adapting to the readers' learning goals and the current state of knowledge. These adaptive textbooks are most frequently powered by internal knowledge models, which associate a list of unique domain knowledge concepts with each section of the textbook. With this kind of concept-level knowledge representation, a number of intelligent operations could be performed, which include student modeling, adaptive navigation support, and content recommendation. However, manual indexing of each textbook section with concepts is challenging, time-consuming, and prone to errors. Modern research in the area of natural language processing offers an attractive alternative, called "automatic keyphrase extraction." While a range of keyphrase and concept extraction methods have been developed over the last twenty years, few of the known approaches were applied and evaluated in a textbook context. In this paper, we present FACE, a supervised feature-based machine learning method for automatic concept extractions from digital textbooks. This method has been created for building domain and student models that form the core of intelligent textbooks. We evaluated FACE on a newly constructed full-scale dataset by assessing how well it approximates concept annotations produced by human experts and how well it supports the needs of student modeling. The results show that FACE outperforms several state-of-the-art keyphrase extraction methods.

With the wide expansion of distributed learning environments the way we learn became more diverse than ever. This poses an opportunity to incorporate different data sources of learning traces that can offer broader insights into learner behavior and the intricacies of the learning process. We argue that combining analytics across different e-learning systems can potentially measure the effectiveness of learning designs and maximize learning opportunities in distributed settings. As a step toward this goal, in this study, we considered how to broaden the context of a single learning environment into a learning ecosystem that integrates three separate e-learning systems. We present a cross-platform architecture that captures, integrates, and stores learning-related data from the learning ecosystem. To demonstrate the feasibility and the benefits of cross-platform architecture, we used regression and classification techniques to generate interpretable models with analytics that can be relevant for instructors in understanding learning behavior and sensemaking of the instructional method on learner performance. The results show that combining data across three e-learning systems improve the classification accuracy compared to data from a single learning system by a factor of 5. This article highlights the value of cross-platform learning analytics and presents a springboard for the creation of new cross-system data-driven research practices.

Purpose: Interest is currently growing in open social learner modeling (OSLM), which means making peer models and a learner's own model visible to encourage users in e-learning. The purpose of this study is to examine students' views about the OSLM in an e-learning system. Design/methodology/approach: This case study was conducted with 40 undergraduate students enrolled in advanced programming and database management system courses. A Likert-type questionnaire and open-ended questions were used to obtain the students' views. System usage data were also analyzed to ensure the richness and diversity of the overall data set. Findings: The quantitative data of the students' views were analyzed with descriptive statistics; the results are presented as graphics. The qualitative data of the students' views were examined by content analysis to derive themes. These themes are organized into four subtopics: the students' positive views, their negative views, their improvement suggestions and their preferences about using similar OSLM visualizations in other e-learning systems. The students' subjective views are discussed in the context of their recorded interactions with the system. Research limitations/implications: Competition due to seeing peer models was considered by participants both as positive and negative features of the learning system. So, this study revealed that, the ways to combine peer learner models to e-learning systems that promote positive competition without resulting social pressure, still need to be explored. Practical implications: By combining open learner models with open peer models, OSLM enhances the learning process in three different ways: it supports self-regulation, encourages competition and empowers self-evaluation. To take advantage of these positive contributions, practitioners should consider enhancing elearning systems with both own learner and peer model features. Originality/value: Despite increasing interest in OSLM studies, several limitations and problems must be addressed such as sparsity of data and lack of study of different contexts and cultures. To date, no published study in this area exists in Turkey. The purpose of this study is to fill this gap by examining OSLM features in an e-learning system from the perspectives of Turkish students by using both their system interaction data and their subjective views.

This research is focused on how to support students' acquisition of program construction skills through worked examples. Although examples have been consistently proven to be valuable for student's learning, the learning technology for computer science education lacks program construction examples with interactive elements that could engage students. The goal of this work is to investigate the value of the "engaging" features in programming examples. We introduce PCEX, an online tool developed to present program construction examples in an engaging fashion. We also present the results of a controlled study with a between-subject design that was conducted in a large introductory Python programming class to compare PCEX with non-interactive worked examples focused on program construction. The results of our study show the positive impact of interactive program construction examples on student's engagement, problem-solving performance, and learning.

Research has demonstrated that people generally think both their knowledge and performance levels are greater than they are. Although several studies have suggested that knowledge and progress visualization offered by open learner modeling (OLM) technology might influence students' self-awareness in a positive way, insufficient evidence exists to show that this is the case. This paper examines the effects of open learner modeling and its extension with social comparison features, known as open social learner modeling (OSLM), on students' knowledge monitoring abilities. We report the results of two semester-long classroom studies, using subjects who were undergraduate and graduate students in Java Programming and Database Management courses at the University of Pittsburgh. During their studies, the students were able to use different versions of an online practice system equipped with both OLM and OSLM. The students' knowledge monitoring abilities were examined in two ways: through absolute and relative assessments. According to the results, although in both OLM and OSLM groups the students' absolute knowledge monitoring ability increased during the semester-long study, relative self-assessment ability (i.e., their ability to compare their own knowledge levels with the knowledge levels of their peers) only increased in the OSLM group. The authors also traced relationships between the students' academic achievement and their absolute and relative knowledge monitoring abilities.