Search

This paper surveys the state of the art on prediction in MOOCs through a systematic literature review (SLR). The main objectives are: first, to identify the characteristics of the MOOCs used for prediction, second, to describe the prediction outcomes, third, to classify the prediction features, fourth, to determine the techniques used to predict the variables, and, fifth, to identify the metrics used to evaluate the predictive models. Results show there is strong interest in predicting dropouts in MOOCs. A variety of predictive models are used, though regression and support vector machines stand out. There is also wide variety in the choice of prediction features, but clickstream data about platform use stands out. Future research should focus on developing and applying predictive models that can be used in more heterogeneous contexts (in terms of platforms, thematic areas, and course durations), on predicting new outcomes and making connections among them (e.g., predicting learners' expectancies), on enhancing the predictive power of current models by improving algorithms or adding novel higher-order features (e.g., efficiency, constancy, etc.).

One of the reported methods of cheating in online environments in the literature is CAMEO (Copying Answers using Multiple Existences Online), where harvesting accounts are used to obtain correct answers that are later submitted in the master account which gives the student credit to obtain a certificate. In previous research, we developed an algorithm to identify and label submissions that were cheated using the CAMEO method; this algorithm relied on the IP of the submissions. In this study, we use this tagged sample of submissions to i) compare the influence of student and problems characteristics on CAMEO and ii) build a random forest classifier that detects submissions as CAMEO without relying on IP, achieving sensitivity and specificity levels of 0.966 and 0.996, respectively. Finally, we analyze the importance of the different features of the model finding that student features are the most important variables towards the correct classification of CAMEO submissions, concluding also that student features have more influence on CAMEO than problem features.

One of the characteristics of massive open online courses (MOOCs) is that the overall number of social interactions tend to be higher than in traditional courses, hindering the analysis of social learning. Learners typically ask or answer questions using the forum. This makes messages a rich source of information, which can be used to infer learners' behavior and outcomes. It is not feasible for teachers to process all forum messages and automated tools and analysis are required. Although there are some tools for analyzing learners' interactions, there is a need for methodologies and integrated tools that help to interpret the learning process based on social interactions in the forum. This paper presents the 3S (Social, Sentiments, Skills) learning analytics methodology for analyzing forum interactions in MOOCs. This methodology considers a temporal analysis combining the social, sentiments, and skill dimensions that can be extracted from forum data. We also introduce LAT[turned E]S, a learning analytics tool for edX/Open edX related to the three dimensions (3S), which includes visualizations to guide the proposed methodology. We apply the 3S methodology and the tool to an MOOC on Java programming. Results showed, among others, the action-reaction effect produced when learners increase their participation after instructor's events. Moreover, a decrease of positive sentiments over time and before deadlines of open-ended assignments was also observed and that there were certain skills, which caused more troubles (e.g., arrays and loops). These results acknowledge the importance of using a learning analytics methodology to detect problems in MOOCs.

Educational computer-based competition environments need to be designed with a set of features that enhance the learning process. Although recently some frameworks for the design of educational computer-based systems (e.g., educational games) have been proposed, they do not focus on the details of the competition. Therefore, the design of educational computer-based competition environments is still an open issue. We propose the 4NESS framework for the design of such competition environments. This framework classifies the competition features into four dimensions: easiness, fairness, awareness, and adaptiveness. We designed the ISCARE educational computer-based competition environment implementing these four dimensions to evaluate and validate some included features for competition, and conducted experiments during three consecutive years. Students considered the computer-based competition environment with the proposed features, including a modification of the Swiss system, to be very easy to use, fair for pairing students, moderately fair for calculating scores, with good awareness of their state during the competition and that of their peers, and with contents of appropriate level, especially for the group working with adaptive contents. Adaptation of contents made students increase their learning gains in 0.5 sigmas during the competition. Pairings were seen as with students of similar levels. And final scores were moderately fair: while the order tends to be similar as compared to the results in the post-test, students grades with high performance tend to be lower in the competition environment.

One of the characteristics of massive open online courses (MOOCs) is that the overall number of social interactions tend to be higher than in traditional courses, hindering the analysis of social learning. Learners typically ask or answer questions using the forum. This makes messages a rich source of information, which can be used to infer learners’ behavior and outcomes. It is not feasible for teachers to process all forum messages and automated tools and analysis are required. Although there are some tools for analyzing learners’ interactions, there is a need for methodologies and integrated tools that help to interpret the learning process based on social interactions in the forum. This paper presents the 3S (Social, Sentiments, Skills) learning analytics methodology for analyzing forum interactions in MOOCs. This methodology considers a temporal analysis combining the social, sentiments, and skill dimensions that can be extracted from forum data. We also introduce LATƎS, a learning analytics tool for edX/Open edX related to the three dimensions (3S), which includes visualizations to guide the proposed methodology. We apply the 3S methodology and the tool to an MOOC on Java programming. Results showed, among others, the action–reaction effect produced when learners increase their participation after instructor's events. Moreover, a decrease of positive sentiments over time and before deadlines of open-ended assignments was also observed and that there were certain skills, which caused more troubles (e.g., arrays and loops). These results acknowledge the importance of using a learning analytics methodology to detect problems in MOOCs. [ABSTRACT FROM AUTHOR]

This paper surveys the state of the art on prediction in MOOCs through a systematic literature review (SLR). The main objectives are: first, to identify the characteristics of the MOOCs used for prediction, second, to describe the prediction outcomes, third, to classify the prediction features, fourth, to determine the techniques used to predict the variables, and, fifth, to identify the metrics used to evaluate the predictive models. Results show there is strong interest in predicting dropouts in MOOCs. A variety of predictive models are used, though regression and support vector machines stand out. There is also wide variety in the choice of prediction features, but clickstream data about platform use stands out. Future research should focus on developing and applying predictive models that can be used in more heterogeneous contexts (in terms of platforms, thematic areas, and course durations), on predicting new outcomes and making connections among them (e.g., predicting learners' expectancies), on enhancing the predictive power of current models by improving algorithms or adding novel higher-order features (e.g., efficiency, constancy, etc.). [ABSTRACT FROM AUTHOR]

One of the reported methods of cheating in online environments in the literature is CAMEO (Copying Answers using Multiple Existences Online), where harvesting accounts are used to obtain correct answers that are later submitted in the master account which gives the student credit to obtain a certificate. In previous research, we developed an algorithm to identify and label submissions that were cheated using the CAMEO method; this algorithm relied on the IP of the submissions. In this study, we use this tagged sample of submissions to i) compare the influence of student and problems characteristics on CAMEO and ii) build a random forest classifier that detects submissions as CAMEO without relying on IP, achieving sensitivity and specificity levels of 0.966 and 0.996, respectively. Finally, we analyze the importance of the different features of the model finding that student features are the most important variables towards the correct classification of CAMEO submissions, concluding also that student features have more influence on CAMEO than problem features. [ABSTRACT FROM AUTHOR]

Educational computer-based competition environments need to be designed with a set of features that enhance the learning process. Although recently some frameworks for the design of educational computer-based systems (e.g., educational games) have been proposed, they do not focus on the details of the competition. Therefore, the design of educational computer-based competition environments is still an open issue. We propose the 4NESS framework for the design of such competition environments. This framework classifies the competition features into four dimensions: easiness, fairness, awareness, and adaptiveness. We designed the ISCARE educational computer-based competition environment implementing these four dimensions to evaluate and validate some included features for competition, and conducted experiments during three consecutive years. Students considered the computer-based competition environment with the proposed features, including a modification of the Swiss system, to be very easy to use, fair for pairing students, moderately fair for calculating scores, with good awareness of their state during the competition and that of their peers, and with contents of appropriate level, especially for the group working with adaptive contents. Adaptation of contents made students increase their learning gains in 0.5 sigmas during the competition. Pairings were seen as with students of similar levels. And final scores were moderately fair: while the order tends to be similar as compared to the results in the post-test, students grades with high performance tend to be lower in the competition environment. [ABSTRACT FROM AUTHOR]

Self-regulated learning (SRL) environments provide students with activities to improve their learning (e.g., by solving exercises), but they might also provide optional activities (e.g., changing an avatar image or setting goals) where students can decide whether they would like to use or do them and how. Few works have dealt with the use of optional activities in SRL environments. This paper thus analyzes the use of optional activities in two case studies with a SRL approach. We found that the level of use of optional activites was low with only 23.1 percent of students making use of some functionality, while the level of use of learning activities was higher. Optional activities which are not related to learning are used more. We also explored the behavior of students using some of the optional activities in the courses such as setting goals and voting comments, finding that students finished the goals they set in more than 50 percent of the time and that they voted their peers’ comments in a positive way. We also found that gender and the type of course can influence which optional activities are used. Moreover, the relations of the use of optional activities with proficient exercises and learning gains is low when taking out third variables, but we believe that optional activities might motivate students and produce better learning in an indirect way. [ABSTRACT FROM PUBLISHER]

A lack of student motivation is a problem in many courses in electrical engineering. Introducing competition between students can enhance their motivation, but it can also generate negative emotions. This paper presents an empirical study of students in a telecommunications engineering degree; it measured their level of motivation, and their emotions about competition in education in general, and in particular about the Information System for Competition based on pRoblem solving in Education (ISCARE). ISCARE makes adaptive matches, pairing students who share a similar knowledge level. The study results show that students had moderate levels of motivation and experienced no negative emotions while using ISCARE. Under certain circumstances, therefore, competition in education, using technology systems that provide competition, can produce motivation without generating negative emotions. Female students and students with low ISCARE scores were less motivated and experienced more negative emotions. In addition, motivation and emotions toward ISCARE were more positive than they had been toward competition in general. [ABSTRACT FROM AUTHOR]

Teaching electrical and computer software engineers how to configure network services normally requires the detailed presentation of many configuration commands and their numerous parameters. Students tend to find it difficult to maintain acceptable levels of motivation. In many cases, this results in their not attending classes and not dedicating enough time to acquire the required competencies. This paper describes a successful experiment that was able to improve student class attendance, performance, and motivational patterns by using mobile devices in class to provide students with contextualized learning pills. A learning pill is a simple exercise that summarizes some of the key concepts explained in class and promotes reflection and self-study. Each student attending a particular lecture or laboratory session automatically receives a learning pill on his or her mobile phone just after the related concept has been presented. This paper details the scenario used in the experiment and analyzes the performance and motivational patterns of 170 students at the Higher Polytechnic School, Carlos III University of Madrid, Leganes, Spain. [ABSTRACT FROM PUBLISHER]

This paper reports the results of an experiment in a Computer Architecture Laboratory course classroom session, in which students were divided into two groups for interaction both with a hinting e-learning system and with human teachers generating hints. The results show that there were high learning gains for both groups, demonstrating the effectiveness of the human teachers as well as of the computer-based hinting e-learning system even without the use of adaptive and personalization capabilities. In addition, in the worst case, the difference in favor of human teachers (with a low student-to-teacher ratio of 13.5 students per teacher) would not be significant with respect to the e-learning system, so the computer-based system can replace teachers without a significant loss of effectiveness. [ABSTRACT FROM AUTHOR]

The use of technology in learning environments should be targeted at improving the learning outcome of the process. Several technology enhanced techniques can be used for maximizing the learning gain of particular students when having access to learning resources. One of them is content adaptation. Adapting content is especially important when using limited devices such as mobile phones. Content can be adapted to restrictive network conditions, to limited terminal capabilities, to different user preferences and learning styles or to external elements in the learning and user contexts. This paper studies and analyzes the impact of modifying and therefore adapting the speed of reproduction of audio content in mobile phones on the learning gain of the user. The paper shows that the optimum speed for Spanish audio is around 206 words per minute. The impact of enhancing the audio reproduction by presenting an equivalent version in text displayed on the screen of the mobile phone is also studied. The paper concludes that this text is only important for students younger than 15. The paper analyzes data from 100 Spanish speaking users that are grouped according to different criteria, such as gender, age, or level of studies. The results are presented and discussed. [ABSTRACT FROM PUBLISHER]