Search

As an innovative evaluation tool, peer assessment is essential in Massive Open Online Courses (MOOCs). In both formative and summative peer assessments in MOOCs, providing reliable feedback is crucial in enhancing learning outcomes. Peer assessment has been highlighted as a reliable tool in both traditional classrooms and small‐scale online learning contexts by a wealth of studies. Factors impacting the result of peer assessment have also been identified. However, such exploration in MOOCs is very limited. Through an examination of 5,722 assignment submissions and 56,794 review scores collected from 18 peer assessments in three courses on Chinese University MOOC platform, this study investigated the inter‐reliability of peer assessment in MOOCs utilizing Krippendorff's alpha and ICC[1]. It also revealed factors affecting peer assessment reliability in MOOCs. The results of this study show that students tended to mark extreme scores, and peer assessment in MOOCs seemed unreliable. In addition, the reliability of the peer assessment in these MOOCs was inversely correlated to the number of reviewers per assignment and reviews completed per reviewer. The assignment type also matters; peer assessment using e‐portfolios format was found more reliable than those using papers and proposals. Suggestions on how to improve the reliability of peer assessments in MOOCs are also provided. Practitioner notesWhat is already known about this topicBoth the formative and summative peer assessment mode, providing reliable feedback is needed, which is of great value for improving the quality of online learning.The reliability and influence factors of peer assessment has been highlighted by a wealth of studies in traditional classrooms and small‐scale online learning contexts.There are still limited explorations in MOOCs.What the paper addsThe students tended to give extreme scores in peer assessment in MOOC, and the peer assessment in MOOC may was not particularly reliable.The Assignment type being rated affected the reliability, and peer assessment using e‐portfolios format was found more reliable than those using papers and proposals.The reliability of peer assessment in MOOC was negative related to the number of reviewers per assignment and the reviews completed per reviewer.Implications for practice and/or policyAvoid using peer assessment as a summative assessment method, and should be assigned relatively low weights to final grades.Using papers and proposals as learning assignments can appropriately reduce the proportion of peer assessment in the final grade.Training on how to identify medium quality work should be provided.Instructional strategies should be designed to encourage learners to actively participate in peer assessments and thoroughly assess. [ABSTRACT FROM AUTHOR]

Situational engagement plays a critical role in promoting students' academic performance. In a smart classroom environment, this study collected longitudinal real‐time data for 105 college students at a university in central China to investigate the relationship among situational engagement, personal characteristics and learning environment perceptions. Hierarchical linear modelling showed that environmental perception and students' personal factors have different effects on situational engagement. Specifically, (1) social support perceptions, autonomous motivation and controlled motivation have a significant impact on behavioural engagement; (2) perceptions of social and media support, autonomous motivation and controlled motivation have a significant impact on shallow cognitive engagement; and (3) perceptions of teacher and social support, self‐efficacy and autonomous motivation significantly predict deep cognitive and emotional engagement. This study suggests that the effect of the perception regarding advanced technology‐supported learning environments on students' situational engagement is limited, and instructors should pay more attention to improving students' perceptions of teacher and social support, self‐efficacy and autonomous motivation to promote students' situational deep cognitive engagement in smart classrooms. Practitioner notesWhat is already known about this topic Compared with overall engagement, situational engagement fluctuates and changes with time and context.Situational engagement is a product of environmental and personal factors.Few studies have focused on the nature of situational engagement and how environmental and personal factors influence situational engagement in smart classrooms.What the paper adds This study contributes to the existing literature by investigating the critical factors that predict situational engagement, using the experience sampling method in a smart classroom at a Chinese university.Environmental perception, self‐efficacy and students' motivation factors have different effects on situational engagement in a smart classroom.Perceptions of teacher and social support, self‐efficacy and autonomous motivation significantly predict deep cognitive and emotional engagement, while perceptions of media support only have a significant impact on shallow cognitive engagement.Personal factors (controlled and autonomous motivation) moderate the relationship between environmental perception factors and situational engagement.Implications for practice and/or policy Rather than only providing external technology‐rich conditions, instructors should focus more on improving students' perceptions of teacher and social support, self‐efficacy and autonomous motivation in the smart classroom environment.Instructors should promote students' perception of teacher support and their autonomous motivation to enhance their deep cognitive engagement. [ABSTRACT FROM AUTHOR]

Computational thinking (CT), which is a cognitive skill used to solve problems with computational solutions, has drawn increasing attention among researchers and practitioners due to the growing recognition of CT competence as a 21st century skill. Collaboration is commonly integrated into CT education to facilitate novice learning, but there is inadequate knowledge regarding the influences of collaboration in CT education. This meta‐analysis examined the overall effects on the cognitive, social and affective competencies of collaborative versus individual problem solving in CT through programming. We identified 33 publications involving 4717 learners, which allowed for 220 effect size comparisons. We found a medium effect size (Hedges' g = 0.562; p < 0.001) in favour of collaborative problem solving on cognitive learning outcomes and a small effect size (Hedges' g = 0.316; p < 0.01) on affective learning outcomes using a random‐effects model. Categorical moderator analysis revealed the moderating effects of educational level, programming environment, study duration, grouping method and group size. The competency model that was generated from the synthesized literature on collaborative computational problem solving yielded insights into the learning design and assessment of relevant activities. Practitioner notesWhat is already known about the topic?Although professional programming usually requires collaboration to accomplish common goals in the work environment, programming is predominantly taught on an individual basis in educational settings.Some intervention studies have reported inconsistent learning outcomes (positive and negative) from integrating collaborative problem solving (CPS) into programming education.There is a need for further scrutiny and exploration of the role of collaboration in the process of learning to program.What does this paper add?Our meta‐analysis showed that CPS is an effective pedagogical tool for developing students' cognitive, affective and social competencies.The moderator analysis revealed that CPS in programming has cognitive benefits irrespective of grouping method, group size and educational levels; however, there is a need for caution in regard to designing collaborative problem‐solving activities for different programming environments and durations.The model generated from the literature synthesis can yield insights into better assessing and designing collaborative problem‐solving activities.Implications for practice and/or policy?Integrating CPS into programming education can prepare students with 21st century skills.Social and affective competencies are crucial to students' learning capacity, which is an issue that needs to be better assessed.Future studies could look more rigorously into the effects of CPS in visual programming and robotics programming among younger age groups, such as primary or middle school students. [ABSTRACT FROM AUTHOR]

Learning analytics (LA) has been widely adopted in research on education. However, most studies in the area have conducted LA after computer‐supported collaborative learning (CSCL) activities rather than during CSCL. To address this problem, this study proposed a LA‐based real‐time feedback approach based on a deep neural network model to improve CSCL performance. In total, 72 university students participated in the study and were randomly assigned to an experimental or control group. The students in the experimental group learned with the LA‐based real‐time feedback approach, whereas the students in the control group learned with the conventional online collaborative learning approach. To analyse the data, both quantitative and qualitative methods were adopted. The results indicated that the LA‐based real‐time feedback approach significantly promoted knowledge convergence, knowledge elaboration, interactive relationships and group performance. The interview results also confirmed the effectiveness of the proposed approach. Practitioner notesWhat is already known regarding this topic Learning analytics (LA) has been widely used in education.Most studies in the area have presented LA results only after collaborative learning and have lacked real‐time analysis and feedback.What this paper adds A LA‐based real‐time feedback approach was proposed and validated in the computer‐supported collaborative learning (CSCL) context.The experimental results indicated that the LA‐based real‐time feedback approach significantly promoted knowledge elaboration, knowledge convergence, interactive relationships and group performance.Implications for practice and/or policy To shed light on progress in CSCL, real‐time LA are recommended.Deep neural network models, such as bidirectional encoder representations from transformers, can be adopted to automatically analyse online discussion transcripts.Real‐time feedback based on LA results can promote CSCL performance. [ABSTRACT FROM AUTHOR]

Designing reflective activities for online video learning could enhance learning outcomes and, to some extent, influence learning processes (ie, attentional processes and emotional experiences). It is necessary to select different levels of reflective activities in online video learning according to learning processes and outcomes. This study examined to what extent different levels of reflection may affect the learning outcomes, and further explored the impact of reflection on learning processes. Participants, comprising 60 undergraduates, were randomly and equally assigned to two different groups: a reflection group and a critical reflection group, and each group member completed specific learning tasks. The results of the learning outcomes showed that critical reflection led to better transfer performance than reflection, but retention performance was the same in both conditions. Regarding process measures, eye‐tracking data indicated that the embedding of critical reflection strategies facilitated learners' visual attention, and controlled the difficulty of processing. Meanwhile, physiological data showed that the learners' reflection was accompanied by stress, and there was no significant difference between the two levels of reflection. Practitioner notesWhat is already known about this topicEmbedding reflections within videos is an effective instructional strategy for online learning which can facilitate students' deep learning.Recent studies have evaluated the effectiveness of reflection mostly through its impact on learning outcomes, yet focusing on the process of reflection is equally valuable.How to choose the level of reflection is critical and challenging. Whether it should combine the processes (including attentional processes and emotional experiences) and outcomes of reflection can be addressed through experimental research.What this paper addsSelecting different levels of reflection (ie, reflection and critical reflection) according to Kember's model and suggesting strategies (ie, retrospective interpretation reports and evaluation reports of problem solving) to guide reflection.The experimental design combined outcomes and processes in order to identify the most appropriate level of reflection in the context of video learning, and both offline and online measurements were taken (including eye tracking and physiological measures).Critical reflection leads to superior learning outcomes and has advantages for attentional processes. Additionally, reflection itself is the main factor influencing participants' stress, and there is no significant difference between the levels of reflection.Implications for practice and/or policyProviding guidance for different levels of reflection (ie, retrospective interpretation reports and evaluation report of problem solving), which helps inform the design of instructional videos and foster learners' reflective awareness.Educators should focus not only on the learning outcomes caused by reflection but also on the learning processes of reflection and the emotional experiences that accompany it, in order to choose an appropriate level of reflection.Whether the teaching goal is knowledge retention or transfer, educators should embed a high level of reflection for learners in instructional videos to promote deep thinking and processing. [ABSTRACT FROM AUTHOR]

Studying the networked nature of social and cognitive aspects of learner interactions is the key to understanding how successful collaborative learning occurs in asynchronous online discussion forums (AODFs). Guided by network science and multiplex network analysis, this study compared the differences of network structure and properties between the social (learners as nodes and commenting on the others' contributions as edges) and cognitive (learners as nodes and explicitly quoting the others' contributions as edges) networks within an AODF. It additionally examined the differences of individual measures between these two monolayer networks and their integrated two multiplex networks. The two multiplex networks were respectively framed in superimposed and unfold approaches. Moreover, correlation analysis was conducted to examine the relationships between individual measures and learning performance across the four networks. Results showed key differences at the network and individual levels between the social and cognitive networks. The two networks had different compositions of participants, and students in the cognitive network occupied more central positions in general. Besides, certain individual measures in the multiplex networks were relatively higher and more related to learning performance than those in the monolayer networks. These results indicate that integrating multi‐aspect information of collaboration may be more conducive to unveiling learners' interaction patterns in asynchronous online discussions. Practitioner notesWhat is already known about this topic? Studying the networked nature of the social and cognitive aspects of learner interactions is the key to understanding how successful collaborative learning occurs.Social network analysis and content analysis are two commonly used methods to analyse learner interactions in asynchronous online discussions.There is a lack of research examining learners' overall interaction patterns by integrating social and cognitive interactions.What this paper adds? This study framed cognitive interactions as a networked phenomenon to unveil the way learners participate in the cognitive aspect of learner interactions.Multiplex network analysis (MNA) was introduced to integrate social and cognitive interactions to understand learners' overall interaction patterns.Integrating social and cognitive interactions led to a more closer relationship between learner interactions and learning performance.Implications for practice and/or policy? The proposed approach for framing the cognitive network and the introduced approach of MNA could be jointly employed to discern learners' overall interaction pattern in asynchronous online discussions.Course facilitators and instructors should be aware of the inconsistencies in social and cognitive interactions and then simultaneously examine them so as to acquire an accurate understanding of learners' diverse interactions.Pedagogical strategies for simultaneously facilitating social and cognitive interactions could be used to better improve learning performance. [ABSTRACT FROM AUTHOR]

A thorough understanding of how ICT support provided by teacher preparation programmes influences preservice teachers' technological pedagogical content knowledge (TPACK) has become increasingly essential. This study aimed at exploring the mediating role of ICT perception, perceived ICT competence and ICT self‐efficacy between university ICT support and preservice teachers' TPACK. Data were collected through an online self‐assessment questionnaire translated and adapted from existing studies and verified with confirmatory factor analysis (CFA). A total of 298 preservice teachers, who were senior students studying at three different normal universities in China, participated in this study. Structural equation modelling revealed the following: (a) university ICT support significantly predicted preservice teachers' ICT perception, perceivedICT competence and ICT self‐efficacy, but had no significant direct effects on their TPACK; (b) ICT self‐efficacy was a strong predictor of preservice teachers' TPACK and ICT perception was a weak one, while perceived ICT competence did not predict TPACK at all; and (c) among three potential mediators, ICT self‐efficacy mediated the most effects of university ICT support on preservice teachers' TPACK. These findings demonstrate strong positive effects of institutional support on preservice teachers' TPACK development, which was strongly underestimated in previous studies. Findings suggest normal universities should create practical opportunities for preservice teachers to perceive the power of technology, especially opportunities that could improve preservice teachers' self‐efficacy while applying technology into instruction. Practitioner notesWhat is already known about this topicICT support from schools is an essential precondition while enhancing in‐service teachers' motivation to apply ICT into instruction.Research revealing how ICT support provided by teacher preparation programmes affects preservice teachers' TPACK is absent.What the paper addsThis study adds to the previous literature by exploring the mediating role of ICT perception, perceived ICT competence and ICT self‐efficacy in the relationship between university ICT support and preservice teachers' TPACK.This study found that university ICT support significantly predicted preservice teachers' ICT perception, perceivedICT competence and ICT self‐efficacy, but had no significant direct effects on their TPACK.Among the three potential mediators, ICT self‐efficacy mediated the most effects of university ICT support on preservice teachers' TPACK.Implications for practice and/or policyNormal universities need to provide effective and professionally targeted ICT support to meet preservice teachers' needs.The mediating role of ICT self‐efficacy should be highlighted on the path from university ICT support to TPACK. [ABSTRACT FROM AUTHOR]