Search

Background: The feedback offered to students in audience response systems may enhance conformity bias, while asking closed-type questions alone does not allow students to externalize and elaborate on their knowledge. Objectives: The study explores how writing short justifications and accessing peer justifications as collective feedback could affect students' performance, confidence and opinions in multiple-choice audience response systems that apply the Peer Instruction model of voting/revoting. Methods: For 8 weeks, 98 students, enrolled in an undergraduate course, attended each lecture following a flipped classroom approach. At the beginning of each lecture, students participated in a quiz with eight multiple-choice questions. Four of these questions included a justification form in which students could elaborate on their answers. The students were randomly grouped into two conditions according to the collective feedback they received: the Shared group (n = 54) could see both the percentage each question choice received from the class and the respective peer justifications, while the Unshared group (n = 44) could only see the percentage information. Results: Analysis showed that students in both groups performed significantly better in questions with the justification form being available. Also, the two groups were comparable in terms of performance and self-reported level of confidence suggesting no main effect for making peer justification available. Despite this, students in the Shared group expressed a significantly more positive opinion in the end-of-activity questionnaire in terms of perceived learning gains and the helpfulness of writing justifications for their answers. Take Away: Writing short justifications can have a positive impact on students' academic performance.

The aim of the present study (n = 113) was to examine how (objective and subjective) information on peers’ preparation, confidence, and past performance can support students in answering correctly in audience response systems (ARS; aka clickers). The result analysis shows that in the “challenging” questions, in which answers diverged, students who received additional information about peers’ self-reported preparation and/or confidence outperformed students who were only given the objective percentage with or without past performance feedback. In addition, students expressed a positive attitude towards the activity, commenting its usefulness in better understanding course material and identifying misconceptions.

Asking questions in classrooms can produce metacognitive judgements in students about their confidence in being able to answer correctly. In audience response systems (ARSs), these judgements can be elicited and used as additional feedback metrics. This study (n = 79) explores how online concurrent item-by-item judgments (OCJ) and retrospective composite judgments of performance accuracy (RJPA) can enhance students’ performance and self-assessing accuracy (i.e., calibration – as measured by sensitivity, specificity, and absolute accuracy index). In each of eight weeks, the students answered a multiple-choice quiz and had to denote their level of confidence that their answers were correct (OCJ) and estimate their final score (RJPA). The quizzes followed the voting/revoting paradigm according to which students answer all the quiz questions, receive feedback, and answer the same questions again before the correct answers are shown. The students were randomly grouped into two conditions based on the feedback they received in the ARS: the OCJ group (n = 41) received the percentage distribution and peers’ OCJs as feedback metrics, while the RJPA group (n = 38) received the percentage distribution and peers’ RJPAs. Data analysis showed a systemic underconfidence that affected students’ OCJ judgments. As a result, students in the RJPA group scored significantly higher than the ones in the OCJ one, were more accurate in self-assessing in the revoting phase, and felt overall more confident in the revoting phase. The study also discusses the relationship between the two judgments employed and the calibration variability between the two study phases.

Aim/Purpose: This study aimed to explore whether students’ self-reported use of various learning strategies affected their perceptions on different course activities as well as their perceived performance in terms of both cognitive learning outcomes and general skills. Background: In a highly active learning environment that incorporates research into teaching, the effective use of various learning strategies is considered of high importance for the successful engagement of students. Yet, this line of research has mainly focused on individual learning. Shifting from individual to collaborative learning settings, the current study investigated whether students’ use of self-regulated learning, peer learning, and help seeking strategies influenced their perceptions on both the group activities and the respective outcomes. Methodology: At the beginning of the course, 81 first semester postgraduate students self-reported the level of use of self-regulated learning, peer learning, and help seeking strategies by filling in the respective subscales of the Motivated Strategies for Learning Questionnaire (MSLQ). Then, groups of 3 or 4 students were formed and instructed to create several learning artifacts of different types and conduct a peer-tutoring session, based on a topic assigned to them by the teacher. Additionally, the same groups conducted a research project of their own choice within course topics. Students’ final grade served as an indicator of their academic performance. At the end of the semester, students filled in a questionnaire eliciting their perceptions on the process and the outputs of the course activities. Finally, through statistical analysis of students’ responses to the questionnaires, the influence of learning strategies on students’ perceptions and their academic performance was examined. Contribution: Our findings contribute to the literature regarding the research-teaching nexus in higher education settings. More specifically, the study shows how students’ self-reported use of learning strategies affects students’ perceptions on the activities they were engaged in, their achievement of cognitive learning outcomes, and their skills development in a research-integrated course design. Findings: Students perceived differently the value of producing and studying learning artifacts. Students who scored higher in the self-regulated learning and peer learning subscales of MSLQ perceived their role as more active in the preparation of the presentation for the peer-tutoring session, which was the artifact that required higher level of interaction among the group members. Students’ final grades were influenced partially by their self-reported use of different learning strategies. Recommendations for Practitioners: Integrating research into teaching through the assignment of research-related tasks to students can promote students’ acquisition of domain knowledge and research skills. The merits of this approach can be further strengthened by having students working in groups and providing the outputs of their involvement in the research-related activities as learning material for their peers. Furthermore, students’ individual characteristics (e.g., use of learning strategies and preferences should be taken into account when designing course activities). Recommendation for Researchers: Researchers should continue to explore the way that various learning strategies influence different aspects of the learning process, especially in the achievement of cognitive learning outcomes and the development of general skills. Impact on Society: Creating learning environments that foster students’ active engagement with the course material and peer collaboration should be a vital goal of higher education institutes as it can improve students’ performance and promote the necessary skills for self-directed and autonomous learning, a key competence in the modern workplace. Future Research: In this study, both cognitive learning outcomes and general skills were assessed by students’ final grade. In a future study, distinguishing these different types of learning outcomes would allow us to examine in more detail the impact of students’ learning strategies and course activities on the accomplishment of cognitive learning outcomes and general skills.

This paper examines ways to enrich the feedback information students receive in closed-type quiz activities that include a revision phase (i.e., students are allowed to change their initial answers after they receive information from their peers, teacher, or system). Typically, in such activities, the information students receive is based on the percentage of students under each possible question choice. The conducted study analyzes the potential of two additional variables, namely the students’ level of preparation and confidence. Both variables are self-reported and, therefore, subjective. During the Fall semester 2016, 91 sophomore students enrolled in an Information Systems course participated in the study for five weeks. The activity was taking place during the first 20 min of each class. Students had to go through three phases and (a) answer a multiple-choice quiz with 8 questions and 4 options for each question, (b) receive feedback based on the whole classroom population, and (c) see the correct answer and discuss them with the teacher in the lecture that follows. The students were randomly grouped into four conditions, based on the feedback they received. The control group only received information on the percentage of students that selected each choice, the Confidence group received feedback on the percentage and the average level of confidence of students that selected each choice, the Preparation group received feedback on the percentage and the average level of preparation of students that selected each choice, and finally the Both group received feedback on the percentage and both the average level of confidence and preparation of students that selected each choice. Result analysis showed that in the most challenging questions (i.e., the ones where students’ answers were diverging) the students in the Confidence, Preparation, and Both groups significantly outperformed the students in the Control group. In addition, both confidence and preparation variables were significantly correlated to students’ performance during the initial phase, suggesting that students were accurate and sincere in describing their preparation and confidence levels. This paper is an extended version of [1], presented at the 9th International Conference on Computer Supported Education.

Higher Education Massive Open Online Courses (MOOCs) introduce a way of transcending formal higher education by realizing technology-enhanced formats of learning and instruction and by granting access to an audience way beyond students enrolled in any one Higher Education Institution. However, although MOOCs have been reported as an efficient and important educational tool, there is a number of issues and problems related to their educational impact. More specifically, there is an important number of drop outs during a course, little participation, and lack of students’ motivation and engagement overall. This may be due to one-size-fits-all instructional approaches and very limited commitment to student-student and teacher-student collaboration. This paper introduces the development agenda of a newly started European project called “colMOOC” that aims to enhance the MOOCs experience by integrating collaborative settings based on Conversational Agents and screening methods based on Learning Analytics, to support both students and teachers during a MOOC course. Conversational pedagogical agents guide and support student dialogue using natural language both in individual and collaborative settings. Integrating this type of conversational agents into MOOCs to trigger peer interaction in discussion groups can considerably increase the engagement and the commitment of online students and, consequently, reduce MOOCs dropout rate. Moreover, Learning Analytics techniques can support teachers’ orchestration and students’ learning during MOOCs by evaluating students’ interaction and participation. The research reported in this paper is currently undertaken within the research project colMOOC funded by the European Commission.

Following the increased interest in students becoming partners in teaching and learning in higher education settings, this study explores students’ perceptions on a collaborative content generation activity, in which students assume also the role of peer-tutors. Students in an Information Systems course collaborated on domain topics, assigned to them by the teacher, and created learning material for their fellow students. In the peer-tutoring session following the creation of the learning material, students presented the topic in class and discussed it with their peers. Each peer tutoring session was assessed by the students as audience (students attending the student-tutoring sessions). The aim of the study is to explore students’ perspectives on the collaborative content generation assignment. Our focus primarily is to examine how students’ experiences regarding the collaboration were affected by learning strategies in self-regulation, peer learning, and help seeking. Results showed that students were more engaged in the course after participating in the co-creating learning material activity and also, four distinct patterns of collaboration were revealed by analyzing student activity. Even though students were in general satisfied with their collaboration, as they could suit the activity to their needs, those students who relied more on their group for help proved to be less satisfied by the communication among group members. In addition, the teacher and the audience evaluated positively the students’ performance as peer-tutors. This paper is an extended version of [1], presented at the 9th International Conference on Computer Supported Education.

This paper investigates the impact of the proposed ‘Make It Explicit!’ technique on students' learning when participating in scripted collaborative activities. The method posits that when asking students to proactively articulate their own positions explicitly, then improved peer interaction is triggered in a subsequent collaborative session. Forty-two (42) students worked collaboratively in dyads on a peer-review-scripted task with the aid of a Web-based technological system. Students were asked to individually study the same material and produce answers to study questions, review their peer's work and, finally, produce collaboratively revised common answers to the study questions. Dyads were randomly assigned in two conditions: low coercion (LC) and high coercion (HC). In the HC condition, students were required to write and submit to the system their reviews, prior to the task of producing the common answers to the study questions. In the LC condition, instead, submitting the individual review to the system was optional. By collecting and analysing both quantitative (pre- and post-test scores, measures of individual and collaborative learning) and qualitative (interviews, log files) data, it is shown that students in the HC condition applied better collaboration patterns and outperformed students in the LC condition in acquiring domain conceptual knowledge. Overall, this work analyses key aspects of implementing the ‘Make It Explicit!’ technique and concludes that it can efficiently improve collaborative learning outcomes by increasing script coercion.

This study was designed to investigate the impact of question prompts that guide students to focus on context-related issues when learning through cases in an ill-structured domain. Three groups of undergraduate students studied cases during a lab-session time period using a web-based environment. The first group studied without any question prompts. The second group studied the same material while prompted to provide written answers to embedded questions in the cases. The third group studied while having only to think of possible answers for the question prompts. In this study, we explored how the questioning intervention affected students’ conceptual knowledge of the domain and their problem-solving ability. Post-tests did not reveal significant statistical differences in the groups’ performance, indicating that under specific study conditions the prompting impact is not traceable in the learning outcomes. This result, however, is discussed in the light of a previous study, which showed that this context-oriented prompting method had a beneficial effect on student learning in a prolonged study-time setting, where students were able to self-regulate their study activity.

This paper investigates the possible benefits as well as the overall impact on the behaviour of students within a learning environment, which is based on double-blinding reviewing of freely selected peer works. Fifty-six sophomore students majoring in Informatics and Telecommunications Engi-neering volunteered to participate in the study. The experiment was conducted in a controlled environment, according to which students were divided into three groups of different conditions: control, usage data, usage and ranking data. No additional information was made available to students in the control condition. The students that participated in the other two conditions were provided with their usage information (logins, peer work viewed/reviewed, etc.), while members of the last group could also have access to ranking information about their positioning in their group, based on their usage data. According to our findings, students’ performance between the groups were comparable, however, the Ranking group revealed differences in the resulted behavior among its members. Specifically, awareness of ranking information mostly benefited students which were relatively in favor of ranking, motivating them to further engage and perform better than the rest of the group members.

Conversational agents that draw on the framework of academically productive talk (APT) have been lately shown to be effective in helping learners sustain productive forms of peer dialogue in diverse learning settings. Yet, literature suggests that more research is required on how learners respond to and benefit from such flexible agents in order to fine-tune the design of automated APT intervention modes and, thus, enhance agent pedagogical efficacy. Building on this line of research, this work explores the impact of a configurable APT agent that prompts peers to build on prior knowledge and logically connect their contributions to important domain concepts discussed in class. A total of 96 computer science students engaged in a dialogue-based activity in the context of a Human-Computer Interaction (HCI) university course. During the activity, students worked online in dyads to accomplish a learning task. The study compares three conditions: students who collaborated without any agent interference (control), students who received undirected agent interventions that addressed both peers in the dyad (U treatment), and students who received directed agent interventions addressing a particular learner instead of the dyad (D treatment). The results suggest that although both agent intervention methods can improve students’ learning outcomes and dyad in-task performance, the directed one is more effective than the undirected one in enhancing individual domain knowledge acquisition and explicit reasoning. Furthermore, findings show that the positive effect of the agent on dyad performance is mediated by the frequency of students’ contributions displaying explicit reasoning, while most students perceive agent involvement favorably.

PurposeThe purpose of this paper is to explore the impact of question prompts on student learning in relation to their learning styles. The context of the study is technology‐enhanced learning in an ill‐structured domain.Design/methodology/approachThe study conditions were the same for all the students in the four learning style groups. Student learning style was the independent variable, while students' attitudes and task performance were the dependent variables of the study. Pre‐test treatment post‐test method was used. Students studied in a web‐based learning environment during treatment.FindingsThe integration of question prompts as student supporting tool in technology‐enhanced learning environments might not improve learning for all students alike independent of their learning styles.Research limitations/implicationsSmall uneven groups because the researcher has no control over the student distribution across the different learning style profiles.Practical implicationsThe suggestion for designers is to consider combining prompting with other scaffolding methods, in order to effectively support all students independent of their learning styles.Originality/valueThe paper combines learning in ill‐structured domains through cases and a scaffolding method based on question prompts focusing on contextual elements. The results of the study inform the designers of TELEs that although prompting can be generally helpful, parameters such as the students' learning style are able to limit the cognitive benefit emerging from the prompting intervention.

This work explores the role of students' motivation and metacognitive skills as moderating factors that influence the impact of an instructional method in the ill-structured domain of Software Project Management (SPM). In order to teach aspects of the SPM domain, we developed a web environment for case-based learning and implemented additionally a questioning strategy to help students focus on important parts of the case material. The paper presents the results from three studies revealing how students' motivation and metacognitive awareness influenced their engagement in the cognitively challenging situations induced by the method. The implication for instructors and designers is that implementing a promising method, to help students efficiently process the complex material in an ill-structured domain, might not always lead to the desired learning outcomes. Students' motivation and metacognitive skills should also be addressed, in order to maximize the potential benefits of instruction.

This work analyzes a case of computer-supported scripted collaboration, focusing on how students' self-organization affected the actual collaboration script during script run-time. Two groups of students studied learning material using a web environment designed for supporting case-based learning. The first group followed a non-scaffolded individual mode of study while students in the second group were guided by a collaboration script to work (in dyads) on the case material. Statistical analysis indicated no significant differences in the learning outcomes of the two groups. Qualitative analysis (based on students' interviews and field observations) revealed that students' self-organization resulted to a broad range of actual script implementation ranging from full conformance to partial violation of the script guidelines. The paper discusses the socio-cognitive role of students' self-organization during scripted collaboration and presents suggestions for the teacher and CSCL designer in order to enhance the engagement of collaborating students to productive learning interactions.