Search

Digital education, now common in higher education, is particularly evident in the expansion of blended and fully online offerings at universities. Central to this expansion are educational developers, staff who support teaching and learning improvement in courses they do not themselves teach. Working closely with staff, students, and the curriculum, educational developers see first-hand how the digital learning agenda is both implemented and experienced. This article reports on findings from a national study of three educational development groups: academic developers, academic language and learning developers, and online educational designers, from 14 Australian universities. Although their institutional settings, roles, and work practices varied considerably, a central theme was the tension arising from a perceived shift in institutional priorities from 'people development' to 'product development': that is, from building human (educator) capacity towards curriculum resource development, particularly for the online environment. Participants reported a decline in autonomy, with institutional strategy and targeted projects increasingly directing both the work that gets done, and the skill sets required to do it. Their observations have implications for how universities conceptualise the development and support of the educational process. Refereed/Peer-reviewed

This article reports on the findings of a study that investigated the effects of instructional conditions and prior experience on students' self-reflection. The study was conducted with the use of a video annotation tool that was used by undergraduate performing arts students to reflect on their video-recorded performances. The study shows a consistent positive effect of previous experience with the video annotation tool for engagement with reflection. Graded instructional conditions with feedback had a positive effect on increasing higher order reflections particularly for students with prior experience with the video annotation tool for reflective purposes. The finding suggests that when including reflection in the curriculum, it is important to consider introducing it at a program or degree level rather than individual courses in order to provide an opportunity for students to gain experience with reflection and any particular tool that is used (e.g., a video annotation tool). Furthermore, reflective tasks should be scaffolded into the curriculum with ample opportunity for formative feedback and summative assessment in order to encourage higher order thinking and foster students' metacognitive awareness and monitoring for increased goal-setting and acknowledgement of the motive or effect of their observed behavior. Refereed/Peer-reviewed

There is little debate regarding the importance of student feedback for improving the learning process. However, there remain significant workload barriers for instructors that impede their capacity to provide timely and meaningful feedback. The increasing role technology is playing in the education space may provide novel solutions to this impediment. As students interact with the various learning technologies in their course of study, they create digital traces that can be captured and analysed. These digital traces form the new kind of data that are frequently used in learning analytics to develop actionable recommendations that can support student learning. This paper explores the use of such analytics to address the challenges impeding the capacity of instructors to provide personalised feedback at scale. The case study reported in the paper showed how the approach was associated with a positive impact on student perception of feedback quality and on academic achievement. The study was conducted with first year undergraduate engineering students enrolled in a computer systems course with a blended learning design across three consecutive years (N2013 = 290, N2014 = 316 and N2015 = 415).

Prior education studies have consistently emphasized the importance of sustained and active student engagement to aid academic performance and achievement of learning outcomes (e.g., Hockings, Cooke,Yamashita, McGinty, & Bowl, 2008; Michael, 2006). The positive impact of such active learning models on academic outcomes has been well established, particularly, in the STEM (Science, Technology, Engineering and Mathematics) disciplines. Refereed/Peer-reviewed

This paper presents the results of a natural experiment investigating the effects of instructional conditions and experience on the adoption and sustained use of a learning tool. The experiment was conducted with undergraduate students, enrolled into four performing art courses (N = 77) at a research intensive university in Canada. The students used the video annotation software CLAS for course-based self-assessment on their performances. Although existing research offers insights into the factors predicting students’ intentions of accepting a learning tool, much less is known about factors that affect actual adoption and sustained tool use. The study explored the use of CLAS amongst undergraduate students in four courses across two consecutive semesters. Trace data of students’ tool use, graph-based measures of metacognitive monitoring, and text cohesion of video annotations were used to estimate the volume of tool use and the quality of the learning strategy and learning products created. The results confirmed that scaffolding (e.g., graded activity with instructional feedback) is required to guide students’ initial tool use, although scaffolding did not have an independent significant effect on the quantity of tool use. The findings demonstrated that the use of the tool is strongly influenced by the experience an individual student gains from scaffolded conditions. That is, the students sustained their use of the learning tool in future courses even when the tool use was not graded nor was instructional feedback provided. An important implication is that students’ tool use is not solely driven by motivation – rather, it is shaped by instructional conditions and experience with the tool use. Refereed/Peer-reviewed

The success of the flipped classroom (FC) is effectively reliant on the level of student engagement with the preparatory activities prior to attending face-to-face teaching sessions. Information about the nature and level of student engagement with these activities can help instructors make informed decisions regarding how to best support student learning. Despite the comprehensive data collection enabled by the increasing presence of education technologies, few studies have used these data to investigate the range of strategies students employ in FC models. This study addresses this deficit by proposing an analytical approach that allows for exploring a) the strategies students use to interact with online preparation activities; and b) evolution of those strategies over the duration of a course delivered with a FC pedagogy. The proposed approach identified eight learning strategies and six trajectories of strategy change over the duration of the course, with statistically significant effects of strategy change trajectories on academic performance. Refereed/Peer-reviewed

Flipped classroom (FC) is an active learning design requiring students to complete assigned pre-class learning activities in preparation for face-to-face sessions. Students’ timely, regular, and productive engagement with the pre-class activities is considered critical for the success of the overall FC design, as these activities serve to prepare students for effective participation in face-to-face sessions. However, there is limited empirical evidence on the strength of association between students’ regularity of engagement with the pre-class activities and their learning performance in a FC course. Hence, the current study uses learning trace data from three consecutive offerings of a FC course to examine students’ regularity of pre-class learning activities and its association with the students’ course performance. In particular, the study derives several indicators of regularity from the trace data, including indicators related to time management and those reflecting regularity in the pattern of engagement with pre-class learning activities. The association with course performance is examined by building predictive regression models with the defined indicators as features. To examine the relevance of incorporating the specificities of the instructional design in predictive models, we designed and compared two kinds of indicators: generic (i.e. course-design-agnostic) and course-design-specific indicators. The study identified several indicators of regularity of pre-class activities as significant predictors of course performance. It also demonstrated that predictive models with only generic indicators were able to explain only a small portion of the overall variability in the students’ course performance, and were significantly outperformed by models that incorporated course specific indicators. Finally, the study findings point to the importance of assisting students in regulating their use of learning resources during class preparation activities in a FC. Refereed/Peer-reviewed

Video-based self-reflection and annotation is receiving increasing attention within the education literature. The importance of such technologies in education relate, in part, to the interactive nature and functionality these tools bring to aid learning engagement. In particular, these tools are well aligned with the need to promote and develop student meta-cognitive skills through the use of self-reflection activities. However, in the context of video-based learning environments, the nature of a students’ self-reflective process is not well understood. We attempt to address this gap in the literature in two main ways. First, we developed a coding instrument to assess the depth of a students’ self-reflection captured through the use of a video annotation tool. We then explored the linguistic and discourse properties of the student self-reflections using Coh-Metrix, a theoretically grounded computational linguistics facility. The adopted approach applies comprehensive analysis of language and discourse features associated with the specificity of students’ internal self-feedback, which is externalized as self-reflections in video annotations. The results suggest that levels of self-reflection have characteristically different linguistic properties, and these differences align with the underlying cognitive mechanisms associated with distinct reflective activities. The paper provides a detailed discussion of the findings in the context of the theoretical, methodological, and practical implications associated with video-based self-reflection and video annotation usc Refereed/Peer-reviewed

The use of standards-based assessment (SBA) has gained prominence in higher education due to a wide range of research evidence that supports its effectiveness in improving learning and teaching. Although the concept is widely used in academic discourse and practice, it has multiple interpretations which have led to diverse assessment practices to the extent that some of these depart from the principles of SBA. To help build a common understanding of what constitutes SBA practices, we explored the construct of academic SBA practices using both theoretical and empirical approaches. We developed a tool, based on the principles of SBA emerging from practices noted in the literature and identified from focus groups. We used factor analysis to extract the dimensions of the construct and to establish empirical evidence for the utilisation of the tool using a data set of 410 academics who engaged in self-assessment. Results showed that a six-factor model is the most parsimonious among other models. We used these dimensions to develop a framework for describing academic SBA practices. The use of the framework and tool has critical implications for curriculum design, professional practice and development, and policy articulation Refereed/Peer-reviewed

Reflective writing has been widely recognized as one of the most effective activities for fostering students’ reflective and critical thinking. The analysis of students’ reflective writings has been the focus of many research studies. However, to date this has been typically a very labor-intensive manual process involving content analysis of student writings. With recent advancements in the field of learning analytics, there have been several attempts to use text analytics to examine student reflective writings. This paper presents the results of a study examining the use of theoretically-sound linguistic indicators of different psychological processes for the development of an analytics system for assessment of reflective writing. More precisely, we developed a random-forest classification system using linguistic indicators provided by the LIWC and Coh-Metrix tools. We also examined what particular indicators are representative of the different types of student reflective writings Refereed/Peer-reviewed

The learning analytics community has matured significantly over the past few years as a middle space where technology and pedagogy combine to support learning experiences. To continue to grow and connect these perspectives, research needs to move beyond the level of basic support actions. This means exploring the use of data to prove richer forms of actions, such as personalized feedback, or hybrid approaches where instructors interpret the outputs of algorithms and select an appropriate course of action. This paper proposes the following three contributions to connect data extracted from the learning experience with such personalized student support actions: 1) a student-instructor centred conceptual model connecting a representation of the student information with a basic set of rules created by instructors to deploy Personalized Learning Support Actions (PLSAs); 2) a software architecture based on this model with six categories of functional blocks to deploy the PLSAs; and 3) a description of the implementation of this architecture as an open-source platform to promote the adoption and exploration of this area. Refereed/Peer-reviewed

Video materials have become an integral part of university learning and teaching practice. While empirical research concerning the use of videos for educational purposes has increased, the literature lacks an overview of the specific effects of videos on diverse learning outcomes. To address such a gap, this paper presents preliminary results of a large-scale systematic review of peer-reviewed empirical studies published from 2007-2017. The study synthesizes the trends observed through the analysis of 178 papers selected from the screening of 2531 abstracts. The findings summarize the effects of manipulating video presentation, content and tasks on learning outcomes, such as recall, transfer, academic achievement, among others. The study points out the gap between large-scale analysis of fine-grained data on video interaction and experimental findings reliant on established psychological instruments. Narrowing this gap is suggested as the future direction for the research on video-based learning Refereed/Peer-reviewed

The rapid growth of blended and online learning models in higher education has resulted in a parallel increase in the use of audio-visual resources among students and teachers. Despite the heavy adoption of video resources, there have been few studies investigating their effect on learning processes and even less so in the context of academic development. This paper uses learning analytic techniques to examine how academic teaching staff engage with a set of prescribed videos and video annotations in a professional development course. The data was collected from two offerings of the course at a large research-intensive university in Australia. The data was used to identify patterns of activity and transition states as users engaged with the course videos and video annotations. Latent class analysis and hidden Markov models were used to characterise the evolution of engagement throughout the course. The results provide a detailed description of the evolution of learner engagement that can be readily translated into action aimed at increasing the quality of the learning experience. Refereed/Peer-reviewed

This chapter focuses on the design of a flipped learning experience and, in particular, focuses on the types of activities and their scheduling. The problem can be described as how to stratify the type of activities and how they are distributed in time such that students are provided a gradual and engaging approach to achieve the learning outcomes. The flipped learning design model described in this section has been deployed in a first-year engineering course on computer systems at a higher education institution. In the remainder of the chapter, we assume that students with respect to a topic need to traverse the levels of the revised Bloom’s taxonomy starting with acquiring basic knowledge about a concept and then making a transition to the point where they can evaluate or create artefacts within that area.

This final issue for 2015 includes a special section of invited papers from the recent Learning Analytics and Knowledge conference (LAK15). The collected papers connect with the conference theme of “Scaling up: Big data to Big Impact” and reflect the emerging trends and future directions of learning analytics research.

Massive open online courses (MOOCs) have become the center of much media hype and have been considered to be disruptive and transformational to traditional education practice. This chapter employs two methods: (i) a clustering technique to characterize users and their behaviors and (ii) a classification model in order to infer participants' goals from the behavioral engagement with the course. It provides the design context for the specific MOOC used in our analysis, learning to teach online (LTTO). The chapter describes the sense-making process required to use the data available from the activity logs. This includes data preparation, the identification of metrics and methods to understand what participants do, and study the relationships between their explicitly stated goals and the behavioral traces emerging from the logs. Finally, the chapter evaluates the freedom in self-determined learning paths and how this may relate to the pedagogical design of the MOOC.

© 2016 Copyright held by the owner/author(s). The pervasive collection of data has opened the possibility for educational institutions to use analytics methods to improve the quality of the student experience. However, the adoption of these methods faces multiple challenges particularly at the course level where instructors and students would derive the most benefit from the use of analytics and predictive models. The challenge lies in the knowledge gap between how the data is captured, processed and used to derive models of student behavior, and the subsequent interpretation and the decision to deploy pedagogical actions and interventions by instructors. Simply put, the provision of learning analytics alone has not necessarily led to changing teaching practices. In order to support pedagogical change and aid interpretation, this paper proposes a model that can enable instructors to readily identify subpopulations of students to provide specific support actions. The approach was applied to a first year course with a large number of students. The resulting model classifies students according to their predicted exam scores, based on indicators directly derived from the learning design.

The rapidly changing higher education (HE) environment has transformed the ways academics work to overcome the challenges of ever-evolving imperatives. Changes in public sector funding models, the increase in the number and range of student capabilities, and the raised importance of university status have contributed to increasing accountability measures, including the wide adoption of standards and quality metrics in all aspects of academic practice. Academics are now at the center of coordinated institutional teaching quality enhancement efforts, putting a strain on their ability to perform equally well in research and teaching. In this chapter, a teaching and course development framework is proposed integrating ideas around reflective practice with the systematic use of evidence generated using methodologies emerging from the field of learning analytics and educational data science. The proposed framework could help academics to develop their teaching practice leveraging on previously untapped sources of evidence readily available in contemporary HE institutions and utilizing the disciplinary research skillset, which is not typically applied to teaching development. A case study illustrating the application of the proposed framework to scholarly evaluation of an academic development program (adopting online and face-to-face activities) is presented and discussed.

Designing, validating and deploying learning analytics tools for instructors or students is a challenge that requires techniques and methods from different disciplines, such as software engineering, human-computer interaction, computer graphics, educational design and psychology. Whilst each of these disciplines has established its own design methodologies, there is a need for methodological frameworks that meet the specific demands of the cross-disciplinary space defined by learning analytics. In particular there is no systematic workflow for producing learning analytics tools that are both technologically feasible and truly underpin the learning experience. In this paper, we present a set of guiding principles and recommendations derived from the LATUX workflow. LATUX is a five-stage workflow to design, validate and deploy awareness interfaces in technology-enabled learning environments. LATUX is grounded on a well-established design process for creating, testing and re-designing user interfaces. We extend this process by integrating the pedagogical requirements, to guide the design of learning analytics visualisations that can inform instructors’ pedagogical decisions or intervention strategies. The workflow is illustrated with a case study in which collaborative activities were deployed in a real classroom. Finally, the paper proposes a research agenda to support designers and implementers of learning analytics interfaces.