Reference frames for learning analytics dashboards: The progress and social reference frame and occupational self‐efficacy.

Background: The potential of learning analytics dashboards in virtual reality simulation‐based training environments to influence occupational self‐efficacy via self‐reflection phase processes in the Chemical industry is still not fully understood. Learning analytics dashboards provide feedback on learner performance and offer points of comparison (i.e., comparison with one's own past performance or comparison with peer performance) to help learners make sense of their feedback. Objectives: We present a theoretical framework for describing learning analytics reference frames and investigate the impact of feedback delivered through dashboards with different reference frames on occupational self‐efficacy, while controlling for workplace self‐reflection. Methods: This experimental study engaged 42 chemical operator employees, aged between 18 and 55 years, each with at least one year of experience. We utilised a two‐group design to ask two research question each with three competing hypotheses related to changes in occupational self‐efficacy, employing Bayesian informative hypothesis evaluation. Results and Conclusions: Results for the primary research question suggest that dashboards with progress reference frames do not elicit greater change to self‐efficacy than those with social reference frames, however, they may elicit equal change. Furthermore, dashboards with social reference frames may elicit greater change to self‐efficacy than those with progress reference frames. Exploratory results found that dashboards with progress reference frames may elicit greater positive directional change than those with social reference frames and that they may elicit equal directional change. These findings contribute to the understanding of self‐efficacy beliefs within the Chemical industry, with potential impacts on skill development. The research may inform the design of targeted interventions and training programs to influence self‐efficacy. From a practical perspective, this research suggests that careful consideration is needed when choosing reference frames in learning analytics dashboards due to their potential consequences on the formation of learner self‐efficacy. Lay Description: What is already known about this topic?: Learning analytics dashboards often aim to stimulate self‐regulated learning by providing feedback.Feedback plays a crucial role in simulation‐based learning and learning analytics dashboards can be a valuable tool to deliver feedback and facilitate the learning process by providing learners with insights into their performance and progress.Previous research has highlighted the importance of reference frames as critical design features of learning analytics dashboards, as they assist learners in making sense of learning analytics feedback.Self‐efficacy is a powerful determine factor of workplace performance and is influenced by mastery experiences and social modelling information. What the paper adds?: A theoretical framework describes the mediating role self‐reflection phase processes of the self‐regulated learning cycle plays in the formation of occupational self‐efficacy.This paper introduces the concept of three learning analytics reference frame components which include the performance outcome component, point of comparison component and score delta component.The paper indicates that learning analytics dashboards designed with a progress reference frame does not elicit greater change to occupational self‐efficacy than the social reference frame.Exploratory results suggest dashboards with progress reference frames might produce greater positive directional change compared to social reference frames or elicit equal change. Implications for practice?: Learning analytics system designers who use reference frames to help learners make sense of their feedback should carefully consider which reference frames they use as these decisions likely have consequences on the formation of learner self‐efficacy beliefs.For LAD designers and stakeholders, recognising the absolute change and directional shift in self‐efficacy is crucial, as LAD designs significantly influence these dynamics and guide the tailoring of effective tools.In industries like the chemical sector, where overconfidence can result in severe consequences, understanding the nuance of self‐efficacy changes is especially pertinent. [ABSTRACT FROM AUTHOR]