What kind of learning designs do practitioners create for mobile learning? Lessons learnt from two in‐the‐wild case studies.

Background: In the field of Learning Design, it is common that researchers analyse manually design artefacts created by practitioners, using pedagogically‐grounded approaches (e.g., Bloom's Taxonomy), both to understand and later to support practitioners' design practices. Automatizing these high‐level pedagogically‐grounded analyses would enable large‐scale studies on practitioners' design practices. Such an approach would be especially useful in the context of mobile learning, where practitioners' design practices are under‐explored and complex (e.g., involving both formal and informal learning activities, happening between physical and digital spaces). Objectives: We inquire about the kind of designs that practitioners create in mobile learning by analysing the entire databases of two m‐learning tools, Avastusrada and Smartzoos, which promote inquiry learning outdoors. Methods: We use supervised machine learning to classify the textual content of the designs based on the cognitive level required from learners, the inquiry‐based learning phases they cover, and their connection with the learning context (e.g., the role played by the situated environment). Results and Conclusions: Results from the in‐the‐wild studies emphasize practitioners' tendency to design contextualized activities, but that include few higher‐order thinking tasks and elements of inquiry learning. This raises questions about the real‐life pedagogical value of similar mobile learning tools and highlights the need for providing pedagogical guidelines and technical solutions that would promote the adoption of good learning design practices. Major takeaways from the study: While we show that machine learning techniques (informed by learning design elements) can enable large‐scale studies and provide useful insights, to best understand and support practitioners' design practices it would be necessary to combine them with other quantitative and quantitative analyses (e.g., a qualitative understanding on why practitioners take specific design decisions). Future research could use similar machine learning approaches to explore other design settings, as well as explore scenarios where similar algorithms can be embedded in design tools, to guide practitioners' design practices. Lay Description: What is already known about this topic: Mobile learning activities demand extensive technical and pedagogical competencies from the practitioners that design them, especially in activities that bridge formal and informal learning, across physical and digital spaces.The field of Learning Design has provided several authoring tools, as well as good practices that support practitioners to design for mobile learning.Supervised machine learning informed by activity traces has been used to understand teaching and learning practices. What this paper adds: An in‐the‐wild study on the kind of designs created by the practitioners of two m‐learning tools, Avastusrada and Smartzoos, in terms of the cognitive level that they require by learners, as well as the role played in their activities by inquiry learning and the learning context.Evidence on the potential use of supervised machine learning techniques, informed by elements of the learning design and the learning context, to understand practitioners' design practices. Implications for practice and/or policy: To support practitioners design practices, it is necessary to provide, not only training offering technical and pedagogical expertise in mobile learning design, but also learning design systems with built‐in guidance and recommendations.Using supervised machine learning informed by the learning design and the learning context can help not only researchers but also practitioners to understand and improve their learning design practices. [ABSTRACT FROM AUTHOR]