Search

Socially shared regulation of learning (SSRL) is a crucial process for groups of learners to successfully collaborate. Detecting and supporting SSRL is a challenge, especially in real time, but hybrid intelligence approaches such as Artificial Intelligence (AI) agents may make this possible. Leveraging the concept of trigger events which invite SSRL, we present a design of an AI agent, MAI, which can detect SSRL and prompt students to raise their group‐level metacognitive awareness with the aim of facilitating SSRL. We present the methodology we used to design MAI, drawing on the Echeloned DSR (eDSR) Methodological Framework and making use of the Wizard of Oz prototyping paradigm. We likewise present empirical results evaluating our initial prototype of MAI, using lexical alignment between speakers as a quantitative measure of the effect of MAI's prompts on facilitating SSRL, the Partner Model Questionnaire as a quantitative measure of perceptions of MAI, and interviews as qualitative context for these perceptions. We found that the first prototype of MAI did not facilitate SSRL as hoped, possibly owing to mixed perceptions of MAI's reliability and lack of clarity about MAI's role in the collaborative learning task. From these findings, we offer revised prompts for the next iteration of prototyping this agent and a refined set of design requirements for future development of metacognitive AI agents for supporting SSRL. Practitioner notes What is already known about this topic Socially Shared Regulation of Learning (SSRL) is recognized as a critical component for the success of collaborative learning, emphasizing the importance of group‐level regulatory processes in achieving shared goals, enacting strategies and monitoring learning progress. Supporting SSRL in face‐to‐face collaborative learning environments presents challenges, including the complexity of coordinating and synchronizing individual contributions and regulatory actions within a group context. What this paper adds This paper introduces the design of Metacognitive Artificial Intelligence (MAI), a novel AI system aimed at enhancing Human‐AI collaboration for supporting and augmenting SSRL processes. Through empirical research, the study offers lessons learned and design considerations for developing artificial agents on facilitating and enhancing SSRL among learners, demonstrating how AI can play a pivotal role in collaborative learning environments. The findings highlight the critical importance of multidisciplinary knowledge in the design of multi‐agent interfaces (MAI) that provide real‐time, adaptive support for group metacognitive processes and decision‐making. Implications for practice and/or policy Educational technologists can utilize the proposed design principles in the development and integration of MAI tools to enhance SSRL. Educators can incorporate the principles of MAI and our relevant findings into their teaching strategies to actively foster and support socially shared regulation of learning among students. Policymakers should consider revising educational frameworks to include the use of AI technologies that support SSRL strategies in collaborative learning. What is already known about this topic Socially Shared Regulation of Learning (SSRL) is recognized as a critical component for the success of collaborative learning, emphasizing the importance of group‐level regulatory processes in achieving shared goals, enacting strategies and monitoring learning progress. Supporting SSRL in face‐to‐face collaborative learning environments presents challenges, including the complexity of coordinating and synchronizing individual contributions and regulatory actions within a group context. What this paper adds This paper introduces the design of Metacognitive Artificial Intelligence (MAI), a novel AI system aimed at enhancing Human‐AI collaboration for supporting and augmenting SSRL processes. Through empirical research, the study offers lessons learned and design considerations for developing artificial agents on facilitating and enhancing SSRL among learners, demonstrating how AI can play a pivotal role in collaborative learning environments. The findings highlight the critical importance of multidisciplinary knowledge in the design of multi‐agent interfaces (MAI) that provide real‐time, adaptive support for group metacognitive processes and decision‐making. Implications for practice and/or policy Educational technologists can utilize the proposed design principles in the development and integration of MAI tools to enhance SSRL. Educators can incorporate the principles of MAI and our relevant findings into their teaching strategies to actively foster and support socially shared regulation of learning among students. Policymakers should consider revising educational frameworks to include the use of AI technologies that support SSRL strategies in collaborative learning. [ABSTRACT FROM AUTHOR]

This article explores the usability and user experience challenges of ARROWS, a novel augmented reality (AR) and wearable technology (WT) safety system for roadway work zones, an area with limited existing usability research. We utilized a mixed-method approach with two complementary experiments in indoor and outdoor settings, using the Wizard of Oz methodology and a high-fidelity prototype. We focused on identifying usability challenges, factors contributing to user experience and the distinct needs of highway workers, documenting results using the system usability scale (SUS), the rating scale mental effort (RSME) and a trust score. Participants rated the usability of ARROWS above average in both settings, while making a reasonable level of mental effort. The findings also indicate a significant correlation between perceived trust and usability, highlighting the importance of trust in user experience. [ABSTRACT FROM AUTHOR]

In this article, we present RISE--a Robotics Integration and Scenario- Management Extensible-Architecture--for designing human-robot dialogs and conducting Human-Robot Interaction (HRI) studies. In current HRI research, interdisciplinarity in the creation and implementation of interaction studies is becoming increasingly important. In addition, there is a lack of reproducibility of the research results. With the presented open-source architecture, we aim to address these two topics. Therefore, we discuss the advantages and disadvantages of various existing tools from different sub-fields within robotics. Requirements for an architecture can be derived from this overview of the literature, which 1) supports interdisciplinary research, 2) allows reproducibility of the research, and 3) is accessible to other researchers in the field of HRI. With our architecture, we tackle these requirements by providing a Graphical User Interface which explains the robot behavior and allows introspection into the current state of the dialog. Additionally, it offers controlling possibilities to easily conduct Wizard of Oz studies. To achieve transparency, the dialog is modeled explicitly, and the robot behavior can be configured. Furthermore, the modular architecture offers an interface for external features and sensors and is expandable to new robots and modalities. [ABSTRACT FROM AUTHOR]

In this article, we present RISE—a Robotics Integration and Scenario-Management Extensible-Architecture—for designing human–robot dialogs and conducting Human–Robot Interaction (HRI) studies. In current HRI research, interdisciplinarity in the creation and implementation of interaction studies is becoming increasingly important. In addition, there is a lack of reproducibility of the research results. With the presented open-source architecture, we aim to address these two topics. Therefore, we discuss the advantages and disadvantages of various existing tools from different sub-fields within robotics. Requirements for an architecture can be derived from this overview of the literature, which 1) supports interdisciplinary research, 2) allows reproducibility of the research, and 3) is accessible to other researchers in the field of HRI. With our architecture, we tackle these requirements by providing a Graphical User Interface which explains the robot behavior and allows introspection into the current state of the dialog. Additionally, it offers controlling possibilities to easily conduct Wizard of Oz studies. To achieve transparency, the dialog is modeled explicitly, and the robot behavior can be configured. Furthermore, the modular architecture offers an interface for external features and sensors and is expandable to new robots and modalities.

This work describes the quantitative analysis of the linguistic difference in human-human and human-chatbot dialogues. The research is based on conducting a set of experiments where respondents communicate with a human or a chatbot in the domain of COVID-19 questions. In the case of the human-human dialogues, the approach of the inverted "Wizard of Oz" experimental setting is used. During the experiments, 35 humanhuman and 68 human-chatbot dialogues in Russian language were performed. The dialogues were collected.

Background: Older African American and Black adults are twice as likely to develop Alzheimer disease and related dementias (ADRD) and have the lowest level of ADRD health literacy compared to any other ethnic group in the United States. Low health literacy concerning ADRD negatively impacts African American and Black people in accessing adequate health care., Objective: This study explored how 3 technological modalities-voice assistants, chatbots, and mobile apps-can assist older African American and Black adults in accessing ADRD information to improve ADRD health literacy. By testing each modality independently, the focus could be kept on understanding the unique needs and challenges of this population concerning the use of each modality when accessing ADRD-related information., Methods: Using the Wizard of Oz usability testing method, we assessed the 3 modalities with a sample of 15 older African American and Black adults aged >55 years. The 15 participants were asked to interact with the 3 modalities to search for information on local events happening in their geographical area and search for ADRD-related health information., Results: Our findings revealed that, across the 3 modalities, the content should avoid convoluted and complex language and give the possibility to save, store, and share it to be fully accessible by this population. In addition, content should come from credible sources, including information tailored to the participants' cultural values, as it has to be culturally relevant for African American and Black communities. Finally, the interaction with the tool must be time efficient, and it should be adapted to the user's needs to foster a sense of control and representation., Conclusions: We conclude that, when designing ADRD-related interventions for African American and Black older adults, it proves to be crucial to tailor the content provided by the technology to the community's values and construct an interaction with the technology that is built on African American and Black communities' needs and demands., (©Cristina Bosco, Fereshtehossadat Shojaei, Alec Andrew Theisz, John Osorio Torres, Bianca Cureton, Anna K Himes, Nenette M Jessup, Priscilla A Barnes, Yvonne Lu, Hugh C Hendrie, Carl V Hill, Patrick C Shih. Originally published in JMIR Formative Research (https://formative.jmir.org), 09.12.2024.)

From automated customer support to virtual assistants, conversational agents have transformed everyday interactions, yet despite phenomenal progress, no agent exists for programming tasks. To understand the design space of such an agent, we prototyped PairBuddy--an interactive pair programming partner--based on research from conversational agents, software engineering, education, human-robot interactions, psychology, and artificial intelligence. We iterated PairBuddy's design using a series of Wizard-of-Oz studies. Our pilot study of six programmers showed promising results and provided insights toward PairBuddy's interface design. Our second study of 14 programmers was positively praised across all skill levels. PairBuddy's active application of soft skills--adaptability, motivation, and social presence--as a navigator increased participants' confidence and trust, while its technical skills--code contributions, just-in-time feedback, and creativity support--as a driver helped participants realize their own solutions. PairBuddy takes the first step towards an Alexa-like programming partner. [ABSTRACT FROM AUTHOR]

Even though games that focus on the development of logical-mathematical skills aimed at maintaining the attention of children with ADHD currently exist, there are no applications available to learn how to solve mathematical problems. In view of this problem, a serious game was designed and developed to enable children to achieve significant learning in this subject. An evaluation was performed of the interface of a section of the game focusing on the concept of counting. Work was performed with 7 teachers by means of a Wizard of Oz experiment. A CSUQ questionnaire was also used to gather general information on the teachers, such as gender and age. The children’s experience in using the application was also assessed. The study indicates that the current interface is usable, though it requires certain improvements in terms of typography, image design, and color.

As social robots continue to show promise as assistive technologies, the exploration of appropriate and impactful robot behaviors is key to their eventual success. Teens are a unique population given their vulnerability to stress leading to both mental and physical illness. Much of teen stress stems from school, making the school environment an ideal location for a stress reducing technology. The goal of this mixed-methods study was to understand teens' operation of, and responsiveness to, a robot only capable of movement compared to a robot only capable of speech. Stemming from a human-centered approach, we introduce a Participatory Wizard of Oz (PWoz) interaction method that engaged teens as operators, users, and witnesses in a uniquely transparent interaction. In this paper, we illustrate the use of the PWoz interaction method as well as how it helps identify engaging robot interactions. Using this technique, we present results from a study with 62 teens that includes details of the complexity of teen stress and a significant reduction in negative attitudes toward robots after interactions. We analyzed the teens' interactions with both the verbal and non-verbal robots and identified strong themes of (1) authenticity, (2) empathy, (3) emotional engagement, and (4) imperfection creates connection. Finally, we reflect on the benefits and limitations of the PWoz method and our study to identify next steps toward the design and development of our social robot.

Abstract Successful language learning requires an understanding of the target culture in order to make valuable usage of the learned language. To understand a foreign culture, language students need the knowledge of its related products, as well as the skill of comparing them to those of their own culture. One way for students to understand foreign products is by making Culturally Situated Associations (CSA), i.e., relating the products they encountered to products from their own culture. In order to provide students with CSA that they can understand, we must gather information about their culture, provide them with the CSA, and make sure they understand it. In this case, a Culturally Situated Dialogue (CSD) must take place. To carry the dialogue, dialogue systems must follow a dialogue strategy. However, previous work showed that handcrafted dialogue strategies were shown to be ineffective in comparison with machine-learned dialogue strategies. In this research, we proposed a method to learn CSD strategies to support foreign students, using a reinforcement learning algorithm. Since no previous system providing CSA was implemented, the method allowed the creation of CSD strategies when no initial data or prototype exists. The method was applied to generate three different agents: the novice agent was based on an eight states feature-space, the intermediate agent was based on a 144 states feature-space, and the advanced agent was based on a 288 states feature-space. Each of these agents learned a different dialogue strategy. We conducted a Wizard of Oz experiment during which, the agents’ role was to support the wizard in their dialogue with students by providing them with the appropriate action to take at each step. The resulting dialogue strategies were evaluated based on the quality of the strategy. The results suggest the use of the novice agent at the first stages of prototyping the dialogue system. The intermediate agent and the advanced agent could be used at later stages of the system’s implementation.

Gesture elicitation studies are a popular means of gaining valuable insights into how users interact with novel input devices. One of the problems elicitation faces is that of legacy bias, when elicited interactions are biased by prior technologies use. In response, methodologies have been introduced to reduce legacy bias. This is the first study that formally examines the production method of reducing legacy bias (i.e., repeated proposals for a single referent). This is done through a between-subject study that had 27 participants per group (control and production) with 17 referents placed in a virtual environment using a head-mounted display. This study found that over a range of referents, legacy bias was not significantly reduced over production trials. Instead, production reduced participant consensus on proposals. However, in the set of referents that elicited the most legacy biased proposals, production was an effective means of reducing legacy bias, with an overall reduction of 11.93% for the chance of eliciting a legacy bias proposal. [ABSTRACT FROM AUTHOR]

Driving simulation is an important platform for studying vehicle automation. There are different approaches to using this platform – with most using scripting or programmatic tools to simulate vehicle automation. A less frequently used approach, the Wizard-of-Oz method, has potential for increased flexibility and efficiency in designing and conducting experiments. This study designed and evaluated an experimental setup to examine the feasibility of this approach as an alternative for conducting automation studies. Twenty-four participants experienced simulated vehicle automation in two platforms, one where the automation was controlled by algorithms, and the other where the automation was simulated by an external operator. Surveys were administered after each drive and the drivers' takeover performance after the automation disengaged was measured. Results indicate that while the kinematic parameters of the driving differed significantly for the two platforms, there were no significant differences in the perceptions of participants and in their takeover performance between the two platforms. These results provide evidence for the use of alternative approaches for the conduct of human factors studies on vehicle automation, potentially lowering barriers to undertaking such experiments while increasing flexibility in designing more complex studies. [ABSTRACT FROM AUTHOR]