Search

This paper reviews horizontal and vertical scaling methodologies for adaptive instructional system (AIS) software architectures. The term AIS refers to any instructional approach that accommodates individual differences to facilitate and optimize the acquisition of knowledge and/or skills. The authors propose a variety of scaling methods to enhance the interaction between AISs and low-adaptive training ecosystems with the goal of increasing adaptivity and thereby increasing learning and performance. Typically, low-adaptive training systems only accommodate differences in the learner’s in-situ performance during training and do not consider the impact of other factors (e.g., emotions, prior knowledge, goal-orientation, or motivation) that influence learning. AIS architectures such as the Generalize Intelligent Framework for Tutoring (GIFT) can accommodate individual differences and interact with low-adaptive training ecosystems to model a common operational picture of the training relative. These capabilities enable AISs to track progress toward learning objectives and to intervene and adapt the training ecosystem to needs and capabilities of each learner. Finding new methods to interface AISs with a greater number of low-adaptive training ecosystems will result in more efficient and effective instruction.

This paper examines the functionality of artificially-intelligent agents as a methodology for supporting automated decisions in adaptive instructional systems (AISs). AISs are artificially-intelligent, computer-based systems that guide learning experiences by tailoring instruction and recommendations based on the goals, needs, preferences, and interests of each individual learner or team in the context of domain learning objectives. AISs are a class of instructional technologies that include intelligent tutoring systems (ITSs), intelligent mentors or recommender systems, and intelligent instructional media. This paper explores various agent-based methods to gauge their impact on four automated decisions within the Learning Effect Model (LEM): 1) determining current and predicting future learner states, 2) making recommendations for new experiences (e.g., courses or problem selection), 3) selecting high level instructional strategies to influence long-term learning, and 4) selecting low level instructional tactics to influence near-term learning.

The Generalized Intelligent Framework for Tutoring (GIFT) is a research prototype with three general goals associated with its functions and components: 1) lower the skills and time required to author Intelligent Tutoring Systems (ITSs) in a variety of task domains; 2) provide effective adaptive instruction tailored to the needs of each individual learner or team of learners; and 3) provide tools and methods to evaluate the effectiveness of ITSs and support research to continuously improve instructional best practices. This special issue focuses primarily on the third goal, GIFT as a research testbed. A discussion thread covers each article within this special issue and discusses its actual and potential impact on GIFT as a research tool for AIED. Our primary motivation was to introduce the AIED community to GIFT not just as a research tool, but as an extension of familiar challenges taken on previously by AIED scientists and practitioners. This preface provides a high level overview of the GIFT functions (authoring, instructional delivery and management, and experimentation) and presents its primary design principles. To learn more about GIFT, freely access the software, documentation, and associated technical papers visit www.GIFTtutoring.org .

The goal of this research was the development of a practical architecture for the computer-based tutoring of teams. This article examines the relationship of team behaviors as antecedents to successful team performance and learning during adaptive instruction guided by Intelligent Tutoring Systems (ITSs). Adaptive instruction is a training or educational experience tailored by artificially-intelligent, computer-based tutors with the goal of optimizing learner outcomes (e.g., knowledge and skill acquisition, performance, enhanced retention, accelerated learning, or transfer of skills from instructional environments to work environments). The core contribution of this research was the identification of behavioral markers associated with the antecedents of team performance and learning thus enabling the development and refinement of teamwork models in ITS architectures. Teamwork focuses on the coordination, cooperation, and communication among individuals to achieve a shared goal. For ITSs to optimally tailor team instruction, tutors must have key insights about both the team and the learners on that team. To aid the modeling of teams, we examined the literature to evaluate the relationship of teamwork behaviors (e.g., communication, cooperation, coordination, cognition, leadership/coaching, and conflict) with team outcomes (learning, performance, satisfaction, and viability) as part of a large-scale meta-analysis of the ITS, team training, and team performance literature. While ITSs have been used infrequently to instruct teams, the goal of this meta-analysis make team tutoring more ubiquitous by: identifying significant relationships between team behaviors and effective performance and learning outcomes; developing instructional guidelines for team tutoring based on these relationships; and applying these team tutoring guidelines to the Generalized Intelligent Framework for Tutoring (GIFT), an open source architecture for authoring, delivering, managing, and evaluating adaptive instructional tools and methods. In doing this, we have designed a domain-independent framework for the adaptive instruction of teams.

Teams and teamwork are ubiquitous in military and civilian organizations. Their importance to organizational success cannot be overstated. This article describes the relationship and effect of three concepts: Intelligent Tutoring Systems (ITSs), shared mental models, and teamwork. The nexus between these concepts is examined to determine its capability to support adaptive instruction of teams, defined here as collectives of interdependent individuals who must communicate and interact with each other in order to perform assigned tasks and missions. An assumption underlying this examination is that augmenting the mental modeling processes of ITS with the mental models shared by members of interdependent teams will allow the considerable and increasingly research-established capabilities of intelligent tutoring of individuals to be applied in training teams. Specifically, we reviewed the learning and performance literature to identify how shared mental models of cognition could be used to enhance the adaptive instruction of teams. Our goal is to develop a methodology to enhance training and educational options for institutions that provide adaptive team instruction at the point-of-need. Toward this end, we discuss the adaptation of the Generalized Intelligent Framework for Tutoring (GIFT), an open source tutoring architecture, to accommodate team models and states. While this article makes a first step toward defining a process for team tutoring, challenges remain. Team tutors must have the ability to manage uncertainty and the dynamic nature of team interaction and communication in order to make effective and timely decisions that optimize team and team member performance.

Accurate tracking of a user in a marker-less environment can be difficult, even more so when agile head or hand movements are expected. When relying on feature detection as part of a SLAM algorithm the issue arises that a large rotational delta causes previously tracked features to become lost. One approach to overcome this problem is with multiple sensors increasing the horizontal field of view. In this paper, we perform a systematic evaluation of tracking accuracy by recording several agile movements and providing different camera configurations to evaluate against. We begin with four sensors in a square configuration and test the resulting output from a chosen SLAM algorithm. We then systematically remove a camera from the feed covering all permutations to determine the level of accuracy and tracking loss. We cover some of the lessons learned in this preliminary experiment and how it may guide researchers in tracking extremely agile movements.

This paper examines the components, functions, and interactions of adaptive instructional systems (AISs) as a method to construct a conceptual model for use in the development of IEEE standards. AISs are artificially-intelligent, computer-based systems that guide learning experiences by tailoring instruction and recommendations based on the goals, needs, and preferences of each individual learner or team in the context of domain learning objectives. IEEE is exploring standards and best practices for AIS modeling, interoperability, and evaluation under its Project 2247 and affiliated working group. This paper was composed to document the interaction of learners with AISs in the context of a domain of instruction. The goal is to identify key interactions within AISs that drive instructional decisions, and to identify the data and methods required to support those machine-based instructions. In other words, we seek to identify methods to assess learner/team progress toward instructional objectives (e.g., knowledge, acquisition, skill development, performance, retention, and transfer of skills from instruction to operational/working environments. As part of the examination of AIS elements, we review a set of popular AIS architectures as a method of identifying what makes AISs unique from other instructional technologies. We conclude with recommendations for future AIS research and standards development.

GIFT Cloud is the recently released web-based application version of GIFT, an open-source computer-based tutoring architecture that supports authoring, deployment, and evaluation of intelligent tutoring system technologies. This paper presents the GIFT Cloud Authoring Tools, through the lens of usability. Each major element within the authoring tools is described, along with usability design considerations that were made in order to reduce occurrence of error, to organize information, and to support end-user goals. The initial release of GIFT Cloud supports an iterative design approach, informed by user data and feedback, with an overall goal of making tutor authoring practical for subject matter experts without computer programming or instructional design knowledge. As such, lessons learned from this release, as well as plans for future research and usability improvements, are discussed.

An advantage of computer-based instruction is that student entered information can be saved and used throughout learning. Self-reference (tying information to the self) has been shown to have a positive impact on memory and learning. This study evaluates the impact of including self-reference and familiar popular culture names during the assessment phase of adaptive instruction. Participants engaged with a computer-based tutorial about solving logic grid puzzles and were assessed by completing additional puzzles. The assessment puzzles included the participant’s and friends’ names (self-reference), popular culture names, or generic names. Participants in the popular culture condition spent significantly less time solving the standard puzzle than those in the generic condition, with no difference in percentage correct. The inclusion of popular culture names may have facilitated more efficient task performance while maintaining quality of performance. It is envisioned that this strategy can be implemented in computer-based adaptive instruction to improve task efficiency.

Realistic augmented reality systems require both accurate localization of the user and a mapping of the environment. In a markerless environment this is often done with SLAM algorithms which, for localization, pick out features in the environment and compare how they have changed from keyframe to current frame. However, human head agility, such as seen in video gaming tasks or training exercises, poses a problem; fast rotations will cause all previously tracked features to no longer be within the field of view and the system will struggle to localize accurately. In this paper we present an approach that is capable of tracking a human head's agile movements by using an array of RGB-D sensors and a reconstruction of this sensor data into 360 degrees of features that is fed into our SLAM algorithm. We run an experiment with pre-recorded agile movement scenarios that demonstrate the accuracy of our system. We also compare our approach against single sensor algorithms and show a significant improvement (up to 15 to 20 times better accuracy) in localization. The development of our sensor array and SLAM algorithm creates a novel approach to accurately localize extremely agile human head movements.

Instruction that adapts to individual learner characteristics is often more effective than instruction that treats all learners as the same. A practical approach to making MOOCs adapt to learners may be by integrating frameworks for intelligent tutoring systems (ITSs). Using the Learning Tools Interoperability standard (LTI), we integrated two intelligent tutoring frameworks (GIFT and CTAT) into edX. We describe our initial explorations of four adaptive instructional patterns in the PennX MOOC "Big Data and Education." The work illustrates one route to adaptivity at scale.

This paper discusses the need and methods to develop community-based persona (learner models) to tie together key learner attributes and learning outcomes (e.g., knowledge acquisition) with the goal of facilitating the validation of adaptive instructional strategies and tactics. Adaptive instruction, sometimes referred to as differentiated instruction, is a learning experience tailored to the needs and preferences of each individual learner or team in which strategies (recommendations and plans for action) and tactics (actions by the tutor) are selected with the aim of optimizing learning, performance, retention, and the transfer of skills between the instructional environment (usually provided by an Intelligent Tutoring System or ITS) and the work or operational environment where the skills learned will be applied. Adaptive instructional systems (AISs) use human variability and other learner attributes along with instructional conditions to select appropriate strategies and tactics. This is usually accomplished through the use of machine learning techniques, but large amounts of data are needed to reinforce the learning of these algorithms over time. We propose a method to develop community models more quickly by enabling diverse groups to contribute the results of their experiments and training data in a common instructional domain to a cloud-based model that could be shared by various instructional applications.

Military organizations worldwide continue to invest heavily in research, development, and fielding of virtual and mixed reality simulations and simulators for training. However, the future fiscal environment will be challenging for both simulation as well as the US Army as a whole. Thus, wise design decisions must be made when developing virtual simulations for training. In order to optimize the effectiveness of these simulations, developers must employ trade-off analysis and scientific methods to derive empirical evidence, in order to ensure that the simulation under development is optimized to meet the training requirements, while still adhering to cost and schedule constraints. This paper specifically focuses on the task of employing the optimal visual modality in virtual and mixed reality simulations. This paper reviews the literature on training simulation benefits and taxonomy, and examines the training efficacy of virtual and mixed reality simulation. Major concepts and applications of virtual and mixed reality simulation training efficacy are discussed. A key component of virtual simulation, visual modality, is examined through a literature review and recommendations for visual display design parameters are provided.

Defence forces have always invested a great deal of their resources in training. In recent times, changes in the complexity and intensity of operations have reaffirmed the importance of ensuring that warfighters are adequately prepared for the environments in which they are required to work. The emergence of new operational drivers such as asymmetric threats, urban operations, joint and coalition operations and the widespread use of military communications and information technology networks has highlighted the importance of providing warfighters with the competencies required to act in a coordinated, adaptable fashion, and to make effective decisions in environments characterised by large amounts of sometimes ambiguous information. While investment in new technologies can make available new opportunities for action, it is only through effective training that personnel can be made ready to apply their tools in the most decisive and discriminating fashion, and by doing so transform military technology into defence capability.There are many factors which can have an impact on the efficacy of training, and there are therefore many issues to consider when designing and implementing training strategies. These issues are often complex and nuanced, and in order to grasp them fully a significant investment of time and energy is required. However, the requirement to respond quickly to ever-changing technology, a high operational tempo and minimal staffing may preclude many in today's defence forces from seeking out all such resources on their own.This edited collection provides brief, easy-to-understand summaries of the key issues in defence training and simulation, as well as guidance for further reading. It consists of a collection of short essays and frequently asked questions, each of which addresses a fundamental issue in defence training and simulation, and features an up-to-date reference list to enable the reader to undertake further investigation of the issues that are addressed. In essence, this book provides the optimum starting point, or first resource, for readers to come to terms with the important issues associated with defence training and simulation. The contributions are written by leading scholars from military research institutions in the US, UK, Canada, Australia and New Zealand as well as selected researchers from academic and private sector research institutions.

This paper describes methods for enhancing the experience application program interface (xAPI) to improve the assessment of domain competency modeling for adaptive instruction. xAPI is an e-learning software specification which allows individual learning experiences and achievements to be amassed in a Learning Record Store (LRS). Adaptive instruction includes tailored training or educational experiences usually delivered and guided by Intelligent Tutoring Systems (ITSs). ITSs can more effectively tailor or adapt instruction when they have more accurate models of the learner's prior knowledge or competency. This paper examines the potential effect of methods to more accurately model learner experiences and domain competency in an LRS. Specifically, we recommend five methods to improve xAPI statements by documenting: 1) achievement types; 2) experience duration; 3) experience source information; 4) domain learning and forgetting; and 5) assessment within learning experiences.

The Generalized Intelligent Framework for Tutoring (GIFT) is an open-source domain-independent intelligent tutoring system (ITS) framework. It provides the tools and capabilities to design a complete ITS, manage student instruction, and serve as a testbed for experimentation and analysis. GIFT was developed to provide a cost-effective and efficient means for developing ITSs that have interchangeable parts and can be reused. Additionally, GIFT provides the ability for human factors professionals to develop and run complete experiments in a computer-based environment. This demonstration will provide an introduction to GIFT, along with the motivation behind its design. Further, a demonstration of the tools that are useful to human factors practitioners and psychologists will be provided, and examples of courses that have been developed using GIFT will be shown.

The Generalized Intelligent Framework for Tutoring GIFT project is partially an effort to standardize the systems and processes of intelligent tutoring systems. In addition to these efforts, there is emerging research in agent-driven systems. Agent-based systems obey software and messaging communication protocols and accomplish objectives to the original system, but have different architectural structure. This paper describes the upcoming research changes for GIFT, from a module-driven system to an agent-driven system, the reasons for wanting to do so, the advantages of the change, some initial technical approaches which encapsulate current functionality, and the types of research that this change will enable in the future.

While human tutoring and mentoring are common teaching tools, current US Army standards for training and education are group instruction and classroom training, also known as one-to-many instruction. Recently, the US Army placed significant emphasis on self-regulated learning (SRL) methods to augment institutional training where Soldiers will be largely responsible for managing their own learning. In support of the US Army Learning Model and to provide affordable, tailored, SRL training and educational capabilities for the US Army, the US Army Research Laboratory is investigating and developing adaptive tools and methods to largely automate the authoring (creation), delivery of instruction, and evaluation of computer-regulated training and education capabilities. A major goal within this research program is to reduce the time and skill required to author, deliver, and evaluate adaptive technologies to make them usable by a larger segment of the training and educational community. This research includes 5 interdependent research vectors: individual learner and unit modeling, instructional management principles, domain modeling, authoring tools and methods, and evaluation tools and methods. This report (1 of 6 interdependent research outlines) focuses on individual learner and team modeling research for adaptive training and education. Learner and team models enable adaptive training systems to determine the most appropriate training content and/or methods for each individual or team. The research focuses on the structure of those models and how those models can be used to best adapt training. This outline describes past and current research on learner and team modeling and identifies research gaps that need to be addressed.

While human tutoring and mentoring are common teaching tools, current US Army standards for training and education are group instruction and classroom training, also known as ???one-to-many??? instruction. Recently, the US Army has placed significant emphasis on self-regulated learning \201SRL\202 methods to augment institutional training where Soldiers will be largely responsible for managing their own learning. In support of the US Army Learning Model \201ALM\202 and to provide affordable, tailored SRL training and educational capabilities for the US Army, the US Army Research Laboratory \201ARL\202 is investigating and developing adaptive tools and methods to largely automate the authoring \201creation\202, delivery of instruction, and evaluation of computer-regulated training and education capabilities. A major goal within this research program is to reduce the time and skill required to author, deliver, and evaluate adaptive technologies to make them usable by a larger segment of the training and educational community. This research includes 6 interdependent research vectors: individual learner and unit modeling, instructional management principles, domain modeling, authoring tools and methods, and evaluation tools and methods. This report \2011 of 6 interdependent research outlines\202 focuses on domain modeling research for adaptive training and education with the goal of guiding learning in militarily relevant training and educational domains.

The purpose of this technical report is to provide an exemplar for conducting an analysis of a qualitative dataset using machine learning techniques. Qualitative data are measured or expressed as a natural language description (e.g., category or attribute) rather than numbers as in quantitative datasets. It is often difficult to evaluate the relationship between qualitative variables of interest and outcomes (dependent variables), but machine learning techniques offer simplified methods to classify these outcomes. WEKA, the Waikato Environment for Knowledge Analysis, is a popular set of machine learning algorithms developed at the University of Waikato in New Zealand, which can be used to analyze both qualitative and quantitative data. To illustrate the use of WEKA on a qualitative dataset, we selected a known set of primate species with the desire to classify them into 1 of 3 classes (prosimians, monkeys, and apes) based on 7 qualitative attributes. When an existing dataset is used with known relationships, it allows us to evaluate a large number of WEKA algorithms in a relatively short time and validate their accuracy with the goal of identifying best practices for analyzing qualitative data.

An important component of metacognition relates to the understanding and use of strategies. Thus, measuring and supporting students’ strategy understanding in complex open-ended learning environments is an important challenge. However, measuring students’ strategy use and understanding is a difficult undertaking. In this paper, we present our design for representing and supporting students in their understanding of strategies while working in complex, open-ended learning environments using the Generalized Intelligent Framework for Tutoring (GIFT). Our approach utilizes a wealth of previous research and relies on three primary instructional interventions: contextualized conversational assessments and feedback; reviewing knowledge and strategies; and teaching through analogies. We believe that incorporating these approaches into GIFT will allow for powerful instruction of complex tasks and topics.

The tutorial described in this report was conducted at the Human-Computer Interaction International Augmented Cognition Conference in Creta Maris, Crete, Greece, in June 2014. The purpose of this tutorial was 3-fold: 1) familiarize participants with the fundamental principles of adaptive Intelligent Tutoring System (ITS) design; 2) illustrate how ITS design influences/enables self-regulated learning; and 3) discuss the need for standards for authoring of ITS, modeling of learners and experts, automated instructional strategies, and methods of analysis for ITS technologies.

During computer-based tutoring sessions, Intelligent Tutoring Systems (ITSs) adapt planning and manage real-time instructional decisions. The link between learner data and enhanced performance is the adaptive tutoring learning effect chain through which learner data informs learner state classification which in turn informs optimal instructional decisions to enhance performance. This paper examines the roles and influence of learner data in both short-term (also called run-time or session) and long-term (also called persistent) learner models used to support adaptive tutoring decisions within the Generalized Intelligent Framework for Tutoring (GIFT). To enhance the usability of tutoring systems and learner performance, recommendations for the design of future learner models are also presented.

Scenario-based training exemplifies the learning-by-doing approach to human performance improvement. In this paper, we enumerate the advantages of incorporating automated scenario generation technologies into the traditional scenario development pipeline. An automated scenario generator is a system that creates training scenarios from scratch, augmenting human authoring to rapidly develop new scenarios, providing a richer diversity of tailored training opportunities, and delivering training scenarios on demand. We introduce a combinatorial optimization approach to scenario generation to deliver the requisite diversity and quality of scenarios while tailoring the scenarios to a particular learner's needs and abilities. We propose a set of evaluation metrics appropriate to scenario generation technologies and present preliminary evidence for the suitability of our approach compared to other scenario generation approaches.

Famously, individual expert tutoring holds the promise of two standard deviations of improvement over classroom-based instruction. Current content-scaling techniques have been able to prove one standard deviation of improvement. However, just as expert tutors take the motivation and emotional state of the student into account for instruction, so too must computer instructors. Differences between individuals and individual baselines make this difficult, but this information is known across one training session. The construction of assessing modules in realtime, from the available performance and sensor datastreams, skirts these problems, but is technically difficult. This research investigates automated student model construction in realtime from datastreams as a solution from which to base pedagogical strategy recommendations.

The development of intelligent tutoring systems (ITS) capable of supporting training experiences for geographically-distributed team members in shared virtual simulation environment presents considerable challenges Even human tutors face challenges in developing team cohesion, coordinating roles and assessing contributions Just as a human tutor might assess collective performance, a team ITS must be capable of passively assessing the trainees' readiness to learn and evaluating their progress toward team objectives Passive sensing methods offer the opportunity for the ITS to understand the team's cognitive and emotional state without interfering with the learning process It also helps determine their any interventions needed to optimize performance This article reviews challenges and hypothesizes functions for computer-based distributed team tutors.

With the Navy leaning towards a smaller but better trained force, using simulators in training becomes a cost effective way of keeping our military edge. Several innovations are being utilized in today's simulators. Hardware and software designs now include distributed computer architectures, real-time, multi-tasking operating systems, flexible generic consoles, commercial off-the-shelf equipment, standard real-time ocean acoustic models, and standard acoustic data bases. This paper describes these technologies and their innovative application in the Navy's newest Anti-Submarine Warfare Team Trainer, Device 14A12. Device 14A12 simulates sonar, fire control, electronic warfare, bridge, ownship aircraft and Naval Tactical Data Systems (NTDS) for 16 surface ownship classes. Device 14A12 also simulates multiple air, surface and subsurface threat, neutral and friendly platforms as well as missiles and torpedoes. The innovative modeling of these systems along with Device 14A12's enormous flexibility will allow the Navy to provide effective tactical training for the surface fleet now and in the future.

The purpose of this paper is to provide a basis for planning, designing and testing distributed exercisesand experiments. Lessons-learned from the Synthetic Theater of War - Europe (STOW-E) exercise conducted in November 1 994 are incorporated to provide system engineers/integrators areas of consideration in the design and testing of Distributed Interactive Simulation (DIS) exercises/experiments.Issues involving DIS compliance in accordance with IEEE 1278.11, and interoperability and compatibilitytesting measures are also discussed. Design and test issues for live, virtual and constructive simulationsare considered. This paper assumes basic knowledge of DIS principles. 1. BACKGROUND "One ofthe emerging capabilities of real-time simulation is the ability to create large virtual worlds in whichmany subjects can interact."2 This virtual or synthetic environment is shared by virtual and constructiveentities, and virtual representations of live entities. Distributed simulations (live, virtual and constructive)share entity information and interaction data via DIS protocols transmitted on local and wide area networksin support of training and testing exercise missions (see figure 1).