1. Mining Artificially Generated Data to Estimate Competency
- Author
-
Robson, Robby, Ray, Fritz, Hernandez, Mike, Blake-Plock, Shelly, Casey, Cliff, Hoyt, Will, Owens, Kevin, Hoffman, Michael, and Goldberg, Benjamin
- Abstract
The context for this paper is the "Synthetic Training Environment Experiential Learning -- Readiness" (STEEL-R) project [1], which aims to estimate individual and team competence using data collected from synthetic, semi-synthetic, and live scenario-based training exercises. In STEEL-R, the "Generalized Intelligent Framework for Tutoring" (GIFT) orchestrates scenario sessions and reports data as experience API (xAPI)statements. These statements are translated into assertions about individual and team competencies by the "Competency and Skills System" (CaSS). Mathematical models use these assertions to estimate the competency states of trainees. This information is displayed in a dashboard that enables users to explore progression over time and informs decisions concerning advancement to the next training phase and which skills to address. To test, tune, and demo STEEL-R, more data was needed than was available from real-world training exercises. Since the raw data used to estimate competencies are captured in xAPI statements, a component called DATASIM was added. DATASIM simulated training sessions by generating xAPI statements that conformed to a STEEL-R "xAPI Profile." This facilitated testing of STEEL-R and was used to create a demo that highlighted the ability to map data from multiple training systems to a single competency framework and to generate a display that team leaders can use to personalize and optimize training across multiple training modalities. This paper gives an overview of STEEL-R, its architecture, and the features that enabled the use of artificial data. The paper explains how xAPI statements are converted to assertions and how these are used to estimate trainee competency. This is followed by a section on xAPI Profiles and on the xAPI Profile used in STEEL-R. The paper then discusses how artificial data were generated and the challenges of modeling longitudinal development and team in these data. The paper ends with a section on future research. [For the full proceedings, see ED623995.]
- Published
- 2022