Search

We introduce the Cognitive Science Community to the challenges faced in human-in-the-loop (HiTL) decision-making for complex and dynamic industrial chemical processes. A microworld is presented which includes advanced dynamic simulations, an optimal control-based decision-support algorithm and a preliminary human machine interface. Three example applications are discussed, including decision-making in plant startup and shutdown, shared control using human and HiTL supervisory Model Predictive Control (MPC), and a pilot study of operator eye-tracking during a complex startup task. The paper also discusses ongoing and future experimental efforts highlighting the utility of the microworld in understanding human cognition and designing advanced operator training software for various process operation related complex tasks.

Tetris is a fast-paced puzzle solving game that requires play-ers to rapidly maneuver falling blocks to clear rows and scorepoints. Skilled Tetris players learn to execute moves in thegame very quickly to keep up with the increasing time pres-sure. But world champion Tetris players employ more complexstrategies that save precious milliseconds that enable them toreach even higher levels of play. Such strategies show mas-tery of the game’s event structure, but also come with a startupcost— a “cognitive speed bump”— wherein they must mo-mentarily decide whether to rotate a block left or right, evenfor scenarios where the distinction is not meaningful for per-formance. We present data showing both the world champions’superior overall action times, but also a preliminary “speedbump” that is consistent both within and between world cham-pion players. Potential underlying memory structures are ex-plored, and implications are discussed for both the Soft Con-straints Hypothesis and the relationship between Hick’s Lawand expertise.

Error detection and correction is a vital part of skill acquisition, but when training a complex, real time, dynamic task,it can be difficult to isolate a true mistake in a sequence of decisions without clear correct choices. We use previouslydeveloped high-performing, human-like models of the video game Tetris (Sibert et al., 2017) to analyze individual pieceplacement decisions for players of high and low skill. In cases where the model’s choice differed from the human’s choice,we examine the eye fixations made during the placement decision to determine if the disagreement is caused due to theplayer performing at lower level than the model (i.e. not being aware of a better placement), the player performing at ahigher level than the model (i.e. deliberately making a suboptimal move in service of a long term strategy), or the playermaking a true error.

We bring together four researchers who study exper-tise in team or in solo (i.e., individual) performance. Teamresearch tends to either collect a lot of questionnaire dataafter performance or a little data, in real-time, by humanobservers. Studies of solo performers are often restrictedto convenience samples of task novices, who often spendless than an hour learning and performing the task. Incontrast, the research of all four of our panelists is no-table for using tasks which require days-to-years of prac-tice and for the quantities of data collected. Discussionswill emphasize the contributions these approaches aremaking to theoretical cognitive science.

How do individuals learn a complex task? Averaging performance over a group of individuals implicitly assumes that onlyone set of methods exists for accomplishing the task and that all learners acquire those methods in the same sequence.Rather than profiling a mythical “average subject”, we focus on individuals using SpotLight – a tool for analyzing changesin individual performance as a complex task is learned. Specifically, we investigate 9 individuals who spent 31 hourslearning the task of Space Fortress (SF). The SpotLight enables us to uncover the evolution of strategies and the iterativeefforts of individuals to explore and devise new ways to improve performance. To our surprise, these players seem tohave followed a common ‘design for the weakest link’ rule, in which after the current weakest link of performance isstrengthened, an individual’s attention turns to the next weakest link.

For Tetris, clearing 4 lines at once (a ”Tetris”) results in 7.5 times as many points as clearing one line four times. Gettinga Tetris requires a solid block of filled cells, 9 columns wide and 4 rows high. That block leaves vacant one column. If anI-beam appears, all 4 rows can be cleared. Finalists at the Classic Tetris World Championships have an explicit subgoalstructure not seen in lesser players. Among the 32 competitors, the 4 finalists are those who are most adept at maintainingor preparing the board for a Tetris by executing one of these subgoals, as needed. We present a video-based analysis whichcompares the proportion of time spent on each activity between those eliminated on the first tournament round and thosewho survive to the final round.

Part-task training is a technique which involves separating the target task into parts and presenting them during training.This approach has been used to train users to perform optimally in dynamic task environments. The present study investi-gated the effects of fractionation, a part-task training approach, versus whole-task training to improve performance in thevideo game Tetris by focusing on an important sub-task element of the game. Seventy-eight young adults were trained onTetris with one of three training regimens: 1) Part-task training with feedback, 2) Part-task training with no feedback, and3) Whole-task training in which participants practiced the whole game to obtain the highest overall score. Results showthat baseline performance influences training gains and feedback may not be helpful for learning. Training gains fromthe different training regimens show that tasks with highly interdependent components may benefit most from whole-tasktraining.

Pupillometry uses pupil diameter as a physiological measure of cognitive effort and load. In static tasks, pupillometry hasrevealed that cognitive effort varies with expertise, and, combined with gaze analysis, shows that experts can exert effort tofocus on non-salient visual input. Much real-life expertise is practiced in dynamic tasks, and expert effort in dynamic tasksremains unstudied. Using tetris as a dynamic task environment, we collected pupil and gameplay data from individuals ofvarying expertise levels. We then use collected data and examine cognitive workload differences across levels of expertise.Consistent with studies of image saliency and gaze, our results indicate that experts and novices engage differently withthe task and do not experience the same cognitive workload. Further inspection will likely reveal strategy-level sources ofthese differences.

Why games? How could anyone consider action gamesas experimental paradigms for Cognitive Science? In 1973,as one of three strategies he proposed for advancing Cogni-tive Science, Allen Newell exhorted us to “accept a singlecomplex task and do all of it.” More specifically, he told usthat rather than taking an “experimental psychology as usualapproach” that, we should “focus on a series of experimentaland theoretical studies around a single complex task” so as todemonstrate that our theories of human cognition were pow-erful enough to explain, “a genuine slab of human behavior”with the studies fitting into a detailed theoretical picture. Ac-tion games represent the type of experimental paradigms thatNewell was advocating and the current state of programmingexpertise and laboratory equipment, along with the emer-gence of Big Data (Griffiths, 2015) and Naturally OccurringData Sets (NODS, Goldstone & Lupyan, 2016), provide thetechnologies and data needed to realize his vision. ActionGames enable us to escape from our field’s regrettable fo-cus on novice performance to develop theories that accountfor the full range of expertise through a twin focus on ex-pertise sampling (across individuals) and longitudinal studies(within individuals) of simple and complex tasks.This Symposium is inspired by the recent Action Gamesas Experimental Paradigms for Cognitive Science (Game-XP), issue of Topics in Cognitive Science (topiCS), April2017. It includes late-breaking work from some of the re-searchers represented in that topic as well as new work bynew researchers.

Tetris can be viewed as a highly complex decision making task, and used as a paradigm for studying human expertise.We hypothesized that models capable of playing Tetris for a long time are doing so by adopting slow but steady strategies toaccumulate points, while human players are much more prone to using high-risk, high-reward strategies that earn more pointsin a shorter time frame. This work used the MindModeling.org computational cognitive modeling platform to develop the bestmodels capable of playing long term games and short term games, and then compared the performance of the two. The bestlong term model adopted the slow and steady strategy, while the best short term model displayed the higher-risk, higher-rewardstrategy that more closely matches behavior observed in human players. Models that ”trained long” but ”played short” didworse than those that both trained and played ”short.”

Does neuroscience have the potential to inform education? In thisdebate, three participants will describe emerging results inneuroscience with translatable links to learning and memory.Experts in learning sciences will discuss connections as well aslimits in the ways that neuroscience can inform education at theindividual and classroom levels. Although significant progress hasbeen made in our understanding of how the brain learns andremembers, the question for this symposium is whether thisprogress does or can provide a direct bridge from brain science toeducation.

In complex task domains, such as games, students may ex-ceed their teachers. Such tasks afford diverse means to trade-off one type of performance for another, combining task ele-ments in novel ways to yield method variations and strategydiscoveries that, if mastered, might produce large or smallleaps in performance. For the researcher interested in the de-velopment of extreme expertise in the wild, the problem posedby such tasks is “where to look” to capture the explorations,trials, errors, and successes that eventually lead to the inven-tion of superior performance. In this paper, we present severalsuccessful discoveries of methods for superior performance.For these discoveries we used Symbolic Aggregate Approx-imation as our method of identifying changepoints withinscore progressions in the venerable game of Space Fortress.By decomposing performance at these changepoints, we findpreviously unknown strategies that even the designers of thetask had not anticipated.

What can a machine learning simulation tell us about humanperformance in a complex, real-time task such as Tetris¬ó?Although Tetris is often used as a research tool (Mayer,2014), the strategies and methods used by Tetris players haveseldom been the explicit focus of study. In Study 1, we usecross-entropy reinforcement learning (CERL) (Szita & Lorincz,2006; Thiery & Scherrer, 2009) to explore (a) the utilityof high-level strategies (goals or objective functions) formaximizing performance and (b) a variety of features andfeature-weights (methods) for optimizing a low-level, onezoidoptimization strategy. Two of these optimization strategiesquickly rise to performance plateaus, whereas two otherscontinued towards higher but more jagged (i.e., variable)plateaus. In Study 2, we compare the zoid (i.e., Tetris piece)placement decisions made by our best CERL models withthose made by the full spectrum of novice-to-expert humanTetris players. Across 370,131 episodes collected from 67 humanplayers, the ability of two CERL strategies to classify humanzoid placements varied with player expertise from 43%for our lowest scoring novice to around 65% for our threehighest scoring experts.

The framework of "plateaus, dips, and leaps" shines light on periods when individuals may be inventing new methods of skilled performance. We begin with a review of the role "performance plateaus" have played in (a) experimental psychology, (b) human-computer interaction, and (c) cognitive science. We then reanalyze two classic studies of individual performance to show plateaus and dips which resulted in performance leaps. For a third study, we show how the statistical methods of "Changepoint Analysis" plus a few simple heuristics may direct our focus to periods of performance change for individuals. For the researcher, dips become the marker of exploration where performance suffers as new methods are invented and tested. Leaps mark the implementation of a successful new method and an incremental jump above the path plotted by smooth and steady log--log performance increments. The methods developed during these dips and leaps are the key to surpassing one's teachers and acquiring extreme expertise.

Abstract Background Varicella zoster virus (VZV) is a neurotropic alphaherpesvirus that infects humans and results in chickenpox and herpes zoster. A number of VZV genes remain functionally uncharacterized and since VZV is an obligate human pathogen, rigorous evaluation of VZV mutants in vivo remains challenging. Simian varicella virus (SVV) is homologous to VZV and SVV infection of rhesus macaques (RM) closely mimics VZV infection of humans. Recently the SVV genome was cloned as a bacterial artificial chromosome (BAC) and BAC-derived SVV displayed similar replication kinetics as wild-type (WT) SVV in vitro. Methods RMs were infected with BAC-derived SVV or WT SVV at 4x105 PFU intrabronchially (N=8, 4 per group, sex and age matched). We collected whole blood (PBMC) and bronchoalveolar lavage (BAL) at various days post-infection (dpi) and sensory ganglia during latent infection (>84 dpi) at necropsy and compared disease progression, viral replication, immune response and the establishment of latency. Results Viral replication kinetics and magnitude in bronchoalveolar lavage cells and whole blood as well as rash severity and duration were similar in RMs infected with SVV BAC or WT SVV. Moreover, SVV-specific B and T cell responses were comparable between BAC and WT-infected animals. Lastly, we measured viral DNA in sensory ganglia from both cohorts of infected RMs during latent infection. Conclusions SVV BAC is as pathogenic and immunogenic as WT SVV in vivo. Thus, the SVV BAC genetic system combined with the rhesus macaque animal model can further our understanding of viral ORFs important for VZV pathogenesis and the development of second-generation vaccines.