Search

The National Academies recently issued a consensus study report summarizing the state of the art and research needs relating to Human-AI teaming, especially in the context of dynamic, high-risk environments such as multi-domain military operations. This consensus study was conducted by the National Academies Board on Human-Systems Integration (BOHSI). This panel, organized by BOHSI, brings together prominent researchers, including several members of the consensus committee, to discuss the state of the art and research frontiers for development of effective human-AI teams that can operate resiliently in complex, data intensive, and dynamically paced environments.

The Behavior Rating Inventory of Executive Function–Adult Version (BRIEF-A) is a standardized rating scale of subjective executive functioning. We provide univariate and multivariate base rates (BRs) for scale/index scores in the clinical range ( T scores ≥65), reliable change, and inter-rater information not included in the Professional Manual. Participants were adults (ages = 18–90 years) from the BRIEF-A self-report ( N = 1,050) and informant report ( N = 1,200) standardization samples, as well as test–retest ( n = 50 for self, n = 44 for informant) and inter-rater ( n = 180) samples. Univariate BRs of elevated T scores were low (self-report = 3.3%–15.4%, informant report = 4.5%–16.3%). Multivariate BRs revealed the common occurrence of obtaining at least one elevated T-score across scales (self-report = 26.5%–37.3%, informant report = 22.7%–30.3%), whereas virtually none had elevated scores on all scales. Test–retest scores were highly correlated (self = .82–.94; informant = .91–.96). Inter-rater correlations ranged from .44 to .68. Significant ( p < .05) test–retest T-score differences ranged from 7 to 12 for self-report, from 6 to 8 for informant report, and from 16 to 21 points for inter-rater T-score differences. Applications of these findings are discussed.

Discussion Panel Abstract: The recent Boeing 737MAX accidents crystalized for the public the complexity of anticipating system and operator performance and developing a system design that prevents catastrophic outcomes. The operational situations, progression of flightcrew actions, and system behaviors that led to the two accidents had not been anticipated by the manufacturer or the regulator. These accidents were only the most recent examples of our failure to anticipate and manage operational complexities and operator performance. The art and science of human factors has yet to perfect risk assessment (or safety assessment) for complex systems. In the not-so-distant past, system risk assessment made estimates of human error probabilities (HEPs) for specific operational tasks, which were combined with estimated equipment failure rates to produce an overall risk estimate. Indeed, these Human Reliability Analysis (HRA) techniques have evolved over decades and are still being developed (e.g., IDHEAS-ECA, Xing et al., 2020), partly because they satisfy the need for a simple quantitative threshold that can be used by industry and regulators: if risk probability is too high, change the design or some other aspect of operations. Through the years, there have been critiques of the HRA approach (e.g., Hollnagel, 1998) that led to revisions, such as focusing on cognitive functions instead of operator tasks, but not to the basic quantitative risk-estimation approach. Other approaches to assessing risk/safety have wandered down other paths: attempting to capture system complexity from an operator’s perspective (Roth, Mumaw, Lewis, 1994; Nuclear Regulatory Commission, 2000), or better documenting the many ways in which system operators manage complexity daily to find ways to improve their capacity (Hollnagel, Woods, & Leveson, 2006). These approaches have used different measures than HEPs; e.g., measures of operator performance, measures of interface usability/design, measures of task complexity, and the analysis of system constraints. In this panel, we offer different perspectives on risk/safety assessment as it relates to operator performance in complex systems. Foundational to assessment is deciding the nature of safety and the role of operator performance. Another important question is, as you move away from simple quantitative measures, how do you establish safety thresholds? That is, what guidance can we give to industry and regulators regarding how to measure safety and how to decide that action is required on the basis of safety.

New systems are often based on optimistic assumptions of how they will improve human performance. In the cognitive engineering tradition, these assumed benefits are regarded as hypotheses that need to be tested. An important element of a system user evaluation is to determine whether the hypothesized benefits are realized. Evaluation may also uncover unsupported aspects of performance or unanticipated side-effects of introducing the new technology that need to be addressed. We present a work-centered approach to user evaluation intended to meet these objectives, focusing specifically on design of tailored user-feedback questionnaires ( work-centered questionnaires) that are intended to be diagnostic of how specific system elements do, or do not, support work. We summarize two recent evaluation studies we have conducted that illustrate our approach and the diagnostic power of work-centered questionnaires. We discuss how the goals and approach of a work-centered evaluation differ from more traditional approaches to usability evaluation that emphasize the use of standardized questionnaires and broad assessments of usability.

Systemic lupus erythematosus (SLE) is a chronic autoimmune inflammatory disease affecting both adults and children. Belimumab is the only biologic approved for SLE, and the first in a class of drugs known as B-lymphocyte stimulator-specific inhibitors. The introduction of intravenous belimumab in 2011 was a major advance, being the first new therapy approved for SLE in over 50 years. As of April 2021, more than 7200 people with SLE have received belimumab in clinical studies, and it is approved in over 75 countries for the treatment of adults with SLE. A subcutaneous, self-injectable belimumab formulation was licensed in 2017 by both the US Food and Drug Administration (FDA) and European Medicines Agency (EMA). Belimumab was then approved for use in children in Europe, the USA and Japan in 2019, and China and Brazil in 2020. Recently, belimumab became the first FDA-approved drug for the treatment of adults with active lupus nephritis (LN), the most-common severe manifestation of SLE. Over the past 10 years, belimumab has established its position as a disease modifier in the SLE treatment paradigms. Robust evidence from randomised clinical studies and observational, real-world studies has demonstrated the tolerability and efficacy of belimumab for reducing disease activity and the risk of new, severe SLE flares. This enables patients to taper their glucocorticoid use, which limits damage accumulation. Significantly more patients with active LN met the criteria for renal responses and were at less risk of a renal-related event or death after receiving belimumab plus standard therapy, compared with standard therapy on top of mandatory steroid reduction. Ongoing clinical studies are evaluating belimumab’s effectiveness in various indications beyond SLE. Post-marketing and registry studies are gathering additional data on key areas such as pregnancy outcomes after belimumab exposure and belimumab co-administration with other biologics.

Objective: To determine where the current literature stands in regard to diet/exercise interventions on cognition in overweight or obese individuals. Data Source: A rapid review was conducted of English-language studies published in Medline from January 1965 to January 2020. Study Inclusion and Exclusion Criteria: Included studies were intervention studies lasting ≥12 weeks, with participants aged ≥65 years, with a body mass index ≥25 kg/m2 Data Extraction: Data extracted included study population, duration, intervention design, outcomes, and results. Data Synthesis: Outcomes were qualitatively measured due to paucity of RTC. Results: 1845 citations were identified, 31 full-text articles were reviewed, and 5 studies were included. Studies had usual care control groups and combined exercise/diet intervention groups with 31-3,526 participants randomized to each arm. Mean age of participants was 69.2-83.4 years. Studies reporting on cognitive changes showed marginally significant positive changes in cognition, and those that reported BMI indicated potential improvements in cognition. Conclusions: The number of interventions assessing the combined effects of both diet and exercise is low. Future studies should evaluate the impact of combined effects to ascertain whether cognitive decline may be reversed in older adults with a BMI ≥25 kg/m2.

The National Academies Board on Human Systems Integration (BOHSI) has organized this session exploring the state of the art and research and design frontiers relating to the application of Artificial Intelligence (AI) technologies to Healthcare. Advances in AI, machine learning, and robotics promise to revolutionize access to and quality of healthcare. At the same time lessons learned from earlier applications of AI technologies suggest a need to carefully consider the user and use context and the need for effective human-machine teaming design to avoid brittle systems that fail to be used or worse that negatively impact patient safety. The Panel brings together prominent researchers from both the Human Factors and Computer Science and AI communities to discuss promising applications, challenges and short-falls and ways forward for achieving the promise of enhanced delivery of health care through application of AI technologies.

The present study sought to examine the relationship between musical preference and aspects of aggression in a United States of America community sample. A total of 400 adults ( M age = 34.14; 50.3% female) completed the Short Test of Musical Preference, Revised (STOMP-R), the Diagnostic and Statistical Manual of Mental Disorders (5th ed., DSM-5) Level 1 Cross-Cutting Symptom Measure, and several aggression questionnaires aimed at assessing normative beliefs about aggression and prior history of aggressive behaviors. Results of correlational analyses revealed significant relationships between all examined musical genres and psychopathology. Results of regression analyses revealed the presence of psychopathology as the main contributor to scores across aggression measures, with age and gender variables also varying in their significance. Importantly, preference for intense and rebellious musical genres (i.e., alternative, rock, punk, and heavy metal) was a nonsignificant predictor across all aggression questionnaires. As such, the current results do not support a correlational nor causal link between intense and/or rebellious musical preference and aggressive behaviors.

A qualitative egocentric social network approach was taken to explore motivations for pre-exposure prophylaxis (PrEP)-related communication between women who inject drugs and network members. Eligible participants were HIV-negative, 18 years or older, and participating in a PrEP demonstration project in Philadelphia, PA, USA. The study employed content analysis of in-depth interviews to identify themes related to contextual and relational factors impacting PrEP communication within networks. Participants ( n = 20) named on average three network members, resulting in a total of 57 unique relationships. PrEP conversations occurred within 30 of the 57 relationships, and motivations were to benefit others, to benefit themselves, and due to a sense of obligation. Some conversations also occurred when a peer unexpectedly found their pills. Taking a qualitative approach to network analysis provided a nuanced understanding of how interpersonal characteristics motivated PrEP conversations. Network interventions that facilitate information diffusion and social support may increase PrEP uptake and adherence among women who inject drugs.

To date, studies have found scattered and inconsistent use of existing opioid prescription support tools, such as Prescription Drug Monitoring Systems (PDMPs) and guidelines, and little research has been done to identify barriers for adopting them and better ways to support clinician decision making. This pilot study aims to better understand the flow, resources used, and challenges faced when prescribing opioid medications. The study will help shape research questions for a larger study geared towards informing future decision aid designs and considerations for clinicians in the emergency department (ED). Semi-structured interviews were conducted with a total of six emergency medicine physicians and nurses using an abbreviated variant of the Critical Decision Method. Discussions were focused around how participants made their decisions and what was difficult about them, in addition to whether current aids and tools are helpful. Common themes were identified from free-text notes in the following categories: cognitive and communication challenges, challenges with existing tools and databases, and decision-making strategies. Nurse responses were most commonly related to gaps in communication between physicians and patients and ensuring patient satisfaction. Physicians most commonly reported challenges with existing tools. While existing resources were said to be very helpful, several improvements were suggested by each group. By understanding these types of challenges and dynamics between physicians and nurses in the ED, we can better identify ways to improve the design of opioid prescription decision-making aids in the future.

The experiences of violence and overdose are highly prevalent among women who use illicit drugs. This study sought to ascertain whether multiple victimizations during adulthood increase the frequency of women’s overdose. The sample comprised 218 women recruited at Philadelphia harm reduction sites during 2016–2017. Victimization was assessed as exposure to 16 types of adulthood violence. Three measures were constructed for multiple victimizations: continuous and categorical polyvictimization, and predominant violence domain. Negative binomial regression estimated the incidence rate ratio (IRR) of lifetime overdoses from multiple victimizations. Lifetime history of opioid use (88.6%) and drug injection (79.5%) were common. Among overdose survivors (68.5%), the median of lifetime overdoses was 3. The majority of participants (58.7%) were victims of predominantly sexual violence, 26.1% experienced predominantly physical abuse/assault, and 3.7% were victims of predominantly verbal aggression/coercive control. Participants reported a mean of seven violence types; the higher-score category of polyvictimization (9–16 violence types) comprised 41.7% of the total sample. In multivariable models, one-unit increase in continuous polyvictimization was associated with 4% higher overdose rates (IRR: 1.04, 95% confidence interval [CI]: [1.00, 1.08]). Compared to women who were not victimized (11.5%), those in the higher-score category of polyvictimization (IRR: 2.01; 95% CI: [1.06, 3.80]) and exposed to predominantly sexual violence (IRR: 2.10, 95% CI: [1.13, 3.91]) were expected to have higher overdose rates. Polyvictimization and sexual violence amplified the risk of repeated overdose among drug-involved women. Female overdose survivors need to be screened for exposure to multiple forms of violence, especially sexual violence. Findings underscore the need to scale-up victimization support and overdose prevention services for disenfranchised women.

Objective The understanding of systemic lupus erythematosus (SLE) and lupus nephritis (LN) pathogenesis remains incomplete. This review assessed LN development in SLE, within-LN progression and progression to end-stage renal disease (ESRD). Methods A keyword-based literature search was conducted, and 26 publications were included. Results Overall, 7–31% of patients had LN at SLE diagnosis; 31–48% developed LN after SLE diagnosis, most within 5 years. Class IV was the most commonly found LN class and had the worst prognosis. Histological transformation occurred in 40–76% of patients, more frequently from non-proliferative rather than proliferative lesions. Cumulative 5- and 10-year ESRD incidences in patients with SLE were 3% and 4%, respectively, and 3–11% and 6–19%, respectively, in patients with SLE and LN. Conclusions Elevated serum creatinine was identified as a predictor of worsening disease state, and progression within LN classes and from SLE/LN to ESRD. This review highlights the substantial risk for developing LN and progressing to ESRD amongst patients with SLE.

This study was designed to expand on a recent meta-analysis that identified ≤42 as the optimal cutoff on the Word Choice Test (WCT). We examined the base rate of failure and the classification accuracy of various WCT cutoffs in four independent clinical samples ( N = 252) against various psychometrically defined criterion groups. WCT ≤ 47 achieved acceptable combinations of specificity (.86–.89) at .49 to .54 sensitivity. Lowering the cutoff to ≤45 improved specificity (.91–.98) at a reasonable cost to sensitivity (.39–.50). Making the cutoff even more conservative (≤42) disproportionately sacrificed sensitivity (.30–.38) for specificity (.98–1.00), while still classifying 26.7% of patients with genuine and severe deficits as non-credible. Critical item (.23–.45 sensitivity at .89–1.00 specificity) and time-to-completion cutoffs (.48–.71 sensitivity at .87–.96 specificity) were effective alternative/complementary detection methods. Although WCT ≤ 45 produced the best overall classification accuracy, scores in the 43 to 47 range provide comparable objective psychometric evidence of non-credible responding. Results question the need for designating a single cutoff as “optimal,” given the heterogeneity of signal detection environments in which individual assessors operate. As meta-analyses often fail to replicate, ongoing research is needed on the classification accuracy of various WCT cutoffs.

Function allocation refers to strategies for distributing system functions and tasks across people and technology. We review approaches to function allocation in the context of human machine teaming with technology that exhibits high levels of autonomy (e.g., unmanned aerial systems). Although most function allocation projects documented in the literature have employed a single method, we advocate for an integrated approach that leverages four key activities: (1) analyzing operational demands and work requirements; (2) exploring alternative distribution of work across person and machine agents that make up a human machine team (HMT); (3) examining interdependencies between human and autonomous technologies required for effective HMT performance under routine and off-nominal (unexpected) conditions; and (4) exploring the trade-space of alternative HMT options. Our literature review identified methods to support each of these activities. In combination, they enable system designers to uncover, explore, and weigh a range of critical design considerations beyond those emphasized by the MABA–MABA (“Men are better at, Machines are better at”) and Levels of Automation function allocation traditions. Example applications are used to illustrate the value of these methods to design of HMT that includes autonomous machine agents.

This paper presents a tradespace framework designed to analyze tradeoffs associated with different crewing configuration options. It was created in support of the United States Army’s Future Vertical Lift (FVL) program whose goal is to develop future Army rotorcraft that incorporate advanced automation and can be flexibly crewed depending on mission context. The tradespace framework provides an analytic tool that can help Army decision makers evaluate the tradeoffs associated with different crewing configurations and compare crewing options for different mission contexts. While the framework was developed to assess crew configurations for military rotorcraft, the use of tradeoffs as an organizing framework to analyze crewing alternatives can be applied to a variety of domains.

The National Academies Board on Human Systems Integration (BOHSI) has organized this session exploring the state of the art and research and design frontiers for intelligent systems that support effective human machine teaming. An important element in the success of human machine teaming is the ability of the person on the scene to develop appropriate trust in the automated software (including recognizing when it should not be trusted). Research is being conducted in the Human Factors community and the Artificial Intelligence (AI) community on the characteristics that software need to display in order to foster appropriate trust. For example, there is a DARPA program on Explainable AI (XAI). The Panel brings together prominent researchers from both the Human Factors and AI communities to discuss the current state of the art, challenges and short-falls and ways forward in developing systems that engender appropriate trust.

This research is a first stage in developing a method for modeling the clinician workload associated with an emergency medicine patient in order to display workload for purposes of managing clinician workload and emergency department (ED) flow. We proposed a multi-stage approach of predicting patient-related drivers of clinician’s workload in the emergency department. We trained the model from one month of electronic health record data (EHR) records of an ED. The model predicts the amount of work that individual patients contribute to the workload of clinicians. It can potentially help to manage clinician workload by supporting the decision of assigning new patients.

We conducted a study to examine the factors that contribute to run through switches (RTS) in passenger rail yard operations. RTS occur when a train runs through a misaligned switch. It can result in damage to the switch that, if not detected and repaired, can increase the risk of derailment of a subsequent train coming in the opposite direction. The study was conducted at one passenger railroad and included interviews and focus groups with employees at different levels in the organization, as well as observations of yard operations. A sociotechnical systems perspective was used to identify physical infrastructure, individual, team and organizational contributors to RTS. This included performing a Systems Theoretic Process Analysis (STPA) to identify how interacting factors can lead to RTS. This paper provides a summary of the main results. It illustrates the value of adopting a sociotechnical lens when investigating safety incidents that involve human performance.

Objective To evaluate the safety, tolerability and efficacy of subcutaneous (SC) belimumab in patients with systemic lupus erythematosus (SLE) beyond 1 year. Methods This was a 24-week, open-label extension following a 52-week, double-blind, placebo-controlled trial of belimumab SC. Patients who completed the double-blind phase were eligible to enter the open-label phase. All patients received weekly belimumab 200 mg SC plus standard SLE therapy. Outcome measures included safety and efficacy (SLE Response Index (SRI) and SLE Flare Index (SFI) rates), and changes in biomarker and B cell levels. Results Of 677 patients who completed the 52-week, double-blind phase, 662 entered the open-label phase; 206 had previously received placebo and 456 had previously received belimumab. Despite differences in total belimumab exposure (24 weeks in the placebo-to-belimumab group versus 76 weeks in the belimumab group), the proportions of patients experiencing more than one adverse event (AE) or a serious AE in the open-label phase were similar between groups (placebo-to-belimumab: 51.5 and 6.8%; belimumab: 48.2 and 5.5%, respectively). Most AEs were mild/moderate in severity. Efficacy was maintained through the extension phase. An SRI response was achieved by 16.1% of patients in the placebo-to-belimumab group and 76.3% patients in the belimumab group. Furthermore, 1.0% of patients in the placebo-to-belimumab group and 2.6% of patients in the belimumab group experienced a severe SFI flare. Conclusion Belimumab SC was well tolerated and efficacy was maintained during the extension phase of this study. The safety profile of belimumab SC is consistent with that of previous experience with belimumab. Trial registration ClinicalTrials.gov identifier: NCT01484496

Objective Intravenous belimumab 10 mg/kg is approved as an add-on therapy in patients with active, autoantibody-positive systemic lupus erythematosus. This study aimed to assess the impact of belimumab on immune response to pneumococcal vaccination in patients with systemic lupus erythematosus. Methods This was a Phase 4, open-label study (GSK BEL115470; NCT01597492) conducted in the United States. Patients were randomized (7:9) to receive a 23-valent pneumococcal vaccination four weeks prior to (pre-belimumab cohort) or 24 weeks after (belimumab-concurrent cohort) commencing four-weekly belimumab 10 mg/kg intravenous treatment plus standard systemic lupus erythematosus therapy. Analyses of vaccine titers were performed on the as-treated population (received ≥1 dose of belimumab). The primary endpoint was the proportion of patients with positive antibody responses (≥2-fold increase from pre-vaccination levels, or post-vaccination level ≥ 0.6 µg/mL if pre-vaccination levels were unquantifiable) to ≥1 of 23 pneumococcal vaccine serotypes, four weeks post vaccination. Other endpoints included the proportion of patients with positive antibody responses to ≥2 to ≥10, and ≥11–23 (post hoc analysis) of serotypes. Safety was assessed by monitoring adverse events. Results Seventy-nine patients received pneumococcal vaccination (pre-belimumab cohort, n = 34; belimumab-concurrent cohort, n = 45). The majority (87.3% [69/79]) completed the study; 10 (12.7%) withdrew (patient request, n = 3; adverse event, n = 3; lost to follow-up, n = 2; other, n = 2). At Week 4 post-vaccination, 97.0% (32/33) and 97.6% (40/41) of patients (pre-belimumab and concurrent belimumab cohorts, respectively) had a positive response to ≥1 of 23 pneumococcal serotypes. Over 85% of patients in both cohorts responded to ≥10 of serotypes, approximately 80% responded to ≥12 serotypes, and approximately two-thirds responded to ≥16 serotypes. Little difference was observed between cohorts across a broad response, up to 23 serotypes. Eight (23.5%) patients experienced an adverse event considered by the investigator to be treatment-related in the pre-belimumab cohort and four (8.9%) in the belimumab-concurrent cohort; seven patients experienced non-fatal serious adverse events (pre-belimumab cohort, 11.8% [ n = 4]; concurrent-belimumab cohort, 6.7% [ n = 3]), and no deaths were reported. Conclusion The proportion of patients generating a response to ≥1 pneumococcal serotype did not differ between the pre-belimumab and belimumab-concurrent cohorts; the proportions were also comparable across a broader response (from ≥2 serotypes to 23 serotypes).

We describe a prototype cognitive work aid for airlift mission allocation and scheduling. A key design challenge was how to generate and visualize multiple operationally distinct options to broaden the set of alternative plans that Mission Planners consider, increasing overall efficiency and effectiveness of the mission planning organization. We first describe the distributed nature of the mission planning domain, where key considerations arise across different organizational boundaries, come in over time and dynamically change. We describe techniques we developed for generating and displaying multiple operationally distinct mission planning solutions, to facilitate contingency planning and collaboration across organizatinal boundaries. This work is part of a longer term program to develop and demonstrate concrete techniques for more effective joint cognitive systems.

The relatively rapid transition from a paper-based system to a digital system in healthcare has not always employed a sophisticated integration of usability concepts. Yet usability is critical to safety and to effectiveness of the electronic health record, and regulators and policy makers have been increasingly focused on this area. This panel will provide a variety of perspectives on this important issue, ranging from a description of the problem based on current vendor usability practices; recommendations regarding domain content rich usability processes including use cases, assessments, and scenarios; and the extension of usability assessments and design improvements to post-system implementation.

In this work, pH-sensitive amphiphilic macromolecules are designed to possess good biocompatibility and drug loading while employing an acid-sensitive linkage to trigger drug release at tumor tissues. Specifically, two pH-sensitive amphiphilic macromolecules were synthesized with a hydrazone linkage between the hydrophobic and hydrophilic segments. The chemical structure, molecular weight, critical micelle concentration, micelle size, and pH-triggered cleavage of the amphiphilic macromolecules were characterized via matrix-assisted laser desorption/ionization time-of-flight, nuclear magnetic resonance, and dynamic light scattering techniques. Drug loading and release as well as cytotoxicity studies were performed using doxorubicin. Hydrodynamic diameters of the micelles formed with pH-sensitive amphiphilic macromolecules were within an optimal range for cellular uptake. The critical micelle concentration values were 10–8–10–6 M, indicating micellar stability upon dilution. The degradation products of the amphiphilic macromolecules after acidic incubation were identified using mass spectrometry, nuclear magnetic resonance, and dynamic light scattering methods. A pH-dependent release profile of the doxorubicin-encapsulated amphiphilic macromolecules was observed. Cytotoxicity studies against two cancer cell lines, MDA-MB-231 human breast cancer cells and A549 lung cancer cells, showed that doxorubicin encapsulated in pH-sensitive amphiphilic macromolecules decreased cell viability more efficiently than free doxorubicin, possibly due to the toxicity of the amphiphilic macromolecule degradation products. Resulting from enhanced release at acidic pH due to hydrolysis of the hydrazone linkage, pH-sensitive amphiphilic macromolecules also had improved efficacy toward cancer cells compared to other carriers (e.g. Pluronics®). These findings indicate that pH-sensitive amphiphilic macromolecules can potentially be applied as anticancer drug delivery vehicles to achieve controlled release and improved therapeutic effects.

Background Belimumab is a recombinant, human, IgG1λ monoclonal antibody that targets B-lymphocyte stimulator. The intravenous formulation is indicated for the treatment of active, autoantibody-positive systemic lupus erythematosus (SLE). Belimumab has been formulated for subcutaneous (SC) administration to improve patient convenience. This post-hoc modeling and simulation analysis characterizes the population pharmacokinetics (PK) of SC belimumab, and compares the exposure profiles of the approved belimumab IV dose—10 mg/kg every four weeks—to the 200 mg SC weekly dose in SLE patients, highlighting key pharmacological differences relevant for clinicians. Methods Data from two Phase 1 studies in US American and Japanese healthy subjects were analyzed with a non-linear mixed effects modeling approach. The resulting SC population PK model and a previously developed IV population PK model were used to conduct simulation trials in a Phase 3 IV belimumab SLE patient population, comparing chronic exposure profiles and exposure ranges stratified by body weight tertiles for IV vs SC dosing. Results The PK of belimumab following SC administration was best described by a linear two-comment model. The estimates for clearance, steady-state volume of distribution, and bioavailability were 208 mL/day, 5250 mL, and 76%, respectively. After four weeks of SC dosing, simulated belimumab concentrations exceeded the steady-state trough concentrations of the IV dosing regimen. At steady state simulated serum profiles demonstrated comparable average belimumab concentrations (Cavg,ss) after IV and SC dosing. Simulated belimumab exposures demonstrated largely overlapping concentration ranges following 200 mg SC weekly and 10 mg/kg IV every four weeks dosing. Discussion The predicted Cavg,ss of belimumab in SLE patients was comparable following 200 mg SC weekly and 10 mg/kg IV every four weeks dosing. The simulated belimumab accumulation following SC weekly dosing indicated that administration of a loading dose was not required. Similar Cavg,ss ranges were predicted for fixed dose SC and weight-proportional IV regimens in the simulated SLE population, albeit with a reversed body-size-to-exposure relationship for the SC regimen. These findings provide rheumatologists with a better understanding of expected differences in belimumab exposure when comparing IV and SC dosing regimens.

The National Academies Board on Human-Systems Integration (BOHSI) has organized this session. An initial presentation by the director of BOHSI will provide information on recent projects and updates. Then a panel will discuss the application of human-systems integration (HSI) to the issue of cyber security. Cyber security attempts to protect devices, networks, applications, and data from unintended or unauthorized access. A cyber security solution should not only focus on technology, but also the malevolent behaviors of those wanting to exploit system weaknesses, the cognitive processes required by cyber analysts, as well as the behavior of every day users who may open the door to vulnerability. The panel will therefore address issues related to the human decision process in both defensive and offensive cybersecurity operations. The panel will present future opportunities and will engage the audience in a discussion of these issues.

Scholars have long debated whether trade liberalization has positive or negative effects on resource use and ecosystems. This study examines the conditions under which resource use increases or decreases in response to reduced trade barriers, specifically after the 2008 World Trade Organization decision that led the United States to reduce anti-dumping duties on Indian shrimp. At the district level in South India, fishing fleet expansion was correlated with access to global market information via mobile phones. Model simulations indicate that increased mobile phone saturation could expand fishing effort sufficiently to deplete multiple marine species groups, while other species benefit from the loss of predators. However, scenario analysis suggests that regulatory interventions could mitigate these ecosystem pressures while still permitting fishers to benefit from increased access to global market information.

This study focused on understanding the coping strategies and related behavioural changes of women who were recently diagnosed with herpes simplex virus type 2. In particular, we were interested in how coping strategies, condom use, and acyclovir uptake evolve over time. Twenty-eight women screening positive for herpes simplex virus type 2 were recruited through a public health STD clinic and the Indianapolis Community Court. Participants completed three semi-structured interviews with a woman researcher over a six-month period. The interviews focused on coping strategies for dealing with a diagnosis, frequency of condom use, suppressive and episodic acyclovir use, and the utilisation of herpes simplex virus type 2 support groups. Interview data were analysed using content analysis to identify and interpret concepts and themes that emerged from the interviews. Women employed a variety of coping strategies following an herpes simplex virus type 2 diagnosis. Of the women, 32% reported an increase in religious activities, 20% of women reported an increase in substance use, and 56% of women reported engaging in other coping activities. A total of 80% of women reported abstaining from sex immediately following the diagnosis, but 76% of women reported engaging in sex again by the six-month interview. Condom and medication use did not increase and herpes simplex virus type 2 support groups were not utilised by participants. All participants reported engaging in at least one coping mechanism after receiving their diagnosis. A positive diagnosis did not seem to result in increased use of condoms for the majority of participants and the use of acyclovir was low overall.

Intelligence analysts—overloaded with complex and disparate data—must incorporate information into cohesive and convincing narratives and explanations. For analysts, data overload can result in developing premature conclusions and limit their ability to effectively conduct comprehensive analyses. Historically, automated decision aids designed to help with these processes have largely failed analysts in managing a fundamental work tradeoff between analytic narrowing and broadening because software tools too often attempt to supplant analyst reasoning, rather than support the iterative process as a whole. Instead of such brittle approaches, previous research suggests analysts would benefit from automation that supports a collaborative partnership throughout the analysis process. The research and development described in this work began with a cognitive analysis of practicing analysts, entailing literature review, interviews, and walkthroughs with a software prototype. Findings from the cognitive analysis were translated into design requirements for decision aids that support representation management across multiple intelligence products. The findings provide insight into under-supported cognitive and collaborative work in modern intelligence analysis with implications for the future design of useful automated decision aids.

Cognitive Task Analysis (CTA) has become part of the standard tool set of cognitive engineering. CTAs are routinely used to understand the cognitive and collaborative demands that contribute to performance problems, the basis of expertise, as well as the opportunities to improve performance through new forms of training, user interfaces, or decision aids. While the need to conduct CTAs has become well established, there is little in the way of available guidance with respect to ‘best practice’ for how to conduct a CTA or how to evaluate the quality of a CTA that has been conducted by others. This is an important gap as the range of consumers of CTAs is expanding to include program managers and regulators who may need to make decisions based on CTA findings. This panel brings together some of the leaders in development and application of CTA methods to address the question: Given the variety of methods available, and the lack of rigid guidance on how to perform a CTA, how does one judge the quality of a CTA?” The goal of the panel is to explore points of consensus with respect to ‘best practice’ in conducting and evaluating a CTA, in spite of differences in particular CTA methods, as well as to draw insights from unique and provocative perspectives.

A strength of the field of cognitive engineering and decision-making lies in its wide applicability across the complex socio-technical systems, which are ubiquitous in modern society. Methods and theoretical advances in CEDM have been both developed through, and adapted across, domains as diverse as nuclear power, health systems, and aviation. While all of these domains clearly differ in terms of their surface characteristics, cognitive engineers are able to make fundamental connections across domains. These connections are supported by the types of methodological tools deployed within CEDM and allow problem solutions to be extended and adapted across domains. This panel brings together researchers and practitioners who have worked in a wide variety of domains to discuss a variety of design and methodological challenges they have and are facing. The panel will focus on synthesizing these challenges across domains – both across the panellists, and members of the audience, with the goal of providing both guidance and direction for future research.

To address issues in enterprise systems, operations research (OR) analysts need to be able to understand, codify, and communicate various aspects of the issues, such as the environment’s conditions and their relationships. Models are a natural way to capture that information, but they must be understandable to a variety of stakeholders involved in improving the enterprise. At the same time, the large amount of available information, such as event history and weather data, can easily overload analysts. To help analysts cope with data overload, it is useful for models to be accessible to computational tools that can provide data processing and visualization capabilities. To support both of those goals simultaneously, we describe an approach that supports the elicitation of qualitative insight from operations researchers and other relevant stakeholders and also provides avenues for computer software to perform semantic labeling and quantitative data processing. This approach directly supports an iterative OR process that satisfies the needs of multiple stakeholder communities, enabling initial qualitative relationships and hypotheses to be further investigated and justified with data-driven conclusions. Building on our previous experiences in knowledge acquisition and quantitative analysis, this paper outlines a new integrated workflow and a collection of graphical representation concepts for operations research and similar domains.

Despite the high prevalence of herpes simplex virus type 2 (HSV-2), testing for asymptomatic infections is uncommon. One population for whom targeted interventions may be prioritized include individuals involved with the correctional system. Here we describe the acceptability of a novel HSV-2 screening program, implemented in a court setting, as a possible intervention for corrections-involved women. Female defendants completed an interviewer administered survey assessing factors associated with uptake/refusal of free point-of-care HSV-2 serologic testing and HSV-2 seropositivity. Participants included 143 women, 18–62 years old (mean 32.85) with diverse ethnicities. The majority (65.7%) accepted testing and 62.4% tested HSV-2 seropositive. Factors independently associated with test acceptance included higher perceived susceptibility to genital herpes infection and not receiving a preventative health screen. Women who were seropositive tended to be older, Black, report having previous STI, and be arrested on a prostitution charge. Findings suggest point-of-care testing in a court setting is acceptable to women and can be implemented to improve case finding of STI.

The cyber security task is an intensely cognitive task that is embedded in a large multi-layered sociotechnical system of analysts, computers, and networks. Effective performance in this world is hampered by enormous size and complexity of the network data, the adaptive nature of intelligent adversaries, the lack of ground truth to assess performance, the high number of false alarms presented by automated alerting systems, by organizational stove pipes thwarting collaboration, and by technology that is thrown at the problem without an adequate understanding of the human needs. Further, the consequences of effective system performance in the cyber security domain are unparalleled because our world is so dependent on its cyber infrastructure. We have assembled a panel of six experts in cognitive engineering to provide perspectives on the cyber security problem and promising solutions.

Automated systems, ranging from robots to intelligent planning aids, are increasingly part of the cognitive landscape. A pressing question facing the human factors community is how to design the automation to foster effective collaboration between the human and automated agents. This panel brings together leading researchers actively engaged in design of decision-support systems that involve some element of ‘cognitive work’ automation. They represent a variety of domains and a variety of approaches to design of collaborative automation. Panel members will discuss unique challenges of their respective domains, including differences in temporal rhythm, and how these have shaped their perspective on ‘collaborative aiding’.

Previous research has identified many effective methods to visualize different types of meta-information, or information qualifiers; however, these methods are often incorporated without understanding how the graphical codes are perceived and how the encoded information is interpreted by display users. This results in display designers selecting graphical codes to represent meta-information without empirical evidence to determine the appropriateness of these selections. To help address this lack of guidance, this paper presents a systematic study of how people perceive two graphical codes (saturation and opacity) and relate those codes to different types of meta-information. Results were generated using psychophysical scaling methods, and provide visualization designers with a means to more appropriately design meta-information representations.

Supporting complex decision making requires conveying relevant information characteristics or qualifiers. The authors tested transparency and numeric annotation for displaying uncertainty about object identity. Participants performed a “missile defense” game in which they decided whether to destroy moving objects (which were either threatening missiles or nonthreatening birds and planes) before they reached a city. Participants were provided with uncertain information about the objects’ classifica-tions. Uncertainty was represented through the transparency of icons representing the objects and/or with numeric annotations. Three display methods were created. Icons represented the most likely object classification (with solid icons), the most likely object classification (with icons whose transparency represented the level of uncertainty), or the probability that the icon was a missile (with transparency). In a fourth condition, participants could choose among the representations. Icons either were or were not annotated with numeric probability labels. Task performance was highest when participants could toggle the displays, with little effect of numeric annotation. In conditions in which probabilities were available graphically or numerically, participants chose to engage objects when they were farther from the city and had a lower probability of being a missile. Results provided continued support for the use of graphical uncertainty representations, even when numeric representations are present.

Cyber Network degradation and exploitation can covertly turn an organization's technological strength into an operational weakness. It has become increasingly imperative, therefore, for an organization's personnel to have an awareness of the state of the Cyber Network that they use to carry out their mission. Recent high-level government initiatives along with hacking and exploitation in the commercial realm highlight this need for general Cyber Situational Awareness (SA). While much of the attention in both the military and commercial cyber security communities is on abrupt and blunt attacks on the network, the most insidious cyber threat to organizations are subtle and persistent attacks leading to compromised databases, processing algorithms, and displays. We recently began an effort developing software tools to support the Cyber SA of users at varying levels of responsibility and expertise (i.e., not just the network administrators). This paper presents our approach and preliminary findings from a CTA we conducted with an operational Subject Matter Expert to uncover the situational awareness requirements of such a tool. Results from our analysis indicate a list of preliminary categories of these requirements, as well as specific questions that will drive the design and development of our SA tool.

There is renewed interest in building new commercial nuclear power plants. Unlike existing plants that have traditional control rooms with large control boards, new plants will have compact digital control rooms. The transition to digital control rooms introduce opportunities for enhanced support (e.g., integrated displays; improved alarm systems; computerized procedures) as well as potential new challenges (e.g., shift from open to private workspaces; changes in workload distribution resulting from reduced control room crew size). This paper describes two simulator tests that were conducted as part of a person-in-the-loop test program to support development and validation of a control room for the US-APWR evolutionary pressurized water reactor plant. While the results are presented in the context of evaluation of a particular digital control room design, they have applicability to design of compact digital control rooms more generally, and point to areas where more research is needed.

Atomic layer deposition (ALD) has recently been demonstrated as a novel method for the creation of nanoscale coatings on fiber-based materials. The ALD process has proven effective in altering the surface energy of both woven and nonwoven materials. In this work, the effect of fabric weave geometry on the behavior of ALDmodified fabrics has been studied. Aluminum oxide ALD was used to modify a series of fabric weaves which were then analyzed using sessile contact angle measurements. The experimental results demonstrated the ability of the ALD process to modify the surface energy of a variety of weave structures, regardless of the inter-fiber spacing within the yarns. At the same time, the amount of fiber spacing, which is influenced by the weave geometry, changes the effect of the ALD coatings on fabric hydrophobicity and hydrophilicity. The results of this work demonstrate the versatility of the ALD process when modifying woven fabric structures and its potential as a method for nanoscale textile finishing.

Summary Rectal sexually transmitted infections (STIs) are common in men at risk for urethral infections with these pathogens, particularly men who have sex with men (MSM). However, for those individuals not regularly seen by a clinician, screening for rectal STI is not currently a widespread option. Qualitative data and samples (i.e. self-obtained rectal specimens) were collected from 75 MSM in a variety of venues. Upon completion of the rectal self-sampling, each participant completed a brief interview regarding their overall experience with the process. Participants reported an overall high level of acceptability and comfort-level involved with self-sampling for rectal STI. Of the majority of men who agreed to provide a rectal self-sample, all reported that they would provide a sample again in the future. However, many men also appreciated the interaction with a health-care provider that a clinical setting offered. In conclusion, self-sampling is a feasible and acceptable option when offered to MSM in a range of community-based venues. Further research is needed to determine which combinations of STI testing and treatment methods (including self-sampling) are most appropriate for diverse groups of men.

Advances in aircraft operational capabilities have led to a dramatic increase in the operational tempo facing air combat aviators, which has in turn led to SA failures, particularly with respect to secondary information. For example, when engaging an air threat, aviators will often overlook key information such as geopolitical boundaries, resulting in potential infractions of rules of engagement. In a previous study, we investigated an Adaptive Border Display designed to maintain awareness of these boundaries, and found that it was not helpful. However, in that study, our scenarios did not create the high workload situations in which pilots lose track of these boundaries. In this study, we used significantly more complex scenarios to investigate this display, creating high workload situations for the aviators. Results showed that in these situations, the Adaptive Border display had a significant effect on improving aviator performance.

We describe the most recent work-centered design for a military airlift organization. Earlier design cycles produced a set of coordinated visualizations to support synchronized air mission replanning. In this phase of the program automated planning support was incorporated to help C2 staff solve complex constraint problems across multiple missions and airfields. We describe our efforts at designing a replanning tool to work in collaboration with the human operator on complex replanning problems. A prototype was developed and used in an empirical evaluation comparing target user performance on complex constraint problems using either the visualizations alone or the visualizations with embedded automated support (collaborative planner). The collaborative planner significantly improved the speed and quality of dynamic replanning solutions. Post-test questionnaire ratings indicated users trusted the support provided by the automated aid. Design principles for effective collaborative automation are discussed.

Bayesian networks (BNs) are probabilistic models frequently used to capture domain knowledge for use in computational systems that can reason about states, causes, and effects. While BNs have many advantages, their complexity can hamper the process of knowledge elicitation and encoding. First, domain experts may not have expertise in artificial reasoning or probabilistic models, and that lack of understanding may complicate the elicitation of probabilities relevant to BN model structure. In addition, BNs require the definition of a priori, conditional probabilities: for complex models, this requires eliciting large numbers of complex probabilities. Multiple “canonical modeling” approaches, such as Causal Influence Models (CIMs), have been developed to address these complexities. However, little progress has been made towards human-in-the-loop evaluation of such approaches - specifically, their accessibility and usability, their related user interfaces, and how they enable a user to correctly create and interpret variables and probabilistic relationships. In this study, we evaluated the CIM approach (implemented in a software application) to determine the effect on user task performance. Results indicate that the model complexity has an adverse effect on performance when users are interpreting an existing model; that semantics of a model may impact performance; and that users were generally successful in creating new models of different situations.

Cognitive systems engineering projects have found a need for and a value in combining particular methods of cognitive task analysis (CTA). A principle reason that is given is that different CTA methods have different strengths in terms of how they inform the study, analysis or design of cognitive work systems. Focus questions include: What specific CTA methods have been combined or merged for some particular research project? What was the rationale for the selection of methods? How did researchers do the combining? Were methods actually merged into a single procedure, or were multiple methods conducted separately, what the resulting analyses or representations merged subsequently?

Bayesian belief networks (BNs) are well-suited to capturing vague and uncertain knowledge. However, the capture of this knowledge and associated reasoning from human domain experts often requires specialized knowledge engineers responsible for translating the expert's communications into BN-based models. Across application domains, we have analyzed how these models are constructed, refined, and validated with domain experts. From this analysis, we have identified key user-centered complexities and challenges that we have used to drive the selection of simplifying assumptions. This led us to develop computational techniques and user interface methods that leverage these same assumptions with the goal of improving the efficiency and ease with which expert knowledge can be expressed, verified, validated, and encoded. In this paper, we present the results of our analysis of BN construction, validation, and use. We discuss how these results motivated the design of a simplified version of BNs called Causal Influence Models (CIMs). In addition, we detail how CIMs enable the design and construction of user interface mechanisms that address complexities identified in our analysis.

We conducted two studies that investigated display characteristics related to color (hue, saturation, brightness, and transparency) and contrast with a background for displaying information qualifiers (termed meta-information) such as uncertainty, age, and source quality. Level of detail (or granularity) of the meta-information and task demands were also manipulated. Participants were asked to rank and rate colored regions overlaid on different map backgrounds based on the level of meta-information the regions displayed. Results from Study 1 indicated that participants could appropriately rank and rate levels of meta-information across saturation, brightness, and transparency conditions, and results from Study 1 and Study 2 showed that the natural direction of ordering is complex and dependent on the relevance of different information to the task and the contrast of the overlay region with the background.

New system design often focuses on opportunities afforded by technology without careful analysis of the needs of the people and work involved. A strong consensus has been found across diverse design traditions regarding the need for in-depth analysis of the context of use to inform system design. Analysis of context of use encompasses identifying and characterizing the range of situations that can arise, the demands they impose, and the motivations and activities of the people involved. This paper presents three case studies in which context of use analysis was a key part of successful system design. The three case studies represent work-centered design, emerging from cognitive engineering and cognitive systems engineering; interaction design, emerging from industrial and communication design; and participatory design, emerging from the Scandinavian labor movement. The case studies highlight commonalities in experience and core tenets across traditions. They argue for the importance of incorporating context of use analyses as part of the systems engineering process and illustrate a variety of approaches for context of use analysis.

Objective: In this article, the author provides an overview of cognitive analysis methods and how they can be used to inform system analysis and design. Background: Human factors has seen a shift toward modeling and support of cognitively intensive work (e.g., military command and control, medical planning and decision making, supervisory control of automated systems). Cognitive task analysis and cognitive work analysis methods extend traditional task analysis techniques to uncover the knowledge and thought processes that underlie performance in cognitively complex settings. Methods: The author reviews the multidisciplinary roots of cognitive analysis and the variety of cognitive task analysis and cognitive work analysis methods that have emerged. Results: Cognitive analysis methods have been used successfully to guide system design, as well as development of function allocation, team structure, and training, so as to enhance performance and reduce the potential for error. Conclusions: A comprehensive characterization of cognitive work requires two mutually informing analyses: (a) examination of domain characteristics and constraints that define cognitive requirements and challenges and (b) examination of practitioner knowledge and strategies that underlie both expert and error-vulnerable performance. A variety of specific methods can be adapted to achieve these aims within the pragmatic constraints of particular projects. Application: Cognitive analysis methods can be used effectively to anticipate cognitive performance problems and specify ways to improve individual and team cognitive performance (be it through new forms of training, user interfaces, or decision aids).

Cognitive task and work analyses are approaches to the analysis and support of cognitive work (rather than primarily physical or perceptual activities). Although a variety of methods exist for performing cognitive task and work analyses, they share a common goal of providing information about two mutually reinforcing perspectives. One perspective focuses on the fundamental characteristics of the work domain and the cognitive demands they impose. The other focuses on how current practitioners respond to the demands of the domain. This includes a description of the knowledge and skills practitioners have developed to operate effectively as well as any limitations in knowledge and strategies that contribute to performance problems. This chapter provides a broad survey of cognitive task analysis and cognitive work analysis methods. Some of the methods highlight techniques for knowledge gathering, whereas others focus on aspects of analysis and representation. Still other techniques emphasize process outputs, such as formal (computational) models of cognitive activities or design artifacts and associated rationales. In this chapter we review specific cognitive task and work analysis methods and describe through illustration how these methods can be adapted to meet specific project objectives and pragmatic constraints.