Search

Decision support systems (DSSs) are being woven into human workflows from aviation to medicine. In this study, we examine decision quality and visual information foraging for DSSs with different known reliability levels. Thirty-six participants completed a financial fraud detection task, first unsupported and then supported by a DSS which highlighted important information sources. Participants were randomly allocated to four cohorts, being informed that the system's reliability was 100%, 90%, 80% or undisclosed. Results showed that only a DSS known to be 100% reliable resulted in participants systematically following its suggestions, increasing the percentage of correct classifications to a median of 100% while halving both, decision time and number of visually attended information sources. In all other conditions, the DSS had no effect on most visual sampling metrics, while decision quality of the human-DSS team was below the reliability level of the DSS. Knowledge of an even slightly unreliable system hence had a profound impact on joint decision making, with participants trusting their significantly worse performance more than the DSSs suggestions.

Data from 15 jewellery students, in their 1st and 3rd years of training, were analysed to show how data collected from work settings can be used to objectively evaluate performance in the use of tools. Participants were asked to use a piercing saw to cut 5 lines in a piece of metal. Performance was categorised in terms of functional dynamics. Data from strain gauges and a tri-axial accelerometer (built into the handle of the saw) were recorded and thirteen metrics derived from these data. The key question for this paper is which metrics could be used to distinguish levels of ability. Principal Components Analysis identified five components: sawing action; grasp of handle; task completion time; lateral deviation of strokes; and quality of lines cut. Using representative metrics for these components, participants could be ranked in terms of performance (low, medium, high) and statistical analysis showed significant differences between participants on key metrics.

User interface (UI) design can affect the quality of decision making, where decisions based on digitally presented content are commonly informed by visually sampling information through eye movements. Analysis of the resulting scan patterns - the order in which people visually attend to different regions of interest (ROIs) - gives an insight into information foraging strategies. In this study, we quantified scan pattern characteristics for participants engaging with conceptually different user interface designs. Four interfaces were modified along two dimensions relating to effort in accessing information: data presentation (either alpha-numerical data or colour blocks), and information access time (all information sources readily available or sequential revealing of information required). The aim of the study was to investigate whether a) people develop repeatable scan patterns and b) different UI concepts affect information foraging and task performance. Thirty-two participants (eight for each UI concept) were given the task to correctly classify 100 credit card transactions as normal or fraudulent based on nine transaction attributes. Attributes varied in their usefulness of predicting the correct outcome. Conventional and more recent (network analysis- and bioinformatics-based) eye tracking metrics were used to quantify visual search. Empirical findings were evaluated in context of random data and possible accuracy for theoretical decision making strategies. Results showed short repeating sequence fragments within longer scan patterns across participants and conditions, comprising a systematic and a random search component. The UI design concept showing alpha-numerical data in full view resulted in most complete data foraging, while the design concept showing colour blocks in full view resulted in the fastest task completion time. Decision accuracy was not significantly affected by UI design. Theoretical calculations showed that the difference in achievable accuracy between very complex and simple decision making strategies was small. We conclude that goal-directed search of familiar information results in repeatable scan pattern fragments (often corresponding to information sources considered particularly important), but no repeatable complete scan pattern. The underlying concept of the UI affects how visual search is performed, and a decision making strategy develops. This should be taken in consideration when designing for applied domains.