Search

This study addressed the cognitive impacts of providing correct and incorrect machine learning (ML) outputs in support of an object detection task. The study consisted of five experiments that manipulated the accuracy and importance of mock ML outputs. In each of the experiments, participants were given the T and L task with T-shaped targets and L-shaped distractors. They were tasked with categorizing each image as target present or target absent. In Experiment 1, they performed this task without the aid of ML outputs. In Experiments 2-5, they were shown images with bounding boxes, representing the output of an ML model. The outputs could be correct (hits and correct rejections), or they could be erroneous (false alarms and misses). Experiment 2 manipulated the overall accuracy of these mock ML outputs. Experiment 3 manipulated the proportion of different types of errors. Experiments 4 and 5 manipulated the importance of specific types of stimuli or model errors, as well as the framing of the task in terms of human or model performance. These experiments showed that model misses were consistently harder for participants to detect than model false alarms. In general, as the model's performance increased, human performance increased as well, but in many cases the participants were more likely to overlook model errors when the model had high accuracy overall. Warning participants to be on the lookout for specific types of model errors had very little impact on their performance. Overall, our results emphasize the importance of considering human cognition when determining what level of model performance and types of model errors are acceptable for a given task., (© 2024. The Author(s).)

Eye tracking is a useful tool for studying human cognition, both in the laboratory and in real-world applications. However, there are cases in which eye tracking is not possible, such as in high-security environments where recording devices cannot be introduced. After facing this challenge in our own work, we sought to test the effectiveness of using artificial foveation as an alternative to eye tracking for studying visual search performance. Two groups of participants completed the same list comparison task, which was a computer-based task designed to mimic an inventory verification process that is commonly performed by international nuclear safeguards inspectors. We manipulated the way in which the items on the inventory list were ordered and color coded. For the eye tracking group, an eye tracker was used to assess the order in which participants viewed the items and the number of fixations per trial in each list condition. For the artificial foveation group, the items were covered with a blurry mask except when participants moused over them. We tracked the order in which participants viewed the items by moving their mouse and the number of items viewed per trial in each list condition. We observed the same overall pattern of performance for the various list display conditions, regardless of the method. However, participants were much slower to complete the task when using artificial foveation and had more variability in their accuracy. Our results indicate that the artificial foveation method can reveal the same pattern of differences across conditions as eye tracking, but it can also impact participants' task performance.

Background: International Atomic Energy Agency (IAEA) safeguards inspectors are faced with the difficult task of learning the layout of complex nuclear facilities while being escorted through the facilities. This study addresses a gap in the literature regarding how to best support the development of inspectors' spatial knowledge, given the constraint that they cannot bring digital devices into most nuclear facilities. We tested whether viewing a map before learning a guided route or carrying a map along the route enabled better spatial learning than having no exposure to a map. Moreover, we tested the impact of carrying maps with different levels of detail (simple 2D, simple 3D, or complex 2D maps) on spatial learning outcomes, as well as interactions between map type and individual differences in sense of direction., Results: The results showed nearly opposite patterns of performance for participants with good and poor sense of direction scores. Participants with a good sense of direction showed higher levels of spatial knowledge when studying or carrying simple maps, whether 2D or 3D, but they did not benefit from using a complex map. Participants with a poor sense of direction showed lower levels of spatial knowledge when using a simple map relative to using no map or a complex map, suggesting that they did not attempt to use the complex map. For both groups of participants, referring to a simple map while learning a route decreased their awareness of their environment, as measured by response times on a memory test that included incidentally learned items.