Automation Error Type and Methods of Communicating Automation Reliability Affect Trust and Performance: An Empirical Study in the Cyber Domain

Antiphishing aid systems, among other automated systems, are not perfectly reliable. Automated systems can make errors, thereby resulting in false alarms or misses. An automated system's capabilities need to be communicated to the users to maintain proper user trust. System capabilities can be learned through an explicit description or from experience. Using a phishing-detection system as a testbed in this article, we systematically varied automation error type and the method of communicating system reliability in a factorial design and measured their effects on human performance and trust in the automation. Participants were asked to classify emails as legitimate or phishing with assistance from the phishing-detection system. The results from 510 participants suggest that learning through experience with feedback improved trust calibration for both objective and subjective trust measures in most conditions. Moreover, false alarms lowered trust more than misses for both unreliable and reliable systems, and false alarms turned out to be beneficial for proper trust calibration when using unreliable systems. Design implications of the results include using feedback whenever possible and choosing false alarms over misses for unreliable systems.