1. Improving Rater Calibration in Aviation: A Case Study.
- Author
-
Holt, Robert W., Hansberger, Jeffrey T., and Boehm-Davis, Deborah A.
- Subjects
- *
MARITIME pilots , *PERFORMANCE standards , *COMMERCIAL aeronautics - Abstract
The assessment of pilot performance relies on systematic observation and assessment by a single trained rater or instructor/evaluator (I/E). Because of the importance of aviation safety, it is imperative that the rating and evaluation processes used by these I/Es yield reliable and valid data. This article describes a case study focused on improving the reliability and validity of crew assessment at a commercial carrier. The process for improving reliability involved the evaluation of current training methods, the construction and evaluation of 5 metrics for assessing interrater reliability, and a standardized process for using these metrics to train I/Es. A separate set of interventions was developed and implemented to improve structural validity. Data collected from 2 fleets in this airline during a 3-year period indicate aspects of reliability and validity that should be the focal points of continuing I/E training. [ABSTRACT FROM AUTHOR]
- Published
- 2002
- Full Text
- View/download PDF