Derived operating conditions for classifier performance understanding

The target classification algorithm community is making a special effort to explicitly treat operating conditions (OCs) in classifier assessments and performance modeling. This is necessary because humans do not intuitively appreciate what makes classification difficult for computers-it just seems so easy to us. In analyzing OCs, some OCs are more direct or primitive while others are more abstract or integrating. These more abstract or "Derived OCs" provide an intermediate step between direct OCs and classifier performance. Similar to the target, sensor, environment partition of OCs, the AFRL COMPASE Center introduces the "Mossing 3" partition of derived OCs into "Clarity," "Uniqueness," and "Conformity." Clarity is primarily concerned with the relevant information content available in the sensor data. Uniqueness is about the inherent separability between the types of objects to be classified (i.e., the library) and between all those types and objects not known to the classifier. Conformity is about the relationship between the OCs of the test instances and the OCs represented in the library types or training data. Furthermore, by analyzing derived OCs from multiple perspectives, informative subpartitions of the Mossing 3 are created. Clarity measures are well developed, particularly as image quality metrics. The other partitions are less well developed, but relevant work exists and is brought into context. While derived OCs and the Mossing 3 partition are not a complete solution to performance modeling, they help bring in powerful existing technologies and should enrich and facilitate dialogue on classifier performance theory and modeling.