1. On the reliability of unitizing textual continua: Further developments
- Author
-
Klaus Krippendorff, Antoine Widlöcher, Stéphane Bouvry, Yann Mathet, University of Pennsylvania [Philadelphia], Equipe Hultech - Laboratoire GREYC - UMR6072, Groupe de Recherche en Informatique, Image et Instrumentation de Caen (GREYC), Centre National de la Recherche Scientifique (CNRS)-École Nationale Supérieure d'Ingénieurs de Caen (ENSICAEN), Normandie Université (NU)-Normandie Université (NU)-Université de Caen Normandie (UNICAEN), Normandie Université (NU)-Centre National de la Recherche Scientifique (CNRS)-École Nationale Supérieure d'Ingénieurs de Caen (ENSICAEN), Normandie Université (NU), and Equipe CODAG - Laboratoire GREYC - UMR6072
- Subjects
Statistics and Probability ,Computer science ,Krippendorff’s alpha ,050801 communication & media studies ,02 engineering and technology ,Unitizing ,computer.software_genre ,Agreement ,0508 media and communications ,Software ,Segmentation ,0202 electrical engineering, electronic engineering, information engineering ,ComputingMilieux_MISCELLANEOUS ,business.industry ,Coding ,05 social sciences ,General Social Sciences ,020207 software engineering ,Reliability ,Annotating ,Content analysis ,[INFO.INFO-IT]Computer Science [cs]/Information Theory [cs.IT] ,Artificial intelligence ,business ,computer ,Natural language processing ,Coding (social sciences) - Abstract
This paper builds on an agreement coefficient proposed by Krippendorff (Content analysis: an introduction to its methodology, 2013) for measuring the reliability of unitizing and coding continuous phenomena, for example, of texts, videos, or sound recordings. It serves three purposes: It modifies Krippendorff’s definition which turned out to behave not as expected when applied to more than two observers, coders, or annotators. It extends this reliability measure to a family of four coefficients able to assess the reliabilities of diverse properties of unitized continua. It adds a way to obtain the confidence intervals of these coefficients as well as the probability of failing to reach targeted reliability levels. And it describes and provides access to free software that calculates all values of this family of reliability coefficients.
- Published
- 2016
- Full Text
- View/download PDF