Comparing Coding Viewing and Recording Methods to Quantify Embedded Instruction Learning Trials

The purpose of this study was to examine the comparability of counts of embedded instruction learning trials when different methods of viewing and recording direct behavioral observations were used. In 13 classrooms, while videotaping embedded instruction implementation for a larger randomized controlled efficacy trial was occurring, teachers’ implementation of trials was coded in situ using pencil-and-paper methods. Videos were later coded using computer-assisted methods. Dependent-samples t tests, Pearson product-moment correlation coefficients, and additional score agreement calculations were conducted. Statistically significant differences were found in the estimates of trial frequency. Correlational analyses showed positive and strong relationships between the coding methods. Coding agreement was higher across the entire observation versus during 10-min continuous event blocks. In situ coding took significantly less time than video coding. Results provide empirical evidence for the advantages and disadvantages of common viewing and recording methods for quantifying behavior as part of systematic observation systems.