Search

The purpose of this study was to support the development and initial validation of the Intervention Selection Profile (ISP)-Skills, a brief 14-item teacher rating scale intended to inform the selection and delivery of instructional interventions at Tier 2. Teacher participants (n = 196) rated five students from their classroom across four measures (total student n = 877). These measures included the ISP-Skills and three criterion tools: Social Skills Improvement System (SSIS), Devereux Student Strengths Assessment (DESSA), and Academic Competence Evaluation Scales (ACES). Diagnostic classification modeling (DCM) suggested an expert-created Q-matrix, which specified relations between ISP-Skills items and hypothesized latent attributes, provided good fit to item data. DCM also indicated ISP-Skills items functioned as intended, with the magnitude of item ratings corresponding to the model-implied probability of attribute mastery. DCM was then used to generate skill profiles for each student, which included scores representing the probability of students mastering each of eight skills. Correlational analyses revealed large convergent relations between ISP-Skills probability scores and theoretically-aligned subscales from the criterion measures. Discriminant validity was not supported, as ISP-Skills scores were also highly related to all other criterion subscales. Receiver operating characteristic (ROC) curve analyses informed the selection of cut scores from each ISP-Skills scale. Review of classification accuracy statistics associated with these cut scores (e.g., sensitivity and specificity) suggested they reliably differentiated students with below average, average, and above average skills. Implications for practice and directions for future research are discussed, including those related to the examination of ISP-Skills treatment utility. [This paper was published in "Journal of School Psychology" 2020.]

Teachers are often called upon to identify behavioral and emotional risk within their students by completing a variety of assessment tools. However, many teachers may lack the requisite skills to reliably identify students at-risk or use data derived from assessment tools to inform intervention. A series of trainings were developed to improve decision-making on the "Intervention Selection Profile-Skills," with a focus on improving accuracy and use of data. Specifically, a two-study randomized controlled design was employed to evaluate the efficacy of a basic informational training and a training with a practice component with regards to a control condition on the collection and use of social-emotional assessment data on the ISP-Skills. Results suggest limited influence of training on the accuracy of data collection, yet significant influence on improving how data are used to inform intervention. Implications for practice and research, as well as limitations, are discussed. [This paper will be published in "School Psychology Review."]

The purpose this study was to evaluate the reliability, validity, and accuracy of scores from the "Intervention Selection Profile--Function (ISP-Function)": a brief functional assessment tool founded upon Direct Behavior Rating (DBR) methodology. Participants included 34 teacher-student dyads. Using the ISP-Function, teachers rated the extent to which students exhibited disruptive behavior, as well as the frequency with which disruptions were met with four consequences. Ratings were completed across three 10-minute sessions, during which a research assistant also collected systematic direct observation (SDO) data regarding the same behavior and consequences. Results indicated adequate temporal reliability (=0.70) was attained for the Adult Attention and Peer Attention target across the three ratings; in contrast, up to 8-18 data points would be needed to achieve adequate reliability across the remaining targets. Findings further suggested that while ISP-Function ratings of Disruptive Behavior, Adult Attention, and Peer Attention were moderately to highly correlated with SDO data, correlations were in the low range for the Access to Items/Activities and Escape/Avoidance targets. Finally, analysis of difference scores showed that on average, mean ISP-Function scores fell within only 0.33 to 1.81 points of mean SDO scores (on the 0-10 DBR scale). Agreement coefficients indicative of exact score agreement were less consistent, suggesting accuracy ranged from poor to substantial. Results are promising, but future research is necessary to support applied ISP-Function use. "Impact Statement": Study results support the continued validation of the "Intervention Selection Profile--Function," with findings speaking to the reliability, validity, and accuracy of three of five scales within the tool. The availability of such tools enhances the likelihood of schools collecting functional assessment data to inform the implementation of Tier 2 interventions. [This paper was published in "School Psychology" (EJ1227784).]

The purpose of the current study was to examine the validity and diagnostic accuracy of the "Intervention Selection Profile-Social Skills" (ISP-SS), a brief social skills assessment tool intended for use with students in need of Tier 2 intervention. Participants included 160 elementary and middle school students who had been identified through universal screening as at risk for behavioral concerns. Teacher participants (n = 71) rated each of these students using both the ISP-SS and the "Social Skills Improvement System -- Rating Scales" (SSiS-RS), with the latter measure serving as the criterion within validity and diagnostic accuracy analyses. Confirmatory factor analysis supported ISP-SS structural validity, indicating ISP-SS items broadly conformed to a single 'Social Skills' factor. Follow-up analyses suggested ISP-SS broad scale scores demonstrated adequate internal consistency reliability, with hierarchical omega coefficient equal to .86. Correlational analyses supported the concurrent validity of ISP-SS items, finding each ISP-SS item to be moderately or highly related to its corresponding SSiS-RS subscale. Finally, analyses indicated three of the seven ISP-SS items demonstrated sufficient diagnostic accuracy; however, findings suggest additional revisions are needed if the ISP-SS is to be appropriate for use in schools. Implications for practice and future research are discussed. [This paper was published in "Psychology in the Schools" (EJ1207669).]