Assessing Impact Submissions for REF 2014: An Evaluation

The Research Excellence Framework (REF) is a new system for assessing the quality of research in UK higher education institutions (HEIs). For the first time, part of the assessment included the wider impact of research. RAND Europe was commissioned to evaluate the assessment process of the impact element of REF submissions, and to explore the strengths and weaknesses of the assessment process in delivering reliable, robust and fair outcomes, identify any unforeseen issues emerging during the assessment process and outline broad implications for the assessment process of impact in future REF exercises. This report provides a summary of our key findings, complemented by a detailed analysis of the process using the following methodologies: focus groups with panel members, users and impact assessors; interviews with panellists; and a survey of all panellists. This evaluation complements another piece of research that RAND Europe undertook for the UK higher education funding councils to evaluate the submission process HEIs underwent to prepare for the impact element of REF 2014 (Manville et al. 2015a, Manville et al. 2015b). Appended are: (1) List of panels and units of assessment for REF 2014; (2) Semi-structured protocol for focus group interviews; (3) NVivo code book for coding interviewee responses from the focus groups; (4) Survey protocols for those involved in the assessment of the impact element of REF 2014; and (5) Protocol for individual interviews with panellists involved in the assessment of impact. [For "Preparing Impact Submissions for REF 2014: An Evaluation. Approach and Evidence," see ED561159; for "Preparing Impact Submissions for REF 2014: An Evaluation. Findings and Observations," see ED561162; for "Assessing Impact Submissions for REF 2014: An Evaluation. Research Brief," see ED561164.]