Does Conflict of Interest Distort Global University Rankings? Research & Occasional Paper Series: CSHE.5.2021

Global university rankings influence students' choices and higher education policies throughout the world. When rankers not only evaluate universities but also provide them with consulting, analytics, or advertising services rankers are vulnerable to conflicts of interest that may potentially distort their rankings. The paper assesses the impact of contracting with rankers on university ranking outcomes using difference-in-difference research design. The study matches data on the positions of 28 Russian universities in QS World University Rankings between 2016 and 2021 with information on contracts these universities had for services from QS -- a company that produces these rankings. The study estimates the effects of the conflicts of interest with two difference-in-difference models. The first model compares the difference in five-year change in rankings between QS rankings and Times Higher Education (THE) rankings across two groups of universities -- those that frequently (five times or more) contracted for QS services, and those that never or seldomly contracted for QS services. The second model compares the difference in five-year change in faculty-student ratios -- between scores in QS rankings, THE rankings, and scores recorded by national statistics -- across the same two groups of universities. The results suggest universities with frequent QS-related contracts had an increase of 0.75 standard deviations (~140 positions) in QS World University Rankings and an increase of 0.9 standard deviations in reported QS faculty-student ratio scores over five years, regardless of changes in the institutional quality. The study calls for universities, governments, and prospective students to reconsider their use of global rankings where conflicts of interest may be generated by the ranker's business activities.