Search

Multi-lab direct replication projects like Many Labs (e.g., Klein et al., 2014) or the Registered ReplicationReports (e.g., Wagenmakers et al., 2016) are a valuable form of scientific collaboration, produceoutstanding data sets and are a great resource for third-party researchers. Their data may be reanalyzedand used in research synthesis. The repositories and code could provide guidance to future projects ofthis kind. But, while large-scale collaborations are similar in their structure and aggregate their data inmeta-analyses, they deploy a variety of different solutions regarding the storage structure in the repositories,the way the (analysis) code is structured and the file-formats they provide. Continuing this trendimplies: Anyone who wants to work with data from multiple of these projects, or combine their datasets, is faced with an ever increasing complexity. Some of that complexity could be avoided. Here, we introduceMetaPipeX, a standardized framework to analyze, document, harmonize and visualize multi-labdata. It features a pipeline conceptualization of the analysis and documentation process, an R-packagethat implements both and a Shiny App that allows users to harmonize and visualize these data sets. Weintroduce the framework by describing its components and applying a practical example to it. Engagingwith this form of collaboration and integrating it more into research practice will certainly be beneficialto psychological science and we hope the framework provides a structure and tools to reduce effort foranyone who creates, harmonizes or learns about multi-lab replication projects.

In this paper, we present a review of how the various aspects of any study using an eye tracker (such as the instrument, methodology, environment, participant, etc.) affect the quality of the recorded eye-tracking data and the obtained eye-movement and gaze measures. We take this review to represent the empirical foundation for reporting guidelines of any study involving an eye tracker. We compare this empirical foundation to five existing reporting guidelines and to a database of 207 published eye-tracking studies. We find that reporting guidelines vary substantially and do not match with actual reporting practices. We end by deriving a minimal, flexible reporting guideline based on empirical research (Section “An empirically based minimal reporting guideline”).