Search

There has been a recent major upsurge in the concerns about reproducibility in many areas of science. Within the neuroimaging domain, one approach is to promote reproducibility is to target the re-executability of the publication. The information supporting such re-executability can enable the detailed examination of how an initial finding generalizes across changes in the processing approach, and sampled population, in a controlled scientific fashion. ReproNim: A Center for Reproducible Neuroimaging Computation is a recently funded initiative that seeks to facilitate the "last mile" implementations of core re-executability tools in order to reduce the accessibility barrier and increase adoption of standards and best practices at the neuroimaging research laboratory level. In this report, we summarize the overall approach and tools we have developed in this domain.

Background: The degree of reproducibility of the neuroimaging literature in psychiatric application areas has been called into question and the issues that relate to this reproducibility are extremely complex. Some of these complexities have to do with the underlying biology of the disorders that we study and others arise due to the technology we apply to the analysis of the data we collect. Ultimately, the observations we make get communicated to the rest of the community through publications in the scientific literature. Methods: We sought to perform a ‘re-executability survey’ to evaluate the recent neuroimaging literature with an eye toward seeing if the technical aspects of our publication practices are helping or hindering the overall quest for a more reproducible understanding of brain development and aging. The topic areas examined include availability of the data, the precision of the imaging method description and the reporting of the statistical analytic approach, and the availability of the complete results. We applied the survey to 50 publications in the autism neuroimaging literature that were published between September 16, 2017 to October 1, 2018. Results: The results of the survey indicate that for the literature examined, data that is not already part of a public repository is rarely available, software tools are usually named but versions and operating system are not, it is expected that reasonably skilled analysts could approximately perform the analyses described, and the complete results of the studies are rarely available. Conclusions: We have identified that there is ample room for improvement in research publication practices. We hope exposing these issues in the retrospective literature can provide guidance and motivation for improving this aspect of our reporting practices in the future.

Background: The degree of reproducibility of the neuroimaging literature in psychiatric application areas has been called into question and the issues that relate to this reproducibility are extremely complex. Some of these complexities have to do with the underlying biology of the disorders that we study and others arise due to the technology we apply to the analysis of the data we collect. Ultimately, the observations we make get communicated to the rest of the community through publications in the scientific literature. Methods: We sought to perform a ‘re-executability survey’ to evaluate the recent neuroimaging literature with an eye toward seeing if our publication practices are helping or hindering the overall quest for a more reproducible understanding of brain development and aging. The topic areas examined include availability of the data, the precision of the imaging method description and the reporting of the statistical analytic approach, and the availability of the complete results. We applied the survey to 50 publications in the autism neuroimaging literature that were published between September 16, 2017 to October 1, 2018. Results: The results of the survey indicate that for the literature examined, data that is not already part of a public repository is rarely available, software tools are usually named but versions and operating system are not, it is expected that reasonably skilled analysts could approximately perform the analyses described, and the complete results of the studies are rarely available. Conclusions: We have identified that there is ample room for improvement in research publication practices. We hope exposing these issues in the retrospective literature can provide guidance and motivation for improving this aspect of our reporting practices in the future.

There has been a recent major upsurge in the concerns about reproducibility in many areas of science. Within the neuroimaging domain, one approach is to promote reproducibility is to target the re-executability of the publication. The information supporting such re-executability can enable the detailed examination of how an initial finding generalizes across changes in the processing approach, and sampled population, in a controlled scientific fashion. ReproNim: A Center for Reproducible Neuroimaging Computation is a recently funded initiative that seeks to facilitate the “last mile” implementations of core re-executability tools in order to reduce the accessibility barrier and increase adoption of standards and best practices at the neuroimaging research laboratory level. In this report, we summarize the overall approach and tools we have developed in this domain.

There has been a recent major upsurge in the concerns about reproducibility in many areas of science. Within the neuroimaging domain, one approach is to promote reproducibility is to target the re-executability of the publication. The information supporting such re-executability can enable the detailed examination of how an initial finding generalizes across changes in the processing approach, and sampled population, in a controlled scientific fashion. ReproNim: A Center for Reproducible Neuroimaging Computation is a recently funded initiative that seeks to facilitate the "last mile" implementations of core re-executability tools in order to reduce the accessibility barrier and increase adoption of standards and best practices at the neuroimaging research laboratory level. In this report, we summarize the overall approach and tools we have developed in this domain. [ABSTRACT FROM AUTHOR]

Background: The degree of reproducibility of the neuroimaging literature in psychiatric application areas has been called into question and the issues that relate to this reproducibility are extremely complex. Some of these complexities have to do with the underlying biology of the disorders that we study and others arise due to the technology we apply to the analysis of the data we collect. Ultimately, the observations we make get communicated to the rest of the community through publications in the scientific literature. Methods: We sought to perform a 're-executability survey' to evaluate the recent neuroimaging literature with an eye toward seeing if our publication practices are helping or hindering the overall quest for a more reproducible understanding of brain development and aging. The topic areas examined include availability of the data, the precision of the imaging method description and the reporting of the statistical analytic approach, and the availability of the complete results. We applied the survey to 50 publications in the autism neuroimaging literature that were published between September 16, 2017 to October 1, 2018. Results: The results of the survey indicate that for the literature examined, data that is not already part of a public repository is rarely available, software tools are usually named but versions and operating system are not, it is expected that reasonably skilled analysts could approximately perform the analyses described, and the complete results of the studies are rarely available.  Conclusions: We have identified that there is ample room for improvement in research publication practices. We hope exposing these issues in the retrospective literature can provide guidance and motivation for improving this aspect of our reporting practices in the future., Competing Interests: No competing interests were disclosed., (Copyright: © 2020 Hodge SM et al.)