Search

Background/Aim: RNA sequencing (RNA-seq) has become an exemplary technology in biology and clinical applications, outperforming other transcriptomic profiling protocols due to its precision and accuracy in characterising transcriptomes. As RNA-seq analysis gains popularity and becomes an essential component of future precision medicine, examining the software development and open science practices used by RNA-seq tools is necessary to ensure and improve the accuracy of generated results. Methods. We performed a comprehensive assessment of the software development and open sciences practices of RNA-seq tools, and benchmarking practices across more than 319 RNA-seq tools developed between the years of 2008 to 2024 by manually data extraction. We extracted features of the RNA seq tools including availability of package managers, number of releases and the benchmarking practises adapted by the researchers. We plan to access the archive stability and engagement of the scientific community to software repositories like Github and Bitbucket to better measure recognition of the tools. Results. Here we show that good software development increases the number of citations for the tool with a statistically significant p-value of 9.1×10−6 (Mann-Whitney U test). Conclusions. In addition to software development practices, it is vital to follow best practices in benchmarking and validation of developed tools. Ideally, a newly developed tool would be compared against all previously developed tools1, but it is currently unknown if this practice is widely adopted in the bioinformatics community. Examining these studies is vital to develop RNA-seq tools that can not only enhance the precision and thoroughness of all future benchmarking studies, but also encourage tool developers to engage in more comprehensive benchmarking when releasing their tools. By highlighting promising trends in usability, accessibility, maintainability, and benchmarking practices, we aim to encourage wider adoption of these best practices within the biomedical research community. This collective effort will ultimately lead to the development of more accurate, robust, and user-friendly RNA-seq tools in the future [ABSTRACT FROM AUTHOR]

Aim. To address the scarcity of high-quality experimental benchmarks in metagenomics by developing a cost-effective, semi-real benchmark generation tool, MetaWiz, which can modify existing experimental data to create diverse and realistic metagenomic benchmarks. Methods. In this study, we employed a series of steps to modify mock data representing known microbial genomes or strains. These steps involved selection, downloading, preprocessing, mapping, calculating mapping statistics, selecting uniquely mapped reads, and making modifications to create diverse benchmarks. The data was downloaded in fasta/fastq formats from the open source repositories like NCBI, MG-RAST, and Github. We preserved the original data quality and length without filtering or trimming to maintain uniformity. For mapping, we used bwa-mem and Minimap2, running both tools with default parameters. We discarded unmapped reads and selected uniquely mapped reads, we filtered out supplementary and alternative alignment. To modify the benchmark dataset, we implemented several functionalities, including the adjustment of strain frequencies, converting long Pacbio reads into short reads, correcting erroneous bases in Nanopore ONT reads using genomes, followed by read length adjustment similar to Pacbio reads. Results. MetaWiz successfully generated diverse benchmarks from real metagenomic data, encompassing various biological and technological characteristics. MetaWiz effectively reduced raw data error rates through mapping and error correction. Specifically, we were able to modify the following datasets into multiple benchmarks: Dataset #1: 9 strains of HIV and 2 strains of HIV (Illumina). We altered the frequency of viral strains and removed strain-specific reads. Dataset #2: 11 strains of bacterial species (Illumina). We altered the frequency of strains and species and removed species-specific reads. Dataset #3: 26 strains of Bacteria and Archaea (Illumina). We selected some of the species/strains from this dataset to mix with Dataset #2, to create multiple benchmarks with different combinations. Dataset #4: 12 strains of bacterial strains (Pacbio and Nanopore). For Pacbio reads, we shortened the read length to 100bp, generating up to 40 million short reads. For Nanopore ONT reads, we corrected the errors using the genomes and shortened the read length to 100bp, generating up to 60 million short reads. Conclusions. MetaWiz offers a cost-effective solution to the challenge of generating high-quality benchmarks in metagenomics. The tool’s ability to create semi-real benchmarks from existing experimental data maintains crucial technological and biological aspects, providing advantages over purely computational simulations. This approach can be extended beyond viral sequences to various sequencing data, enhancing the comprehensiveness of metagenomics benchmarking studies across different biological contexts. [ABSTRACT FROM AUTHOR]