Research reproducibility and the need for Open Science

9:15 AM - 9:40 AM

We are all aware of the need for transparent reporting of preclinical and clinical research, and as part of this, ensuring accuracy in methodology and results reporting and addressing publication bias. Despite this awareness there is ongoing concern regarding the lack of reproducibility in biomedical research, particularly in preclinical stages – most publicised in psychology and oncology. This year, mainstream media and public interest in biomedical and clinical research has heightened significantly due to the global pandemic, and science must, now more than ever, strive to withstand scrutiny and respond to any deficiencies in the system that inhibit reproducibility.

Many initiatives around the world are serving to enhance research reproducibility. International clinical research registries, IRBs and international reporting guidelines have assisted in standardizing clinical trials, including advance registration of outcome measures, proper randomization, blinding and reporting of statistical analysis. Data-sharing initiatives (of which there are over 300 in biomedicine alone) – as well as the availability of open source software and workflow tools - are promoting access to and utilization of, existing electronic data, algorithms, and analytical tools. Data sharing statements are being promoted by some journals, however whether these reach their full potential in practice and increase the actual rate of data re-analysis and reuse remains to be seen; they seem under-utilised currently.

There are areas where further work is required. Peer review – often using two reviewers to check on the resulting paper - is meant to screen and protect against publication of erroneous findings. However, peer review in this form is outdated and can be subject to the failures and biases of writers and reviewers. Peer review can be hindered by a lack of transparency such as full access to the data, and a lack of infrastructure to reliably check the actual statistical methodology, data collection, analysis and conclusions. Preclinical research registries do exist, however, there are multiple additional challenges in biomedicine to administer. For new discoveries and drug development in particular, intellectual property and competition issues may prevent some aspects of research being pre-registered and entering the public domain.

The use of Open Science tools and frameworks is enhancing rigour and reproducibility in research, taking it out of traditional silos and into an environment of more open sharing and scrutiny of methods, results and data. Open access publication is increasingly chosen by authors and funders. However, for Open Science to truly succeed in biomedical and clinical research, a shift in funding trends is required – from funding research silos to funding collaborative research networks. Additionally, maintaining patient confidentiality in clinical research must be prioritized, along with upfront management of intellectual property matters and a comprehensive review of the incentive structures that underpin research culture. To this end, a fundamental re-design is required of institutional and academic research quality benchmarks, and of the traditional publication model.