Skip to main content

Preprints in times of COVID19: the time is ripe for agreeing on terminology and good practices

Abstract

Over recent years, the research community has been increasingly using preprint servers to share manuscripts that are not yet peer-reviewed. Even if it enables quick dissemination of research findings, this practice raises several challenges in publication ethics and integrity. In particular, preprints have become an important source of information for stakeholders interested in COVID19 research developments, including traditional media, social media, and policy makers. Despite caveats about their nature, many users can still confuse pre-prints with peer-reviewed manuscripts. If unconfirmed but already widely shared first-draft results later prove wrong or misinterpreted, it can be very difficult to “unlearn” what we thought was true. Complexity further increases if unconfirmed findings have been used to inform guidelines. To help achieve a balance between early access to research findings and its negative consequences, we formulated five recommendations: (a) consensus should be sought on a term clearer than ‘pre-print’, such as ‘Unrefereed manuscript’, “Manuscript awaiting peer review” or ‘’Non-reviewed manuscript”; (b) Caveats about unrefereed manuscripts should be prominent on their first page, and each page should include a red watermark stating ‘Caution—Not Peer Reviewed’; (c) pre-print authors should certify that their manuscript will be submitted to a peer-review journal, and should regularly update the manuscript status; (d) high level consultations should be convened, to formulate clear principles and policies for the publication and dissemination of non-peer reviewed research results; (e) in the longer term, an international initiative to certify servers that comply with good practices could be envisaged.

Peer Review reports

Background

Over recent years, the research community has been increasingly using preprint servers to share manuscripts that have not yet been peer-reviewed, enabling quick dissemination of research findings, and in some cases to obtain peer feedback to improve the final version submitted to a journal [1]. Relevant international stakeholders provide guidance on ethics and integrity in scientific publications, in particular the Committee on Publication Ethics (COPE) and the International Committee of Medical Journal Editors (ICMJE). However, issues related to publication ethics and integrity become more complicated, when a form of publication has already occurred before peer review and is available to the public. Furthermore, calls to rapidly share research results to inform public health emergencies including COVID19, have introduced additional layers of complexity [2,3,4]. For instance, according to the World Health Organization (WHO) Working Group on Ethics and COVID19, researchers generating information with the potential to aid response efforts have an ethical obligation to share it “without waiting for publication in scientific journals”, and as soon as the information “is quality-controlled for release (e.g., peer-reviewed)” [5]. These recommendations have an inherent conflict, as the quality control of the peer-review process is managed by the scientific journals whose publication timelines should not be waited for. The pursuit of rapid dissemination also comes into conflict with the need to comply with adequate publication standards [2].

Preprint platforms: benefits and risks

Preprints (i.e. preliminary reports of work not yet subject to peer review) are increasingly posted online on websites such as the well-known MedRxiv, arXiv and bioRxiv [6,7,8], as well as other servers. These can be valuable mechanisms to facilitate rapid communication within the international scientific community, and trigger early, fruitful and transparent discussion among peers, while waiting for the scientific work to undergo a journal’s peer review. A cross-sectional study of preprint policies among the 100 clinical journals with the highest impact factors showed that 86% of journals allow for submitted articles to be previously posted as preprints, making researchers less concerned that posting a manuscript on a preprint server will disqualify it from further publication [1]. However, it remains important to thoroughly understand and accurately describe the nature and purpose of preprints. This is why arXiv explicitly states that material is “not peer-reviewed by arXiv” and” they should not be relied upon without context to guide clinical practice or health-related behavior and should not be reported in news media as established information without consulting multiple experts in the field” [7]; and bioRxiv clarifies that “articles are not peer-reviewed, edited, or typeset before being posted online” [8]. Preprints posted in MedRxiv, arXiv and bioRxiv include a small header or footer on each page with a warning that the article has not undergone peer review [6,7,8]. MedRxiv also explicitly states that due to the inherent nature of “preliminary reports of work that have not been certified by peer review”, preprints “should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information” [6]. This is an important caveat as there is no assurance that preprints have not had any external quality control when they are made publicly available. However, it is unlikely that it will always be carefully read—for instance, when preprints are circulated via social media.

Though peer review remains the de facto source of quality-assurance in scientific publishing [9], it does not always prevent inaccurate reports from entering the scientific literature [10,11,12], and mechanisms to ensure reviewer access to original data sets or codes for analysis to strengthen the quality of review are not systematically in place. This has become particularly evident during the pandemic. Smith and colleagues noted in March 2021 that in a rush to disseminate information about COVID19, diverse inaccuracies have been published, causing inappropriate changes in clinical care, ineffective public health responses, and increasing anxiety in communities [13]. It is also likely that the rush to publish COVID19 manuscripts is causing shortages of qualified reviewers. The vast and growing volume of submissions related to COVID19 is precipitating reviewer fatigue. Despite the limitations, peer review can help to mitigate overinterpretation and misreporting of results, and reduce the proportion of poor-quality and inaccurate publications [11,12,13]. Therefore, when it comes to pre-prints, both authors and readers would benefit from guidance about their appropriate dissemination and interpretation, and on reliable servers for posting and finding preprints.

The WHO has established a platform bringing together approved clinical trial registries [14], and a similar initiative for pre-print servers may be necessary. The Committee on Publication Ethics (COPE) and the International Committee of Medical Journal Editors (ICMJE) could play an essential role in setting high ethical standards for reporting and disseminating research findings—including those that have not yet been peer-reviewed-, intending to promote integrity along all phases of research dissemination.

Preprints: what’s in a name

Despite the caveats provided in various servers, preprints can still be confused with peer-reviewed manuscripts. The name “pre-print” itself may further encourage misunderstanding. The term implies that the manuscript will be printed or published, thus inferring a level of quality sufficient for publication. It may also raise expectations that the manuscript should, or ever will in any case be printed. Additionally, “pre-print” sounds very close to “Epub ahead of print", which indicates a peer-reviewed article that is being listed electronically before being typeset [15].

For some users, especially those who are not involved in the scientific publishing process, such as journalists, politicians, policy-makers, and the general public, this language implicitly suggests a sort of imprimatur, i.e. that the contents have already be approved, which may increase the probability that they are widely promoted and disseminated before a robust external evaluation. A clearer term should be chosen instead of ‘preprint’, such as ‘Unrefereed manuscript’ or “Manuscript awaiting peer review” or ‘’Non-reviewed manuscript”, to avoid preventable misunderstandings.

Hunger for scientific information in times of COVID19

The COVID19 pandemic has created a global “hunger” for scientific information to inform policy and the general public. In the 5 months prior to the COVID19 era (1/8/2019 to 31/12/2019), 14,078 preprints were posted in MedRxiv and bioRxiv. This number increased by 61.2% to 22,691 in the first 5-months of the COVID19 era (1/1/2020 to 31/5/2020), including both COVID19 and non-COVID19-related preprints. Preprints have become an important source of information for a variety of stakeholders interested in COVID19 research developments, including traditional media, social media, and policy-makers. But the philosophy of preprints, i.e. that errors will get fixed over time as the scientific community crowdsources and opines on the findings [22], may be unknown to or misunderstood by many of their new users, including politicians, journalists, “influencers”, prescribers, policy-makers—and sometimes even researchers. Nowadays, research appraisal and synthesis often occur before a decision is reached on publication, with the risk of leading to “irresponsible dissemination, as flawed studies are picked up by the media” [2]. For example, suppose clinical trial findings are only available as preprints. In that case, a cautionary approach should be taken to the interpretation of their findings, which should not be used as the sole basis for introducing a new therapeutic or preventive intervention into practice and policy. Lack of caution may have been a factor in the publication and withdrawal of early evidence on hydroxychloroquine efficacy in COVID19 [16,17,18,19,20]. Numerous papers related to COVID19 cite preprint materials, and it is not always explicitly stated that these are not peer-reviewed and should be cautiously and critically interpreted [21]. Additionally, if unconfirmed but widely shared first-draft results later prove to be wrong or misinterpreted, it can be very difficult for people to “unlearn” what they thought was true [22], because of ‘anchoring bias’, which comes into play when individuals prioritize information and data that support their initial impressions, even when first impressions are wrong [23]. Complexity may further increase if such unconfirmed findings had been used to inform guidelines; and if pre-prints remain pre-prints indefinitely and never get formally published, either because they are rejected or not even submitted.

We need a balance between early access to research findings and its unwanted negative consequences, such as the rushed adoption of policies unsupported by evidence on efficacy and safety, and misplaced or unrealistic expectations from the public, policy-makers and stakeholders [16, 24,25,26,27]. Therefore, we contend that all preprint servers should state, as does MedRxiv [6], that “Preprints are preliminary reports of work that have not been certified by peer review. They should not be relied on to guide clinical practice or health-related behaviour and should not be reported in news media as established information”. The caveat should not be divorced from preprints when they circulate, thus this statement should be prominent on the first page of each preprint, and each page should include a red watermark stating ‘Caution—Not Peer-Reviewed’ [10, 15]. This call is in line with a statement from the Editors of The Lancet, who confirmed the journal policy to make preprints a permanent offering, while applying “a more obvious watermark stating that these are preprints and not peer-reviewed” [28].

A call for scientific integrity standards

While preprints can help to make new evidence-based knowledge rapidly available to the scientific community, and perhaps improve scientific transparency [15], these benefits can be undermined by harms caused by the release, dissemination, and misuse of unreliable evidence. It is not a foregone conclusion that the potential benefits of preprints always outweigh the risks of harm [10, 29]. It is also crucial that preprints are accompanied by a minimal set of essential, ethics-related information. However, this is not always the case. For instance, only two-thirds of research paper authors who submitted to the Lancet Group journals and opted to post pre-prints at submission, had posted all this information, i.e. ethics approval if needed, declaration of interests, funding statement, and prospective registration for randomised controlled trials [28]. Thus, the various stakeholders that currently encourage the use of ‘preprints’, including research institutions and research funding agencies [30], should also encourage improvement of ethical standards, appropriate terminology, warnings regarding interpretation and dissemination, and recognition of limitations. Given the rapid and growing importance and use of preprint servers, particularly concerning the ongoing pandemic, the time is ripe for the scientific community to agree on “Good Preprint Practices” formally. Adequate and harmonized scientific quality and integrity standards [10] should be defined, as they have been for other critical research and reporting activities [31], and appropriate mechanisms for implementation and enforcement of such good practices should be identified and implemented. In particular, we propose the following specific recommendations:

  • As the term “preprint” may cause misunderstanding about the nature of these manuscripts, consensus should be urgently sought on a clearer term, such as ‘Unrefereed manuscript’ or “Manuscript awaiting peer review” or ‘’Non-reviewed manuscript”.

  • Key caveats about unrefereed manuscripts (i.e., that they are preliminary reports of work that have not been subject to peer review; that they should not be relied on to guide clinical practice or health-related behaviour; and that they should not be reported in news media as established information), should be prominent on the first page of each preprint, and each page should include a red watermark stating ‘Caution—Not Peer Reviewed’.

  • Pre-print servers should require authors to certify that the manuscript will imminently be -or is- submitted to a peer review journal; to regularly update the status of the manuscript (e.g. submitted, rejected, re-submitted, published with DOI); and to disclose their pre-print history.

  • High level consultations should be convened with relevant stakeholders, including but not limited to the ICMJE, COPE and WHO, to formulate clear principles and policies for the publication and dissemination of non peer-reviewed research results, and to further disseminate such principles and policies to the scientific community.

  • In the longer term, an initiative to certify servers that comply with agreed good practices could be envisaged.

Availability of data and materials

Not applicable.

Abbreviations

COPE:

Committee on Publication Ethics

ICMJE:

International Committee of Medical Journal Editors

WHO:

World Health Organization

References

  1. Massey DA, Opare MA, Wallach JD, Ross JS, Krumholz HM. Assessment of preprint policies of top-ranked clinical journals. JAMA Netw Open. 2020;3(7):e2011127.

    Article  Google Scholar 

  2. Glasziou PP, Sanders S, Hoffmann T. Waste in covid-19 research. BMJ. 2020;369: m1847. https://doi.org/10.1136/bmj.m1847.

    Article  Google Scholar 

  3. Modjarrad K, et al. Developing global norms for sharing data and results during public health emergencies. PLOS Med. 2016;13:e1001935.

    Article  Google Scholar 

  4. Moorthy V, Henao Restrepo AM, Preziosi M-P, Swaminathan S. Data sharing for novel coronavirus (COVID-19). Bull World Health Organ. 2020;98:150.

    Article  Google Scholar 

  5. World Health Organization. Ethical standards for research during public health emergencies: distilling existing guidance to support COVID-19 R&D. WHO/RFH/20.1.WHO Geneva, Switzerland, 2020.

  6. https://www.medrxiv.org/. Accessed 27 July 2021.

  7. https://arxiv.org/. Accessed 27 July 2021.

  8. https://www.biorxiv.org/.

  9. Sizo A, Lino A, Reis LP, Rocha Á. An overview of assessing the quality of peer review reports of scientific articles. Int J Inf Manag. 2019;46:286–93.

    Article  Google Scholar 

  10. Ravinetto R et al. Rapid response: Time is ripe for “Good preprint practices”. BMJ 2020. https://www.bmj.com/content/370/bmj.m3111/rron1/9/2020. Accessd 27 July 2021.

  11. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124. https://doi.org/10.1371/journal.pmed.0020124.

    Article  Google Scholar 

  12. Smith MJ, Upshur REG, Emanuel EJ. Publication ethics during public health emergencies such as the COVID-19 pandemic. Am J Public Health. 2020;110:947–8.

    Article  Google Scholar 

  13. Ochodo EA, et al. Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “Spin.” Radiology. 2020;267(2):581–8.

    Article  Google Scholar 

  14. International Clinical Trials Registry Platform (ICTRP). https://www.who.int/clinical-trials-registry-platform (who.int). Accessed 27 July 2021.

  15. Ravinetto R, Guerin PJ, Zaman MH, Caillet C, Singh JA, Newton PN. Rapid response to “Waste in covid-19 research”. https://www.bmj.com/content/369/bmj.m1847/rapid-responses

  16. Singh JA, Ravinetto R. COVID-19 therapeutics: how to sow confusion and break public trust during international public health emergencies. J Pharm Policy Practice. 2020;13:47.

    Article  Google Scholar 

  17. https://retractionwatch.com/2020/05/21/french-hydroxychloroquine-covid-19-study-withdrawn/. Accessed 27 July 2021.

  18. https://retractionwatch.com/2020/06/04/lancet-retracts-controversial-hydroxychloroquine-study/. Accessed 27 July 2021.

  19. https://retractionwatch.com/2020/07/10/a-month-after-surgisphere-paper-retraction-lancet-retracts-replaces-hydroxychloroquine-editorial/.. Accessed 27 July 2021.

  20. https://www.medrxiv.org/content/10.1101/2020.05.13.20094193v2. Accessed 27 July 2021.

  21. Tripathy JP. Is publication ethics becoming a casualty of COVID-19? Indian J Med Ethics. 2020. https://doi.org/10.20529/IJME.2020.092.

    Article  Google Scholar 

  22. Weiis R, Moreno J. Boosting the promise—and reining in the peril—of Covid-19 preprints. STAT, published on 28 July 2020. Cited 26 August 2020 from https://www.statnews.com/2020/07/28/boosting-promise-reining-in-peril-covid-19-preprints/.

  23. Molony DA. Cognitive bias and the creation and translation of evidence into clinical practice. Adv Chronic Kidney Dis. 2016;23(6):346–50.

    Article  Google Scholar 

  24. Africa Centres for Disease Control and Prevention (Africa CDC). Statement on medications to treat novel coronavirus disease (Covid-19). Africa CDC, Addis Ababa, Ethiopia. 2020. https://media.tghn.org/medialibrary/2020/04/STATEMENT_ON_MEDICATIONS_TO_TREAT_COVID-19_-_ENG.pdf (tghn.org). Accessed 27 July 2021.

  25. Lenzer J. Covid-19: US gives emergency approval to hydroxychloroquine despite lack of evidence. BMJ. 2020;369: m1335. https://doi.org/10.1136/bmj.m1335.

    Article  Google Scholar 

  26. Abena PLM, Decloedt EH, Bottieau E, et al. Perspective piece. Chloroquine and hydroxychloroquine for the prevention or treatment of novel coronavirus disease (COVID-19) in Africa: caution for inappropriate off-label use in healthcare settings. Am J Trop Med Hyg. 2020. https://doi.org/10.4269/ajtmh.20-0290.

    Article  Google Scholar 

  27. Newton PN, Bond K, on behalf of 53 signatories from 20 countries. COVID-19 and risks to the supply and quality of tests, drugs, and vaccines. Lancet Glob Health. 2020. https://doi.org/10.1016/S2214-109X(20)30136-4.

    Article  Google Scholar 

  28. Kleinert R, Horton R, on behalf of the Editors of the Lancet Group. Preprints with The Lancet are here to stay. The Lancet. 2020;396:805.

    Article  Google Scholar 

  29. van Schalkwyk MC, et al. The perils of preprints their use and platforms require greater scrutiny. BMJ. 2020;370:m3111.

    Article  Google Scholar 

  30. Sharing research data and findings relevant to the novel coronavirus (COVID-19) outbreak. Press release of 20.01.2020.  https://wellcome.org/press-release/sharing-research-data-and-findings-relevant-novel-coronavirus-ncov-outbreak. Accessed 27 July 2021.

  31. https://www.equator-network.org/. Accessed 27 July 2021.

Download references

Acknowledgements

Not applicable

Funding

There was no specific funding for writing this “debate” manuscript.

Author information

Authors and Affiliations

Authors

Contributions

RR, CC and PNN wrote the first draft of this debate article. MHZ, JAS, PJG, AA, CED, AJ, AP, LM, PWH, EB and TH critically read and contributed to the initial and following draft version. All authors approved this submission.

Corresponding author

Correspondence to Raffaella Ravinetto.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

We declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ravinetto, R., Caillet, C., Zaman, M.H. et al. Preprints in times of COVID19: the time is ripe for agreeing on terminology and good practices. BMC Med Ethics 22, 106 (2021). https://doi.org/10.1186/s12910-021-00667-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12910-021-00667-7