Carpentries Community Discussion on PREreview.org

Scholarly peer review is broken

As defined by the Merriam-Webster English dictionary, peer review ¹is “a process by which something proposed (as for research or publication) is evaluated by a group of experts in the appropriate field”. But who gets to decide who is the “expert” and who isn’t? What are the criteria by which a proposed research article is selected or rejected? Different journals have different policies, but for the most part this process is opaque and done behind closed doors. This opaqueness might also be the root of a system whose players are disproportionately male (Helmer et al. 2017²; Lerback, American Geophysical Union, and Brooks Hanson 2016³; “Nature’s Sexism” 2012⁴; Fox, Sean Burns, and Meyer 2016⁵) and from North America or European countries (Chawla 2018⁶; Murray et al. 2018⁷). As part of a global push for transparency in scholarship, we at PREreview believe we need better ways to find, train, and engage researchers in peer review.

What can we do about it?

PREreview is a grassroots initiative that seeks to diversify peer review by crowdsourcing preprint feedback to improve the quality of published scientific output, and to train early-career researchers in how to review other researchers’ work.

To do that we are working on different initiatives. We are currently developing a new, open-source, free and independent platform that will allow all researchers to review preprints and be recognised for their contributions.

Additionally, we are developing a cohort-based peer review mentorship program aimed at training early-career researchers to give (and receive) constructive feedback on scholarly outputs, and get recognition for their contributions. In fact, even if peer review is the bottle-neck of scientific dissemination, researchers are rarely trained in how to do it or recognised for it.

Why we need your help?

We want to foster a culture change and to do that we need to build a diverse and resilient community. The Carpentries have demonstrated on multiple fronts their ability to engage people across professional backgrounds, cultures, languages, and geographical locations. We would love to learn from you how we can successfully reach our audience and empower all researchers in becoming active reviewers, connecting with peers, and develop long-lasting collaborations. Join our next community call on August 28, 2019 (sign up on this Etherpad) to ask and help us answer questions, share your experience, and suggest ways we can work together to achieve these goals.

To learn more about this initiative, read our latest blog post.
Follow us on Twitter @PREreview_
Email us at contact@prereview.org

PREreview is a grassroots initiative founded in 2017 led by three women scientists, Drs. Daniela Saderi, Monica Granados, and Samantha Hindle. The project is fiscally sponsored by the non-profit organisation Code for Science & Society and so far has been funded by the Alfred P. Sloan Foundation, the Wellcome Trust, and Mozilla.

1: Merriam Webster’s definition of Peer Review: https://www.merriam-webster.com/dictionary/peer%20review
2: Helmer, Markus, Manuel Schottdorf, Andreas Neef, and Demian Battaglia. 2017. “Gender Bias in Scholarly Peer Review.” eLife 6 (March). https://doi.org/10.7554/eLife.21718.
3: Lerback, Jory C., American Geophysical Union, and R. Brooks Hanson. 2016. “GENDER BIAS IN PEER REVIEW AND SCHOLARLY PUBLISHING.” https://doi.org/10.1130/abs/2016am-281633.
4:“Nature’s Sexism.” 2012. Nature. https://doi.org/10.1038/491495a.
5:Fox, Charles W., C. Sean Burns, and Jennifer A. Meyer. 2016. “Editor and Reviewer Gender Influence the Peer Review Process but Not Peer Review Outcomes at an Ecology Journal.” Functional Ecology. https://doi.org/10.1111/1365-2435.12529.
6:Chawla, Dalmeet Singh. 2018. “Huge Peer-Review Study Reveals Lack of Women and Non-Westerners.” Nature 561 (7723): 295.
7:Murray, Dakota, Kyle Siler, Vincent Larivière, Wei Mun Chan, Andrew M. Collings, Jennifer Raymond, and Cassidy R. Sugimoto. 2018. “Gender and International Diversity Improves Equity in Peer Review.” Scientific Communication and Education. bioRxiv.