Pushing the reset button on peer review

As part of my doctoral programme in sociology, I investigate purported resistance to new scientific ideas by editorial readers at scientific and medical journals. The research is nestled within my wider interest in the sociology of ignorance.

Scientific ignorance here is non-pejorative and refers to the limits and the borders of knowledge – what scientists know that they do not know yet. New scientific ideas typically propose new ignorance.

Interestingly, early on I hit a surprising roadblock. It became apparent that research into journal peer review performed by natural and social scientists did not always carefully construct journal peer review as an object of study (object). Rather, it was frequently framed as common experience or as a pre-constructed and self-evident object.

Studying peer review as a pre-constructed object of study

Studying journal peer review as self-evident holds many drawbacks, not least of which is that this approach is not scientific – in the natural or the social sciences. Already in the 17th century, Francis Bacon referred to pre-constructed objects of study as ‘prenotions’ or ‘vulgar notions’ based on a failure to go beneath the surface to construct an object scientifically.

The use of pre-constructed objects in alchemy and astrology, for example, created a boundary between these non-scientific activities and chemistry and astronomy that purposefully and painstakingly constructed scientific objects.

I previously outlined a few contemporary examples of pre-constructed objects in the natural sciences.

One example involved Nobel prize laureates Barry Marshall and Robin Warren who proposed the H pylori bacterium as a cause for peptic ulcers. They encountered resistance as they challenged two pre-constructions for peptic ulcers: first, that these were mostly due to stress and second, that bacteria could not survive in the highly acidic stomach.

I also broached examples of how social scientists likewise investigate pre-constructed objects of study using common experience criteria.

Even if natural and social science researchers meticulously apply scientific methods to journal peer review such as statistical or narrative analysis and rigorous research designs such as clinical trials or case studies, these remain inadequate. When the object itself is not carefully tended to, rigorous analysis cannot compensate for porous foundations.

Neglecting how the object of study is shaped

A second aspect of journal peer review appears to have been neglected by researchers – how the object came to be, and changes. In sociology we refer to this as the shaping of an object.

Looking at shaping can help researchers not fall into the trap of assuming a social object to ‘simply be what it is’ without first investigating the social conditions, dynamics, processes and contexts that contribute(d) to shaping.

For example, an underlying assumption in most research into journal peer review is that it was borne of a need for rationality at the first journals in 1665. Historical and contemporary research into shaping helps do away with such pre-constructed notions.

Pushing the reset button on journal peer review research

The above highlight a need to push the reset button on how we investigate journal peer review. I propose an approach building on the concept of ‘social form’. A social form helps capture how individuals relate around a particular content.

In an historical investigation (or unabridged versions parts I and part II ), and building on existing research, I proposed two precursor social forms of early journal peer review: inquisition and censorship.

The link among inquisition, censorship and journal peer review is not superficial – it is based on analogy. By analogy, the proposed form of ‘boundary judgement’ where individuals aggregate around a common content – decisions as to what can be deemed scientific (or not) – fosters judgments for scientific written texts held against an overarching knowledge system.

For inquisition and censorship, religion as an overarching knowledge system helped determine what was deemed scientific, or not. Sometime in the 19th century religion receded in science and the overarching knowledge system became self-regulating science.

Looking at journal peer review as social form means analysing structural properties that make up a form. For pre-publication journal peer review, for example, I proposed a model and also performed a brief comparative analysis with a contemporary post-publication journal peer review social form.

An investigator who views the object through a social form lens no longer assumes that journal peer review must include structural properties of anonymity for referees, be performed pre-publication, or require a criterion for originality.

Pre-constructed or common experience conceptions of journal peer review become irrelevant and are replaced by an understanding for ever-mutating structural properties.

For example, historical research into shaping frames anonymity for referees as a deviation from journal peer review’s legal roots in the censorship function of inquisition. Historical and contemporary research into shaping helps delve into why anonymity for referees and structural secrecy for editorial judgments and decisions can be exploited by ‘unscrupulous’ journals.

Finally, viewed through a social form lens, editorial readers do not appear to resist new scientific ideas per se – rather, those engaged in an editorial reader role hold a relation of accountability to empiricism where they question specific elements of scientific explanations for new scientific ideas.

Pushing the reset button on journal peer review research is an opportunity for natural and social scientists to join forces in order to investigate and propose scientifically-founded changes for a shared object of study.

* Joanne Gaudet is a doctoral candidate (ABD) in sociology at the University of Ottawa.