I know from my own experience with faculty reviewing my papers as I proceeded through my Ph.D. program that peer review can be wonderful. The work that emerged from this process was immensely improved.
But Paul Thacker and Jon Tennant are absolutely right to point to some shortcomings[1] and I encourage you to read their article. I would add that peer reviewers are much too often from the same fields or disciplines as authors and thus share biases that lead to mistakes.
I first noticed this in my undergraduate work. I took a human development class where the textbook astonishingly emphasized nature over nurture. In the social sciences, the bias is very much the other way around: We assume nurture which, it must be admitted, has not helped the LGBTQ+ crowd gain acceptance. My feeling remains that if you are going to rely on nature as a cause, you need to carefully consider nurture.
Perhaps, as apparently with LGBTQ+ folks, you can rule nurture out as a cause for deviance from the binary “comfortable in skin” (cis) heterosexual “norm” that queer theory denies is a “norm.” Sexual orientation and gender identification look increasingly to be biological in origin, even if we still don’t understand how or how to reconcile the non-binary nuance with what appears to be the predominant binary.
But do the homework: Don’t just assume nature, as that textbook seemed to do, when social reproduction, the process by which culture is passed down from generation to generation, is such a major factor in who we are and how we view ourselves. Including how we define gender, gender roles, and what genders we recognize. Nature apparently causes LGBTQ+ folks to be LGBTQ+ just as it causes binary cis heterosexuals to be binary cis heterosexuals. Nurture almost certainly accounts for all the rest of it, especially stigmatization.
Similarly, when taking an Introduction to Communication class, the first theory we were taught was that of linear communication. This is about sender, receiver, channel, and noise (that interferes with communication from sender to receiver across the channel). A refinement on the theory allows for feedback, where the receiver signals to the sender how well the communication is working, enabling the sender to adjust the message.
The very second thing we were taught is that the theory of linear communication is outdated, that “nobody” goes by it anymore.
So I was astonished when years later, I read a study that just assumed linear communication. The author was not from the communication field and had plainly not consulted with anybody from the communication field.
Mistakes such as these are understandable because the boundaries between the social sciences are arbitrary and ambiguous. Researchers can’t help transgressing them because, really, these boundaries don’t make any sense to begin with.
The trouble is that researchers from one discipline or another lack the preparation to transgress the boundaries that they inevitably transgress. So they make bonehead mistakes. Like the nature versus nurture thing. Or the linear communication thing.
Peer review needs, much more often, at least to be interdisciplinary and preferably transdisciplinary—the latter beginning not from within the perspective of any one discipline but rather standing apart from them.
For instance, when I looked at a lot of medical research on chronotypes, I saw right away that authors had effectively confounded correlation with causation. (Sadly, this seems to be the case with a lot of medical research.) This is a huge no-no. In every research methods class, every statistics class I took, professors and textbooks specifically warned against this, sometimes ruefully conceding that it happens anyway.
I was also specifically concerned that these studies had not adequately accounted for social factors: Find some folks at a conference looking bleary-eyed and ask them who rules the world. If it isn’t that they had been partying the previous night, they might respond that early risers rule and that they are most certainly not early risers, that they, “night owls,” feel an oppression from having to conform to a “lark’s” schedule. Such oppression can obviously have knock-on effects, ultimately including all those dreaded psychiatric and medical conditions chronotype researchers had correlated with late and even middle chronotypes.[2]
Indeed, it turns out that “about 40 percent of the population are morning people, 30 percent are evening people, and the remainder land somewhere in between.”[3] But
Society likes morning people. Loves them, actually. Early risers tend to be more punctual, get better grades in school and climb up the corporate ladder. These so-called larks are celebrated as the high achievers, the apple polishers, the C.E.O.s.[4]
Because researchers had settled for correlation, they failed to adequately consider or control for variables other than chronotype. Like the demands of society that presumes Benjamin Franklin had it right with his “early to bed, early to rise” bullshit.
I was right.[5] But no medical journal editor would even think of turning to a human scientist for peer review because the paradigm calls for an increasing refinement of knowledge, that is, among specialists within the field.
Scholars are well aware of the problem. Where I started off seeing disciplinary boundaries as “blinders,” limiting one’s field of view, the term I see more often in publications such as the Chronicle of Higher Education is “siloization.”
Think of a silo: It is tall, surrounded by metal walls with a roof on top. If you are outside it you can’t see in. If you are inside it, you can’t see out. And the contents are homogenous: They might be corn, wheat, or some other grain. But those contents are all the same.
And it is a lousy metaphor for knowledge. The very metaphor of peer review.
- [1]Paul D. Thacker and Jon Tennant, “Why we shouldn’t take peer review as the ‘gold standard,’” Washington Post, August 1, 2019, https://www.washingtonpost.com/outlook/why-we-shouldnt-take-peer-review-as-the-gold-standard/2019/08/01/fd90749a-b229-11e9-8949-5f36ff92706e_story.html↩
- [2]Alex Williams, “Maybe Your Sleep Problem Isn’t a Problem,” New York Times, August 25, 2018, https://www.nytimes.com/2018/08/25/style/sleep-problem-late-night.html↩
- [3]Alex Williams, “Maybe Your Sleep Problem Isn’t a Problem,” New York Times, August 25, 2018, https://www.nytimes.com/2018/08/25/style/sleep-problem-late-night.html↩
- [4]Alex Williams, “Maybe Your Sleep Problem Isn’t a Problem,” New York Times, August 25, 2018, https://www.nytimes.com/2018/08/25/style/sleep-problem-late-night.html↩
- [5]Bryan Clark, “No, Night Owls Aren’t Doomed to Die Early,” New York Times, May 23, 2019, https://www.nytimes.com/2019/05/23/smarter-living/no-night-owls-arent-doomed-to-die-early.html↩
Pingback: The Irregular Bullshit