Elizabeth Oldfield speaks to lecturer and author Dr. Stuart Ritchie. 05/01/2022

Stuart is a lecturer at the Social, Genetic, and Developmental Psychiatry Centre at King’s College London. He’s been a researcher in human cognitive abilities, and his most recent book is ‘Science Fictions’, a popular science book that presents an insider’s view on how fraud, bias, negligence and hype affect scientific research.

In this episode he talks about being an overconfident frontman in a band in his teenage years, having a similar rhetorical style to Boris Johnson, the role science plays in public, and how scientists could use a little bit more ethics training. 

Each episode in this series includes an additional reflection from Elizabeth at the end, so keep listening if you’d like to. We are also publishing full transcripts of each episode so scroll down on this page if you’re a reader rather than a listener.

Elizabeth
Hello, and welcome to The Sacred. My name is Elizabeth Oldfield and this is a podcast in which I speak to people who have some kind of public voice to try and get a sense of their deepest values, what’s driving them in the ways they use their platform, and what they’ve learned about how we can navigate our deep differences in our fractured and conflicted common life. As usual, we love hearing from you – please get in touch via Twitter @sacred_podcast or @ESOldfield. And as you are, I’m sure, very bored of being asked to do, do please leave us a review or a rating on iTunes. 

In this episode, you’ll hear a conversation I had with Dr. Stuart Ritchie. Stuart is a lecturer at the Social, Genetic and Developmental Psychiatry Centre at King’s College London. He’s been a researcher in human cognitive abilities, and his most recent book is ‘Science Fictions’ which has been published in July 2020. It’s a popular science book which presents an insider’s view on how fraud, bias, negligence and hype affect scientific research. I really liked learning that he was an overconfident frontman in a band in his teenage years. And also the moment when he confesses to a similar rhetorical style to Boris Johnson. We had a really interesting chat about the role science plays in public, the way that power is wielded for good and for ill. And we ponder together about how scientists could maybe have a little bit more ethics training. As usual, you can have some further reflections from me on the conversation at the end, and I really hope you enjoy listening.

So, on this podcast I don’t do chit chat, ease you in. We’re going to go straight for the hefty question, which I’m hoping you had a briefing email about. What is sacred? So lots of people have particularly religious connotations with this word, you can just strip that away. And what I’m trying to get to is a principle or a value that feels very deep, if someone offered you money to give it up, you’d at the very least feel a bit compromised.

Stuart
I think increasingly, the idea of, of scepticism is really important to me and I, you know, in science, we talk about the Mertonian norms, Robert Merton, the sociologist in 1942, had these four norms of how science should be done. The one, the final one, is this organised scepticism thing, where you’re constantly questioning everything. I think so many of our problems come from people just not asking questions about stuff, whether that’s through social awkwardness, or being scared to question authority, or being scared to just to speak up. And I think so many of our problems within science and also, you know, you’ve seen this happen in the pandemic and various other places, come from just not wanting to ask questions and be sceptical. Whereas, you know, as Robert Merton would have said, being sceptical and forcing skepticism on absolutely everything is an integral part of what science is all about. 

Elizabeth
Interesting. We’re definitely gonna come back to that but before we come to that, we’re gonna put you in a bit of context, we’re going to hear about your story, get a sense of where you come from, and the things that have shaped you. So I’m interested in particular, were there any big values, big ideas in the air in your childhood, political, philosophical, religious, other?

Stuart
I don’t think there were any like very strong ideas. I’m from a fairly boring, nicely boring, background in Scotland just south of Edinburgh. I guess no. In some weird sense, moderation has been you know – whenever anything goes too far in one direction my parents would always be these mad people saying this or saying that and so I feel like maybe that is what taught me to be sort of questioning whenever something goes off into an extreme in whatever way, but that also could just mean that sort of boring Lib Dem voting parents have not given me any particularly strong ideas in my life. I think finding a Richard Dawkins book on my shelf from my mum when I was kind of late high school made me realise that you can have interesting arguments with people and we set up a debating club in our high school, where I very much upset several of the other students who weren’t perhaps used to the idea of robustly debating things like euthanasia and abortion and the existence of God and all that sort of stuff. And so I think it’s one of those kind of ‘found a book on the shelf that that made the person you are’ types of situations that I had.

Elizabeth
Give me three words that describe Stuart as a teenager

Stuart
Confident or perhaps overconfident. I joined various bands and would be the lead singer in various bands. I was, from about 15, I was in various rock bands, and we played in, rehearsed in, garages and things like that. And we did that. I used to jump around on the stage and I really, you know, what must have been very embarrassing to look at and embarrassing to watch, but people would always say, Oh, you’ve got so much energy – I didn’t know if that was a sort of a way of them saying just calm down a little bit. But I noticed Boris Johnson does this thing where when he’s talking in front of an audience, he’s kind of joking a little bit with the audience. And he’s kind of saying, not in so many words, but he’s sort of saying, this is all a bit of a laugh. We’re all having a laugh here. This isn’t that serious. But he’s playing that it is serious to the other person that he’s talking to or whatever. But he’s kind of winking at the audience not deliberately. And I realised I do that as well.

Elizabeth
I’ve obviously been reading a lot of what you’ve written and listening to podcasts. And one of my interests in is in how we kind of cross these divides, how we have conversations with people we disagree with. And lots of people in that space, lots of people who are scientists, or who are public figures with a voice in these kind of things, have a certain kind of scrappy energy about them. I’ve got lots of friends who are rationalist, or self–described sceptics. But there’s much more of a like, this is a serious battle that we’re in guys. Whereas your energy is very playful and calm. Even when you’re having conversations about COVID data. Yeah, how do we navigate these life and death risks and what do we owe to each other as society? Do you think it’s temperamental? Is it from your parents? Do you see it? Do you have a hunch about where it comes from?

Stuart
I realised early on that if you if you talk in that sort of style, people get on side and I don’t have any, you know, stage fright or worry about public speaking like, like many people do. I mean, I was reading about, just recently, about – there’s a psychology paradigm like a sort of, in social psychology, like a type of experiment that they do, where the thing they use to induce anxiety is to make people do a speech in front of a panel of people because it’s just so common to be – and I thought to myself, I wouldn’t bother with that. I can sort of extemporaneously speak on stuff and it doesn’t really matter. I mean, I’m not saying it’s high quality, but it doesn’t make me feel anxious. 

Elizabeth
What does make you feel anxious?

Stuart
Letting people down, people expecting things and you haven’t done them yet. And I unfortunately say yes to almost everything people ask in terms of work related things – writing. ‘You want to work on this paper?’ ‘Yeah, sure.’ ‘Do you want to help supervise my PhD student?’ ‘Yeah, sure.’ ‘Do you want to come to a different country and do a talk though you’ve got several other things the day before and the day after?’ ‘Yeah. All right.’ And I – and then the problem is I say yes to so many things that I then worry about, I then can’t do everything and I worry about letting everyone down. That’s my major anxious – It’s all very social anxiety. 

Elizabeth
You’re being very patient with me asking personal questions, which I know academics tend to hate, but your discipline almost felt like it gave me a little bit more leeway because you are in psychology, it’s your kind of overarching discipline Tell me what drew you to it. What was the road to psychology?

Stuart
It’s very embarrassing because psychology has this reputation as like the thing that if you don’t really know what you want to do at university, you kind of choose psychology. And, and that was me. So I remember having a chat with the careers counsellor at the end of high school. And she said, Well, you do okay in science things and you seem to be doing okay, in you know, English and all that sort of stuff, and you want to speak to your interest in people and so on. How about psychology? And I thought, oh, right then and then I took it, I think I actually spelled psychology wrong on my application form to university, I think I put fi–cology or something like that. And that’s how much I knew about the topic. We didn’t have it in high school. So it was very much a kind of a, here’s something that seems like a halfway house between, you know, being interested in people and being interested in, you know, science things, maths, data, whatever. I chose psychology because I didn’t really know what else to do. And then it turns out that actually it’s not bad actually, it’s quite fun. I started realising, you know, there were lots of really interesting controversial things in psychology that I can, you know, get interested in. There are lots of debates. And then you know I have just followed debatable controversial topics in psychology ever since.

Elizabeth
What do you love about something that’s causing a bit of heat?

Stuart
First of all, it’s a really good way to learn. If there’s some very controversial issue, it’s a really good way to learn about things. So I know everything I know about evolutionary biology, because I got interested in creationists and arguing with creationists while I was an undergrad, and we had a creationist, there was a church nearby, that had very, very hardcore young earth creationists, who would bring in a speaker every week and do a talk, and me and a couple of pals would go and sit in the back, and then we would debate them in a very, I think we became quite friendly, not in an aggressive way at all, not in a shouty way that you associate with students. And in order to properly debate these things, you’ve got to go and learn how things work in terms of the biology and so I learned everything about that. It’s the same with learning, it’s much more fun to learn, to me anyway, to learn about climate science from seeing what, you know, obsessive cranks on the internet say about it, and seeing how that has been debunked, rather than just a book about the dry, you know, science of climate change. So that’s always much more interesting. And then, you know, that spurred my interest in science as well, because one of my major interests in in psychology is individual differences, people’s differences in their intelligence, their cognitive abilities, and their personalities, and so on. And obviously, especially when you start bringing in genetics and the brain, and so on, that becomes extremely controversial. And I think it helps you sharpen the arguments when a field is controversial. So my colleague, Robert Plomin, was doing a lecture just yesterday for our students, he was kind of reflecting on why behaviour genetics as an area of psychology, although lots of areas of psychology have kind of had their had their findings questioned in the past few years, the basic core findings of behaviour genetics haven’t really been. I mean, there are people who have always been sceptical of them, but within the scientific field, there’s not been this huge revolution, in the same way that it has been in social psychology, say, where a lot of the experiments have been kind of undermined. And one of the reasons for that he thinks is that it’s been criticised for decades and decades, we’ve had this constant debate, constant argument, constant criticism for so long, that it helps you sharpen up the argument, the standards are very, very high, because people are perhaps offended or whatever by the results. And I think in many cases, due to misunderstandings of results, they’re offended. But that offence causes people to have a very high bar for evidence. And baby geneticists have had to kind of go and leap over that. And so I think that’s one really important reason why you know debate is worth doing because you come at the end of it, and you think, Well, actually, I have sharpened up my arguments. And I don’t mean debate in the sense of…

Elizabeth
Personal attacks

Stuart
Right, exactly, exactly. I don’t mean, you know, just aggro for the sake of aggro, and I also don’t mean, someone does a 10 minute opening, a next person does a 10 minute opening, five minutes rebuttal, like that’s entertaining, but it’s not actually really helpful in terms of sharpening things up. And it’s not what, you know, Robert Merton would have talked about in terms of the scepticism in science, that’s you know vulnerable to all sorts of strategies where you just throw out tonnes and tonnes of arguments and then your opponent doesn’t debunk them all. And you say, well, you only debunked some of my arguments and therefore I’ve won the debate. That’s not the way to get to the truth but debate in general, and longer form arguments and so on, I think is worthwhile.

Elizabeth
And you mentioned shouty students and controversy so I would be remiss if I didn’t ask you about the current academic climate, and the context of this is I really struggled to get a handle on what is actually happening. And I’ve read, you know, the open letter signed by the academic saying they’re being censored and the open letter signed by the academic saying, stop being such babies, you’re not being censored, and the various statistics that are thrown about, about instances of people feeling professionally squashed and, yeah, there’s the Netflix series at the moment, The Chair, which is all about that. It’s, it’s not sort of necessarily empirically, particularly amenable question, is there a kind of chilling of academic freedom? 

Stuart
Almost, by definition, you can’t know.

Elizabeth
So what – with the data available to you/your personal experiences as someone who has just come from teaching students, do you think you’re more nervous about wading into controversies now than you were? Do you get a sense that your colleagues are? Or will this all just come out in the wash?

Stuart
Yes, I’m more nervous about talking about my own research interests, I think, now than I was, even though I’ve always taken a very moderate stance on human intelligence, and so on, I think I’ve always, you know, without throwing everything out, which is what a lot of people want and just saying that none of this makes any sense, or becoming obsessed with it and saying that, it’s the only thing that makes, you know, that is worth studying on the other hand, and there are people who represent both those extremes. I’ve always tried to take a fairly moderate stance of saying, intelligence tests are useful in some contexts, they’re part of a full psychological evaluation. And, you know, shouldn’t be understood to be a measure of worth or anything moral like that, but are just an interesting predictive tool. And I think I think I am more nervous now to say them because I think– this possibly is because I have more followers on Twitter now than I used to, and so there’s more people responding. But I think you get more pushback for saying things, which I think, you know, five years ago, you wouldn’t get the same amount of pushback. But I do think there’s a kind of, a certain set of people who are out there to misinterpret and, and kind of view this as a sort of a cultural war, it’s a culture war. I mean, they view themselves as being at war, in some sense, and when you’re at war, you don’t need to make, you know, you don’t need to consider other people’s arguments in very much detail or be charitable to them, you just need to smash them down, and accuse them of being, you know, all sorts of terrible things. So yeah, I think I’m somewhat more nervous. I do think the climate on places like Twitter is pretty bad for discussing controversial topics, I think the idea of getting piled on and so on is pretty, it’s pretty horrible. I’m a really junior academic, and I wouldn’t want to, you know, risk my career and so on. So I do think there’s a bad atmosphere about that sort of stuff. But I just would be a wee bit nervous now about, you know, just talking about, you know, some new paper I’ve done on cognitive development or something without adding lots of caveats, and being very careful about how it’s said and, you know, to some extent, you might say, adding caveats, and being careful is really good as well. That’s good science. I’m interested in what’s, you know, in the last few years become known as socio genomics, that’s linking genetic variables to what would be considered social outcomes, like education, for instance, there’s lots of research on that. And Kathryn Paige Harden has written this book recently called ‘The genetic lottery’ that focuses very much on this kind of thing of how could genes be linked to education? What does that mean? What does that mean for social equality and so on, so she’s getting all the fire from both sides of that debate currently, but I’m interested in that topic. And recently, people have started adding FAQ documents to their scientific papers to say, you know, you might think that this means this, but however, I just want to let you know that this doesn’t mean that people are determined in some sense, or it doesn’t mean that people have their worth designated to them at birth from the DNA, and that’s it, and people are being much clearer. And so to that extent, I think the sort of atmosphere is quite good in that, it allows scientists to be better at communicating their results, and to think more about how they write things down. And there are ways, you know, in scientific papers, that things can be written which do generate controversy and generate, you know, get people’s backs up, essentially. Um, so to that extent, it’s good. Having said that, if the criticisms are valid, then it doesn’t matter if the person that’s making them hates your guts, right? If the criticisms are valid, then you should respond to them or try and hit them off or anticipate them in some way when you’re writing. So those controversial areas are perhaps they’re reacting to the criticism in some, you know, some good ways. They’re reacting to them in some other – in some bad ways, too. I think there are certain topics now that you’re just – so there’s all this open data, open genetics data that you can use to, you know, do various research studies. And some of those databases are now kind of being closed off, if you’re interested in certain questions. It does seem to be odd when I go on to the social genomics website, and it says, You can’t have these data if you’re going to do any research to do with differences between groups of people, you just can’t have it. And in fact, we’ll basically complain to your university and might even sue you if you use our data to do this. 

Elizabeth
Hunt you down!

Stuart
Yeah, if you make racist claims based on this data. It doesn’t seem very open science. And I feel like it merely validates the conspiracy that lots of the kind of race obsessed far right people out there who are interested in this research have, which is geneticists have something to hide and they don’t want anyone to do this research. And, you know, I think it’s, I think the block is there for a well–meaning for, you know, what they think is a good reason, which is that this research is so inherently confounded by all sorts of social and other factors that you’ll just never get anything worthwhile. And if you put that result out, it’ll cause lots of social controversy and pain and so on. And it’s and it’s actually not really based on good science. And I’m sympathetic to that. But I do think you’re kind of, you’re seeding the ground somewhat to the bad researchers, the ones that are only interested in causing that controversy by saying all the good researchers are not allowed to do any of this research ever.

Elizabeth
When you encounter controversy or pushback, I know most recently you wrote a review of a book and the people took against that on their podcast – it was the first time I’ve seen you in all of the things I’ve read and listened to about you where I could sense the exasperation. Where do you go with your emotional life on that kind of thing? Do you have practices, do you have people, do you do exercise? Do you meditate, like what are the things that you use to navigate that kind of whole person, not just a scientist?

Stuart
It’s very much other people, so I think sharing things with people, not in public on Twitter and so on, but sharing things with other people, with friends who are kind of already interested in these topics or know the people who are going to cause controversy and so on, is massively helpful because you share something and they’ll say, Oh, don’t worry about it so much and there’s a social aspect to it as well, right? If you get criticised by certain people online or whatever like these, like these people, if their social status is a certain way, then it becomes water off a duck’s back to be criticised by them because you’re just, it’s just not something that’s ever going to cause you any problems. Whereas if you were being criticised by people who are much higher status in your own friend group, or whatever, you really would feel that, you would worry and you would lay awake at night. I don’t really – it doesn’t bother me at all that that fans of Brett Weinstein and Heather Haying are criticising me online because I don’t really consider them – if you were a fan of those people then you’re not the sort of person that I really care about the view of I must say. Having a set of people who are kind of interested in this topic who are always talking about it in private helps you get around it because you can make jokes, make in–jokes about these sort of things and get over any insults and so on. I wasn’t bothered by that stuff at all. I was exasperated certainly but I wasn’t bothered in the sense of being worried that it would you know, ruin my life or whatever. 

Elizabeth
I would love to hear what you think is science’s social role right now, depending on who you listen to it to either, you know, is in crisis, and there’s so much misinformation. or it has become the new priesthood and it’s the only truth anyone is interested anymore. Obviously you’re not going to say yes to either of those extremes. How would you narrate it?

Stuart
I think the conformity priesthood thing is very worrying because that tends to be bought into by people who are powerful in science and so on. And I’ve talked about this in many different articles over the pandemic. What happened at the start of the pandemic was this really horrible display of this conformity with, you know, on the topic of masks, on the topic of herd immunity, on the topic of whether the Coronavirus was dangerous at all. And in my own field, back in February, March 2020, in psychology, there was this weird group thing that developed that, you know, if you’re worried about this Coronavirus thing, then there’s something wrong with your brain, your brain is biased in some way. You’re not understanding risk properly. You’re not dealing with the numbers properly. And we know and you had very, very big name figures in the world of, you know, risk psychology or judgement and decision making, they would call it psychology, writing articles saying, you know

Elizabeth
‘Everyone calm down’

Stuart
Yeah, calm down and it’s not just calm down, but that there’s something wrong with you. If you find – if you’re worried about this, there’s something wrong with you. And I criticised them in an article that I wrote that was one of my first articles of the sort of writing about Coronavirus thing and it was called ‘don’t trust the psychologist’ and I got in big trouble…

Elizabeth
Bet they loved that

Stuart
I really think that was an amazingly depressing, you know, application of sometimes lab based psychology to the real world where it just fell flat on its face. And we all know what happened after that we had, you know, we showed that people were indeed right to be, you know, really concerned and worried about their families and buying toilet roll and all the stuff that we were doing at the start of the pandemic, which we all remember so well. And so I think there’s this group think and then it moved into different groups think about masks – ‘masks don’t work. And in fact, if you wear a mask, it’s bad and it’s gonna make you more likely to get Coronavirus.’ And it’s just that overconfidence. That, as you said, this kind of priesthood thing is, is that the priests who have told you this and we are now going to be super confident in it and really vehement in the message. ‘You do not wear them’, the US Surgeon General, ‘Do not wear a mask’ in capital letters he wrote, you know. And you know, the thing that I pointed out in this article with my friend Michael Story was that it’s the vehemence of it, it’s the lack of uncertainty that we were really complaining about – not that people got things wrong, because getting things wrong is inevitable. Everyone is going to get something wrong and have to change their opinion and that’s completely fine. I think you can trust people’s intelligence to a much greater degree, you can say like, we’re not 100% sure of this. There’s some evidence this way and there’s some evidence that way, you know, probably precautionary principle you should wear a mask would have been a good – would have been a decent argument. I think you could have backed that up. So I think science was used in the initial stages of the pandemic to… in this very, very tribal social sense of you know, there are all these idiots out there who are wearing masks, how dare they do something that goes against the consensus, and now, of course, it’s the other way around. And I don’t think it’s ‘anti–science’ or, or ‘anti–institution’ or whatever to point that out now, that there was this major U turn. I think that U turn would have been a lot easier, and you would have had less pushback and less people saying, ‘Well, I don’t need to wear a mask because you were saying a few months ago yourself, very strongly that we shouldn’t wear a mask, and what’s changed?’ I wasn’t hugely impressed by the way that science was used at the start and that’s before we get into all the debate about the herd immunity

Elizabeth
Tell me about what led you to write the book because you’ve basically turned your kind of sceptical instinct on science itself. Was there a kind of, what do they call it, a narrative theory? Was there kind of a triggering incident or a moment where you were like this is terrible. What do we do?

Stuart
Yeah, there was, it was during my PhD. One of the things I’m interested in is para psychology. When I was at the University of Edinburgh, we had a pair of psychology units, we had academics who were actively researching psychic abilities. Telekinesis, so moving things of your mind, pre cognition, you’re predicting the future, clairvoyance, so seeing into things which you can’t physically see. Pre sentiment is another one is, you know, having a feeling something before it – you feel upset, and then you find out that someone’s died. And you can do all that kind of thing. So it was particularly interesting when this paper appeared in 2011 when I was doing my PhD, by Daryl Bem, who’s this very famous social psychologist at Cornell University in the US, and he claimed that his undergraduate students had psychic powers. He claimed that he could show in the lab that his undergraduate students could predict the future using some kind of pre cognitive means that were unknown to science. And it was experiments like, I mean, the classic one is you’re shown two curtains on the screen, and you’re told that there’s a picture behind one of them. And you kind of go well, I wish I could have possibly known you know, just whichever one you feel the pictures behind. So you click one of them at fairly random. And if it’s just a picture of something boring, like a table or something, then you get it 50% of the time as you would expect. But what he claimed in this paper was if you put a pornographic picture behind one of the curtains, then people get it at like 53.1% of the time or something. 

Elizabeth
Lustfulness is activated. 

Stuart
Yes, their lustfulness activated into the future, they have a future sense that they’re about to see some sort of erotic material. And it makes them more likely to, to like to click the one where they will know that it has appeared. And then it also, he also showed the opposite direction too. Where if you have a violent picture behind one of the curtains, people are less likely to click that one. And so obviously, I was sceptical of this when it came out. But this was published in the mainstream scientific literature in a very prestigious, mainstream psychological journal. And we thought we’d replicate it. So actually, with Chris French, and Richard Wiseman, and I, we got together and we ran the same experiment again, three times. So not the pornographic one. But one which was about words, much less, you know, exciting to talk about. But it was basically about seeing a bunch of words, and then writing down as many of them as you could remember. And then seeing some of the words later. And having seen some of those words later made you remember them better in the past – my head exploding at the thought of that, it’s like studying for an exam, doing the exam, and then going home and studying a bit afterwards. And that extra studying actually helping you in the in the exam, that’s what he claimed was the case.

Elizabeth
Wow

Stuart
And we found absolutely nothing. So we found no evidence that this was happening at all, that our undergraduates had any psychic abilities whatsoever. And that was obviously, you know, very disappointing for some people. And the kind of trigger moment for me, which has been a long way of getting to this question, but the trigger moment for me was when we sent the paper that we wrote up of that negative replication, or null replication, failed replication, we sent it to the same journal that published Darryl Bem’s original paper. And they said, we’re not publishing this. And in fact, we wouldn’t be interested in any replication study whatsoever. So we’ll publish the original finding, the very exciting finding about psychic powers. But we won’t publish any replications whether they’re positive or negative, that’s just not a thing we’re interested in. And that really got me thinking, that really got me thinking, is this how science should work – that journals are interested in the flashy, exciting stuff, and not the perhaps more boring, but more reliable research that comes that comes later? And we eventually did get it published somewhere else. But that really made me think that the incentives are not quite right in science. I was doing my PhD, and then, you know, the early part of my postdoc at a very pivotal time in the field of psychology, psychological science, where we really were starting to say, Wow, the foundations of what we thought we knew about social psychology or, and many other areas, too, are just crumbling. The whole edifice might come down, and in some subfields of psychology, it really did. 

Elizabeth
Tell me the four ways that science goes wrong, in layman’s terms as much as you possibly can

Stuart
Yes, yes, well, so the four ways that I identify in the book are fraud, bias, negligence, and hype. So fraud is fairly straightforward. There are far too many scientists, I mean, a minority, but far too many who just make up their results. So who, instead of you know going out and collecting data and an experiment will open up an Excel spreadsheet and just type in the numbers they want for their experiment, and then often hand that data in inverted commas to their PhD students and say, Look, I’ve collected some data, use this in your PhD dissertation. And it was never, it never existed to begin with 

Elizabeth
How widespread do you think this is?

Stuart
Well, if you ask people, if you just ask scientists, then about 2% of them admit that yes, I have committed some sort of data fabrication or falsification in my career. And if you ask them, have any of your colleagues ever done this? Do you know someone who’s done this, about 14% of them will say that they have some suspicion or some knowledge of fraud. And that was from a survey study that was done quite a while ago, 10 years ago, but you know, that needs to be updated. But that was the best evidence we have from that. It’s a minority, but it’s far too many. Then fraud blurs into the second thing, which is bias, which is where scientists can often in many cases, when they’re setting up their experiment, or when they’re doing their analysis of their data, can basically put their finger on the scale and push the results in the direction that they want. Scientists are not these 100% objective disinterested robotic machines. They’re human beings, and they often want a hypothesis to be true. They want, for very good reasons, their drug to work to cure whatever disease they’re looking at, they want some, you know, economic aspects to relate to some other aspects of well–being, whatever it happens to be, because they, you know, that would help them explain the world, that would help them help people.

Elizabeth
Also for some less good reasons too right, just because it’s, that’s what the incentive structure is? 

Stuart
Sure. Exactly, exactly. So and that all becomes blurred and merged and in people’s careers, you know, they set out to try and help people but also, if you’re helping people, that’s you getting positive results and getting positive results gets you more papers and gets you more grants and gets you more promotions and gives you more fame and gets you more prestige. And that’s not how science should work. I think the insidious thing about that is, people don’t realise, so people don’t realise that they’re putting their thumb on the scale, they do it unconsciously. And it’s this unconscious bias that is kind of pushing them towards positive results, over rigorous results, just interesting, flashy, cool results, rather than things that which are real. And I think you can you can convince yourself post hoc, you can say to yourself, oh, well, you know, that experiment that gave us flat, completely no results, the drug didn’t work, actually, you know, for whatever reason that wasn’t a very good experiment. So I won’t even bother, I won’t bother pushing that one up and pushing that one into a journal, you know, I’ll work on another one. So you can start convincing yourself. So the incentive structure is bad in that respect, then I’ll move on to talk about just basic, just slips, errors, negligence, where scientists are not checking whether their numbers are correct. And there are these studies of where an algorithm checks the statistics in a paper, goes through thousands of scientific papers, and checks whether the statistics are all actually numerically consistent with each other and finds that in some huge proportion of cases they’re not. And this can only have come about through, you know, widespread error, probably copy and paste errors, typos. And then the final one being hype, which is, I think, is, you know, scientists going beyond the data, exaggerating the results beyond what the data can actually show and I think this is extremely common in popular science, people writing books that are based on some interesting studies where there is a kernel of truth and a kernel of good research and good quality stuff. But of course, the incentive to sell more books and to make your books sound more exciting is to say, this is the cure for cancer or this is the way that you will revolutionise your life and it will help you study or change your relationships, that will help you know whatever it is, is to hype things up way beyond what the often very patchy data can do. All these things together. I think they come about because of the way that academia is set up. So the way academia and things attached to academia, like Popular Science writing, and so on are set up, which is that there are, you know, I have a chapter on perverse incentives, the incentive is not to find the truth necessarily, the incentive is to find something that’s publishable. Find something that we can get into a journal, and a high impact prestigious, glamorous journal at that. And that really goes against all these principles of being rigorous, checking things, being sceptical, and open and transparent. Because you know, why be all these things, when you can hoard your own data and publish lots of good stuff out of it? The average person who reads about science reads about in the news, rather than in scientific journals. It’s been put through the…

Elizabeth
Interesting filter

Stuart
Yeah, exactly, exactly.

Elizabeth
Can you give me example of where you either have done one of those four, or you felt the strong temptation to? Because presumably, it’s the kind of thing that once you’re in the machine, these temptations are all over the place? 

Stuart
Where you’re most likely to be negligent, is not necessarily with your own work, though, it’s as a peer reviewer. And this is the scary thing, because peer review is meant to be the quality filter. And I’ve definitely found myself when peer reviewing people’s work, found myself looking at my watch and thinking, Oh, God…

Elizabeth
Skim skim skim

Stuart
You know, I have other things to do, I need to read them. And you know, you submit your review, and then, and then I do try and be rigorous in peer reviews, and try to spend a lot of time on them. But everyone’s got their own stuff on and you know, you don’t get paid to do peer reviews, or any other incentive, really, other than just the goodness of your heart to help the scientific process. And you do sometimes feel afterwards, you know, I could have just taken a few more hours to dig into this, I could have asked the journal to get the author to send me their data set and really dig into it. But who has that time? 

Elizabeth
I don’t know much about the history and philosophy of science. But I gather that lots of the origins of the scientific method were related to a particular kind of puritan Christian anthropology about sin about – we are sinful and self–deceitful. And so we put these guards in the scientific method to help us get closer to the truth because we can’t trust the human heart, which is a particularly kind of bleak version of a Christian anthropology. And lots of what you’re saying strikes me as kind of – they’re character, they’re character questions, you know, virtue questions, trying to reach for less old fashioned sounding language, but I can’t find it. And I know because I’ve read it that your answer is a very good one about the scientific method can be turned on itself to fix some of these problems, you know, better incentives and better safeguards. But is there something about how, how do we want to be better? How do we seek these very intangible things like truth, humility, you know, self–sacrifice of our own good for the good of the wider community? Or the good of truth? Or the good of society? How can science teach these things? And if not, where do you go looking for them? Where do you think we can build that muscle as people interested in truth? 

Stuart
That’s a very good question. I think it is – There are some you know, all those aspects of being humble and intellectual humility, and all that stuff. It’s never explicitly taught, right? It’s very rarely explicitly taught, you do a statistics class. Only recently if people started you know, bringing in all these kind of open science type ideas and saying, you know, we need to share things with each other and be ready for people to criticise you, and so on, as the kind of open science movement has had more of an effect, and it’s still very much a minority thing, the average person is still just being taught in a very practical and pragmatic way how to do statistics that will help them do their analysis and not thinking about it in a kind of stepping back and thinking about it in a more in a broader sense that the reason that we’re doing statistics in the first place is that we can’t trust our own judgement about how a particular experiment works, or you can’t just ask someone how they feel about something or, or whatever, you want to use some actual attempt or an objective measurement of that. And that’s very much what you were saying about, you know, we’re all fallible in various ways, and the scientific method corrects that. And, you know, my argument in the book is that human fallibility has sort of eaten away at the scientific, not necessarily the method or the process of science, because those are principles which stand aside, but certainly at the institutions of science, where we do this stuff, and I think people are implicitly and sometimes explicitly taught to go against all these intellectual humility positions, and all these things which would make the science more rigorous. They’re taught to tell a nice story with their results, they’re taught to just dig around until you find something, they’re taught to value publishing research in very glamorous high impact journals and go for the exciting gold, you know, results regardless, really, of whether those are, are true or not. You know, you hear people all the time saying, My supervisor pressured me into, you know, running an analysis that I knew was kind of wrong, but it got the results that he or she wanted. And, and so when you have an institution, often the senior figures in are – they’ve made their career in the old way of doing science, or the flawed way of doing science, they’ve become really successful and got, you know, very long CVs, have lots of nice papers and stuff based on, you know, often very, very, you know, practices that are no good for science. So how do you, how do you push back against that? And I think, you know, we have this kind of bottom up and top down set of incentives that are changing, people are talking about this stuff in science much more, people are being educated as they start their PhD. But we need to also do something about the more senior academics who have kind of failed to learn these skills or have gotten themselves into a position where they no longer really need to think about 

Elizabeth
Is there – because when you study journalism there is an ethics module, is there a sort of standard ethics module in undergraduate science? 

Stuart
Well, in psychology, the ethics tends to be much more focused on…

Elizabeth
Your participants

Stuart
Yeah, exactly. Don’t exploit them don’t make them feel worse when they leave. And when they come in, you know

Elizabeth
Don’t traumatise them

Stuart
Don’t give them an enormous electric shock, or whatever, whatever it is, like, like, it’s very much, it’s very much focused on that. But I think you could all be like, it’s very obvious that you could make the argument that wasting the taxpayers money on doing crap experiments that don’t replicate is unethical. Getting participants to come in to your study, and making them spend their time on something which is just going to be a load of nonsense is unethical.

Elizabeth
Just for your own status

Stuart
Right? And certainly running randomised controlled trials where you’re giving people drugs, and then, you know, if that trial was set up in such a way that it could never actually produce any useful information. You know, in any case, that’s unethical. Surely. So I think this is much, this could be a much bigger part of, you know, just as scientists training, lots of universities do these kind of doctoral training programmes now when going to do your PhD, you don’t just get taught by your supervisor, you get taught by the University in general, there’s a bit more kind of continuing professional development, you know, wherever you want to call it. And I think you know, ethics, as you say, alongside all these things, which clearly merge into each other – open science and the ethics of doing things all kind of overlap in various ways. That could be taught more explicitly. It’s just that we still have an incentive structure and still have an old guard of scientists who are kind of pushing back against it. 

Elizabeth
Well, maybe roping in the theology and philosophy departments to help would be a good way forward. Stuart Ritchie, thank you so much for talking to me on The Sacred. 

Stuart
Great pleasure. Thank you. 

Elizabeth
Wow, what a lovely guy. I think Stuart Ritchie might be the least tortured and conflicted person I have ever interviewed. And he still seemed really a lot like the teenager that he said he used to be – confident, energetic, and calm. And it was really interesting that he clearly enjoys debates and controversies. As a teenager, I can just imagine him – the debating society kind of ringleader wanting to debate creationists or to take someone’s argument apart and feeling funny, real glee in those debates, and that he’s clearly quite immune to the emotional temperature of controversies. In fact, maybe he’s drawn to the emotional temperature of those controversies. And he gave some really good arguments for why in terms of sharpening of data and arguments and positions, which was really convincing, but it made me think again, about the way our temperaments and our personalities self–select maybe who’s in those conversations or who can tolerate being in those conversations. Long term, it reminded me of a quote from Haruki Murakami, who said ‘always remember that to argue and win is to break down the reality of the person you are arguing against. It is painful to lose your reality. So be kind, even if you’re right.’

I loved Stuart comparing himself to Boris Johnson. And actually, I could see it the sort of smiling half joke, and I’m reminded how effective it is with Stuart and with Boris Johnson – it’s one of the reasons lots of people love him, that there’s always a twinkle and I imagine that it’s really effective, even when you’re talking about quite serious things, to help people control their own emotional reaction to things. And he was really honest about the fact that there are just people he doesn’t respect. So he doesn’t care if they criticise him. I think that’s probably true of a lot of people and it was quite refreshing to hear it is not just that he doesn’t care if people he doesn’t respect criticise him, but actually in some ways that it maybe builds his social status if certain groups of people criticise him, it is a boon for his standing amongst his friends. And that’s made me notice that dynamic playing out a lot actually in our public conversations. And some less scrupulous people are clearly stoking the critique of outgroups or tribes that they don’t respect as a way of playing to the gallery of their own team. 

We really got to the heart of the matter, I think, in terms of the reliability of science. Because of, and this is, from my perspective, a real sense of a lack of moral formation for scientists, you know, Stuart’s book really argues that they’re just as susceptible to exaggeration and self–promotion and outright fraud as the rest of us. But we’re basing so many decisions and stories and worldviews on their findings. And yes, the scientific method itself is one kind of tool in that battle. But I certainly would come from the perspective that the human heart is the heart of the matter, and how you form it and shape our desires and our courage and our virtue. 

One thing I didn’t get to ask him which I really wish I had was when your sacred value is skepticism, how do you prevent scepticism becoming the universal acid that burns through everything? If the kind of negating, critiquing posture is necessary, which he persuaded me that it is, how do you build what is the positive? Who do you trust? Maybe that’s about community, different people having different callings. Anyway, those are some of my thoughts. Thanks for listening.

Interested in this? Share it on social media. Join our monthly e–newsletter to keep up to date with our latest research and events. And check out our Supporter Programme to find out how you can help our work.