Plain English with Derek Thompson: How the Media Failed Its COVID Test: The Truth Behind the Lab Leak and Masking Debates

Did your favorite NFL team win the Super Bowl?

No, then the NFL Draft is your Super Bowl.

I'm Danny Heifetz and for now until the draft,

we are turning our fantasy football show

feed into the Ringer NFL Draft Show.

Every Tuesday and Thursday, we talk about

the top players and most important

storylines for the NFL Draft.

So join us on the Ringer NFL Draft Show.

Today's episode is a big one.

It's about the debate over media coverage of COVID.

Three years after a fateful March of 2020

when it felt like the whole world was shutting down,

we are revisiting two of the most

contentious debates in this space.

Number one, the lab leak hypothesis,

which is the debate over the possibility

that COVID originated at a laboratory in China

and not as the official story went

at a wet market in Wuhan.

And number two, the mask debate,

which touches on a seemingly simple question,

do masks work, which turns out to be

very, very, very hard to answer.

So why these topics and why now?

The answer is that news in these domains

simply won't stop breaking.

Last month, the Department of Energy

revised its prior assessment and announced

that the coronavirus likely did emerge from a laboratory.

The FBI shares that assessment for other agencies

and the National Intelligence Council

have come to the opposite conclusion,

that COVID likely started with natural exposure

to an infected animal at perhaps a wet market.

So if you're doing the lab leak math at home,

the lab leak theory itself is still an underdog

trailing five to two among these government institutions,

none of which, by the way, have reached their conclusions

with a high degree of confidence.

The lab leak is interesting to me for two reasons.

First, it's a pretty important question.

How did a pandemic that has killed millions and millions

and millions of people actually start?

That's a biggie.

Not just because we don't have perfect information,

or just because we don't have perfect information,

doesn't mean we shouldn't be curious about this.

Remember, as a polysine major at Northwestern,

I think I took four separate classes about why World War

I started, and the answer in every single class

was like, no one's really sure, but there's

a lot of interesting theories that

have informed political science.

Well, if we take the lab leak theory seriously,

as I think we should, it should make

us deeply skeptical about many policies

that are active in the world, like funding

gain-of-function research, sort of viral engineering,

that should it escape the lab, could

cause precisely the global catastrophe

that we saw, or perhaps something even worse.

The second reason the lab leak theory is interesting to me

is that this is really, in many ways,

a story about the media.

How I, we, the press, choose what to cover,

and choose who to listen to.

How we choose which stories are considered information,

and which stories are considered misinformation,

or disinformation.

For many months, especially in 2020 and early 2021,

a lot of journalists, smart journalists,

trying, I think in many cases, to do the right thing,

kind of assumed that the lab leak was a racist conspiracy.

In fact, many prominent journalists and outlets

simply said, scientists don't take this seriously,

neither should you.

But now, in 2023, I think if we're being honest,

and if we're really interested in the truth,

the idea that the lab leak was merely

discredited racist conspiracy theorizing,

that itself was a kind of misinformation.

It was a story that kept readers and audiences

from appreciating the actual uncertainty

of this important question.

There's another story that deserves a reappraisal,

and that is the media's treatment of masks.

I think it's fair to say the media and the science community

have been all over the place on this one.

In March, 2020, the Fauci famously

told us we didn't have to wear masks.

And then months later, of course, wearing a mask was a marker,

maybe the marker of how seriously you took COVID.

And some scientists very quickly seemed

to revise the estimation of the essential effectiveness

of masks.

But then a few weeks ago, in February, 2023,

a meta-analysis of masking research

published by the esteemed health organization Cochrane

was widely reported in the media

as proving that actually masks do nothing or close to nothing,

and mask mandates do not work, period, end of story.

But as you're about to hear, even that review, even that

summary of the evidence, is extremely misleading.

So what are you supposed to do about all this?

The lab leak is neither a fact nor a myth.

Masks work, except very often they don't,

and asking people to wear masks can work,

except very often it doesn't work at all.

You try to keep all this in your head, and it's a mess.

It is a mess.

Journalists sometimes like to clean up a story,

make for a simple headline.

I think we need to be better at reporting on uncertainty,

especially when there is a political or ideological

undertow that is pulling us to one side of that story.

When we see a mess, we have to call it a mess.

So don't trust people who, in their handling

of complex questions with imperfect data,

do this trick where they manufacture simplistic answers

with perfect confidence.

Trust people who get in the weeds.

Trust people who see the mess for what it is.

Trust people who change their mind

when the evidence changes.

Today's guests are Dan Engber, science writer and editor

at The Atlantic, who has chronicled the ups and downs

of the media's relationship to the lab leak,

and Jason Abeluck, a Yale economist who

has conducted some of the more famous trials of masking,

and who, as you're about to hear,

is actually not only objected to the Cochrane meta-analysis,

the famous Cochrane review, but has actually

talked to a member of the Cochrane team

and just maybe convinced him that he's right.

I'm Derek Thompson.

This is Plain English.

Dan Engber, welcome to the podcast.

Thanks for having me.

Before we start, a brief story.

I don't know if you remember this,

but I think you edited the first article that I ever

wrote as a professional journalist.

No, I'm sorry.

You were a slate.

I was an internet slate.

I was interested in writing an explainer

about how the government knows how many miles we drive

every year.

Is this an explainer that you have any memory of having

edited?

I mean, Derek, I edited or written hundreds, possibly

thousands of explainer columns.

No impression.

It's a good topic.

Congratulations to us both on producing that story.

Does the story hold up?

I believe it does hold up.

Yeah, and in a weird way, it kind of

connects to what we're going to talk about today,

because it's fundamentally about the question of epistemology.

How do we know what we know of whether it's miles

that Americans drive on roads or exactly where

a novel coronavirus originated?

So I've been interested in this question for a while.

And you, I think, have been one of the most careful

journalistic voices on piecing through the evidence

on both sides of this question and being really clear

about what level of certainty we should

have about answering this question.

But I want to start with 2020.

When did you first start following the lab leak theory?

When do you remember hearing about it,

having any emotional reaction to it?

So I remember hearing about the work that had been done

at the Wuhan Institute of Virology

before I heard about the idea of the lab leak hypothesis,

actually.

And so I think as most people would,

when they hear that in 2020, they go, that's weird.

Wait a second, hold on.

How likely is it that this kind of research would

be happening in Wuhan, China, of all places,

and now we've got this pandemic unfolding?

So actually, the blatant coincidence

hit me first before I was aware that there was this shadow

discourse happening about how likely it was.

And I will say, I was totally tuned out on the politics stuff.

I wasn't aware of Tom Cotton wrote an op-ed about saying

it was a Chinese bioweapon.

All of that stuff got folded in for me.

And I think also, we can talk about this for the media,

got folded into this Trump versus the science narrative

about the China flu and how Trump was blaming

all of these failures of his own administration on China.

So I slipped very easily into that story

of this was the thing that was happening

in terms of the politics of it.

And that was disconnected from any underlying scientific truth.

But I still wondered about that coincidence.

And I thought, hey, that's really odd.

And then I think I kind of didn't dig too deeply into it

until the Nicholson Baker story came out in New York Magazine.

And I remember feeling relief that someone had done that story

and gone big on it.

And then, of course, that set off a bunch of angry reactions

and we went from there.

But that was the very end of December 2020,

if I remember correctly.

And how would you characterize the media's reaction

to the lab leak theory in 2020 and early 2021,

say just before the Baker piece comes out in New York Magazine?

And to my recollection, really crystallizes this sense

that despite the fact that in the mainstream media,

there hasn't been much talk about taking the lab leak theory

seriously, introduces this idea that actually there's

been a kind of shadow discourse happening, where people have

been poking around and asking, can we

find smoking gun evidence that this came from the Wuhan,

from WIV?

Yeah, I mean, my recollection as an editor, editing stories

about the pandemic then, was that it was just incorrect.

We just knew it was incorrect.

There was almost like a copy-paste macro

you could put into a story if this were an issue.

Scientists say this is not the case.

And particularly what I think was missing there,

and I take responsibility for this as an editor editing COVID

stories at the time, was deep thought

about the different shades of what lab leak theory or hypothesis

could mean.

So again, this was all lumped together into the most extreme

version of it.

That was easy to dismiss.

That had been dismissed in prominent venues

by leading scientists.

And that would be the Chinese bio-weapon theory of this.

So once that was all swirling together in your head,

and the politics of it made it very easy for that

to be one's notion of what lab leak meant,

it just was like you just knew that was just a false narrative,

one of many false narratives that were swirling around

at that time.

And so it was just not something to cover.

The Baker piece really goes into much more nuance about what

kind of lab accident might have been in play,

what was the research that was going on,

and even just the history.

That was, I think, if you hadn't been paying attention,

you didn't have this in your head about lab accidents

in recent years, or the moratorium on gain of function

research that had been put in place during the Obama

administration.

So I just thought it was incredible

that Baker brought all of that to the fore,

told the story of these arguments about the dangers

of doing this kind of urology work,

and just forced everyone to look at this.

I mean, still, I would say it would be another five months

or so before, really, the mainstream media

was looking closely.

But that was the first one where it just, at least for me,

I was like, OK, I need to actually go beyond my initial

thought of, hey, that's a weird coincidence

to start taking this very seriously.

I'm really glad that you pointed out

that the media's reaction to the lab leak early on

was a kind of mess of conflation.

There's all these things that you

have to sort of keep in the air,

that Trump was explicitly anti-China in a way

that many liberals found to be racist, number one.

Number two, that many Republicans

were getting over their skis, suggesting that COVID was

a bio-weapon, and that it also was a bio-weapon that

emerged from a lab.

So right there, you have the conflation of lab leak

equals bio-weapon, plus lab leak equals normal virus,

or not engineered virus that comes out of a lab.

And I think that there was a liberal or mainstream media

leaning liberal eagerness to disprove the lab leak hypothesis

that was basically just displaced eagerness

to reject Chinese racism and bio-weapon rumors.

But this created a really weird discourse space.

I remember in the fall of 2020, I

was having a conversation with my wife and her friends.

I remember we were in the bathroom.

Her friends were on speakerphone.

And we were just having a conversation

at the end of some Friday, where my wife and I

had made dinner together.

And they said, we were talking about conspiracy theories.

They said, Derek, what conspiracy theory do you believe?

And I'm not a conspiracy theorist.

Like, I just don't dabble in them generally.

All conspiracy theorists say that, by the way.

But go on.

OK.

That's going to set up my next thing I'm going to say very well.

I said, I have a lot of time for the theory

that this virus came from a lab.

And the reaction was like, wait, we know you're not racist.

But that theory is kind of racist.

And my feeling was, you already articulated this.

Well, look, it's not racist to say

that a good candidate for the emergence of a bat

coronavirus is a local laboratory that

studies bat coronaviruses, right?

I'm not saying here's the truth and the doubters

are a bunch of idiots.

I'm saying we have a crime.

And this is a reasonable murder suspect

that we should consider in the investigation.

But it's really interesting to think back to that period

and remember just how strange it was to take the theory

seriously.

So let's continue the TikTok.

The Baker article comes out in New York Magazine.

It inspires a pretty fierce backlash among some people.

But in the months that follow, this approach of,

I want to describe this carefully,

taking the lab leak seriously without saying,

I believe it to be true in any kind of probabilistic, more

than 70% kind of way, that became more

common among certain journalists.

Would you agree?

I would.

I mean, I wonder if you wouldn't mind doubling back

for just a minute, because actually, in this last piece

I wrote about the lab leak, this little behind the scenes

thing, but I wanted to represent it

as having been so politicized from the start

that it was Democrats and Republicans

disagreeing fundamentally on this question from the get-go.

And then I went back to find, was there

a speech that Chuck Schumer gave in 2020

where he was talking about this?

And the answer was no.

This was something that Republican lawmakers

were talking about a lot, Republican and conservative

columnists and such.

But it's not something that Democratic lawmakers

were talking about.

It was sort of like not Republicans versus Democrats,

but Republicans versus the media, basically.

And then having been in the media at the time,

we were taking our cues from the scientists.

So I edit science stories, work with science journalists

who talk to scientists.

And so I think what was happening

was you were getting a lot of news stories

that reflected the quote unquote scientific consensus

as probed by journalists calling a bunch of people

who are scientists who are prominent on Twitter, maybe.

But in any case, and they were being told,

there's no doubts here.

We know where it came from.

And they were reporting that accurately.

And then it was the segments of society

that are distrustful of elite authority,

including scientists who are saying,

we don't know if we believe that.

So I felt like that was, I don't know if this is worth

bringing up, but I actually, I thought

it was interesting that in writing my last piece about this,

I realized I had distorted my own memory of what

had happened was distorted.

I thought it had been overtly politicized from the get-go.

I think, in a sense, it was.

But not in this explicit left-right way.

It was like this elite's right way, kind of,

which is, I think, a little bit different.

And I think important as it plays out now.

But we can go back.

I'm sorry to divert.

No, that's really interesting.

Let me just dig into that a little bit further.

What I said earlier is that a lot of left-leaning people,

left-of-center people in the media,

might have conflated their insistence

on rejecting Trump racism and downplaying the odds

that this was a bio weapon.

They conflated that with discrediting a lab leak entirely.

You're saying something that, to my ear,

sounds a little bit different, which

is that it's not that the media was necessarily or exclusively

biased against Trump or insistent,

upon disproving the bio weapon theory.

It's also that the media was taking their cues

from scientists who were prominent for whatever reason.

Maybe they had the most Twitter followers.

Maybe they were the co-authors of whatever

was just published in the Lancet the previous month.

But they were taking their cues from prominent scientists

who were insisting that the lab leak was

an improbable, if not impossible, theory

to explain the pandemic.

Is that right?

Yeah, I think that's right.

I agree with both parts of it.

I mean, I think this was over-determined,

and that's why you got this effective media

blackout on the idea.

You had both the political inclinations

were lined up with what scientists from Fauci on down

were saying, and so easy to make the judgment then,

if you're not being careful.

Have those scientists recanted in any way?

I know that Fauci in the last few months

has said at least something along the lines

of we don't know for sure whether this came from a lab.

We can't prove for sure if that it didn't come from a lab.

He seems to have walked back his 2020 position a bit.

Is it your sense that those scientists now

talk about the question of the lab leak in a slightly different

way than they did in 2020 and 21?

Yeah, I mean, I think for sure, Fauci's a great example.

But other scientists, too, there's

been a lot of cases of some new piece of information.

I remember when it came out that some of the key research

that was being done at WIV was being done at biosafety level

two instead of three or four.

That led to some prominent scientists saying,

hey, wait a second here.

If I'd known about that or when the DARPA grant proposal came

out, this was a grant proposal that involved WIV that

was outlined to experiments that sound, in retrospect,

pretty scary.

Now, there's no indication that these experiments were actually

done to be clear.

But just to have this stuff on paper in official grant

documents, I mean, I interviewed scientists

who were like, now I'm beginning to say, at the very least,

there's such a lack of transparency here.

Why wouldn't this information have been put out there

from the beginning?

It's so clearly relevant to the question of pandemic origins.

So I just, in the course of reporting on this since whatever,

at the beginning of 2021, I've heard scientists describe one

said to me, the delta is shrinking.

I remember that phrase, meaning his assessed probability

of a natural origin versus a laboratory origin,

he still felt it was more likely a natural origin,

but the delta is shrinking.

So I think about that as the overall trend

since the beginning of 2021.

What I love about your coverage at the space

is that I think you're very good at pointing out

how amazingly coincidental both theories are,

both the lab leak theory and the natural origin theory.

And I want to just quote from your last article

in the Atlantic, because I think it sums up something

very important very succinctly.

Quote, if COVID really started in the lab when

position holds, then it would have

to be a pretty amazing coincidence

that so many of the earliest infections happened to emerge

in and around a venue for the sale of live animals, which

just happened to be the exact same sort of place where

the first SARS coronavirus pandemic might have started

20 years ago.

But also, if COVID really started in a live animal market,

it would have to be a similarly amazing coincidence

the market in question happened to be across the river

from the laboratory of the world's leading bat coronavirus

researcher, which happened to be running experiments that

could, in theory, make coronaviruses more dangerous.

Dan, we are in the realm of crazy and likely thing one

versus crazy and likely thing two,

both of which require an extraordinarily unlikely role

of the dice for them to cash out in a global pandemic.

Right, and we know for a fact that one of those two things

is really just a coincidence.

Do we know for sure that it can either

be only natural origin in Wuhan or a lab leak at WIV?

Is it conceivable that the actual origin of the pandemic

is some category three phenomenon?

Frozen fish imports into China.

I mean, sure, it's conceivable.

But I mean, I'm stuck on those two coincidences personally.

So I think it's got to be related to one of those two things.

I would also say just while we're

stating where we stand on these things,

I've said in the past the fact that there

are these two enormous coincidences that

have to be reckoned with doesn't mean that it's necessarily

the evidence is a toss-up, it's a coin flip.

There are many other things to take into account.

And probably the most significant for me

would be what's the history of pandemics?

I mean, basically every single one

has started with a so-called natural origin, with one

possible known exception.

So I mean, if you're using that to decide to judge the tie,

tie goes to natural origin, right?

And I don't even know if the evidence really

is quote unquote tied otherwise.

But that's the backdrop here, and I

think that has also contributed to the attitudes in 2020

and attitudes today amongst scientists, right?

We know this story.

It's a story we've heard before, and the evidence

is consistent with that story, so why not?

Aside from the fact that, as you said,

almost every pandemic for which we

have a high degree of confidence in the origin,

the story is always animal to human spillover.

Other than that, what do you consider

the most persuasive facts or maybe even most interesting

rumor that should make a reasonable person lean

toward this pandemic started from natural spillover

from a wild animal?

OK, aside from that fact, which I do think

is a very significant fact, right?

Again, it's going back to that coincidence, this market.

Why should, everyone agrees, even people

who think it started in a lab, everyone

agrees that this market was one of the first, if not

the first major cluster of infections, right?

So that's fine.

Maybe a scientist left work infected with the coronavirus

and then went to pick up some groceries at the market.

You can tell that story.

But again, this is a market where

we have photos of these wild animals, live wild animals

for sale from 2019 from a paper that

was published much subsequent to the start of the pandemic.

There are many other places in the city

where that first big outbreak could have happened.

We don't think about in the, since then,

we haven't thought in the US about market, big markets

as being a place, as the place where super spreader events

happen, we think about it as like conference centers

or concert venues or restaurants or bars or airplanes

or whatever.

Those are the stories we keep hearing.

So why was it a market selling live animals in Wuhan?

I find that the most compelling part of this, aside from,

again, the long history.

And on the other hand, what would you

say is the single most compelling fact or most

interesting rumor that might make a reasonable person,

let me be careful here, either lean toward the lab leak

theory or dramatically increase in their own minds

the odds that this was a lab leak?

I mentioned the DARPA proposal before.

I don't know how useful that is in particular,

because again, we don't know that those experiments were

ever carried out.

But I do think as an elaboration on the mere fact

of there being research on bat coronaviruses in Wuhan,

1,000 miles away from caves where closely related

viruses were found.

And we know that samples were carried from those caves

back to Wuhan.

So we have all that stuff.

We also have all this evidence that experiments were

being done at that institution to tinker with the viruses

and see what happens.

What if you combine this bat coronaviruses with that one

or with some element of the MERS virus?

So again, we don't have any evidence

that such experiments were done using viruses that could

conceivably be a precursor to SARS-CoV-2,

the one that caused the pandemic.

So there's no direct evidence that research

was being done of the type that could lead to the pandemic.

But there's lots of evidence that research

that's kind of in the realm of mixing and matching viruses

was being done right there.

So I find that just in, again, going

past the initial giant coincidence thing,

that extension of it, I find worrisome to say the least.

I really like the way that you set up those coincidences.

And it makes me think, John Stewart

had a very famous segment on The Late Show with Stephen Colbert,

where he put forth his theory for why he thinks

the lab leak idea is probable.

And he said, I'm butchering it, but I

think this is an OK summary.

If you hear that there's an outbreak of chocolatey,

gooey goodness from Hershey, Pennsylvania,

you have to be an idiot to not rule in the fact

that it came from the big chocolate factory at Hershey.

But what you're saying is actually takes that joke metaphor

and usefully edits it.

It's more like we have an outbreak of gooey chocolatey

goodness from the nation of Switzerland.

And there's many possible points of origin,

whether it's Lint or Nestle, Toblerone.

There's a bunch of different chocolate factories

that would be absolutely ideal candidates

for the origin of an outbreak of chocolatey gooeyness.

What you're saying is, it seems to me,

Wuhan has these two points of origin

that one might reasonably rule in.

A research facility that researched

exactly these kind of viruses and a wet lab, a wet market

that we know from the history of pandemics

is a plausible place of origin for viruses

that spill over from a bat to a pangolin to a human.

So actually, I hadn't thought of it in exactly these terms,

but it's more like the chocolatey outbreak

came from Switzerland than the chocolatey outbreak

came from Hershey, Pennsylvania.

Yeah, I don't think that got distracted

by all the different chocolate manufacturers you mentioned.

But just to complicate it a little bit, two things.

First of all, it's not like there are 100 wet markets

like the Huanan market in Wuhan.

There is a small handful that were

known to be trafficking in these sorts of animals.

So I think that's important too, just to kind of build out

in your mind an understanding of what kind of coincidence

this would be if it didn't actually start there.

Similarly, the Wuhan Institute of Virology

is not the only place in the city

where potentially dangerous research might

have been carried out.

In fact, though the details are mysterious and classified,

it has been said or has been reported

that when the Department of Energy

updated its assessment of the source of the pandemic

and said with low confidence that they think it's a lab leak,

that had to do with some idea that it took place

at a different lab, a branch of the Chinese CDC, I think,

which is rather close to the market, I'm told.

So this is sort of like a quality of this whole conversation

is that as soon as you can sort of say, oh, well,

there are these two big coincidences.

And then everything gets parsed a step further.

And now it's complicated.

Now maybe it's not this institution, research institution.

It's this one, which is not really across the river

and so on and so forth.

I think that the overall structure of the debate

is still the same, two big coincidences.

But I'm just bringing this up.

I don't know how to work this into your chocolate analogy.

But I do think it starts to get pretty complicated pretty

quickly when you start adding in these other pieces

of information.

Yeah.

I'm going to stick with the Switzerland metaphor,

because what you're basically saying

is there's a bunch of chocolate manufacturers

in the analogy.

There's a bunch of different plausible origins

for this virus in the same city.

And that just makes it difficult, especially

in the absence of anything approaching

a agreeable and open Chinese government to arrive

at a final truth.

I want to bring in that as the last piece of evidence.

Do you consider the reluctance of the Chinese government

to play ball a useful piece of evidence

for either side of this debate?

I don't.

I don't.

And I will say that from the beginning,

it has seemed to me not great from China's perspective

for the answer to come out in either direction.

I mean, as we said before, if SARS started 20,

SARS-1 started 20 years ago, possibly through a wet market

like this, and now it's happened again,

and this time it's killed millions and millions of people,

like why the hell was this allowed?

Who's responsible for that?

So it's not like the natural origin

has the Chinese government off the hook for this.

So it seems to me that leaving it undetermined

is probably the best possible outcome,

as far as China is concerned.

So that has always been my assumption for what's going on

here, that there's really no interest in resolving it

one way or another.

It doesn't look good either way.

So why do it?

Why let the studies happen?

Put on your media professor hat.

You're teaching a class at wherever, Columbia,

Medell, about the lessons that responsible young journalists

should take from the lab leak frock us.

What's the lesson?

That is a fantastic question.

And I think it goes back to there are some easy lessons

I could, I'm tempted to draw.

But I really want to grapple with the fact that, again,

you're a science journalist in summer 2020.

You want to know what's the thinking on this.

And every scientist you call tells you the same thing.

How are you supposed to arrive at the,

do you then call Tom Cotton?

Like, it just doesn't make sense.

You want to report the science of this.

It's really tricky.

And I would say because of that I'm, and this may be self,

self-interest here, but I'm not,

I have trouble blaming science journalists at least

as much as maybe I ought to.

It just, it's tough.

Like there's a lot of interesting work on this was done

by this ragtag group of independent researchers

communicating on Twitter through anonymous handles.

It's like, as it putting on my, my journalism professor hat,

am I going to tell students to reach out to, you know,

lab leak seeker 10 on Twitter?

Like, you don't know who the person is.

You're going to quote them.

Like, it's, it's tricky.

I mean, I think the, the, the lesson here as far as I'm

concerned is more about what we do next,

which is continuing to cover this doing accountability

journalism about the lack of transparency from the start,

about the lack of transparency that still exists,

not entirely due to, you know,

a Chinese coverup of some kind,

but also due to the lack of transparency

in terms of what was happening here.

I mean, I think we're going to find out in a couple of days

when these hearings start in the house.

There's probably going to be a lot of crazy shit thrown

around at those hearings,

but I think there are substantive questions about

what ideas were taken seriously behind closed doors

and how those conversations related or didn't relate

to what was said in public by, you know,

science officials within the administration

and by scientists in the know.

That's a really interesting lesson.

One thing that I took from it is that

it's hard as a journalist to keep in mind

two things, one, your sources are smarter than you,

they know more than you, you have to rely on them,

but also they're not smarter than God.

People with PhDs and fancy resumes are just people

and people are wrong all the time

and consensus is within industries or disciplines

can be wrong all the time

and you can be captured by sources.

The same way we're familiar with the fact

that some writers are captured by their audience,

their audience expects them to be, I don't know,

anti-trans and so all they do is write just bullshit

about being anti-trans.

I'm reminded of two different events

that I'm not trying to directly analogize

to the question of the lab leak,

but there are a lot of reporters around 2003

who I think were captured by their sources

around the issue of the Iraq war

and they reported that WMDs existed

because there were a lot of people

in the Bush administration around the Bush administration

who were going to tell them that

if that's what your roster of sources was,

then you were just going to hear one message

over and over and over again.

You could be captured by those sources

to report that which was not true.

To a certain extent in my own,

the closer to home and my own sort of domain

in economic analysis,

it was something close to an article of faith

in mid to late 2021

that inflation was just not a serious problem,

that it was gonna go away very quickly,

that it wasn't gonna pose much of a problem to people,

that it wasn't gonna get that high,

that there were just a few blips

in terms of supply chains,

but everything was gonna come down

to normal relatively quickly.

And the next year, 2022,

inflation and rising interest rates

were probably the most important phenomenon in economics.

And you could create a roster,

a bank of really, really smart academic voices

who for whatever reason,

their rooting interests in Joe Biden,

the fact they were a little bit more left leaning,

they were a little bit more captured by MMT,

just did not believe that the US was anywhere near

an inflation crisis.

And I think it's really hard and important to remember

that even the people that you talk to as sources,

even they feel kind of like little gods

when you're reaching out to them

to dig yourself out of ignorance for peace,

they can be catastrophically wrong as well.

Might just be an important piece of intellectual humility

to keep on your shoulder.

I think that's exactly right.

I mean, I think there's even a specific way

which this relates to science journalism,

so that if I were putting on my science journalism

professor hat,

which is a little different from just journalism,

professor in general.

There is this way in which there is kind of

an ideology among science journalists

that they hold in alignment with many of their sources

that you should follow the science.

There's an elision between small S and big S science, right?

And this gets really snagged on political divides as well,

especially in recent years.

But even leaving the politics out of it,

there's this, the difference between science communication

and science reporting can kind of fade.

And one can get confused about,

am I, is what I'm doing good for science

as opposed to getting at the truth.

One can forget that science, capital S science,

which your sources are almost always representing,

is an institution of its own accord

that needs to be held to account,

that has its own motivations and its own biases.

So I think, again,

putting all the politics aside, putting China out of it,

there was in the background,

this sense that the idea that scientists

cause the pandemic is bad for science.

That's like, it's in the same world

as anti-vaccine sentiments.

It's like fanning the flames of conspiracy theorists

that don't want us to listen to science.

And so as a science journalist,

I think a lot of science journalists feel like

it's part of the mission of their job to promote

rational thinking and rational thinking,

in this case, is aligned with what the scientists are saying

and what's in the best interests of further study.

And like, I think in 2020,

probably there was an idea of,

not science got us into this mess,

but we need to be as science-y as possible

to get us out of this mess.

Like, we need those vaccines.

We need to figure out as soon as this is over.

We need to use science to figure out

how to prevent the next one.

So the idea that, oh, the scientists

who are trying to prevent pandemics,

because that's what the research we're talking about

was about, they actually cause the problem.

I think it's just sort of on this deep level,

politics aside, when against the core beliefs

and inclinations of most of the journalists

who are covering that question.

So totally agree with you.

Breaking out of that mindset is crucial

and not just in covering pandemic origins,

but in doing any science coverage,

you need to understand that it's not a big business,

but it's kind of like a big business in a lot of ways.

Science is hard.

It's the simplistic, but unfortunately,

entirely true conclusion.

You cannot just be so much of a contrarian

that you become a crank,

but you also just cannot simply believe someone

by virtue of the fact that they have credentials

with more initials than you have.

It's a very, very difficult thing to get right.

The truth is changing constantly

and that at the end of the day is what this episode is about.

Dan Engber, thank you so much.

Thanks a lot for having me, Derek.

That was Daniel Engber, writer and editor at The Atlantic.

And next up, we have Jason Abeluck of Yale University.

Jason Abeluck, welcome to the podcast.

Great to be here.

Jason, before we talk about masks,

tell me a little bit about what it is

that you study at Yale.

Sure, so I'm a professor of economics

at the Yale School of Management.

Most of my work is in the areas of health economics

and public policy.

So I do a lot of work about different ways

that we can evaluate the quality of people's choices

and how we can design institutions

like health insurance markets, for example,

in light of the fact that people often might not be

well-informed about what impact choosing

different health insurance plans

would have on their health outcomes.

So questions of that nature,

when people sort of make mistakes,

how can we design institutions that still work well

in light of mistakes?

And what is your recollection

of the conventional wisdom in 2020 around masks?

Because to me, when I think back,

and I do not pretend to have perfect recollection

about that incredibly chaotic year,

this is a really chaotic question to answer.

My recollection is that the WHO and Fauci came out

initially in the middle of March and said,

masks, not so sure that they work.

And then for some reason within three to six months,

the fact that masks did work became an article of faith

among people that share my general ideology

and proclivities.

And then later you had this backlash among people,

some of whom are firmly in the science community,

who said, nope, masks actually don't work at all

and all you people covering your face

are participating in some kind of apocalypse cult.

What is your own memory

of the pendulum swinging on the mask issue

during the pandemic?

So I can tell you a few stories

about what I was involved in in 2020,

but I think the way you described it sounds about right

that the pendulum swung back and forth many times,

but basically my introduction to this,

I think in like March of 2020,

when there were maybe a few dozen cases of COVID

in the United States,

one of my colleagues came up to me and was like,

Jason, there's this debate on Twitter

about whether people should start wearing masks about COVID.

Like, what do you think about this?

And my initial response was something like,

well, we probably should,

like in the sense that it's a super low cost thing.

It's like at the time,

I didn't realize COVID would be endemic

and this would go on for years.

At the time there was some possibility that it's like,

oh, there's just going to be this wave of cases

over the next six months.

And maybe we should wear masks to do something about this.

And the idea was, well, it's super low cost.

It might be effective.

Why not?

So then I sort of started to look into it a little bit

in March and April of 2020.

And I looked at some of the existing evidence,

which I believe actually included an earlier version

of the Cochrane report, the Cochrane review,

which was an article, basically a summary article written,

and there were several summary articles.

And I might, by the way, be misremembering

whether it was a Cochrane review or other similar reviews.

But basically the upshot of those reviews was typically

that like there's been a bunch of studies of masking

in hospitals and in what's called community semics.

And in hospitals, they would do things like,

oh, let's randomize people to surgical or cloth masks,

or let's randomize them to N95 or surgical masks.

And typically they would find, yes,

when we gave people the higher quality masks,

they were more protective against, in those cases,

like influenza.

Then there were community studies.

And the consensus in most of these articles at the time

was something like, well, masks worn by people

in hospitals work and community mask wearing doesn't work.

And that's kind of a weird thing.

And it's like, what does it mean to even say

like community mask wearing doesn't work?

Like what were they basing that on?

And what they were typically basing that on

were studies that would do things like,

oh, you know, we went to a college campus

and we wanted to see if masks were protective

against influenza.

So we did a randomized experiment

where we took people in the intervention group

and we sent them masks and we gave them instructions

about how masks can protect you against influenza.

And we said, hey, can you wear this mask

during the flu season?

And typically they would do those studies

and they would find, hey, there's no difference

between the people we sent masks

and the people we didn't send masks.

Now, when I looked at this initially,

my reaction was like, well, that's kind of a weird study

because they don't tell us how many people actually wore masks.

So given that masks were effective in hospitals,

presumably they'd be effective in the community as well

if people actually wore them.

And in these studies, they just don't wear them.

But of course, now that we have this COVID epidemic

that's gonna kill millions of millions of people,

people will actually wear masks and that would be protective.

So I actually wrote this paper in March and April of 2020

where I was like, look, I don't think that a lot

of these existing studies actually tell us much about masks.

So what we're gonna do is compare countries

with historical mask norms like Japan or Taiwan.

And we're gonna see if COVID is spreading

at a different rate in those countries.

And the answer was, yeah, it seems to be spreading more slowly

in the countries that have historical mask norms.

Although of course, things are difficult.

Why is it difficult?

There's many reasons to have.

One reason it's really difficult is

because those countries might be different

in lots of other respects, right?

So it's like, in Japan, sure they wear masks,

but also maybe they do more testing

and maybe they do more contact tracing

and all these things like,

and maybe they were just more cautious

and all these things could have led to slower spread.

So it's not totally clear if that was a tribunal to masks,

but there was some indirect evidence that it was.

So that was kind of the early 2020 period.

I wrote that paper.

I actually went to the Yale COVID task force.

My colleague and I, and we were like,

hey, why don't we put out an announcement

that suggests in light of this,

that we would recommend that people start wearing masks

as COVID is spreading in the US.

So this was, I believe, in late March of 2020.

And the Yale COVID task force, there were two groups there.

There were epidemiologists who were like,

yeah, this seems like a pretty reasonable idea.

And then there were clinicians who were like,

you economists, this is crazy.

Like the FDA would never approve this,

given the existing evidence.

We can't recommend that people wear masks

if the FDA would never approve it.

And also they were like,

but we know they're important in hospitals.

And if you recommend everyone wear masks,

then we're not going to be able to have masks

for people in hospitals.

So we were like, okay, what about if we recommend

they wear cloth masks so that they still have enough masks

for people in hospitals,

although surgical masks are really cheap to produce.

Anyway, so we got some people to sign on to like a statement

like everyone should wear cloth masks.

That was like April of 2020.

I think by, you know, May or June,

like there was this rapid shift

in sort of the conventional wisdom about this,

where all the public health agencies went from basically

being like, oh, you know, we're skeptical of masks

to basically saying like, oh, actually this is something

like we're going to start recommending everyone wear it.

And that sort of like changed the polarization.

And then for I'd say the next like year and a half

that was the conventional wisdom

among public health agencies,

I still think that is mostly the conventional wisdom

that they've kind of public health agencies.

If you talk to people like the World Health Organization

or whatever, or the Center for Disease Control,

they're still pretty much, yeah,

masks are probably pretty effective against COVID.

Although I agree with you,

there are some people like the authors of the Cochrane Report

who are, you know, epidemiologists and legitimate scientists

who have reached a different conclusion,

although we can talk about, let's talk more about why.

Let's hold the Cochrane Review for a few more minutes.

I want to ask you about this study

that you did in Bangladesh.

Tell me about the study.

Tell me briefly why this study was importantly different

than the community studies that you have just blasted

in the last few minutes.

And what did you find in the Bangladesh study?

Yeah, okay.

So there are a few major differences.

So basically what we did first of all,

is we went to Bangladesh, we went to 600 villages,

and in 300 of those villages,

we did a really intensive campaign

to try to get people to wear masks.

So we sent everyone masks,

we gave them information about masks,

but then we also worked with a bunch of community leaders,

like imams and like village leaders

to try to promote mask wearing.

And maybe most importantly of all,

we had people walking around in every village

in crowded public areas and in masks,

saying, hey, suppose you see someone who's not wearing a mask,

walk up to them and be like,

hey, here's a mask, can you please put this on?

Right, so it's not like they're arresting them

or something, but they're just applying social pressure.

They're being like, hey, can you please put this on?

And if people say no, they say no,

but like there's not that much militant resistance

to mask wearing.

So most people say yes.

Okay, so we did this experiment.

How is this different from what came before?

First way it's different is we actually observed

whether people wore masks in crowded public areas.

So we can see if we changed their behavior.

In our initial pilots,

we didn't change their behavior very much

because we just sent them masks and gave them information.

And mask wearing increased by something

like nine percentage points,

which is actually high relative to other studies

that have done similar stuff.

There was one study in Kenya

that found like a one percentage point increase

from giving people information and handing out masks.

So we felt like a nine percentage point increase.

So then we're like, we need to do more than this.

So then we started doing the people walking around

asking people to wear masks,

that got us to a 30 percentage point increase.

Okay, so that's a pretty appreciable increase.

So what else is different about this study

relative to all the other ones I was talking about,

including my own earlier study?

Well, it was randomized.

So I mentioned before the problem

that it's like Japan, Taiwan, et cetera.

We see lots of mask wearing,

but they might do lots of other things differently.

In these villages in Bangladesh,

the 300 villages where we did this campaign,

those were randomly chosen.

There's one thing that's systematically different

about those villages,

which is we did this campaign

to get a bunch of people to wear masks.

Okay, so then we could see

what's the impact of mask wearing?

And the answer is we saw basically a 10% decline

in COVID symptoms.

And we did blood tests to see if it was actually COVID.

And we also saw a 10% decline

in what's called symptomatic seropositivity.

So people who are symptomatic for COVID

and also their blood was seropositive for COVID.

So what does 10% mean, by the way?

How do we interpret that magnitude?

So we're saying a 30 percentage point increase in mask use

led to a 10% decline in COVID.

So how do you extrapolate that

to what if everybody wore masks, right?

It's not completely obvious,

but a back of the envelope simple thing to do

is just to like assume everything scales.

So if 30% wearing masks gets you 10%,

maybe 100% is about three times,

maybe you get like a 30% decline in COVID infections.

Is a reasonable conclusion to draw from this study?

Because I know that there are some mass skeptics

that read your study and they say,

okay, it took Jason and his team

an unbelievable amount of money.

You guys got millions of dollars from GiveWell.

It took you an unbelievable amount of money,

extraordinary amounts of enforcement,

like annoying amounts of enforcement.

You're there on the streets pointing at people saying,

uh, mask, uh, mask, and all of this,

all of this only increased mask wearing by 20, 30%.

I mean, that's not, that suggests

that the typical mask policy

is not going to be very successful.

So do you, how do you feel about the critique of this study

that says that the effect size is not large enough

to prove to us that the average mask mandate

is going to do anything?

Yeah.

So first of all, it's a really good question.

So one thing we need to keep in mind is,

so one takeaway, you might be like,

oh, well, it's just impossible or just really hard

to get people to wear a mask.

But of course, we know that there are some places

where lots of people wear masks.

So first of all, it's like Japan,

like almost everyone wears masks.

Now you might say, okay, fine.

In countries with historical mask norms, it's possible,

but it's just too hard everywhere else.

But even in the United States,

there were places that had really, really high compliance

with mask use.

There were places where 90% of people

were wearing masks in 2020 and in 2021.

Now, it's important to draw the distinction

between can you change behavior

and how many people do with it?

So what is hard about doing this type of study

that we did and what is a major deficiency

of the earlier studies is getting people

who aren't motivated on their own to wear masks

to change their behavior, that requires doing something.

Now that's something might be that you mandate masks.

You make it a rule with different kinds of enforcement.

Like in the United States,

a lot of states mandated mask wearing, what did that mean?

Well, typically if they saw you in public,

not wearing a mask, it's not like they're arresting you

and dragging you to jail immediately.

It's kind of like this recommended thing

that's variably enforced, right?

Like maybe in the post office, they say,

hey, sir, please put on a mask.

And if you're just adamant that you won't,

then they say, please leave and they'll take you out.

Right?

So those kinds of things do,

if you have a church or a post office

or whatever, a public area

and you ask people to please put on masks,

most people actually comply with that, right?

So what we're trying to do in these studies

is to figure out the answer to two separate,

but both important questions.

One question is just what would happen

if people actually did comply and wear masks?

And then a second question is like,

what kinds of things are actually effective

for getting people to wear masks in practice?

So my answer to the second question first

is if people actually did wear masks,

yeah, you probably get something like what we found

in this study, if in public areas, people wore masks.

Now that doesn't mean you're wearing a 95,

24 hours a day, right?

You go home, you're probably not wearing a mask, right?

What we see in the studies is this 30% point increase

in wearing masks in the mosque

and wearing masks in the crowded market

and things of that nature, right?

So if people wear masks in these public areas,

but not necessarily at home or anything,

you get this 30% decline in COVID in the medium term.

Now, what does that mean?

That's actually complicated

because one thing we can revisit is okay,

you have this 30% decline.

One thing that might happen is R was greater than one before,

now the rate of transmission is less than one.

And so instead of COVID spreading to everyone,

it doesn't spread to anyone

because they have a war mask all the time.

Or alternatively, it's so contagious,

it still spreads to everyone

and then our masks doing anything.

Let's revisit that question.

That's an important question, okay?

So one question is,

what are the long-term impacts of this 30% decline?

But the question we posed a moment ago was,

what about other policies

to try to get people to increase mask use?

And my answer is, well, it really depends.

It's like if Alabama today said,

hey, we're recommending everyone wear a mask.

Probably no one would.

Nothing's gonna happen.

It does nothing, right?

But on the other hand,

if we have another respiratory pandemic

and tomorrow in two years, there's new COVID

and New York state is like,

hey, new COVID is killing a tremendous number of people.

We recommend that everyone put back on masks.

Like probably you get reasonably high compliance,

at least among the people who aren't politically resistant.

And there's a further point,

which is even today, what about symptomatic people?

If you're coughing and sneezing

and you have to go out in public,

maybe you should be wearing a mask.

What about the elderly people?

People with comorbidities.

Maybe they should still be wearing masks.

So we definitely still wanna understand

if when you successfully wear a mask, it works.

And I think our study strongly suggests in light of,

especially some additional other evidence

from all the other studies that like it probably does.

Two rapid fire questions about this study

before we get to the Cochran meta-analysis,

which was the subject of all these news stories

for the last few weeks.

Question number one,

there was a difference in your study

in the effect size for older people versus younger people.

And it's hard, or at least my read of your study,

says that it's hard to explain exactly what happened there.

Maybe older people were more conscientious

because they were at higher risk.

But does that gap make you concerned about confounders

that you can't explain in this study,

that something else happened

that's just not being captured

by the variables that you've talked about?

I mean, I find that gap especially concerning.

I agree with you.

There's many possible explanations

we could give for why it happened,

many that have nothing to do with confounds

or anything like that.

Like it could be, for example,

it could be elderly people were massed at similar rates,

but what happened was that elderly people

have like fewer social connections.

So wearing a mask is more likely

to sort of cut off the transmission vectors.

And therefore, it has a bigger impact

on the likelihood of getting COVID.

Or it could be that elderly people

are more vulnerable to low viral loads.

And so, and that's mass, you know,

turn low into nothing or whatever.

And then it prevents them from getting COVID.

Or it could be that elderly people,

we actually couldn't observe by age.

We could have in retrospect, I wish we did,

but we didn't observe by age,

whether the elderly people just increased mass use by more.

That's certainly another possible explanation.

So, but it's an interesting finding

because obviously the elderly bear the bulk

of the morbidity and the mortality from COVID.

So if it is generally true,

for example, the explanation I gave earlier

that elderly people get infected by low viral loads

and mass prevent that,

then that would be a huge thing to know.

But I think from the study, it's hard to,

like we would want sort of multiple studies of this type

before we concluded that, yes,

mass are definitely more effective

in the elderly or something like that.

Second follow-up question, when we say masks,

most people are referring to a bunch of different things.

Some people wear bandanas, some people wear cloth masks,

some people wear surgical masks, some people wear N95s,

some people wear K and 95s.

What is just briefly,

because they really do want to get

to the conference stuff in a second,

what's the difference?

Is there a major difference between these categories

that we use the noun mask for?

So in terms of the effectiveness,

so in our study, we had in one third

of the treatment villages cloth masks,

and in two thirds, we had surgical masks.

We find stronger evidence

for the effectiveness of surgical masks,

although both cloth and surgical masks

appear to have reduced symptoms.

We have a much more precise estimate

for the impact on the blood test of COVID for surgical masks.

For cloth masks, it's actually, it's just imprecise.

Some can't rule out that they're about

as effective as surgical masks,

but from other studies, like studies in hospitals,

it seems like surgical masks are better

also from just laboratory studies

when people cough into a petri dish.

In our study originally,

the reason we did both surgical and cloth,

surgical are actually cheaper,

but people at the time, this was, you know, 2021,

especially in countries like Bangladesh,

they sort of regarded surgical masks

as these cheap throwaway masks.

And so we were worried that people wouldn't bother.

They would wear the surgical mask once and throw it out

and they keep the cloth mask.

In fact, we found they wore the surgical masks

as much or more, so it's not really,

that wasn't really a concern.

So like if you can, higher quality masks

are more productive when you're looking for protection.

And by the way, another point I should make

that I realized we haven't made yet,

we haven't talked about like,

what I think are the actual policy consequences

of this or anything, but just to be clear,

I'm not saying, oh, mass work,

therefore everyone should be wearing masks all the time,

everywhere they go, everywhere in the world.

Like there's a different question

of when are mask mandates called for?

You know, COVID fatality rates are 20 times lower in the US

than they were in like July of 2021.

They're 40 times lower globally.

That makes a big difference to the cost benefit analysis

of when we need mandates.

But anyway.

And we're gonna get to the difference between masks.

Do they work as a product and mask mandates?

Should they be implemented as a policy

at the very end of our conversation?

But all right, let's finally turn to the Cochrane meta-analysis.

This famous or infamous meta-analysis

that was reported in the New York Times,

probably most prominently in a column

by Brett Stevens and opinion columnist for the Times

under the headline, the mask mandates did nothing.

Will any lessons be learned?

He quotes journalists saying

that masks don't make any difference, full stop.

He quotes co-authors of this meta-analysis saying

that masks make no difference, none of it.

Tell me, let's go into it this way.

What is the question that the Cochrane analysis

was trying to answer?

And why do you think the studies that it used

don't provide a high quality answer to that question?

Yeah, absolutely.

So there's a difference between the question

they were trying to answer

and the question they did answer.

So it's hard for me to speak to the question

they were trying to answer from some of the quotes

in the study and some of the quotes

I've seen publicly from the authors.

By the way, I have some anecdotes about that.

I've now spoken at length to one of the authors

and I'm trying to speak to others.

So we'll get to that in a moment.

But from their public comments,

it seems like they wanted to get at this question

of basically, are masks effective against COVID?

If a person wears a mask,

does it make them less likely to be infected?

If a population wears a mask, does it slow this,

or wears masks more, doesn't slow the spread of COVID?

That's not the question they answered

because the vast majority of the studies

that they are summarizing

are the studies that I was talking about earlier,

which are the studies that, for example,

send people on a college campus mask

and they ask, hey, please wear this mask during flu season

and then they don't check if you actually wear the mask.

And can I just jump in here

because you pointed me to a couple of studies

that make this point very, very clearly.

There's a famous Danish study

which was often used in the media

to suggest that mask interventions did not work.

When those researchers reached out to the participants,

they found that fewer than half of the masking group

said they, quote, wore the mask as recommended,

end quote.

There was a study in Uganda, 2022,

that when researchers called the participants by phone,

97% said they, quote, always or sometimes wore a mask,

but these researchers also observed people in Uganda

on the street and only 1.1% of the people they observed

were seen wearing masks correctly.

That suggests that the share of people saying

they wore masks versus the share

actually wearing them correctly

was a factor of 88, 88 times more were likely to say

on the phone they were wearing masks than actually were.

It's very difficult in studies like this

to know what you are actually measuring

because what you are measuring is a failure of adherence

and a failure of self-reporting

rather than a failure of a product.

In terms of policy, we'll get to that in a second,

but I was very motivated by that finding.

Back to you.

So I totally agree with you, I would say there are three

big deficiencies of the data study in this regard.

So the first deficiency is exactly what you mentioned,

that it's like probably, yes,

whatever half the people said they wore masks,

but we know that self-reports vastly overstate

the fraction of people who actually wear masks.

Now, the second thing about the data study

is if you actually look at their bottom line,

they find that there was like 18% less COVID

in the treatment group than the control group, right?

Now, that's not a statistically significant difference.

In science, we care a lot about statistical significance

for good reason, because if it's an insignificant thing,

it could have just happened by chance.

But what you absolutely should not infer from that study

is like, oh, the mask didn't do anything

because just the best estimate they have granted

being very imprecise is pretty similar

to what we end up arriving at in the Bangladesh study.

Now, why is it more imprecise?

Well, because the sample size was much smaller

and because probably they got a lot fewer people

to actually wear masks, okay?

So what should we infer from this?

Well, if anything, we should say,

here's one very imprecise signal

that suggests potentially similar effects

than what we see in the Bangladesh study,

not like, oh, masks don't work.

That would be a crazy inference to draw from this

because even the imprecise point estimate

suggests that masks work.

And of course, the true effect

would have been considerably larger

if you think that only a small fraction

of the people in the treatment group actually wore masks.

So that's like the second deficiency.

The third issue with this study,

relative to our study in Bangladesh,

is that it is only designed to figure out

mask protect individuals.

It doesn't tell you even in principle

if mask prevent infected individuals

from transmitting the virus to others.

Our study in Bangladesh,

where we increase mask wearing at the village level,

identifies the joint effect of mask protecting individuals

and preventing them from transmitting the virus.

So even in principle, if all the other problems were solved,

the study in Denmark can only get the protective effect,

which nonetheless, very imprecisely,

suggests might be there.

But this highlights another answer to your question earlier,

of why is this hard?

Well, it's hard because as common as COVID is,

most people over any several months period

are not infected with COVID.

So if you do a study where you monitor people

for a couple of months,

it's not like 70% of people are gonna get COVID.

It's like, oh, maybe one to 2% are gonna get COVID.

And then if you do an intervention,

such that you increase mask wearing

by even the 30 percentage points that we managed to get,

how much of that COVID do you think

you're gonna be preventing?

Well, maybe you prevent a 10th of that.

So now you've gone from one and a half percent

to a 10th of that 0.15.

And then you actually have to do blood tests

and not everyone is gonna consent to have their blood drawn.

So now you get an even smaller number.

So you need a really giant sample

if you're going to detect any impact of any policy.

So look, you have done the mass research.

I have not.

You have now spoken to the Cochrane authors.

I am so interested to know how that conversation went

because at least in terms of their statements in the media,

they seem very clear on what their position is,

which is that masks don't work the end.

You're telling me the exact opposite thesis.

So how did this conversation go?

Okay, so I've now spoken to one of them

and I'm trying to schedule a call.

There's the lead author, Tom Jefferson,

who I have not spoken to yet,

who is the main one making these remarks.

But I will hopefully speak with them in the next few days.

Maybe we'll do a two minute addendum

if you need an update on that.

But for the author that I did speak with,

so first I'll tell you the way I expected

the conversation to go before it went,

which is when I was in grad school,

I had a professor, Jerry Hausman.

And Jerry Hausman told us the story where he's like,

when there's a Nobel Prize winner, Clive Ranger,

and he met with Clive Ranger

and they had this technical dispute

and Jerry Hausman was like, yeah,

we had one of these long drawn out academic conversations

where I pounded him into the ground.

Right?

And I was like, oh, this is what's good.

But in fact, it was the exact opposite.

So I talked to the guy from the Cochran Report

and I was like, hey, here's what I think the issues are.

And he's like, yeah, those are really good points.

And I was like, oh, and I was like,

would you be willing to co-author an editorial

like making these points that sort of the Cochran Review,

I keep calling it the Cochran Report,

we'll just stick with that.

I like that more.

The Cochran Report is that, you know,

it's been misinterpreted in the press coverage

and it's kind of the conclusion that mass don't work,

just isn't warranted given the studies you've included.

And he was like, yeah, sure,

I'd be happy to sign off to that.

Wait, this feels like breaking news.

This feels like breaking news.

An author of the Cochran Report slash Cochran Meta Analysis

is about to co-author an op-ed with you

saying that the report broadly interpreted by the media

and quoted by some of the lead authors of the report,

saying that mass managed to work,

he's about to say it didn't actually show that.

Yeah, I feel like now you have me

making such a big deal of it,

where it makes me worried that he's gonna like,

hear this podcast.

This is coming on Tuesday morning, I think.

That is the plan right now,

where I sent him a draft, we're chatting more at five,

so hopefully that'll happen.

Jesus, well, it is for, listen,

it's 2.37 PM Eastern Standard Time on Monday.

We'll see what happens tomorrow

and if he responds to that email by saying, nevermind.

So give me a sense of what you think

the smartest mask critics get wrong.

Because I'll say this, we emailed,

I talked to an aerosol researcher,

Professor Jimenez about how masks actually work.

His, the fact that he's very persuaded

by the lab reports that, look,

if COVID is an aerosolized disease

and we know clearly that masks reduce aerosols

going in and out, they have to work.

This has to be an issue of adherence

rather than an issue of public policy.

So I wrote this up and I got a really nice response,

frankly, from some conservatives and some mass skeptics.

And I got what I would characterize

as a really not very nice response

from some mask critics and mask skeptics.

I would say there were two kinds of criticism

of my attempt to synthesize some of your research.

One line of criticism was that

perfect mask adherence in a community

is kind of like telling everyone they have to just like,

like trying to like banning sugar

or like mandating that babies eat broccoli

or mandating that people just sort of fast

when the sun is out every single day.

It is, it's not that those things won't be,

won't like reduce weight.

It's that it's almost impossible to enforce.

So why are we even heaping

in the arsenal of public policy interventions

something that we broadly understand to be unenforceable?

Let's start with that.

Do you agree with the contention

that maybe masks work, but mask mandates don't.

And we should just acknowledge

that that's the state of the world.

Yeah.

So let me first just draw a distinction

between if everybody wore masks,

what mask mandates do

and what the studies in the Cochrane review do,

which is the studies in the Cochrane review,

it's like, oh, we're going to send people masks

and give them information.

We know that that has very little impact on adherence.

What about mask mandates?

Mask mandates, I would say,

that is one name for a wide variety of different things

that sometimes increase mask use and sometimes don't.

Like when airlines are like, hey, please put on a mask

or we'll kick you off the plane, they're really good

at getting people to wear masks.

When the post office is like, hey, please put on a mask

or we'll kick you out, they're really good

at getting people to wear masks.

If the governor of a state is like, hey,

we're recommending that everyone put on masks,

you're not going to get to 100%, right?

Like there's going to be resistance

depending on the state.

It might be that you get in some states very little effect

and in some states more of an effect.

Like one study I saw, just a correlational study

looking across states that has its own deficiencies

suggested maybe like a 25 percentage point increase

in mask wearing from mask mandates.

That's comparable to what we saw

from our intervention in Bangladesh.

So my view is not mask mandates don't do anything.

They sometimes get people to wear masks more,

but it's hard like at a national or a state level

to suddenly change norms and get every person to wear masks.

Now there is nonetheless the question of like,

I think there are things depend

on the underlying objective circumstances.

If there were new respiratory disease

that were five times as deadly as COVID, I would predict,

people would see their relatives dropping dead,

they would want to do something about it.

Now it might be just because the way politics have worked

that the US would be weird in Alabama,

they still wouldn't wear masks.

But in most of the world,

people would pretty rapidly adopt masks

if there were a situation like that

with such high fatality rates.

So I think the answer is,

depends on the circumstance, it's certainly possible

sometimes to get people to wear masks.

And I mean, I don't think the analogy with like,

vegetables is even that great, maybe for infants,

it's kind of weird, but it's like, you know,

we do research, we try to figure out what foods are healthy.

I bet that people eat a lot more vegetables than they would

if there had never been any research showing that, you know,

vegetables were healthy and help you live longer,

help you have fewer heart attacks and cancer and everything.

And it's like, you know, that's why we do the research

to figure out what it does,

not because we think that tomorrow,

everyone is suddenly gonna start eating only vegetables,

even not even because we necessarily think

they should do that because we wanna know,

what should they do in different circumstances?

The other objection that I've heard from the mass skeptics

is that they look at a country like, say, Japan,

and you talked about how, you know,

you look at the COVID transmission rates in the West,

in Europe, in the United States,

and you compare it to Japan and Taiwan in 2000,

and it's not even close.

It's astonishing how much more it was transmitting in the West.

But then Omicron comes around

and Japan, Taiwan, Hong Kong,

they definitely had Omicron waves,

even despite the fact that, and look,

I'm not on the ground in Hong Kong, Taiwan,

but I'm guessing that there was still

a decent amount of mass coherence.

Do you have like even a stylized door in your head

of how these things can both be true?

It's so funny, because I often see this argument on Twitter,

and then I'm like, okay, so when I see an argument like this,

my instinct is to be like, well, you know,

let me look at the numbers and try to do a comparison.

So it's like, so then I like look up the fatality rates,

and it's like vastly lower in Japan than the United States.

If you look at like how many people have died

of COVID per capita, I'm trying to remember offhand,

I'll probably get it wrong,

but maybe it was like five times lower,

10 times lower, something in Japan than the United States.

So it's like, are we saying that if everyone wore masks,

no one would ever get COVID?

Like, no, what the studies say

is that it probably protects you.

Now, we haven't spoken much about the question

of the long-term consequences yet,

where I think there really is a lot of uncertainty

about a number of factors

of what the long-term consequences of more mask-wearing is.

But like, I think the conclusion that, hey,

these East Asian countries,

the conclusion we drew in 2020,

that the East Asian countries had slower COVID growth

is born out even more so today

when you look at the long-term fatalities

for the countries that had historical mask-wearing norms.

Let's get to that right now.

In the article that I wrote for the Atlantic,

I concluded going through all this mask research

by saying, you know,

there's still a cloud of uncertainty here,

but we have to make discreet and irreversible decisions

sometimes, even in clouds of uncertainty.

So I'm just gonna tell readers what I'm going to do.

And what I'm going to do is,

especially during periods of high COVID transmission,

that, and I would not necessarily consider

this moment to be one of them,

I'm gonna wear N95 masks in public indoor spaces.

Furthermore, I think that Washington DC, Northwest Washington DC,

which is my neighborhood,

would probably benefit from a mask mandate

because I have observed the social norm

of mask-wearing in my neighborhood.

And there are a lot of high-quality masks

worn well in this area.

And I would expect that if more people

wore high-quality masks well,

it would probably reduce levels of transmission

within this neighborhood.

Even as I suspect that you're right,

that mask mandate policies are not gonna do much

in, you know, Alabama.

But what are the risks?

What are some of the costs that are important to consider

when we're evaluating mask mandates as a public policy?

Because, you know, you and I have both said a couple of times now,

like, this is not something that's risk-free.

And I've written about, you know, all these debates

about masks in schools.

Tell me about some of the costs

that are most top of mind to you

when you think about masks as not a product,

but as public policy.

Let me first rewind to April of 2020,

where I was trying to convince the epidemiologists

that the ones who were reticent to recommend mask wearing,

one of the biggest arguments they made was,

oh, if we recommend that people wear masks,

they will think they are fully protected

and they won't socially distance anymore.

And what we think they really need to do

is socially distance,

which I thought was a very interesting argument.

Like in Bangladesh, we actually found the opposite.

We found that in the villages where we did the mask promotion,

they socially distance more.

We think we can separate the effects.

We talked about that.

That's a separate issue.

But I mean, I think that's an open question.

It might vary across contexts.

There's different effects that go in different directions.

I'd be surprised if the risk compensation

were enough to outweigh the direct effect.

Like in many settings, like with seatbelts and airbags,

this is something people have brought up in economics.

It's called a Peltzmann effect, that it's like,

oh, you know, because you wear a seatbelt,

you're not driving this carefully

because you know your protection.

Yeah, maybe that happens a little bit on the margin,

but probably people are gonna be safe

for wearing seatbelts and not wearing seatbelts.

So yeah, okay.

That's one thing, but that was the big issue,

like during large COVID waves,

more generally, what are the costs?

I mean, the biggest, most prevalent cost is just,

it's kind of uncomfortable to wear a mask, right?

I don't mean to be poo-pooing that.

Like it's like, how big would the benefit have to be

for it to be worth, you know, wearing a mask in,

first in like public areas.

Suppose you're just going there for a few hours, right?

During the day, you know, you're going to a mall

or something and you're gonna wear a mask in that mall.

It's like, well, if you get the equivalent of, you know,

$100 worth of benefit, it's probably worthwhile to do it.

You would probably do it for $100.

But if you get $3 worth of benefit,

maybe you'd just be like, no, I'd rather be comfortable.

I don't wanna wear a mask.

So it is really important the magnitude of the benefits.

So when I was saying earlier, you know,

fatalities from COVID are 20 times lower

than they were in July of 2021.

You know, if the magnitude of the benefit

was $100 in July of 2021, now it's $5.

So that changes things, right?

So my general view of public policy is, hey,

we should do calculations to see

if the benefits would see the cost.

Now, these calculations are very, very hard to do.

As soon as you sit down

and you try to actually quantify costs and benefits,

what you find is you have to make lots and lots of assumptions

for which you don't have good data.

And one response that people have to this

is anyone who does a calculation, everyone yells at them,

because everyone's like, oh, look,

your assumption about X, Y, and Z is not supported

and I disagree with you about this, this and that.

But guess what?

The alternative to doing a calculation

is making shit up and guessing.

And that is worse.

That is even harder than making assumptions

and trying to do a calculation.

So my view is, you know, we should sit down.

So what are the costs you talked about?

For elderly people, or for most people,

maybe the biggest cost is just mask wearing is uncomfortable.

Now, the other, in some settings,

what we worry about is communication.

So for example, if you're wearing masks in classrooms,

does that make it harder to speak?

If the teacher is wearing a mask,

can you not hear them as well?

If the students are wearing a mask,

are they less able to focus?

Like if you're wearing masks in indoor business areas,

are they harder to focus or you're less productive?

I honestly think we just have poor quality evidence

on these questions.

We just don't know much about how big those costs are.

And then of course, in younger children,

we worry about various developmental things,

where again, we just have poor quality evidence

and we don't know much about the magnitude of the cost.

So I think we know more about the magnitude of the benefits,

but only in the medium term.

The long-term benefits are hard to assess

for reasons we can get into.

Earlier in the episode, I had my colleague, Dan Engber,

who's a science journalist who's written a lot

about the lab leak.

And we talked a little bit about how the media's treatment

of the lab leak in the last three years

is an interesting lesson for science writers and journalists

in dealing with clouds of uncertainty.

Not ruling out candidates, not ruling out stories

simply because they seem like they might be racist

or it seems like it might give credence to a side

that you don't belong to,

or that it seems like it might undercut

the benefit of science.

It might have been hard for scientists in 2020

to say that actually maybe this pandemic started

because of scientists making a mistake.

That might have been a hard thing for some people to admit.

I'd love to ask you a similar question,

which is, what do you think the debate about masking

reveals about the way science

and scientific communication is conducted today?

Like, do you see ways in which this fight that we're having,

not you and I, but that we as a country are having,

this way, a way that this fight is a microcosm

of a really important story about how science works

in America?

So my view is, I don't think there's anyone

making an obvious mistake.

My view is kind of like doing science properly

is really, really difficult.

There are lots and lots of things you can get wrong.

And so when you have a highly politicized topic like masking,

the world is always just gonna be flooded with bullshit

and the signal from the actual signal from the truth

is always going to be kind of weak.

And what happens is like,

maybe over the very long span of history,

as more research accumulates and everything,

then eventually sort of people figure out what's what,

but it's just kind of the nature of things

that like doing science properly is really, really difficult.

And there's just a lot of subtle distinctions

that are gonna get lost in any political discussion.

So you don't, journalists have a super hard job too.

So it's not something where it's like,

oh man, everyone's being so stupid.

Why don't they just do this?

It's kind of like, yeah,

that's the way the world is gonna be

when you have a highly politicized issue.

It's just really, really hard

to figure out what the truth is.

One thing I would say is there are certain communities

that have like really good norms

and it's those norms that sort of allow science

to progress over the longterm.

Like there are communities where,

and I would say like, I think like people who do,

I consider my field of being

what's called like applied microeconomics.

Like we try to do empirical research

and we try to design studies

to sort of tease out the truth of things.

I think it actually has very good norms

in the sense that if you go to a seminar in our field,

of course, everyone is political.

It's not like people are perfectly objective

and can get away from all the kinds of things

you're talking about.

But there's just a norm that it's like,

we're really trying to figure out what's true.

And so if someone has sort of like a counterintuitive result

on like a hot button political issue,

the questions aren't like, how dare you say this?

And I can't believe you would say this.

It's like, the questions are really going to be focused

on like technical questions of methods.

Like, could you do this to maybe,

maybe you should do this additional check

because it might suggest whether this thing went wrong

or you could get this additional data to try to do that.

So I think there are some communities like that.

And the more of those there are, the better.

And like one thing journalists can try to do

is to like figure out where those communities are

and try to get a little bit of signal from those.

But you guys don't have an easy job

because of course, there's a very hard meta problem

in the world of how do you decide who has expertise

and who you listen to?

And that's hard to do.

A part of me wants to make fun of you for,

like saying like, microeconomist colon,

the world should be much more like the field

of microeconomics.

At the same time, in all honesty,

and this is not just like blowing smoke up your ass,

I'm in a lot of different communities on Twitter.

I dabble in politics Twitter, I dabble in COVID Twitter,

I dabble in entertainment Twitter and media Twitter.

Economics and finance Twitter is one

of the best bubbles to be in.

And I don't know why the people,

at least that I follow in that space,

are so not perfectly unideological,

but so curious about understanding problems with numbers.

Like it's a lot of people posting graphs

of the direction of used car inflation

and saying, what do you think that is?

And then there's like a list of theories

and some of them get retweeted.

And then someone's like, actually,

I don't think that this theory holds

because if you refer back to something

that Abeluck wrote in like 2020,

it actually turns out that it's a lot of numbers detectives

that are kind of just trying to like be little,

number Sherlock Holmes and solve numbers mysteries.

And that leads to some problems.

The economics Twitter misses a ton of shit

and has a ton of problems,

but it's better along this vector,

I think, than a lot of alternative communities.

Last question for you.

You said, you know, studying this is hard,

figuring out exactly how masks work

and how especially they work in community settings,

it's really hard.

What do we need more of for this question to be easy?

So I think one thing that I would like

to generally see more of in the world,

and I think that public health agencies

as currently structured are not well equipped to handle

is sort of like rapidly funded,

but very large scale experiments.

So similar to what we did in Bangladesh for masks,

but it's like there's similar kinds of things

you could do for ventilation

and for all these other things

that we really just have not done

because like public health agencies are not equipped

to do these in the short run

and they're kind of ill equipped

to do them in the long run too.

So we got the funding for our experiment

just from a private funder give well, right?

And it's like, you know, we managed to convince them.

It's like they're trying to do good in the world.

We managed to convince them

that this was a useful source of funds.

But it's like, if we had tried to get that funding

from the National Institutes of Aging, the NIA,

you know, it would have taken a four year application

or something like that before any funds materialize.

So I think setting up agencies

that basically encourage two things.

Like one is making decisions quickly.

So instead of being like,

you apply and in two years you might get some funding

being like, there are certain types of problems

where we need to respond more quickly.

And second, funding more like large scale projects

instead of like a hundred smaller projects.

So instead of like a hundred people being like,

oh, I need a hundred thousand dollars

so I can spend this year doing,

I'm gonna try to survey people and do this sort of like,

I know it's not the best study

but at least it'll be kind of suggestive,

doing more things which are like,

no, we're gonna get a hundred people together

to do the best possible study,

to really get the best data to answer this question.

I think that is like, there are much higher returns

to doing like the large scale ambitious thing

that collects the right data

and really answers the question you care about

as opposed to writing many, many low quality studies

that don't necessarily have data on the outcome you care about

and aren't necessarily designed

to answer the right question.

And then doing what the counter review did and saying,

oh, let's aggregate up a hundred of those studies

or something instead of doing a handful of the studies

that are really well designed to do that.

Yeah, more speed, less bullshit, more big studies.

That seems like a pretty interesting

and promising formula for the future science.

Jason Abelik, thank you very, very much.

Yeah, this was great.

Thank you for having me.

Thank you for listening.

Playing English is produced by Devon Manzi.

If you like the show, please go to Apple Podcast or Spotify,

give us a five star rating, leave a review

and don't forget to check out our TikTok

at plain English underscore.

That's at plain English underscore on TikTok.

And until next time, goodbye.

Machine-generated transcript that may contain inaccuracies.