How Accurate Are Antibody Tests?

Published May 29, 2020, 7:00 AM

Dr. Alex Marson, the Director of the Gladstone-UCSF Institute for Genomic Immunology, explains what antibodies tests can and cannot tell us.

Learn more about your ad-choices at https://www.iheartpodcastnetwork.com

Pushkin from Pushkin Industries. This is Deep Background, the show where we explore the stories behind the stories in the news. I'm Noah Feldman. As the COVID nineteen pandemic continues, antibody tests are gradually becoming increasingly available to ordinary people. You can get one online, and some employers are even requiring that people take them to go back to their jobs. But how accurate are these tests really? And if it turns out that you do have antibodies that show you've been exposed to coronavirus, what does that actually mean about what you should or shouldn't be doing. Next. Here to answer these questions is doctor Alex Marson. He's a biologist and an infectious disease doctor. He's a tenured professor of microbiology and immunology at the University of California in San Francisco, where he's the director of the Gladstone Institute of Genomic Commune Knowledge. Alex, thank you so much for joining me. Alex, your lab ordinarily does high throughput genomic engineering research, which is very cutting it stuff. And then when the COVID nineteen pandemic started, you went to your lab team and you said, okay, we're completely changing course. We're going from the highest tech to relatively lower tech, and we're going to look at the antibody tests that are out there and we're going to see if they work. So, first of all, what gave you the idea to do this? It's a great question, and looking back, it was really it was a confluence of a few things. It was necessity, and it was really in many ways motivated more by the people in my lab, or equally by the people in my lab as it was by me. Our lab was shut down. People weren't able to go in and work on their normal projects, but there was a real feeling in the lab that people had expertise and motivation to try to figure out what they could to be useful in the midst of this pandemic. It was really inspiring for me to see grad students and post docs and technicians who were looking for ways to contribute, and if anything, I was able to help channel and put together collaborations to enable their desire to help out. And so we saw a huge flood into the market of antibody tests that were becoming available. And actually one of the motivations for me was I got a text message from a friend of mine not in science here in the Bay Area, who showed me that she was testing herself for antibodies on one of these home diagnostic kits, And so I started wondering, what are the basic test performance characteristics of these kits? There were so many that were becoming available. We wanted to see could we assign some rough numbers to how reliable these tests actually are? And so that was really the fundamental goal of this study was to say, could we get our hands on as many of these test devices as possible and see what kind of information they actually give and don't keep us in suspensey longer? How were they so? There was a range. They're so polite. This is the boring answer that which was not totally unexpected. Some were reasonably good and some were not going to be very useful in this And one of the major determinants is how specific they are. And especially in a disease like this, where there's in many parts of the world is still relatively low prevalence, the chance of misinterpreting these results due to false positives is very high, and so one of the major things that we wanted to check was are these giving us results that are going to be confused by false positives, perhaps because they're misinterpreting antibio against other viruses, like other common coronaviruses that cause common colds. So we wanted to see, in a population where we know there's no Saris CoV two infection, how many people in that negative population had antibodies on each of these devices, And so this was really, in many ways the meat of this study. We took one hundred and eight blood specimens that had been frozen down well before the pandemic, going back to twenty eighteen and before, and we asked how many of those blood samples showed evidence of antibodies with these commercial tests that were popping up, and we saw a range. Many of them found a lot of evidence of antibodies against saris CoV two in these samples where we know that it shouldn't be there, and so those results are really disabling for a proper interpretation of these tests. If we want to use these antibody tests to measure in a population how many people have actually been infected, they would cloud that picture strongly, and many of the cases detected would actually be false positives. And if we wanted to give an individual patient information. We could give them very misleading information if we gave them false positives and said that they had been infected with stars Cove two when in fact they had it. But I want to caveat that there were a handful that seemed reasonable, and even one test showed one hundred percent specificity, meaning no false positives at all, meaning no false positives in the limited number that we tested. I've been thinking about the fact that in many ways, the way that people responded to our results was a Rorschach test of how they wanted to see these results or how they were predisposed to see these results. And I've been struck by some of the news coverage, which really ranged from saying antibody tests show great promise to these are complete hooks. And my true interpretation is actually somewhere in between that in some ways we got out in front of ourselves where in response to a pandemic, many many suppliers started racing into this market, and it wasn't a total shock to me that we saw arrange. I think what was a bit surprising was that these tests were becoming available to individuals in some cases before this basic information was available, and so we felt like we were just building in that gap. One thing I've been sensing a lot of recently is that as more and more states begin gradual kinds of opening, lots of people are now saying, gee, you know I was sick in March at one point, or I had a long lunch with someone whom I found out later turns out to have had it. Maybe I should look into having an antibody test now. And one question that they asked me, probably because they have heard me talking about your results, is how reliable are these tests now? I don't know what to tell them, So I'm asking you, what would you say under these circumstances. I have companies on the whole taken the lessons of your research and figured out so that more of the tests that are available now are like the better ones that you saw. So there were probably at least one hundred different companies that are offering different tests, and these will pop up in different in different settings. We tested sampling of these. We tested about twelve of these different tests, and we have a website available so that people could compare our results to whatever test information may be becoming available. How would people find that website? What should they be looking for the COVID Testing Project dot org. Now, keep in mind it's a preprint, it has not yet been peer reviewed, and it's a small sample, and so it's not intended to guide any kinds of clinical interpretation, but it provides some basic information. We've been in conversation with a larger testing effort that has now come up as part of a governmental effort. The National Cancer Institute is now doing a large test in concert with the FDA, where FDA is going to be assessing tests for antibodies going forward with the National Cancer Institute. And there's another website on the National Cancer as website and on the FDA website that you can look and find information about commercial tests that are now undergoing evaluation by the FDA. So the short answer is, let the buyer beware. Right, you may not even be able to find reliable information online about whether a given test you're taking is reliable or not. That's true. I think we're starting to see now some of the larger commercial vendors that have traditionally been major suppliers of lab diagnostics entering into this field and playing a bigger role. That these large vendors get into it. There's a hope that there's more consistency and more quantitative information on the levels of antibodies and also on the test performance characteristics. With the rapid diagnostics, not only is there variation among the different vendors, but there's some anecdotal reports that even within something bearing a label of one particular vendor, there may be batch variation, and so the picture is even more clouded with these rapid diagnostics. So the one takeaway there would be, if you really really feel like you have to get the test, send it away. Don't do one of these rapid diagnostic tests at home. If it looks like a pregnancy test, probablesion rely on it. So we've been thinking a lot about this going forward, about what are the possible ways that you could have an efficient testing algorithm. So if someone does a home pregnancy test, what's the first thing they do with that information. If it's a positive, well, they go to their doctor and they get a lab based test. And so maybe there's some opportunity to do something like that where there's a more complex algorithm that could be devised, where there's multiple tests that are used for confirmatory testing, where it's either a combination of home diagnostics or a combination of home diagnostics and lab based to expand the testing infrastructure without sacrificing sensitivity or specificity. That requires some more thought about exactly how that algorithm is designed, but I think these types of test performance numbers are the basic building block that you would use to design an algorithm like that. Going forward, let's talk about what someone could actually do if they did one of these tests it was reasonably reliable and they got a positive. Given the relative uncertainty that's out there, what would it mean for someone who said, well, I've tested positive. If anything. I think that there's really two measurements of what it means. One is how likely is it that it's actually giving reliable information about whether or not you've been infected? And the other implicit question, which I think is what people really care about, is what information is it giving you about how likely you are to get reinfected in the future. And so let me tackle both of those. The first is we are getting to a point where some of these better tests, especially the lab based diagnostics, are starting to give reasonably reliable information about whether or not there are in fact antibodies present in an individual's blood. The second piece is much more complicated, What do we actually hell someone who has a positive antibody test. I think no matter how many times we say in the news that we can't yet tell someone if they have a positive antibody test, they're actually protected from future infection. People have such a strong intuition and desire for antibodies to mean immunity that I'm concerned that there will be an implicit behavioral message that people are safe from prior infection and should take on risks and go into the community and do things that they wouldn't otherwise do without real science to back up to that behavior. So I think right now we're starting the next round as a community of scientific fact finding to start saying how can we advise people about risk of future infection if they do test positive for antibodies. There's really a big range of what infectious disease doctors can come to expect from what antibodies and prior infection mean for the prospects of symptoms and contagion on reinfection, there's a few lines of evidence that we as scientists are really looking for that will firmly tell us that we can give the recommendation to someone that they will be protected. The first are starting to emerge now, and these are animal studies. There have now been a few animal studies, including one published just recently in Science by Dan Baruk's group that looked at Reese's macaques infected with SARS CoV two and then reinfected upon infection, The monkeys developed signs of immunity and reinfection was far less severe. There might be small amounts of virus that actually infected the monkeys, but they seem to clear it relatively and quickly and didn't have signs of infection similar to the first infection, So that's highly promising. How that will translate to humans remains to be seen, and there's two levels of questions. One is will people clear the virus and not have severe symptoms and will they be contagious because that's also something people care about. Can you go back to work and not worry about spreading it to more vulnerable people that you also come in contact with, and that will require time. We'll be right back in the monkey studies. Were they able to determine whether they were infectious to others or were they only able to determine how much they showed symptoms or clear the virus. It looked like they were really only able to see evidence of symptoms and quick viral clearance and really pretty limited levels of virus infection, if any. Because that seems hugely significant, right, I mean, if it were to turn out that what was observed in monkeys was also replicated in humans, so that you would get a much milder case the next time around, that would be very reassuring to individuals. But if you had a mild case and we're still infectious to others, we don't want you going back out into public at that point. And we definitely don't want you walking around and thinking that you're effectively immune because you could be as much of a spreader as the next person. Right, So that these data are incredibly important and to my knowledge or not yet available for this particular virus. So the upshot of that for an ordinary person, just to bring it back to the you know, our hypothetical person who's thinking, So, now this person takes a test, it's a send away test from a reliable deliverer. Having done due diligence, our person now thinks that he's been exposed and has antibodies, and he says, Okay, now I'm going to go out and go about my business and interact with people, and the takeaway for that person is not so fast. Yeah, you know this was brought home poignantly to me recently. I had a conversation with a close family friend who called me up and said, should I take an antibody test? I really want to be able to see my grandson? And I said, look, I would love to be able to tell you that a positive antibody test could safely mean this, that you could you would be protected, but we don't yet have that information. I so deeply understand the yearning to have that level of security, but trained as a doctor, I don't feel that I yet have enough information and to say that the test result would allow me to actually recommend that you'd be safe to go and change your behavior in any way that you wouldn't otherwise. And although it's hard to put numbers on these things, how confident would you have to be? I mean, I understand that as a physician, you want to be cautious, right. You don't want to say to somebody you know what, you'll be fine and then have it turn out to be the case that they're in some small tale of the data where they actually were still able to get it again or to give it again. But how confident would you have to be to say to somebody, yeah, you know what on the whole once you've had the positive antibody test, this is basically almost certainly going to be all right. Though I'm making you no promises. I would like to see the human data. So I just told you in detail about the experiment in the monkey model. I think that we really need some basic information from humans. Now. Some people have been advocating strongly for actually doing what I just told you was done to the monkeys, to actually doing that in humans, and there's been a group of scientists that have signed letters talking about advocating for human trials of actually active infection. Now, this has been done for other coronaviruses in the past, the kind that caused common colds, and for something that causes mild symptoms like a common cold, that may be an acceptable risk. The question is would it be an acceptable risk here, perhaps in a young, healthy individual. I would advocate that that's not necessary and perhaps not ethical. In this case. I think that there's still high enough rates of transmission that well designed studies and high risk individuals should be able to give us this information, not quite as rapidly, but rapidly enough that we can interpret them. So I think what we really need is a carefully designed epidemiological study that aggregates all the data from everyone who's had antibody testing and watches them carefully over time, especially in situations where they'd be high risk if they're healthcare workers and high incidence regions, and ask the question very carefully and numerically, what degree of protection do we actually see in the people who have antibodies? Do they get infected and do they spread to their context? Are we moving in a direction where we're going to have to rely on two kinds of tests simultaneously, where we're going to have to rely both on swab testing of whether people have the virus in real time and also on an antibody test. Or are we heading for a world where one of these will predominate over the other. I strongly believe that we need both, and we need to be very clear about what information we'll get from each of those types of tests. The virus testing is the gold standard for seeing who is infected with this virus SARS CoV two, and that is what happens when people get the nose or throat swab. Increasingly this is moving to saliva testing, which I think is very promising and perhaps more scalable. Those are looking for the presence of the virus itself inside of an individual. After about a week on average, we start to see the evidence of the virus itself will wane, and by about two weeks or three weeks with some variability, we'll see that people who have been infected will start to produce antibodies against the stars covi two and that's where we'll be able to detect the presence of antibodies, and those will stay up for some period of time, although the exact period of how long they'll be detectable we still don't know. We have to trace that out farther. But by looking at both of those, we'll get information both about early infection and later information about who had been infected in the past. And if we want to accurately put together measurements of prevalence across a population, transmission dynamics, mortality, we really need information from both of those, and likewise, for back to work, we also need information from both of those. We've talked a lot now about will the antibody testing help us determine who's safe to go back to work, and we've talked about the pieces of knowledge that we still need about immunity which will start to guide whether antibody tests can tell us whether some people are safe to go back to work. But virus testing itself is also important for knowing who it goes back to work. One of the key things that we know about transmission of this virus is that it often occurs from asymptomatic individuals before or they even know that they have symptoms, or even some people who may never go on to have symptoms. And the only way that we could really tell whether people are infectious is if we screen for the presence of the virus itself. Now this may sound unusual, but there was an article in stat News recently about a model for this from the adult film industry. It's almost too good, yes go on. In response to the HIV epidemic in a group of very high risk workers, there had to be a model for how to figure out how to pe get people back to work with relative safety, because most HIV testing that done is actually based on antibodies against HIV, but those come up relatively late. Again, they don't come up in the earliest acute phases of infection for HIV, they wouldn't come up for the earliest phases of SARS CoV two. So that means that there would be an individuals who could be acutely infected with HIV who would be infectious before they would be detectable with an antibody. And so the adult film industry has come up with a very aggressive testing strategy that depends not on the antibody tests used for most individuals, but on actual viral testing that would sensitively detect the presence of virus at earlier time points. And individuals have to get tested at very regular intervals to be cleared to go back to work. On the set of an adult film industry, I think it's every fourteen days they have to get clear to go back to work. And there's a whole infrastructure set up with thought about privacy and also coming up with a plan for how individuals would get treated if they were to test positive, and also how contact tracing would be done to identify people that have been exposed. And so all this has been actually carefully thought about in one industry where there's a high risk of viral infection. Now, as a result of this pandemic, all industries are at high risk of infection as people go back to work, and I think we have to give some similar thought. I don't mean to over extend this analogy. HIV is a chronic virus, sarrys. CoV two is not is something that will get cleared, and so the dynamics are quite different. But I think we have to think about how can we detect early cases, asymptomatic cases and have infrastructure in place to make sure that positive cases get treated and traced, to limit the spread of infection and to limit mortality and morbidity. The idea that we could borrow a protocol from the adult film industry is its delicious and it would certainly be fascinating if that ended up being being applied more broadly. Can I see a totally outside the box quirky question that I've noticed in talking to different people, I've noticed that serious scientists all say stars CoV two, and they almost never say COVID nineteen. Why it seems to me like a little insider outsider code. I've actually realized this recently talking to some people that this is confusing. I think the simplest way to explain this is by analogy to HIV and AIDS. AIDS is the disease caused by HIV and STARS. CoV two is the virus that causes COVID nineteen the disease, And so when we're really talking about the mechanics of infection, scientists will gravitate to talking about the name of the virus rather than the syndrome that it causes, since there's a wide range of outcomes of what the virus may actually do, and so COVID nineteen is actually just the name of the syndrome. So it's the person is suffering from COVID nineteen having been infected with the stars CoV two exactly. And I think that actually has been a genuine source of confusion. When will your lab be able to be up and running and doing its ordinary but not normal science. We have started going back to work now at a drastically reduced capacity. One eighth of the lab is able to work now, and we're able that people are going in with masks and keeping their distance from each other and working in isolation. To start returning to their long term projects. I again was inspired during this period until this where people were so excited to have a chance to do the science that they're passionate about, even if it wasn't their actual PhD project, to be able to take the underlying skills that they're cultivating and apply them, and people express real gratitude to have the ability to work on stars copy two testing, COVID nineteen testing during this epidemic. And so we're continuing some of this but also transitioning back to the more long term science that we've been focused on on crispergino engineering of human immune cells to try to think about treating a wide range of human diseases. And I think people are also glad to be continuing to work on this pandemic, but think broadly about how they can make contributions to human health even beyond that too. Well, let me take the opportunity to thank your lab members and you for the contribution you've made on czars Cove two, but also for the work you're doing all the time, and to wish you guys a good opportunity to get back into the lab at more than one eighth and go back to continuing to try to make the world a little bit of a better place. Thank you very much, Alex for your time. I walked away from my conversation with Alex with a kind of mixed picture of developments in the antibody testing space. On the one hand, I heard some measured optimism from Alex that the originally not that impressive tests that he discovered when he and his lab went to measure the effectiveness of the antibody tests at first are starting to get a little bit better, especially those that are send away laboratory tests. That said, despite this progress, Alex thinks it is still not soon enough to tell people that once they have those antibodies that they can go around treating life as normal, because we still do not know what degree of immunity, if any, is being conferred by the antibodies. For that will need more time and more research. Alex also points out that going forward, to get people back to work and get the world open, we're going to need much more testing, both testing of people who currently have the virus and also antibody testing. So more tests remain a crucial desideratum for reopening the economy, and last and definitely not least Alex suggested that we may actually already have an available model from industry that helps us know how to get people back to work, the model of the HIV testing protocols put in place by the adult film industry. If it turns out that the adult film industry has something significant to contribute to getting people back to work, that will be one of the great surprises. Let us just say of this entire strange COVID nineteen pandec and although this may sound a little different immediately following a conversation of adult films, until the next time I speak to you, be careful, be safe, and be well. Deep background is brought to you by Pushkin Industries. Our producer is Lydia gene Cott, with research help from zooe Win and mastering by Jason Gambrel and Martin Gonzalez. Our showrunner is Sophie mckibbn. Our theme music is composed by Luis Guerra. Special thanks to the Pushkin Brass, Malcolm Gladwell, Jacob Weisberg, and Mia Loebell. I'm Noah Feldman. I also write a regular column for Bloomberg Opinion, which you can find at Bloomberg dot com. Slash Feldman. To discover Bloomberg's original slate of podcasts, go to bloomberg dot com slash podcasts. And one last thing. I just wrote a book called The Arab Winter, A Tragedy. I would be delighted if you checked it out. You can always let me know what you think on Twitter about this episode, or the book or anything else. My handle is Noah R. Feldman. This is deep background

Deep Background with Noah Feldman

Behind every news headline, there’s another, deeper story. It’s a story about power. In Deep Backgro 
Social links
Follow podcast
Recent clips
Browse 160 clip(s)