-
Notifications
You must be signed in to change notification settings - Fork 0
/
Bayes_COVID_self-tests.html
96 lines (87 loc) · 10 KB
/
Bayes_COVID_self-tests.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
<!DOCTYPE html>
<html lang="en">
<head>
<title>What Bayes' Theorem and COVID-19 self-tests can tell you</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<style>
th{
border: 1px solid black;
background-color: #FFDD99;
}
td{
border: 1px solid grey;
}
</style>
</head>
<body>
<h2>What Bayes' Theorem and COVID-19 self-tests can tell you</h2>
<p>The result of a self test can be regarded as additional information, given an initial probability of having COVID. Bayes' theorem is all about adjusting our estimates based on the introduction of new information. All diagnostic tests can be interpreted using Bayes' theorem, and COVID self-tests are no exception.
</p>
<p>We use Bayesian reasoning all the time. For example: the train I take to work is normally on time, but runs late about 10% of the time. If I am running a little late, I might decide to hurry rather than to wait for the next train, because I know there's a chance the train will also be late. I get to the station and I see that there are still a lot of people on the platform. Using this additional information, I adjust my estimate: instead of thinking that there's a <em>small chance</em> that the train is late, I now believe that the train is <em>probably</em> late.</p>
<p>Calculating exactly how much new information changes the probability is a little trickier, but we can get some reasonable estimates with simple math.</p>
<p>The easiest way to understand Bayes is to think about the Monte Hall problem. There are 3 doors. Behind one is a new car. Behind the other two doors are goats. You are asked to pick one door; you pick door A. There is a 1/3 chance that you have picked correctly. Monty will open one of the other two doors: it has a goat. (It will always have a goat - Monty won't open the door with the car.) Do you now stick with your original guess or change to door C? The answer is that you should change to door C, because Monty will never open the door that has the car behind it, so you now know that of the two remaining doors, one of them has the car. There was only a 1/3 chance that you picked it on the first try, which means there is now a 2/3 chance that door C has the car.<br>
<a href = "https://brilliant.org/wiki/monty-hall-problem/" >https://brilliant.org/wiki/monty-hall-problem/</a>
</p>
<p>Let's bring this logic back to COVID testing.
</p>
<p>Let's say that the current prevalence of COVID-19 is 1%; that is, 1% of people currently in your country have COVID-19. That means there's a 1% probability that any random person has COVID, including yourself. You take a self-test; it is positive. What is the probability that you actually have COVID?
</p>
<p>We cam calculate a rough probability using simple math: If the test is 99% accurate, then if we randomly test 100 people we would expect to get 1 who has a false positive test. Our prevalence is 1% so we also expect to get 1 with a true positive test. That means we have two people with a positive test: one false positive and one true positive. Our experienced accuracy is 50%. In other words, there's about an equal chance of the test being right or wrong.
</p>
<p>However, these numbers change with the prevalence. If 2% of people currently have COVID, then out of 100 people, you'd expect 2 true positives and 1 false positive, thus a 66% or 2:1 chance that the test is right. In Bayesian terms, we have changed the <em>prior probability</em>.</p>
<p>Other things can change our prior as well - for example, the numbers from the Dutch government indicate that about 20% of people who get tested without a self-test turn out to have COVID. (Based on <a href = "https://www.rivm.nl/coronavirus-covid-19/weekcijfers">https://www.rivm.nl/coronavirus-covid-19/weekcijfers</a>, - note that this is the weekly report so the numbers will be different from week to week.) If we guess that these people are getting tested because they have symptoms, then we can say that this week, for <em>symptomatic</em> people, the prior probability is 20% or 0.2.
<p>Using our napkin-math, this means we'd expect 20 people out of 100 to be true positives and 1 out of 100 to be a false positive, meaning there's about a 95% chance that the positive test is correct if you have symptoms.</p>
<p>Calculating the actual numbers using Bayes theorem is a little more complicated. The easiest way to calculate this is to use an online calculator, such as this one:<br>
<a href ="http://medcalc.com/bayes.html">http://medcalc.com/bayes.html</a><br>
(the calculations are explained at the bottom of the page).
</p>
<p>To make the calculation, we need the current prevalence and/or current positive test rate in symptomatic people, and the <a href="https://en.wikipedia.org/wiki/Sensitivity_and_specificity">sensitivity and specificity</a> of our self-test. Broadly speaking, sensitivity is a measure of how likely the test is to <em>catch positive cases</em> and specificity is the <q>true positive rate</q>, or how likely it is that someone who has a positive test actually has the condition. The sensitivity and specificity vary, let's start with the numbers from the <em>Nederlands Tijdschrift voor Geneeskunde</em> (Dutch Medical Journal): they say the self-tests are 85% sensitive and 99% specific:<br>
<a href = "https://www.ntvg.nl/artikelen/het-gebruik-van-de-coronazelftest-perspectief">https://www.ntvg.nl/artikelen/het-gebruik-van-de-coronazelftest-perspectief</a>
</p>
<p>If we enter the values of 0.01 (indicating a 1% prevalence), 0.85 sensitivity and 0.99 specificity, we see that the probability of having COVID if you have a positive self-test is 0.462, or a little less than half.<br>
To run through the calculation by hand:<br>
Post-test Probability = Post-test Odds / (1 + Post-test Odds) <br>
Post-test Odds = Pre-test Odds x Likelihood Ratio <br>
Pre-test Odds = Pre-test Prob / (1 - Pre-test Prob) <br>
Positive Likelihood Ratio = SENS / (1-SPEC)<br>
Thus:<br>
Positive Likelihood Ratio = .85 / (1 - .99) = 85<br>
Pre-test odds = .01 / (1 - .01) = 0.010101010101<br>
Post-test odds = 0.010101010101 x 85 = 0.858585858585<br>
Post-test probability = 0.858585858585 / (1 + 0.858585858585) = 0.461956521739
</p>
<p>However, if you have symptoms, it is much more likely that you have COVID. Remember, this changes our prior from 1% to 20%, or 0.01 to 0.2. If we enter 0.2 for the prior in our calculator, we get a probability of 0.9551. Here our napkin math was pretty good: if you have symptoms and a positive test, there's a 95% chance that you actually have COVID.
</p>
<p>We can also make calculations for negative tests:<br>
No symptoms + negative self-test = 99.8% chance that you do not have COVID (the "Negative Predictive Value" in the outcome of the calculator).<br>
Symptoms + negative self-test = 95% chance that you do not have COVID ...but still a 5% chance that you do, which is still 5x higher than the 1% for any random asymptomatic person with no test. This is why you should stay home if you are symptomatic, even if you have a negative self-test.
</p>
<p>Arguably, the negative self-test didn't give us much additional information: we were already 99% sure we didn't have COVID, now we are only 0.8% more sure. Still, that extra reassurance can mean a lot in practice: if you teach a class of 50 students 1x/week without self tests, odds are that you will be exposed to COVID 25 times in the course of a year. If everyone is doing self-tests, that goes down to 5 times.</p>
<p>It's important to keep in mind that these numbers will change dramatically with varying prevalence:
</p>
<table>
<tr><th></th><th colspan=2>Prevalence 0.5%</th><th colspan=2>Prevalence 1%</th><th colspan=2>Prevalence 2%</th></tr>
<tr><th></th><th>Has COVID</th><th>No COVID</th><th>Has COVID</th><th>No COVID</th><th>Has COVID</th><th>No COVID</th></tr>
<tr><th>Symptoms + positive test</th><td>93%</td><td>7%</td><td>95%</td><td>5%</td><td>97%</td><td>3%</td></tr>
<tr><th>Symptoms + negative test</th><td>3%</td><td>97%</td><td>5%</td><td>95%</td><td>6%</td><td>94%</td></tr>
<tr><th>No symptoms + positive test</th><td>30%</td><td>70%</td><td>46%</td><td>54%</td><td>63%</td><td>47%</td></tr>
<tr><th>No symptoms + negative test</th><td>0.1%</td><td>99.9%</td><td>0.2%</td><td>99.8%</td><td>0.4%</td><td>99.6%</td></tr>
</table>
(Based on historical data, I estimated the prevalence to be 15% in symptomatic people when the national prevalence is 0.5%. For a prevalence of 2%, I used a prevalence of 30% in symptomatic people.)
<p>As you might imagine, these numbers will also change with the accuracy of the test. The accuracy of COVID-19 self tests varies widely depending on the manufacturer.<br>
<a href="https://bmcinfectdis.biomedcentral.com/articles/10.1186/s12879-021-06528-3">https://bmcinfectdis.biomedcentral.com/articles/10.1186/s12879-021-06528-3</a>
Let's grab the low end of the numbers from this article: a sensitivity of 79% and a specificity of 92%.<br>
<a href="https://www.mininggazette.com/news/features/2021/12/sensitivity-and-specificity-taking-the-measure-of-at-home-tests/ ">https://www.mininggazette.com/news/features/2021/12/sensitivity-and-specificity-taking-the-measure-of-at-home-tests/ </a>
</p>
<table>
<tr><th></th><th>Has COVID</th><th>No COVID</th></tr>
<tr><th>Symptoms + positive test</th><td>71%</td><td>29%</td></tr>
<tr><th>Symptoms + negative test</th><td>5%</td><td>95%</td></tr>
<tr><th>No symptoms + positive test</th><td>9%</td><td>91%</td></tr>
<tr><th>No symptoms + negative test</th><td>0.3%</td><td>99.7%</td></tr>
</table>
(Using a prevalence of 1% in the general population and 20% in symptomatic people.)
<p>As we can see, even with a kind of lousy test, a negative test + no symptoms is reasonably reassuring. A negative test with symptoms is somewhat reassuring too, but who wants a 5% chance of giving Grandma something horrible? If you have symptoms, it's better to get a PCR test (which is much more accurate). Similarly, if you have no symptoms, a positive test is still worrisome enough to warrent getting a PCR test, especially if the tests on the market in your area are of reasonable quality.
</p>
</body>
</html>