The CDC: Worse than useless
The Director of the CDC doesn't seem to understand the implications of her own statements. Or she thinks we're all dumb. Who knows?
There appears to have been some kind of Human Resources mix up at the CDC and one of the interns was inadvertently promoted to Director.
I was planning a whole post on Number Needed to Vaccinate (NNV) but the intern, I mean Director, beat me to the punch. Commenting on the incidence of myocarditis in children post-vaccination, the intern quoted some statistics that have important implications beyond the comparison to rates of myocarditis.
According to this statement, vaccinating one million 12-17 year olds will prevent 8,000 cases and one death. Before discussing the implications of this, let’s talk about where these numbers come from.
To understand this we’ll need some background on number needed to vaccinate (NNV) and infection fatality rate (IFR).
So what is number needed to vaccinate (NNV)?
The idea of NNV (sometimes called NNT, number needed to treat) is a common measure for determining the cost/benefit ratio of a treatment. This paper gives us a quick primer:
How and why researchers use the number needed to vaccinate to inform decision making--a systematic review (Hashim et al. 20151)
The number needed to vaccinate (NNV) is used as a simple summary calculation to evaluate the possible benefits of immunization programmes in preventing and controlling communicable diseases. It is defined as the number of persons needed to vaccinate in order to prevent one outcome, and it combines both vaccine effectiveness and incidence of disease. Generally, the NNV is calculated as NNV = 1/(annual incidence of event in the unvaccinated × vaccine effectiveness (VE)). This is equivalent to the reciprocal of the annual absolute risk reduction, since the VE measures the relative risk reduction.
NNV is the number of vaccinations required to eliminate one case. In the quote above, we are told that 1,000,000 vaccinations will eliminate 8,000 cases. This means the intern is using an approximation of 125 for the NNV of this age group (125 times 8,000 equals 1,000,000). Presumably there is some rounding involved so the exact NNV may be slightly different.
So what’s the NNV from the actual studies?
This paper provides us with a summary of the clinical trial results for several different vaccines.
COVID-19 vaccine efficacy and effectiveness-the elephant (not) in the room (Olliaro et al. 20212)
Before discussing NNV, let’s look at relative risk reduction. This looks great, especially for Pfizer and Moderna.
Vaccine efficacy is generally reported as a relative risk reduction (RRR). It uses the relative risk (RR)—ie, the ratio of attack rates with and without a vaccine—which is expressed as 1–RR. Ranking by reported efficacy gives
relative risk reductions of 95% for the Pfizer–BioNTech, 94% for the Moderna–NIH, 90% for the Gamaleya, 67% for the J&J, and 67% for the AstraZeneca–Oxford vaccines.
But not everyone in the studies became infected, so relative risk reduction doesn’t tell us the whole story. To determine NNV we need to look at absolute risk reduction.
To help understand these concepts, let’s make up an example:
Imagine a virus that only infects 10 out of every 1,000 people (so 1%).
The vaccine for that virus reduces infections to 2 out of every 1,000 people (so 0.2%).
Total infections went from 10 to 2, so the relative risk reduction (RRR) is 80% (great!).
But the absolute risk of each person getting infected went from 1% to 0.2%, so the ARR is only 0.8%. Expressed as a fraction this is .008.
The NNV is therefore 1 divided by .008, or 125 (for every 125 vaccinations there is one less infection).
So if your study group is really, really large then the reduction in number of infections looks impressive, but each one of those required a lot of vaccinations.
Getting back to the paper, if we look at the absolute reduction in risk for the COVID-19 vaccines, things are not quite so rosy:
However, RRR should be seen against the background risk of being infected and becoming ill with COVID-19, which varies between populations and over time. Although the RRR considers only participants who could benefit from the vaccine, the absolute risk reduction (ARR), which is the difference between attack rates with and without a vaccine, considers the whole population. ARRs tend to be ignored because they give a much less impressive effect size than RRRs: 1.3% for the AstraZeneca–Oxford, 1.2% for the Moderna–NIH, 1.2% for the J&J, 0.93% for the Gamaleya, and 0.84% for the Pfizer–BioNTech vaccines.
Okay, so the absolute reduction is way less impressive because (just like in our example) the chance of getting infected is nowhere near 100%. So how many people need to be vaccinated to eliminate one infection?
NNVs bring a different perspective: 76 for the Moderna–NIH, 78 for the AstraZeneca–Oxford, 80 for the Gamaleya, 84 for the J&J, and 117 for the Pfizer–BioNTech vaccines. The explanation lies in the combination of vaccine efficacy and different background risks of COVID-19 across studies: 0.9% for the Pfizer–BioNTech, 1% for the Gamaleya, 1.4% for the Moderna–NIH, 1.8% for the J&J, and 1.9% for the AstraZeneca–Oxford vaccines.
The final NNV numbers change slightly for each vaccine when adjusted, and are given in the paper as 119 (Pfizer), 81 (Moderna/NIH), 108 (Gameleya), 84 (J&J), and 78 (AstraZeneca). To make things simple we’ll just use the average of these (94).
This is a little better than the 125 vaccinations per case prevented that we reverse-calculated from the CDC statement, but it looks like we’re both in the same range.
Before we move on, the paper we’re looking at (Olliaro 2021) gives us a brief warning about interpreting the data:
Unfortunately, comparing vaccines on the basis of currently available trial (interim) data is made even more difficult by disparate study protocols, including primary endpoints (such as what is considered a COVID-19
case, and when is this assessed), types of placebo, study populations, background risks of COVID-19 during the study, duration of exposure, and different definitions of populations for analyses both within and between
studies, as well as definitions of endpoints and statistical methods for efficacy.
Since we don’t know how each of these studies defined what a case is, we’ll need to accept that further calculations are only estimates. But for now let’s assume we have 94 vaccinations to prevent one infection (rather than one case, which would be a symptomatic infection).
Next, what’s the IFR? What’s the probability of a minor dying from COVID-19?
This number is low - very low. Axfors and Ioannidis compiled data from multiple countries, focused on the elderly, but they also give us data for minors.
Infection fatality rate of COVID-19 in community-dwelling populations with emphasis on the elderly: An overview (Axfors & Ioannidis 20213)
The authors have compiled data from multiple countries and U.S. states in order to estimate the infection fatality rate (IFR).
Across all countries (Figure 3), the median IFR was 0.0027%, 0.014%, 0.031%, 0.082%, 0.27%, and 0.59%, at 0-19, 20-29, 30-39, 40-49, 50-59, and 60-69 years, using data from 9, 9, 10, 9, 11, and 6 countries, respectively.
The IFR for anyone under 19 years old is .0027%. This means one in every 37,037 infections results in a fatality (.0027% as a fraction is .000027, and 1/.000027 is 37,037).
We already said the NNV for the COVID-19 vaccines is, on average, 94. So 94 vaccinations must be given to prevent one infection, but only one in every 37,037 infections will result in the death of a minor. Multiplying these, we would need to vaccinate 3,481,481 minors (<19) to prevent one death.
This is a 19 year range, but the quote from the CDC intern is about 12-17 year olds - so a little less than a third of this range. We don’t have IFR data for the 12-17 year olds as a group, so it may be slightly higher for them than for the whole 0-19 crowd. Of course these are all just estimates, and we may also be looking at the difference between infection fatality rate and case fatality rate (not every one who is infected gets sick).
But we have an idea where the intern’s numbers are coming from. It looks like a combination of the NNV (number needed to vaccinate) and the IFR (infection fatality rate) for this age group, and it’s consistent with data from published studies. Maybe we need to vaccinate 2-3 million to save one kid, but for the purpose of this discussion let’s just take the statement from the CDC as roughly correct.
The current U.S. population of the 12-17 age group is estimated at 25.2 million4. If for every 1 million vaccinated, we save one life, then the entire 12-17 vaccination program will save 25 lives.
This next part inspired by Thomas Sowell, our greatest living economist and philosopher
Thomas Sowell likes to point out that when evaluating a government program it’s important to ask what else we could have done for the same amount of money. After all, we don’t have an infinite supply of resources, and prices are really just a way of allocating those limited resources.
So how much are those 1,000,000 (or 25,000,000) vaccinations going to cost us?
Someone at Forbes5 was nice enough to make some calls and get us a few estimates. For the two dose Pfizer vaccine, the Medicare approved amount for each dose is $144 - although according to the article Medicare pays an average of 80% which is $115.20. For the two doses, the total payout should be $230.40. However, the author tells us that his billing statement says Medicare paid $49.71 for his two doses (the writer isn’t really sure why).
On the low end, we have 1,000,000 two-dose vaccinations costing $49.71 each, or $49,710,000. Or, if the vaccinations cost $230.40, a total of $230,400,000.
So somewhere between $50 million and $230 million per life saved, or between $1.2 billion and $5.7 billion to vaccinate the entire 12-17 age group of 25 million kids, saving 25 lives. Of course if we quadruple boost everyone then these numbers double.
The key question here isn’t really “are those kids lives worth $50 million (or more) each?” If we had infinite resources, their lives would be worth infinite money. But we don’t have infinite resources.
The CDC intern is pushing to spend several billion dollars to save a handful of lives, while ignoring other issues of child mortality. An economist would ask a better question: “Can we save more than one life for $50 to $200 million?”
Leukemia is a disease too
Let’s take an extreme example. How many new cases of Leukemia do we see each year?
Acute Lymphocytic Leukemia (Puckett & Chan 20226)
It is diagnosed in about 4000 people in the United States each year with the majority being under the age of 18. It is the most common malignancy of childhood.
Treatments costs will vary a lot, but we can at least get an estimate. We’re not trying to be precise, we just want a rough idea what the total annual cost would be for all leukemia treatments of newly diagnosed patients.
Inpatients Costs of Cancer Treatment Among Children and Young Adults with Acute Lymphoblastic Leukemia (ALL) Treated at Specialized Cancer Centers in California (Keegan et al. 20187)
Results: The mean cost for children receiving all care at SCCs vs non-SCCs was $216,439 (median=$121,039) vs $191,082 (median=$84,529) (p mean = 0.008; p median = <0.001).
Let’s just round this to about $200,000. If we multiply this by 4,000 new cases diagnosed each year, we need $800 million annually to treat all new cases of leukemia. This is a very rough estimate, but it looks like treating ALL leukemia patients each year would cost substantially less than the proposal to vaccinate just a subset of children (12-17 years old).
I understand that most of these costs are borne by insurance, but that’s just another method of allocating resources. And there are definitely cases that aren’t covered (but which we could treat for a fraction of the vaccine money) and there are definitely other childhood mortality issues (hunger, for example) that we could also put this money toward.
It takes very little creativity or life experience to think of better ways to spend billions of dollars, that will save many more children’s lives, than a mass vaccination program that benefits fewer children than fill a single school bus.
Is this your idea of morality? It’s not mine. If I give a foundation $50 million to save the lives of children, I expect them to save more than just one. The CDC, of course, is somehow less competent than any private organization (or Girl Scout troop) but even they should be able to do better.
The intern Director of the CDC has inadvertently told us something much more important than the relative risks of COVID versus vaccine induced myocarditis. She has given us solid evidence that the COVID response is really more about power and profits than health.
Hashim A, Dang V, Bolotin S, Crowcroft NS. How and why researchers use the number needed to vaccinate to inform decision making--a systematic review. Vaccine. 2015 Feb 4;33(6):753-8. doi: 10.1016/j.vaccine.2014.12.033. Epub 2014 Dec 25. PMID: 25543164.
https://pubmed.ncbi.nlm.nih.gov/25543164/
Olliaro P, Torreele E, Vaillant M. COVID-19 vaccine efficacy and effectiveness-the elephant (not) in the room. Lancet Microbe. 2021 Jul;2(7):e279-e280. doi: 10.1016/S2666-5247(21)00069-0. Epub 2021 Apr 20. PMID: 33899038; PMCID: PMC8057721.
https://pubmed.ncbi.nlm.nih.gov/33899038/
Infection fatality rate of COVID-19 in community-dwelling populations with emphasis on the elderly: An overview Cathrine Axfors, John P.A. Ioannidis medRxiv 2021.07.08.21260210; doi: https://doi.org/10.1101/2021.07.08.21260210
https://www.medrxiv.org/content/10.1101/2021.07.08.21260210v1
Population data:
https://www.childstats.gov/AMERICASCHILDREN/tables/pop1.asp
Archived copy:
https://web.archive.org/web/20220214051636/https://www.childstats.gov/AMERICASCHILDREN/tables/pop1.asp
Article on cost of vaccination:
https://www.forbes.com/sites/johnlamattina/2021/04/15/surprising-cost-for-covid-19-vaccine-administration/?sh=6ea31adc362e
Archived copy of article:
https://web.archive.org/web/20220113162302/https://gum.criteo.com/syncframe?origin=publishertag&topUrl=www.forbes.com
Puckett Y, Chan O. Acute Lymphocytic Leukemia. [Updated 2022 Jan 2]. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2022 Jan-. Available from: https://www.ncbi.nlm.nih.gov/books/NBK459149/
https://www.ncbi.nlm.nih.gov/books/NBK459149/
https://ashpublications.org/blood/article/132/Supplement%201/324/264474/Inpatients-Costs-of-Cancer-Treatment-Among
https://doi.org/10.1182/blood-2018-99-111437
She also ignored the fact that her 30-40 cases of so-called "mild" myocarditis would likely result in 6-8 deaths over the next 5 years, which totally destroys her math.