Flubar: When the CDC meets the flu
This time I had help, because alpacas are science nerds.
Things were going well at the Palo Verde Alpaca Ranch (the only ranch in the world with glow-in-the-dark alpacas thanks to the nearby nuclear power plant) when influenza season rolled around. Today the alpacas are all feeling woozy, which I personally think is related to the sketchy-looking enchiladas they had for lunch.
The enchiladas came from the kitchen at the Chicken House Solar Plant and the alpacas are convinced it’s some kind of bird flu instead, so they have embarked on their own independent research into influenza. (Bad alpacas! CNN says not to think for yourselves!) Although herd animals by nature, they are sometimes capable of independent thought.
Alpacas are also gluttons for punishment so they started with the CDC. Since the CDC doesn’t keep data on alpacas, they decided to look at humans instead. But they immediately discovered that they need to know what ICD codes are before they can make sense of anything.
ICD codes: Probably the cause of several suicides by medical billing agents every year.
ICD stands for International Classification of Diseases. Published by the WHO1, the CDC is currently using the tenth version (ICD-10-CM) which includes around 70,000 codes for different diseases and injuries.
These can be surprisingly specific. For example, does the patient have:
J09.X2 Influenza due to identified novel influenza A virus with other respiratory manifestations
J09.X3 Influenza due to identified novel influenza A virus with gastrointestinal manifestations
So wheezy baby, or gassy baby?
Gastrointestinal manifestations caused by under-cooked enchiladas does not have its own code.
There are lots of codes for influenza and pneumonia, because they can be caused by lots of things. Anyway, someone combed through this mess and compiled a report, using statistical models, of how many people they think had influenza and died.
The alpacas, having both literally and figuratively consumed this really, really thick handbook of ICD codes, moved on to:
Estimates of deaths associated with seasonal influenza --- United States, 1976-2007 (CDC 20102)
This data was published in something called the Morbidity and Mortality Weekly Report, which is almost certainly one of the most depressing titles ever used for a government report. It’s published along with Kittens Who Didn’t Manage to Just Hang in There.
Note that these are all “influenza-associated” deaths. Determining actual cause of death is tricky, so they just tell us how many people they think had the influenza virus.
Estimates are provided for three age groups (<19 years, 19–64 years, and ≥65 years) and for two categories of underlying cause of death codes: 1) pneumonia and influenza causes and 2) respiratory and circulatory causes.
The first group includes all types of pneumonia (with and without a diagnosis of influenza) and influenza, from various causes.
The second group (respiratory and circulatory) includes things like hypertension, angina, myocardial infarction, coronary artery aneurysm, and many more.
(ICD-10 is very detailed. There are also codes for things like falls, which might be how that kitten report was compiled.)
What they are trying to do is figure out how many of the deaths in these categories involved people who also had influenza, because it’s often not listed on death certificates. And of course:
Many deaths associated with influenza infections occur from secondary infections such as bacterial pneumonia or complications of chronic conditions such as congestive heart failure and chronic obstructive pulmonary disease.
The report includes two tables, giving year-by-year numbers for influenza-associated deaths. The tables are:
Estimated number of annual influenza-associated deaths with underlying pneumonia and influenza causes.
Estimated number of annual influenza-associated deaths with underlying respiratory and circulatory causes. Includes cause of death codes for influenza and pneumonia.
So the second group includes the totals from the first group. They give averages for each category, and rather than insert the whole chart the alpacas have summarized those averages in a little table:
For animals without fingers, alpacas are surprisingly good with computers. The report also gives these as a fraction of the population. For example:
In the younger than 19 age group:
for pneumonia and influenza only, they estimate 0.1 deaths per 100,000 people (so about 1 death per million people).
for all respiratory, circulatory, pneumonia and influenza, they estimate 0.2 deaths per 100,000 people (so about 2 deaths per million people).
This is their estimate for the number of people <19 years old who died, and who also probably had the flu (which may or may not have been the cause of death).
The alpacas are careful readers and therefore noticed that these are all described as estimates. Mathematical models are used to make these estimates.
Estimating influenza-associated deaths in the United States. (Thompson et al. 20093)
This is one of the references given in the report for the statistical techniques. Thompson and pals tell us why models are used, rather than just the totals for all deaths coded for influenza.
For several reasons, the number of influenza-related deaths cannot be determined solely by reports of influenza-coded deaths. First, most adult patients with symptoms consistent with influenza infection are not tested for influenza. Those who are generally receive rapid tests of only modest sensitivity. In addition, many influenza-associated deaths occur one or two weeks after the initial infection (when viral shedding has ended), either because of secondary bacterial infections or because the influenza has exacerbated chronic illnesses (e.g., congestive heart failure or chronic obstructive pulmonary disease).
So to summarize:
Most people don’t get tested for the influenza virus
Some of the tests aren’t very good at detecting influenza
Many deaths occur weeks after initial infection and…
Many deaths are due to secondary infections
Well, we already knew about the secondary infections (see 1918 flu epidemic), and recall from our discussion of pneumonia that the underlying cause of the pneumonia is only determined for 30%-40% of cases. Alpacas are really bad at math but they assure me that 40% is less than half.
The alpacas, who are also quite excitable animals, highlighted this part for us:
In the spring of 1968, the number of deaths for which pneumonia was listed as the underlying cause sharply decreased concurrently with a sharp increase in the number of deaths for which influenza was listed as the underlying cause. These changes in mortality coding practices occurred after the emergence of the new pandemic strain was detected in Asia and publicly announced but before the 1968 pandemic influenza A(H3N2) virus actually arrived in the United States. (emphasis added by a herd of alpacas)
Once we started warning doctors about the emergence of a new pandemic influenza strain, they started listing it as the cause of death (instead of pneumonia) even though that strain hadn’t actually reached the U.S.
I wonder if that little nugget has any relevance today?
What about those flu vaccines we take every year? Don’t they make us superhuman?
Not really. The CDC is very excited about them, other people less so. Here is one of the papers that is very much on the optimistic side.
The efficacy of influenza vaccine in elderly persons. A meta-analysis and review of the literature. (Gross et al. 19954)
This paper summarized existing studies, but unfortunately there weren’t many randomized controlled trials – they only referenced one.
Only cohort observational studies with mortality assessment were included in the meta-analysis. In addition, 3 recent case-control studies, 2 cost-effectiveness studies, and 1 randomized, double-blind, placebo-controlled trial were reviewed.
What did they determine?
Vaccine efficacy in the case-control studies ranged from 32% to 45% for preventing hospitalization for pneumonia, from 31% to 65% for preventing hospital deaths from pneumonia and influenza, from 43% to 50% for preventing hospital deaths from all respiratory conditions, and from 27% to 30% for preventing deaths from all causes. The randomized, double-blind, placebo-controlled trial showed a 50% or greater reduction in influenza-related illness.
Did you see the part about flu vaccines preventing 27% to 30% of deaths FROM ALL CAUSES? That is one hell of a claim - it implies that giving annual flu vaccines to all elderly people would eliminate almost 1/3 of their deaths from all diseases.
An effect that large should be easy to detect in a randomized controlled trial.
Despite the paucity of randomized trials, many studies confirm that influenza vaccine reduces the risks for pneumonia, hospitalization, and death in elderly persons during an influenza epidemic if the vaccine strain is identical or similar to the epidemic strain.
“Paucity” means “very few.” - We don’t conduct randomized controlled trials on these vaccines, but they work great. Trust us. Can we interest you in a 1982 Pontiac? High mileage, but the interior is pure Corinthian leather. The previous owner was a gnome with one too many DUIs.
Me and the alpacas aren’t the only skeptical ones.
Impact of influenza vaccination on seasonal mortality in the US elderly population. (Simonsen et al. 20055)
The NIH funded this study:
This study was funded by Unmet Needs grant NVPO-01-N55 from the National Vaccine Program Office, Washington, DC.
They were looking at the difference between the expectation and the reality of flu vaccines. From 1980 to 2001, vaccination rates among the elderly increased but so did mortality.
Observational studies report that influenza vaccination reduces winter mortality risk from any cause by 50% among the elderly. Influenza vaccination coverage among elderly persons (> or =65 years) in the United States increased from between 15% and 20% before 1980 to 65% in 2001. Unexpectedly, estimates of influenza-related mortality in this age group also increased during this period. We tried to reconcile these conflicting findings by adjusting excess mortality estimates for aging and increased circulation of influenza A(H3N2) viruses.
Okay, so vaccination rates increased but so did the number of influenza-associated deaths. Oops.
The authors note that mortality did improve after the 1968 pandemic, but doubt it was due to vaccination. They think it was due to acquired immunity, since after 1980 vaccination rates don’t correlate with mortality.
We attribute the decline in influenza-related mortality among people aged 65 to 74 years in the decade after the 1968 pandemic to the acquisition of immunity to the emerging A(H3N2) virus. We could not correlate increasing vaccination coverage after 1980 with declining mortality rates in any age group. Because fewer than 10% of all winter deaths were attributable to influenza in any season, we conclude that observational studies substantially overestimate vaccination benefit.
Did you get that? They couldn’t find a correlation between how many people are vaccinated and how many people die, for any age group.
Alpacas generally don’t like people they haven’t met but they are big fans of a guy named Peter Doshi. He’s been causing trouble for quite some time by pointing out the discrepancy between what we claim for flu vaccines and what we actually get.
Influenza: marketing vaccine by marketing disease (Doshi 20136)
Remember those impressive sounding numbers from earlier? Some studies claim as high as 50% reduction in all cause mortality. Doshi points out:
If true, these statistics indicate that influenza vaccines can save more lives than any other single licensed medicine on the planet. Perhaps there is a reason CDC does not shout this from the rooftop: it’s too good to be true. Since at least 2005, non-CDC researchers have pointed out the seeming impossibility that influenza vaccines could be preventing 50% of all deaths from all causes when influenza is estimated to only cause around 5% of all wintertime deaths.
He goes on to point out that it’s very hard to account for the “healthy-user effect” where healthy people are also more likely to get vaccinated than unhealthy people. How to resolve this? Clearly some randomized controlled trials (RCTs) would help.
But no one seems to be interested.
Perhaps most perplexing is officials’ lack of interest in the absence of good quality evidence. Anthony Fauci, director of the US National Institute of Allergy and Infectious Diseases, told the Atlantic that it “would be unethical” to do a placebo controlled study of influenza vaccine in older people. The reason? Placebo recipients would be deprived of influenza vaccines—that is, the standard of care, thanks to CDC guidelines.
Hey, I’ve heard of that guy Fauci. He spends a lot of time on the news talking about people injecting something or other. I knew a kid like that in high school, but he’s in prison now.
The CDC went ahead and made flu vaccines the standard of care without the RCTs, and now it’s unethical to do a trial. Nice little circular loop there, Tony. If you can get your drug listed as “standard of care” without evidence, you then get out of the requirement to provide evidence.
Doshi throws the flu vaccine cheerleaders at the CDC a bone here:
This is not to say influenza vaccines have no proven benefit. Many randomized controlled trials of influenza vaccines have been conducted in the healthy adult population, and a systematic review found that, depending on vaccine-virus strain match, vaccinating between 33 and 100 people resulted in one less case of influenza.
So one less case of the sniffles for every 33-100 people, assuming you’re vaccinating for the correct strain. But how many people were saved from something more serious than a fever and sore throat? In other words, what was the change in mortality?
No evidence exists, however, to show that this reduction in risk of symptomatic influenza for a specific population—here, among healthy adults—extrapolates into any reduced risk of serious complications from influenza such as hospitalizations or death in another population (complications largely occur among the frail, older population).
Oh. So maybe zero. Zero is the answer. And Peter Doshi isn’t alone in this opinion.
Influenza vaccination: policy versus evidence. (Jefferson 20067)
Jefferson asks a basic question:
Each year enormous effort goes into producing influenza vaccines for that specific year and delivering them to appropriate sections of the population. Is this effort justified?
Three points from the summary are relevant to our discussion today:
Evidence from systematic reviews shows that inactivated vaccines have little or no effect on the effects measured
Most studies are of poor methodological quality and the impact of confounders is high
Little comparative evidence exists on the safety of these vaccines
The author gives us some potential explanations. One big factor is that it’s hard to tell influenza from illnesses that resemble influenza.
The large gap between policy and what the data tell us (when rigorously assembled and evaluated) is surprising. The reasons for this situation are not clear and may be complex. The starting point is the potential confusion between influenza and influenza-like illness, when any case of illness resembling influenza is seen as real influenza, especially during peak periods of activity.
Remember earlier, when we found out that doctors were listing people as having a strain of influenza virus before it was even in the U.S.?
Some surveillance systems report cases of influenza-like illness as influenza without further explanation. This confusion leads to a gross overestimation of the impact of influenza, unrealistic expectations of the performance of vaccines, and spurious certainty of our ability to predict viral circulation and impact.
And so maybe we should take the advice of the CDC with a little skepticism.
The consequences are seen in the impractical advice given by public bodies on thresholds of the incidence of influenza-like illness at which influenza specific interventions (antivirals) should be used.
Jefferson is being really nice there. I’ll be less nice. The CDC wants you to vaccinate everyone (and maybe your alpacas, too) every year for the seasonal flu, but has not proved this will do us any good.
And why do we need to get re-vaccinated every year for the same thing? Because every annual flu vaccine is actually different.
Annual flu vaccines - like a big hamster wheel of injections
The contents of the annual flu vaccine change every year. A panel at the FDA meets to approve the new concoction - here’s the notice for one of those meetings:
Vaccines and Related Biological Products Advisory Committee March 4, 2020 Meeting Announcement8
They have a standing committee that meets to discuss and recommend whether to approve different vaccines. For example:
On March 4, 2020, under Topic I, the Center for Biologics Evaluation and Research’s (CBER) VRBPAC will meet in open session to discuss and make recommendations on the selection of strains to be included in the influenza virus vaccines for the 2020 to 2021 influenza season.
They do this every year, approving changes to the influenza strains in the annual vaccine as the influenza virus evolves.
The alpacas are all hurling now. Is it the enchiladas or the CDC?
Very few people under 19 years old die with the influenza virus, but we still jab our kids with a different flu vaccine cocktail every year.
The data provided by the CDC tells us the number of “influenza-associated” deaths that occur, but actual cause of death is hard to determine. Either way, for the pre-retirement crowd it’s a small number.
It’s not clear that annual flu vaccination programs save any lives at all. Some people think so, others (including people contracted by the NIH) have concluded that there is no correlation between vaccination rates and mortality.
Unlike other drugs, which require evidence from randomized controlled trials, flu vaccines get a special pass. A guy me and the alpacas already don’t trust tells us it would be unethical to have a control group that’s not vaccinated.
Is this true? Or is it that the control group would have the same mortality as the vaccine group? We’ll never know – the manufacturers aren’t required to do those tests.
Are these vaccines safe? The CDC says so, but these are the same people who don’t require testing to approve things.
The only good news here is that the system for reporting adverse reactions from vaccines, VAERS, shows very few reports each year from annual flu vaccines.
We’ll need to look into VAERS another time - the alpacas are getting snacky from all this reading and are eyeing the enchiladas again. They never learn.
CDC document Uses of Coded Clinical Data:
Centers for Disease Control and Prevention (CDC). Estimates of deaths associated with seasonal influenza --- United States, 1976-2007. MMWR Morb Mortal Wkly Rep. 2010 Aug 27;59(33):1057-62. PMID: 20798667.
Thompson WW, Moore MR, Weintraub E, Cheng PY, Jin X, Bridges CB, Bresee JS, Shay DK. Estimating influenza-associated deaths in the United States. Am J Public Health. 2009 Oct;99 Suppl 2(Suppl 2):S225-30. doi: 10.2105/AJPH.2008.151944. PMID: 19797736; PMCID: PMC4504370.
Gross PA, Hermogenes AW, Sacks HS, Lau J, Levandowski RA. The efficacy of influenza vaccine in elderly persons. A meta-analysis and review of the literature. Ann Intern Med. 1995 Oct 1;123(7):518-27. doi: 10.7326/0003-4819-123-7-199510010-00008. PMID: 7661497.
Simonsen L, Reichert TA, Viboud C, Blackwelder WC, Taylor RJ, Miller MA. Impact of influenza vaccination on seasonal mortality in the US elderly population. Arch Intern Med. 2005 Feb 14;165(3):265-72. doi: 10.1001/archinte.165.3.265. PMID: 15710788.
Doshi P. Influenza: marketing vaccine by marketing disease BMJ 2013; 346 :f3037 doi:10.1136/bmj.f3037
Jefferson T. Influenza vaccination: policy versus evidence. BMJ. 2006;333(7574):912-915. doi:10.1136/bmj.38995.531701.80