online.wsj.com
THE NUMBERS GUY
By CARL BIALIK
This is the first installment of The Numbers Guy, which will examine numbers and statistics in the news, business, politics and health. Some numbers are flat-out wrong, misleading or biased. Others are valid and useful, helping us to make informed decisions. As The Numbers Guy, I will try to sort through which numbers to trust, question or discard altogether. And I'd like to hear from you at numbersguy@wsj.com.
Just How Deadly Is Bird Flu? It Depends on Whom You Ask
January 13, 2005
This is the first installment of The Numbers Guy, a new column on the way numbers and statistics are used – and abused – in the news, business and politics. I welcome your questions and comments, and will post and respond to your letters soon. Write to me at numbersguy@wsj.com.
The World Health Organization has a big problem: It needs to alert the public to the dangers of a virus that has killed very few people, yet could, in some scenarios, devastate nations across the globe.
So, the group's doctors and scientists have lately been forecasting truly alarming numbers from the so-called Asian bird flu -- up to 100 million deaths. One researcher has gone much further, suggesting the toll could be up to a billion people.
But projecting death counts from such a bug isn't just an inexact science; it's more like educated guesswork. The truth is, scientists don't know the rates at which this hypothetical flu -- derived from a bird flu that so far in Asia doesn't spread well from human to human -- could infect and kill. They base guesses on prior flu pandemics, but there's no way to quantify how much better we're prepared in 2005, thanks to improved vaccine production and antiviral medication, than we were in 1968, when the last flu pandemic struck.
Then again, the next pandemic could be worse than that relatively mild one, and even worse than the deadliest of the past century, in 1918, which killed at least 20 million people at a time when the world had a smaller population which traveled less.
The most responsible answer, then, to the question of how many people the flu will kill is, "We don't know." But big numbers get headlines while honest uncertainty usually doesn't. And the WHO has been sharing big numbers, like two million to seven million people dead world-wide. At a press conference in Hong Kong two months ago, one official went further, saying this hypothetical pandemic could kill as many as 100 million people. The WHO always cautions that these aren't sure numbers, but the group shouldn't be surprised that the press often skips the complexity.
The 100 million figure was reported widely, including in the New York Times, The Wall Street Journal Online, CNN, Newsweek and the U.K.'s Observer; but without much caution about how arbitrary it is.
At issue is H5N1, a new strain of bird flu that so far has killed a few dozen people in Asia -- nearly three quarters of the number of people known to have been infected. Scientists fear the virus will spontaneously mutate or swap parts of its genetic code with another virus, and thereby become more transmissible. They hope that in doing so, it also will become less lethal. But there is no way to know. (Dutch researchers recently found that a different strain of bird flu had spread widely among humans in 2003, but killed few of those infected.)
Henry Niman, who studies viruses and criticizes the WHO for being underprepared, says that in the true worst-case scenario, one billion people could die. That figure was reported in the New York Times. Dr. Niman's reasoning: The current mortality rate among those known to have been infected is nearly 75% and the WHO is estimating that one billion to two billion could be infected world-wide. But Dr. Niman, a medical researcher in Pittsburgh whose company, Recombinomics Inc. seeks to develop vaccines for viruses, adds, "There are a lot of variables. The concept that you can't really put a number on [the estimated death toll] at this time, is certainly valid."
This all matters a great deal to companies deciding whether to develop a flu vaccine, to hospitals making worst-case plans, to scientists seeking research grants and even to public citizens who have never heard of bird flu. Steep death-toll projections are very powerful numbers that can drive action. If the numbers are accurate and shared responsibly, they are a force for public good. If they are misleading or hide uncertainty, they can lead to wasted resources and poor decisions.
Martin Meltzer, a senior health economist with the office of surveillance in the National Center for Infectious Diseases at the Centers for Disease Control and Prevention in Atlanta, who made the calculations for the WHO, based his model on the 1968 flu pandemic for a 1999 paper estimating the potential death toll in the U.S. (The model is outlined in an online appendix.)
Here's how Dr. Meltzer and his colleagues arrived at the upper estimate of seven million dead: They estimated the rate of infection in the U.S. could be between 15% and 35% (in somewhat circular logic, they chose a minimum of 15% simply because anything lower wouldn't be much of a pandemic). Then they assigned the expected cases to three age groups, based on U.S. Census data and two models of how infection rates would distribute among age groups, based on prior flu pandemics. (In one model, more people under 20 are infected). Next they used estimates from prior studies to calculate those members of each age group who were at high risk of suffering adverse effects from the flu -- generally people with certain pre-existing medical conditions.
To calculate deaths, they used rates from the 1968 flu pandemic. They used different death rates for different age groups and number of risk factors, based on a study of outcomes for Oregon patients who contracted the prior flu. The estimate for deaths in the U.S. ranged from 89,000 to 207,000, depending on the rate of infection. Then last year Dr. Meltzer, in an unpublished calculation for the WHO, extrapolated the U.S. numbers to the world, using estimates of population, age range and risk factors.
Of course, there is plenty of uncertainty in this model. The distribution of risk factors is not known, for instance, in many parts of the world. And the effects of different risk factors on likelihood of death also are unclear. Just one example of the guesswork involved: "Data regarding the death rate among 0-19 years old with high risk conditions are scarce." So the authors use a model of how pneumonia death rates among people age 44 and under are affected by risk factors -- a different disease and different age group. Also, historical numbers may be the best ones we have, but they come from a different virus, a different era and a different medical system. (Then there's the question of how we know exactly how many died in prior pandemics; perhaps that's a topic for a future column.)
These calculations aren't driven by any specific aspect of H5N1, but that virus's lethality is fueling a growing fear that we are due for a pandemic. Historically, we have had three or four each century, yet the last one to strike came in 1968. But this is essentially the gambler's fallacy of assuming that 10 straight bad bets mean a winning wager is imminent.
The research itself isn't at fault; striving to make estimates based on imperfect data is typical in science. What's questionable is the way the estimates have been used by public-health officials.
Dr. Meltzer acknowledges that his initial paper was based on many assumptions, and calls the extrapolation of U.S. numbers to the world "rough estimates." He urges people to create their own numbers by plugging different starting figures for things like infection rate and mortality into his online software (here): "My estimates may not be correct, but the public should understand the quality of the estimates." Klaus Stohr, a WHO flu virologist, adds, "There are a large variety of different models, all equally right and equally wrong."
Nonetheless, both men defend the WHO's decision to share numbers. "We have come up with this information because we feel it is important to share with the world," Dr. Stohr says. "There are a number of decisions governments and regional authorities can take that can have an impact on pandemic preparedness."
Dr. Meltzer points out that, beyond death tolls, his study also estimated that hundreds of millions could become ill, which could overwhelm hospitals. Thus it's important to prepare the public for rationing and triage, and spark debate about how to allocate scarce resources, should a pandemic strike. "One can cry wolf too many times," Dr. Meltzer says. "This isn't it. I don't think we've cried wolf enough about pandemic flu. The problem is, how do you sustain interest?"
But a faulty number can be worse than no number, especially when it can evoke fear in the general population without advising any evasive actions people can take. Richard Schabas, Ontario's former chief medical officer, contrasts influenza warnings with those about tobacco. "Scaring people about avian influenza accomplishes nothing, because we're not asking people to do anything about it. I don't see the point of that," says Dr. Schabas, now chief of staff at York Central Hospital in Richmond, Ontario, who wrote a column in the Globe and Mail early last year criticizing bird-flu warnings.
Striking the right balance of fear is a delicate public-relations art. In October 2003, New York University clinical associate professor of medicine Marc Siegel thought more fear about the flu would be a good thing. He wrote in a USA Today column, "To contain it, we need more vaccinations, more isolation of those who are sick, more hand washing -- and even, for once, a little more fear."
But since then, thanks to numerous WHO warnings and the scare about vaccine shortages for the more-common strain of flu this winter, Dr. Siegel thinks the balance has shifted. "Flu has now become the opposite of what it was then," he says. "Now we probably have too much fear of the flu."
Dr. Meltzer's desire for robust public examination of his numbers, and debate about public health is admirable. But realistically, since neither public funds and people's attention are infinite, too much fear about the flu can distract from other, less-novel but more-dangerous diseases.
Lee Reichman, executive director of the New Jersey Medical School's National Tuberculosis Center in Newark, shows a slide at seminars comparing deaths world-wide in 2003 due to TB (between two and three million annually) to those due to SARS, Ebola, the West Nile virus and other highly publicized diseases -- a total of just barely 1,000.
Now he says he may add bird flu to the slide. "Someone in the U.S. is far more likely to die of TB than of bird flu," says Dr. Reichman. "But there's a perception: [Tuberculosis] an old disease. It's under control."
I welcome your questions and comments. You can e-mail me at numbersguy@wsj.com. I'll respond to your letters in future columns.
EK!!! |
