Tim, my thought's on Richard Miniter's piece are as follows:
1. He states: "The survey teams did not ask for death certificates, trusting Iraqis to tell them who had died"
The study states:
Within clusters, an attempt was made to confirm at least two reported non-infant deaths by asking to see the death certificate. Interviewers were initially reluctant to ask to see death certificates because this might have implied they did not believe the respondents, perhaps triggering violence. Thus, a compromise was reached for which interviewers would attempt to confirm at least two deaths per cluster. Confirmation was sought to ensure that a large fraction of the reported deaths were not fabrications. Death certificates usually did not exist for infant deaths and asking for such certificates would probably inflate the fraction of respondents who could not confirm reported deaths. The death certificates were requested at the end of the interview so that respondents did not know that confirmation would be sought as they reported deaths.
Confirmation of deaths was attempted at 78 households and death certificates were provided in 63 of them.
2. Miniter states: The survey team asked how many deaths per "house-hold" were known to Iraqis. Americans think of a household as a nuclear family. Arabs think of a household as including the extended family, which can easily number in the hundreds. As a result, one hears about the deaths of distant in-laws and third cousins, even though many of these "relatives" might live far from the surveyed area.
The study states:
Households were informed about the purpose of the survey, were assured that their name would not be recorded, and told that there would be no benefits or penalties for refusing or agreeing to participate. We defined households as a group of people living together and sleeping under the same roof(s). If multiple families were living in the same building, they were regarded as one household unless they had separate entrances onto the street. If the household agreed to be interviewed, the interviewees were asked for the age and sex of every current household member. Respondents were also asked to describe the composition of their household on Jan 1, 2002, and asked about any births, deaths, or visitors who stayed in the household for more than 2 months. Periods of visitation, and individual periods of residence since a birth or before a death, were recorded to the nearest month. Interviewers asked about any discrepancies between the 2002 and 2004 household compositions not accounted for by reported births and deaths. When deaths occurred, the date, cause, and circumstances of violent deaths were recorded. When violent deaths were attributed to a faction in the conflict or to criminal forces, no further investigation into the death was made to respect the privacy of the family and for the safety of the interviewers. The deceased had to be living in the household at the time of death and for more than 2 months before to be considered a household death.
3. Miniter further states: Another bias is something statisticians call "cluster sampling." Two-thirds of the deaths reported in the Roberts study were located in one cluster in Fallujah. Journalist Michael Fumento comments: "That's it, game over, report worthless." Fallujah, the scene of heavy fighting for more than two years, is simply not representative of Iraq as a whole.
The study states:
If we exclude the Falluja data, the risk of death is 1·5-fold (1·1–2·3) higher after the invasion. We estimate that 98000 more deaths than expected (8000–194000) happened after the invasion outside of Falluja and far more if the outlier Falluja cluster is included
Miniter was wrong about (a) death certificates, (b) the composition of a "houehold" and (c) the fact that Falluja results were not included in the analysis. Given those three major errors it is not likely that Miniter read the Lancet study he supposedly "debunks" As for his comments on the methodology of the study I'll defer to epidemiologists.
from: chronicle.com
On its merits, the study should have received more prominent play. Public-health professionals have uniformly praised the paper for its correct methods and notable results.
"Les has used, and consistently uses, the best possible methodology," says Bradley A. Woodruff, a medical epidemiologist at the U.S. Centers for Disease Control and Prevention.
Indeed, the United Nations and the State Department have cited mortality numbers compiled by Mr. Roberts on previous conflicts as fact -- and have acted on those results.
and
The reception of the Iraqi mortality study by scientists has been far friendlier than by the news media.
Scientists say the size of the survey was adequate for extrapolation to the entire country. "That's a classical sample size," says Michael J. Toole, head of the Center for International Health at the Burnet Institute, an Australian research organization. Researchers typically conduct surveys in 30 neighborhoods, so the Iraq study's total of 33 strengthens its conclusions. "I just don't see any evidence of significant exaggeration," he says.
David R. Meddings, a medical officer with the Department of Injuries and Violence Prevention at the World Health Organization, says any such survey will have uncertainty because of extrapolation based on small numbers, and because of the possibility that people gave incorrect information about deaths in their households.
"I don't think the authors ignored that or understated" those factors, he says. "Those cautions I don't believe should be applied any more or any less stringently to a study that looks at a politically sensitive conflict than to a study that looks at a pill for heart disease."
The uncertainty leads to the breadth of the so-called 95-percent confidence interval -- in other words, the 95-percent chance that the number of deaths in Iraq resulting from military activities is between 8,000 and 194,000.
Critics like the Slate writer seized on that range, says Dr. Woodruff, the government epidemiologist. "They thought, 'Well, it's just as likely to be 18,000 as 100,000.' That's not true at all," he says. "The further you get away from 100,000, the probability that the number is true gets much smaller."
from: time.com
Richard Peto, professor of medical statistics and epidemiology at the University of Oxford, and other experts have called the methodology sound.
from: economist.com
Nan Laird, a professor of biostatistics at the Harvard School of Public Health, who was not involved with the study, says that she believes both the analysis and the data-gathering techniques used by Dr Roberts to be sound. She points out the possibility of “recall bias”—people may have reported more deaths more recently because they did not recall earlier ones. However, because most people do not forget about the death of a family member, she thinks that this effect, if present, would be small. Arthur Dempster, also a professor of statistics at Harvard, though in a different department from Dr Laird, agrees that the methodology in both design and analysis is at the standard professional level
I am not personally qualified to comment on the methodology but it seems to me that in general the epidemiology professionals attest to the soundness of the study. It is recognized that the confidence interval is large and that further study would be required to increase the precision of the estimate.
However, I would point out that since heavily damaged areas like Ramadi, Najaf and Tallafar were not selected and Falluja was excluded it may be that the study skewed the mortality rate downward.
|
