In times of war, accurate figures on the civilian death toll are almost always hard to come by. With few exceptions, demographers and epidemiologists have not applied their expertise to making rigorous, credible estimates of civilian mortality and morbidity. Sometimes, a lack of professional freedom prevents those who may be most familiar with the data – for example, analysts whose livelihoods depend on the government(s) involved in the conflict – from using their expertise for purposes that could be politically damaging.
But there are other challenges as well. Isolating the conflict’s impact from that of other interventions (e.g., economic sanctions) may be impossible. Moreover, the high-quality population data needed for credible estimates may not be available due to their “sensitive nature,” or because they never have been collected (sometimes the case in developing nations), or because refugee movements have made data obsolete. As a result, the degree of uncertainty in such estimates may be unacceptably high, making them of little real worth.
Consider the different approaches that have been used to examine the Iraq war. The Iraq Body Count aims to tally only deaths from violence during the current war by creating a data set based on media reports. If there is no double counting, and if the incidents included in the data were reported correctly, their tally represents a minimum number, because media reports may not be comprehensive.
Another approach estimates the total change in mortality that the war caused (including deaths due to the war’s direct and indirect effects) by calculating the change in the death rate from the pre-war period. This requires data upon which to base the rise in mortality, usually derived by conducting a household survey on a random sample of the population. Typically, interviewers ask the head of the household to disclose the number and demographic characteristics of pre-war household members, whether any of the people in the pre-war household had died between the pre-war period and the time of the survey, and the date of any household member’s death.
If household surveys are carried out properly, the number of excess deaths during the war can be estimated within a range of statistical uncertainty. But, when conducted during wartime, risks abound. Aside from the risks to interviewers collecting such data during a conflict, these include the selection bias of households in the sample, a lack of credible population data to which to apply the changed mortality rates, and mistaken or misleading accounts by participants.
The survey approach was used twice by a group of researchers based primarily at Johns Hopkins University, who published their results in the medical journal
The Lancet
. Their estimates have been lauded, but also questioned because of their misinterpretation of their own figures.
Secure your copy of PS Quarterly: The Climate Crucible
The newest issue of our magazine, PS Quarterly: The Climate Crucible, is here. To gain digital access to all of the magazine’s content, and receive your print copy, subscribe to PS Premium now.
Subscribe Now
For example, in a summary of the 2004 study, they wrote, “Making conservative assumptions we think that about 100,000 excess deaths, or more, have happened since the 2003 invasion of Iraq.” But the first study yielded very imprecise estimates of the number of deaths, which the authors glossed over. They should have said, “We can say with 95% certainty that between 8,000 and 194,000 excess Iraqi deaths occurred during the period.”
The group essentially repeated the study in 2006, using a larger sample size. Again, the researchers had interviewers administer what resembled a typical survey of a random sample of households. Appropriately, at the end of the article in
The Lancet
, the authors discuss issues that may have resulted in a sample that in fact did not meet the “random” threshold.
Problems with interpretation also plagued that second effort. The authors used crude death rates (CDR’s), which reflect the number of deaths per thousand people, in explaining the rise in mortality. But demographers rarely use CDR’s, thinking instead in terms of age- and sex-specific mortality rates, usually summarized as “life expectancy.” That being said, the group reported that the CDR increased from 5.5 per thousand people in 2002 to 13.3 per thousand in the post-invasion period (March 2003-March 2006).
To put the pre-invasion figure in perspective, consider United Nations Population Division figures, which are generally considered to be of very high quality. The UN estimates that Iraq’s pre-invasion CDR was 10 per thousand, not the 5 per thousand estimated from the two studies. Comparing internationally, the UN reports that Iran’s CDR in the 2000-2005 period was 5.3 per thousand. Prior to the war, most observers thought that the situation in Iraq was considerably worse than in Iran.
So the pre-war CDR that the two
Lancet
studies yield seems too low. It may not be wrong, but the authors should provide a credible explanation of why their pre-war CDR is nearly half that of the UN Population Division. If the pre-war mortality rate was too low and/or if the population estimates were too high – because, for example, they ignored outflows of refugees from Iraq – the resulting estimates of the number of Iraqi “excess deaths” would be inflated.
More fundamentally, what purpose do these numbers serve? Certainly, after the dust has settled, numbers play a role in evaluating the costs and benefits (if any), of a war. But in real time, do the numbers really add to the debate? Do they really provide us with more information than the Iraq Body Count figures provide? Do we have the appropriate context to help us interpret the numbers? The war in Iraq has been exceptionally bloody. For now, that is about all that statistics can safely tell us.
To have unlimited access to our content including in-depth commentaries, book reviews, exclusive interviews, PS OnPoint and PS The Big Picture, please subscribe
If UK Chancellor of the Exchequer Rachel Reeves tried to meet all the political demands that have been placed on her, her budget would likely result in disappointing growth and financial instability. Instead, the new government's first budget should be judged according to four longer-term criteria.
lists four criteria for assessing the chancellor's attempt to meet a long list of unrealistic demands.
At COP16 in Colombia this month, delegates from nearly 200 countries will discuss how to accelerate action to protect 30% of the planet’s land and maritime areas by 2030. Central to that goal will be unlocking more investment through public-private partnerships, de-risking mechanisms, and other innovative strategies.
points out that biodiversity loss and ecosystem damage jeopardize all other global development goals.
In times of war, accurate figures on the civilian death toll are almost always hard to come by. With few exceptions, demographers and epidemiologists have not applied their expertise to making rigorous, credible estimates of civilian mortality and morbidity. Sometimes, a lack of professional freedom prevents those who may be most familiar with the data – for example, analysts whose livelihoods depend on the government(s) involved in the conflict – from using their expertise for purposes that could be politically damaging.
But there are other challenges as well. Isolating the conflict’s impact from that of other interventions (e.g., economic sanctions) may be impossible. Moreover, the high-quality population data needed for credible estimates may not be available due to their “sensitive nature,” or because they never have been collected (sometimes the case in developing nations), or because refugee movements have made data obsolete. As a result, the degree of uncertainty in such estimates may be unacceptably high, making them of little real worth.
Consider the different approaches that have been used to examine the Iraq war. The Iraq Body Count aims to tally only deaths from violence during the current war by creating a data set based on media reports. If there is no double counting, and if the incidents included in the data were reported correctly, their tally represents a minimum number, because media reports may not be comprehensive.
Another approach estimates the total change in mortality that the war caused (including deaths due to the war’s direct and indirect effects) by calculating the change in the death rate from the pre-war period. This requires data upon which to base the rise in mortality, usually derived by conducting a household survey on a random sample of the population. Typically, interviewers ask the head of the household to disclose the number and demographic characteristics of pre-war household members, whether any of the people in the pre-war household had died between the pre-war period and the time of the survey, and the date of any household member’s death.
If household surveys are carried out properly, the number of excess deaths during the war can be estimated within a range of statistical uncertainty. But, when conducted during wartime, risks abound. Aside from the risks to interviewers collecting such data during a conflict, these include the selection bias of households in the sample, a lack of credible population data to which to apply the changed mortality rates, and mistaken or misleading accounts by participants.
The survey approach was used twice by a group of researchers based primarily at Johns Hopkins University, who published their results in the medical journal The Lancet . Their estimates have been lauded, but also questioned because of their misinterpretation of their own figures.
Secure your copy of PS Quarterly: The Climate Crucible
The newest issue of our magazine, PS Quarterly: The Climate Crucible, is here. To gain digital access to all of the magazine’s content, and receive your print copy, subscribe to PS Premium now.
Subscribe Now
For example, in a summary of the 2004 study, they wrote, “Making conservative assumptions we think that about 100,000 excess deaths, or more, have happened since the 2003 invasion of Iraq.” But the first study yielded very imprecise estimates of the number of deaths, which the authors glossed over. They should have said, “We can say with 95% certainty that between 8,000 and 194,000 excess Iraqi deaths occurred during the period.”
The group essentially repeated the study in 2006, using a larger sample size. Again, the researchers had interviewers administer what resembled a typical survey of a random sample of households. Appropriately, at the end of the article in The Lancet , the authors discuss issues that may have resulted in a sample that in fact did not meet the “random” threshold.
Problems with interpretation also plagued that second effort. The authors used crude death rates (CDR’s), which reflect the number of deaths per thousand people, in explaining the rise in mortality. But demographers rarely use CDR’s, thinking instead in terms of age- and sex-specific mortality rates, usually summarized as “life expectancy.” That being said, the group reported that the CDR increased from 5.5 per thousand people in 2002 to 13.3 per thousand in the post-invasion period (March 2003-March 2006).
To put the pre-invasion figure in perspective, consider United Nations Population Division figures, which are generally considered to be of very high quality. The UN estimates that Iraq’s pre-invasion CDR was 10 per thousand, not the 5 per thousand estimated from the two studies. Comparing internationally, the UN reports that Iran’s CDR in the 2000-2005 period was 5.3 per thousand. Prior to the war, most observers thought that the situation in Iraq was considerably worse than in Iran.
So the pre-war CDR that the two Lancet studies yield seems too low. It may not be wrong, but the authors should provide a credible explanation of why their pre-war CDR is nearly half that of the UN Population Division. If the pre-war mortality rate was too low and/or if the population estimates were too high – because, for example, they ignored outflows of refugees from Iraq – the resulting estimates of the number of Iraqi “excess deaths” would be inflated.
More fundamentally, what purpose do these numbers serve? Certainly, after the dust has settled, numbers play a role in evaluating the costs and benefits (if any), of a war. But in real time, do the numbers really add to the debate? Do they really provide us with more information than the Iraq Body Count figures provide? Do we have the appropriate context to help us interpret the numbers? The war in Iraq has been exceptionally bloody. For now, that is about all that statistics can safely tell us.