UPDATED: Depressing News of the Day

British medical journal The Lancet estimates 655,000 "excess deaths" in Iraq in the first 40 months of the war. (Here's an easier to read version.) They attribute 600,000 to violence, with (conservatively) three-tenths of those due to Americans and allies. The violent death toll in the third year of the war is more than triple what it was in the first year.

Unnamed Iraqi interviewers did door to door surveys around the nation so that

... data from 1849 households that contained 12,801 individuals in 47 clusters was gathered. 1474 births and 629 deaths were reported during the observation period. Pre-invasion mortality rates were 5.5 per 1000 people per year (95% [Confidence Interval] 4.3–7.1), compared with 13.3 per 1000 people per year (10.9–16.1) in the 40 months post-invasion. We estimate that as of July, 2006, there have been 654,965 (392,979–942,636) excess Iraqi deaths as a consequence of the war, which corresponds to 2.5% of the population in the study area. Of post-invasion deaths, 601,027 (426,369–793,663) were due to violence, the most common cause being gunfire.

I'm going to ramble on this topic, because I think it's more useful to toss out various reality checks rather than to decide pro or con and then construct a polemical argument in favor of one side or the other. You can get those lots of other places. As I've thought about this, I've become a little more skeptical, but that's less important than showing some of the ways of approaching the problem of evaluating this study.

Upon hearing that huge number of excess deaths, I had hoped that heat-related deaths due to power outages and lack of air conditioning would have been the big killer, since they mostly kill old people without many years left to live in any event. The 1995 heat wave in Chicago, when the temperature peaked at 106 F and caused a blackout, killed something like 525 people in five days, but most of them already old and frail. It's regularly over 106 in Iraq all summer (the forecast for tomorrow, 10/13, is 99F). But I guess I was wrong. Perhaps Baghdad houses are built more appropriately for triple digit heat than Chicago houses?

A reader writes:

It is not even remotely plausible that 500 Iraqis died from violence every day since the war began without anyone really noticing more than a fraction. The same "study" concluded that over 200,000 Iraqis died in Falujah alone in 2004, than removed the number from the sample, because it was not possible. They nevertheless used the exact same method on the rest of Iraq to get the 655.000 figure now. Scientific?

What they did is ask a small sample "did anyone in your household die in this period" and extrapolate the number to the whole population (each death multiplied by about 2000, insurgents are clearly included). Now ask yourself:

1. Are Arabs generally honest and trustworthy sources? 2. Do Arabs generally define family/household the same way that Westerners do?

It was clear last time that the anti-americans surveyed in Falujah were lying to the surveyors, who got the result that half the population of 500,000 were killed in a few weeks of fighting(!). Strangely the 200.000 corpses in the small town were not noticed by the legions of journalists, not by sanitations workers. Is there any reason to believe this bias has not changed in this survey?

Okay, but the researchers excluded Fallujah in their 2004 study precisely because there were so many deaths there in 2004 (the Marines pretty much flattened it in November 2004) that it would be easy to accidentally come up with an unrepresentative sample of survivors that could then over-inflate the national total. So, the researchers' behavior in 2004 of excluding Fallujah, which then lowered the national death estimate, was erring on the side of caution, which should count in the researchers' favor.

Here is a little plausibility test, that no one else in the media will do for a new item too good to not be true. The deaths are roughly:

340,000 from gunshots
80,000 Allied air strikes
80,000 car bombs
80,000 other explosives

Now you can argue that no one reports the gunshots. But even in Iraq every major car bomb is reported. If we are to believe the Lancet study each day since the invasion 135 Iraqi civilians were blown up in a car bomb/explosion, day after day. But why does the American and Iraqi media only report a fraction of these deaths? The Brookings Iraqi index reports 8,300 death from multiple fatality bombings, compared to 160,000 for Lancet. Is the difference of 20-fold between reported deaths and their estimated deaths most likely due to underreporting from the media or bias in their survey?

You can also ask yourself how it is possible that allied bombing continued to kill 30-40,000 Iraqis from 2005, where there has been limited allied bombing. According to Lancet each day 50 civilians were killed by allied bombing, again day after day for two years, without anyone really noticing. Are these figures even remotely consistent with actual US bombings?

This study is just propaganda that uses scientific lingo to fool people.

Well, the researchers saw death certificates for the majority of reported deaths (501 out of 629):

The study population at the beginning of the recall period (January 1, 2002) was calculated to be 11 956, and a total of 1474 births and 629 deaths were reported during the study period; age was reported for 610 of 629 deaths, sex reporting was complete. During the survey period there were 129 households (7%) that reported in-migration, and 152 households (8%) reported out-migration. Survey teams asked for death certificates in 545 (87%) reported deaths and these were present in 501 cases. The pattern of deaths in households without death certificates was no different from those with certificates.

That renders unlikely my idea that the death toll could be highly inflated by interviewees counting additional dead guys who weren't exactly part of the interviewed household -- third cousin Ali who stayed with us a couple of times but mostly stayed with his parents, that sort of thing. But, presumably, there's only one death certificate.

It could be the interviewers were flat out lying, or that they chose particularly hard hit neighborhoods.

It could also be that the overall death rates are pretty accurate, but that the causes of death are not, with survivors attributing Ali's death to something more random (car bomb or air strike) when he was really shot.

Here's an AP article from 12/2005:

"The number of U.S. airstrikes increased in the weeks leading up to last Thursday's election, from a monthly average of about 35 last summer [of 2005] to more than 60 in September and 120 or more in October and November."

So, if say there have been two airstrikes per day and on average they killed ten each, that would be about 7,300 deaths by airstrike per year, or about 30% of the airstrike reported numbers in the Lancet study.

Perhaps in a culture with stringent codes of vengeance, it can be prudent for the survivors to attribute a relative's violent death to an unknown or inaccessible killer rather than to being shot by somebody in particular because the latter might require honorable men to avenge the death? Maybe wreaking vengeance on the US Air Force for killing a son is socially considered impossible so therefore not mandatory, while staging a revenge attack on the U.S. Army is considered right and fitting? That would incline a man to blame airplanes rather than soldiers for his son's death. I don't know, I'm just speculating.

What about U.S. small arms fire?

A US government report says that US forces are now using 1.8 billion rounds of small-arms ammunition a year. The total has more than doubled in five years, largely as a result of the wars in Afghanistan and Iraq, as well as changes in military doctrine.

Estimating how many bullets US forces have expended for every insurgent killed is not a simple or precisely scientific matter.

John Pike, director of the Washington military research group GlobalSecurity.org, said that, based on the GAO’s figures, US forces had expended around six billion bullets between 2002 and 2005

“If they don’t do body counts, how can I? But using these figures it works out at around 300,000 bullets per insurgent. Let’s round that down to 250,000 so that we are underestimating.”

That six billion sounds too high for Iraq because we were using up a little under a billion bullets per year five years ago, presumably in training or as their "sell-by dates" expired. And some of that remaining billion per year now is fired off in Afghanistan. So we are probably using somewhere around 730 million rounds (to pick a number that's easy to work with) per year in Iraq, or 2 million per day. Let's say half of that is expended in additional training, so that is 1 million American bullets per day fired in anger in Iraq.

Oh, man, that is a big number. Our firepower is so huge these days. (On the other hand, here's a 2004 Army claim of using only 5.5 million bullets in Iran per month, but other services also use a lot of bullets, and the second half of 2003 and early 2004 were the Good Old Days of peace and harmony in Iraq. So, that's not radically off from an overall rate of a million per day.)

Does that million per day number make sense? If there are 150,000 troops there, that's only 7 bullets per day per soldier. If say, one out of 15 soldiers get in a clash every day, that would be about 100 bullets per soldier per fire fight. You can fire 100 bullets in, oh, ten seconds, but with an M-16 you'd have to reload fresh clips several times to fire 100 bullets, so you can't actually fire that fast, but certainly could do that in a minute or two. With a mounted machine gun like the M2 .50 Caliber, you can fire 550 bullets per minute, with an effective range of 2 kilometers, so it's pretty easy to kill bystanders many blocks away just by squeezing your finger for 10 seconds.

In WWII, an American estimate was 6,000 bullets per enemy hit (not killed). At a million bullets per day, that would be 160 Iraqi casualties per day. But we were using a lot of single shot M1s, so we were likely a lot more efficient back then compared to now when we just hose down the general area. But, on the other hand, we are doing more fighting in cities with normal daily life going on, while in France in WWII, civilians tended to flee from the front or hunker down in cellars. There's no front in Iraq.

In July, 2006, a bad month, there were about 1,200 IED bombings against Americans. How many Iraqis died on average in the minutes following each attack? I don't know, but, considering that Americans mostly venture around only in heavily armed convoys, I wouldn't be surprised if our soldiers on average killed several Iraqis in the moments of terror and rage following each bombing. I just don't know what typically happens, but the combination of adrenaline and automatic weaponry would suggest something fairly lethal would be normal.

Another way to look at it is that there were 38 Americans killed last July. At the roughly 50 to 1 local to American kill ratio estimated for the "Black Hawk Down" firefight in Mogadishu in 1993 (about 1,000 Somalis killed versus about 20 Americans), that would be 1,900 Iraqis, which is perhaps a quarter of the Lancet estimate for that month.

But the American death rate in Iraq is historically low relative to the wounding rate, due to armor, advanced medicine, and other factors. There were a total of 546 American causalities in July (38 deaths and 508 wounded). What's the Iraqi death rate per American casualty? A 14 to 1 ratio would add up to about 8,000 Iraqis killed in July by Americans. I have no idea if that is realistic.

The great majority of violent deaths were among male Iraqis, typically younger:

The male-to-female ratio of post-invasion deaths was 3.4 for all deaths, and 9.8 for violent deaths (all deaths: 144 female, 485 male; violent death: 28 female, 274 male). In general, deaths by age group followed the expected J-shaped demographic curve; however, by contrast, most deaths in males were in the middle age groups

That suggests that not too many civilians are getting killed in their own homes by bullets penetrating their walls. So, most of the people getting killed are the usual suspects: fighting-age men. Are they mostly insurgents or just guys hanging out on the street when an improvised explosive device went off or a sniper shot at a convoy and we responded by hosing down the neighborhood? I don't know.

Getting a representative sample for this study is obviously hugely difficult. It could be that the researchers gravitated to the most shot up neighborhoods, or that those families having suffered losses were most likely to talk to the researchers. On the other hand, it's equally easy to theorize that the opposite biases would be true -- researchers would stick to the least violent neighborhoods out of self-preservation urges and families whose menfolk were involved in this war of all against all would be least likely to speak to strangers, who might be connected to an opposing group. (Think of the scene in "The Departed" where the cops try to get the mother of a thief murdered by the Boston Irish mob to talk to them, but she won't.)

A reader points toward the this line in the report:

"In 16 (0·9%) dwellings, residents were absent; 15 (0·8%) households refused to participate."

Out of 1849 households surveyed, those seem dubiously low nobody-at-home and refusal rates. So, maybe they are just making up the data. Who knows ...

What's going on here is that the interviewers were supposed to go to 47 neighborhoods around the country and start interviewing until they tallied 40 households, which equals 1880 households. But they ended up with only 1849, which is 31 (16 not-at-homes plus 15 refusals) short of 1880. So, the question is whether the 31 households mentioned above comprise all the not-at-homes and refusals, which seems implausible, or are they just a subset? For instance, perhaps the interviewers finished up a day thinking they had 40 households but when they got home they found they'd miscounted and did only 39, so they marked one down as refused. Once again, who knows, but the study sponsors should provide an explanation.

Maybe what happened is that the interviewers didn't actually go much door-to-door at random, but instead arrived in a neighborhood, put the word out, and then either had people who wanted to talk to them come see them or were invited to the homes of people who wanted to see them. That might account for the very high % of people with death certificates available.

Or it could be that the interviewers got in contact ahead of time with neighborhood leaders to see if their presence would be welcome to reduce their chances of being killed. (That's not good random surveying hygiene, but are you going to blame them?) Then, in a neighborhood where the local big shot wanted their presence, he might have passed the word around to aggrieved families to get ready to tell their stories to the interviewers when they showed up. This could cause a bias upward in the number of deaths reported.

The more I think about the mechanics of carrying out the survey on the street without getting killed, the more I suspect that the Iraqi interviewers didn't actually implement the purely random survey design that the American professors from MIT and Johns Hopkins dreamed up for them. It would be nuts to to let luck determine which streets you'd choose, as the report claims they did. You'd want to only go where you knew you'd be safe. Then you'd tell the Americans you did exactly what they told you to do.

How that would bias the result, I don't know.

The overall point, however, is that nobody else appears to be doing this kind of study because it is so hideously dangerous, which ought to tell us something.

More analysis is necessary, but, after a few hours of kicking the tires, these numbers don't strike me as obviously implausible. I wouldn't put tremendous confidence in them either, though, due to the savage conditions under which this heroic effort was carried out.

If I had to guess, I'd say 600,000 deaths from violence was too high. But what if you cut that number in half, down to "only" 300,000? I can't say that sounds improbable. And will the death toll sooner or later to get to 600,000? That seems quite likely.

My published articles are archived at iSteve.com -- Steve Sailer