Media Lens's errors on Iraq Body Count

David Edwards and David Cromwell (editors of Media Lens) have published several articles criticising Iraq Body Count (IBC). Their claims have been widely circulated as part of a sustained and vigorous campaign against IBC. But Media Lens's case against IBC is based to a large extent on errors and misrepresentations, and is contradicted by recent research (eg from the Centre for Research on the Epidemiology of Disasters, Iraq Family Health Survey, etc) as outlined below.

Before I list Media Lens's errors, a note about context. There have been several recent attempts to quantify the bloody slaughter resulting from the US/UK-led invasion and occupation of Iraq. The Lancet journal, in October 2006, published a study estimating 654,965 excess Iraqi deaths. A poll from Opinion Research Business (ORB), September 2007 (updated in January 2008), estimated that over a million Iraqis had been murdered. Another study, published in January 2008 by the New England Journal of Medicine, estimated 151,000 violent Iraqi deaths upto June 2006.

Unlike the above figures, IBC's count is not a statistical estimate, but a record of actual, documented deaths (specifically violent civilian deaths). Obviously, then, it doesn't include unreported, undocumented deaths. From the outset IBC stated (in its Quick FAQ) that "it is likely that many if not most civilian casualties will go unreported by the media". IBC's database currently contains details of over 100,000 documented civilian deaths. IBC has always stated that its figures are bound to be an undercount of civilians killed by violence, due to gaps in reporting and recording. And, clearly, Iraq mortality figures will be higher (possibly much higher) to the extent that they include other categories of deaths, eg "excess" deaths from disease or other non-violent causes, plus violent non-civilian deaths, including combatants and also Iraqi military personnel killed during the "shock and awe" phase of the invasion.

IBC is not alone in providing a tally (rather than a sample-based estimate) and should be viewed alongside other comparable counts of war dead (eg independently arrived-at figures based on morgue and death-certificate records, etc¹). The method of documenting and tallying deaths has a long pedigree, going back at least to the First World War. Both this approach and the more recently introduced cluster-sample surveying methods have their advantages and disadvantages, but in the context of the debate raised by Media Lens some of the aims of the documenting approach have been obscured. Most importantly, perhaps, these include humanising the victims of war – giving names, identities and other individual/circumstantial details, where possible.

Basic errors by Media Lens

• One of the main premises of Media Lens's criticism of IBC is that "IBC is not primarily an Iraq Body Count, it is not even an Iraq Media Body count, it is an Iraq Western Media Body Count" (David Edwards/Media Lens Alert, ZNet, March 14, 2006).

This is entirely mistaken. IBC uses non-Western media sources and its database includes hospital, morgue and NGO data. It is able to monitor around 70 major "non-Western" sources, along with 120 "Western" sources. (IBC)

Many incidents/deaths in IBC's database are from the major wire agencies. This reflects the real-world fact that these organisations pick up the highest percentage of incidents. For example, of all incidents from July 2006 to March 2007 (as documented by IBC), Reuters picked up approximately 50%, compared to 35% from Al Sharqiyah TV (another IBC source), with much lower coverage by other media sources, "Western" or "non-Western" (IBC). Note also that at the level of reporting utilized by IBC, the dichotomy of "Western" vs "non-Western" is false, as agencies such as Reuters employ (for example) Iraqi journalists in covering Iraqi incidents ("We mainly use local reporters, Arab reporters can go out and talk to people" – Reuters Baghdad bureau chief).

• Media Lens's incorrect "Western Media Body Count" premise leads to a more serious fallacy – the claim that IBC's database is biased towards under-reporting deaths caused specifically by US/UK forces. For example:

"Our research revealed that the IBC database consistently features the same bias - massive numbers of deaths caused by insurgents as compared to a tiny number caused by the ‘coalition’." (David Edwards/Media Lens Alert, ZNet, March 14, 2006)

This is a bizarre statement, as the IBC database rarely attributes deaths to "insurgents". We can see where Media Lens went wrong by checking the descriptions of their research:

"We reported how we had searched the Iraq Body Count (IBC) database for incidents involving the mass killing of Iraqi civilians by US-UK forces between January-June 2005. We found, for example, 58 incidents of a minimum of 10+ deaths. Of these, just one was attributed to ’coalition’ action - a US airstrike. By contrast, 54 incidents of 10+ deaths were clearly attributed to the insurgency." (David Edwards/Media Lens Alert, ZNet, March 14, 2006)

"[Of the 57 incidents not attributed to the 'coalition',] 25 were attributed to suicide bombers and a further 29 were attributed to insurgent actions targeting Iraqi government troops, government officials, religious groups, and so on. The few remaining cases described corpses shot at close range, bodies blindfolded and shot, and executed bodies that had been dumped." (David Edwards/Media Lens Alert, ZNet, January 27, 2006)

Media Lens's claim that 54 out of the 58 incidents "were clearly attributed to the insurgency" can quickly be shown to be false. It's a simple matter to download a copy of IBC's database and check the period in question (January-June 2005). For the whole of this period (covering over 1,000 incidents) I found only one incident directly attributed to insurgents (d2662: one person reported killed near Tal Afar, 7 May 2005, 'insurgent attack'). I found no incidents of 10+ deaths "clearly attributed to the insurgency". It's worth noting that whilst Media Lens was ostensibly making a point about "mass killing" incidents, its focus on cases involving a minimum of 10 deaths obscures the fact that around 90 incidents during this period (for any number of deaths) were attributed directly to US forces (mostly from "US gunfire"), compared to only one incident directly attributed to insurgents. Note that we are not talking here of incidents involving "clashes between US and insurgents" (etc) in which civilians are killed by unknown perpetrators (eg "crossfire").

So, it seems that the above "massive" attribution of deaths to insurgents is Media Lens's, not IBC's. In most cases IBC's database doesn't directly identify perpetrators, but simply lists the reported target and type of attack. Of deaths unattributed to a perpetrator, cases listed as "suicide bomb" might suggest a category of perpetrator, but they aren't "clearly attributed to the insurgency" (as Media Lens puts it). In other cases, the target of an attack ("Iraqi government troops, government officials, religious groups", etc) might be suggestive, but there's no "clear" identification of perpetrators. Contrary to what might be inferred from Media Lens's statements, deaths not clearly attributed by IBC to the 'coalition' are not attributed by default to the 'insurgency'.

IBC pointed out in a press release (March 9, 2006) that, in the past year, "anti-occupation activity" had reportedly resulted in 2,231 civilian deaths (with 370 reported civilian deaths from "military action by US-led forces"). Importantly in this context, IBC added that "the majority of media reports do not allow a clear identification of the perpetrators or their motives". IBC stated that "unknown agents" did most of the killing, and that this could include US-led forces. Note also that a search of IBC's database reveals that the terms "insurgent(s)", "suspected insurgent(s)", etc, appear more often as reported targets than as reported perpetrators.

In another analysis [p23-26], IBC focused on deaths "definitely attributable to coalition forces", and concluded that "IBC and Lancet [2004] show broadly comparable proportions of deaths attributable to coalition forces". According to this analysis the Lancet 2004 estimate shows that 43% of violent deaths (for the whole country outside Falluja) were directly caused by US-led forces, compared to IBC’s 47% over the same period. Both Lancet 2004 and IBC indicated that "the majority of killings by US-led forces were caused by or involved air strikes".

• Some of Media Lens's initial criticisms of IBC were based on incorrect comparisons with the 2004 Lancet study on Iraqi deaths. For example, in their first article criticising IBC, Media Lens wrote:

"Whereas the Lancet report estimated around 100,000 civilian deaths in October 2004, IBC reported 17,000 at that time." (David Edwards/Media Lens Alert, ZNet, January 27, 2006)

This is incorrect in two ways. First, the Lancet study didn't estimate "civilian" deaths as Media Lens claims (its estimate includes "combatants" as well as civilians – see p7 of Lancet study). Second, IBC records only violent deaths, so the comparison should be between 57,600 and 17,687² (57,600 being the Lancet study's estimate of violent deaths, according to Lancet co-author Richard Garfield). But even that isn't comparing like with like, since IBC does not include combatant deaths, whereas the Lancet study does.

• Media Lens also wrote, in the same article:

"But anyway, as we have seen, the IBC figure is selective in its sources, is the lowest estimate of eight serious studies, and relies on 'professional rigour' in the Western media that does not exist." (David Edwards/Media Lens Alert, ZNet, January 27, 2006)

The claim that IBC provides the "lowest estimate" of "eight serious studies" has been widely circulated but is completely mistaken. It's based on a collection of errors and misconceptions (which were exposed in detail by IBC in 2006), including an error from Les Roberts (Lancet study co-author) which Roberts has acknowledged to be an error ("I said the IBC count was 17 deaths per day over the period 3/1/03 - 2/1/05. That was wrong." – Les Roberts, email to Gabriele Zamparini, June 2006).

• Media Lens has promoted the claim that IBC may capture only 5% of the true death toll (which would currently suggest a figure of 2 million dead), but this is typically supported with errors and inconsistencies:

"How many know that leading epidemiologist Les Roberts recently estimated that between 200,000 and 300,000 Iraqi civilians may have been killed since the invasion? Roberts argues that the most commonly cited source for Iraqi civilian casualties - amateur website Iraq Body Count (IBC) - may have captured less than five per cent of the true total." (David Edwards and David Cromwell, The First Post, July 4, 2006)

The first problem with this is basic arithmetic. IBC's minimum figure for the period in question (approx 35,000) was 17.5% of Roberts's lower figure and nearly 12% of his upper figure. Not "less than five per cent". The second problem is that it's inconsistent with the findings of the Lancet 2004 study (co-authored by Roberts). IBC's earlier count of violent civilian deaths (17,687 for the relevant period – see above) was 30% of Lancet 2004's estimate of violent deaths (57,600). It's interesting to note that later estimates, eg from the Iraq Family Health Survey and the Centre for Research on the Epidemiology of Disasters, suggest that IBC is capturing around a third, or more, of violent deaths – contrary to the claims promoted by Media Lens. An earlier estimate, from the large-scale (over 21,000 households surveyed) Iraq Living Conditions Survey, indicated that IBC may have been capturing over a half of violent deaths over the period surveyed.³

Latest errors by Media Lens

After issuing several 'Media Alerts' criticising IBC in 2006, Media Lens followed up with another piece, titled Iraq Body Count: "A Very Misleading Exercise", in October 2007. Unfortunately, this too contained several errors and misrepresentations, as I list below:

"In the past, IBC's response to the suggestion that violence prevents journalists from capturing many deaths has been, in effect, 'Prove it!'" (Media Lens Alert, ZNet, October 11, 2007)

This is clearly untrue. As I noted above, IBC have always stated that "many if not most civilian casualties will go unreported by the media". (Media Lens had actually quoted this statement from IBC in an earlier Media Alert). IBC have issued similar statements throughout their website and press releases – for example: "Gaps in recording and reporting suggest that even our highest totals to date may be missing many civilian deaths from violence" (IBC's front database page).

"It is striking that IBC link to a high-profile media report that so badly misrepresents its figures. As so often, this opening sentence [this referred to an article in The Independent] gave the impression that IBC are recording the total number of civilian deaths, rather than merely recording deaths from violence as reported by the media." (Media Lens Alert, ZNet, October 11, 2007)

This is misleading. The purpose of IBC's link (titled "Lists of victims or victim categories to signal the pervasive impact on every sector of Iraqi society") is to provide an example of how media have used IBC's data on individual victims (see lower section of the cited article, which is clearly titled "Victims' Stories"). Whether Media Lens's assertion that the article "misrepresents" IBC figures has any merit or not is irrelevant to the point of the link. IBC doesn't endorse misrepresentations of its figures. (Given Media Lens's advocacy for the Lancet studies on Iraq mortality, it's "striking" that they fail to mention a similar "misrepresentation" of IBC's figures by the Lancet study's authors, in an article for Slate magazine: "Today, IBC estimates there have been 45,000 to 50,000 violent deaths").

"It was [Marc] Herold's Afghan Victim Memorial Project that inspired John Sloboda to set up IBC. Herold's 'most conservative estimate' of Afghan civilian deaths resulting from American/NATO operations is between 5,700 and 6,500. But, he cautions, this is 'probably a vast underestimate' [...] There is no reason to believe that the application of the same methodology in Iraq is generating very different results." (Media Lens Alert, ZNet, October 11, 2007)

Again, this is mistaken and misleading. IBC uses the same general approach as Marc Herold used for Afghanistan, but it doesn't use the same methodology. Herold's count includes civilian victims directly killed by US/NATO bombings and military action, while excluding victims of the Taliban or other perpetrators. IBC includes killings by any perpetrators in Iraq. There are several other differences in the methodologies, and there are also reasons to believe the approach in Iraq is generating somewhat different results than in Afghanistan. But it's unlikely that Media Lens's editors have looked into the matter in enough depth to know the reasons. They have not looked into the matter closely enough to know that there are differences in the methodologies, or even to know that it is not Herold's "Afghan Victim Memorial Project" (begun in 2004) that inspired IBC, but rather his "Daily Casualty Count of Afghan Civilians Killed by U.S. Bombing" – begun in 2001), two completely different projects.

In any case, Professor Herold wrote to ZNet [see footnote at this link] stating that the paragraph written by Media Lens had inaccuracies which needed to be corrected, and that the inference drawn from it regarding IBC was unwarranted.

"Well, the bureau chief of one of three Western media agencies providing a third of IBC's data from Iraq sent this email to a colleague last year (the latter asked us to preserve the sender's anonymity): ... [an anonymous email critical of IBC then follows]" (Media Lens Alert, ZNet, October 11, 2007)

The Media Lens editors also cited an "anonymous epidemiologist" in their earlier pieces criticising IBC. It was noteworthy then, as it is now with this anonymous "bureau chief" and "colleague", that these unnamed sources weren't able to send their comments directly to IBC (who would, of course, have treated them in confidence), or stand behind them publicly.

Methodological Credibility

IBC's count is not derived from epidemiological survey methods – it's a completely different type of study, involving corroboration and cataloguing of documented deaths. In their fourth Alert criticising IBC, Media Lens asked:

"How many journalists are aware that IBC is not in fact run by professional epidemiologists? What would we say if, in discussing climate change, politicians and journalists consistently highlighted information supplied by a group deemed by professional climate scientists to be 'amateurs'?" (David Edwards/Media Lens Alert, ZNet, April 10, 2006).

These questions would have been more relevant with respect to ORB's poll on Iraqi deaths (which Media Lens promoted as "credible") since unlike IBC's study, it depended (for the credibility of its estimate) on having a "nationally representative sample" – a challenging requirement in Iraq, even for epidemiological surveys. ORB's director, Allan Hyde, confirmed to me by email (July 24, 2009) that no epidemiologists were involved in conducting ORB's poll.

Despite the low relevance of epidemiology to IBC's methods, the Media Lens editors wrote, in a letter to New Statesman magazine (October 16, 2006), that, "to our knowledge, IBC has not been able to demonstrate support for its methods from a single professional epidemiologist". It's a curious remark, suggesting that IBC (or Media Lens) tried, and failed, to solicit (or search for) epidemiological support for IBC's non-epidemiological methods. To my knowledge, IBC has never focused its energies on seeking approval from epidemiologists. Nevertheless, several prominent epidemiologists and demographers do appear to have consistently supported IBC's methods/data. Here are a few recent examples of such support (postdating Media Lens's New Statesman letter):

"While each approach has its drawbacks and advantages, this author puts the most credence on the work that the Iraq Body Count has done for a lowerbound estimate of the mortality impact of the war on civilians. The data base created by IBC seems exceptional in its transparency and timeliness. Creating such a data base carefully is an incredibly time-consuming exercise. The transparency of IBC’s work allows one to see whether incidents of mortality have been included. The constant updating of the data base allows one to have current figures." (Wartime estimates of Iraqi civilian casualties, by Beth Osborne Daponte, the renowned demographer who produced authoritative death figures for the first Gulf War)

"The Burnham [Lancet 2006] estimates of deaths in the post invasion period are much higher than any other estimate. Even the lower limit of its 95% CI is higher than the highest estimate from any other source (Table 1). Further, weaknesses cited earlier as well as several inconsistencies in their published work undermine the reliability of their estimates. [...] While IBC is undoubtedly missing some deaths in Baghdad, it is unlikely that they would miss an average of over 100 violent deaths a day, given the level of media coverage in the city. We therefore conclude that their Baghdad mortality estimate is close to complete, further corroborated by the ILCS estimates [...]" (Estimating mortality in civil conflicts: lessons from Iraq, by Debarati Guha-Sapir and Olivier Degomme, from the Centre for Research on the Epidemiology of Disasters, Brussels)

 

Conclusion

Some of Media Lens's supporters have expressed the belief that the antiwar cause is damaged by media reporting of IBC's count in preference to higher estimates.⁴ It's worth remembering here that millions of people demonstrated against the war before it started – and it seems reasonable to assume that in most cases opposition to the war was not, and is not, conditional upon a given death toll. It's always possible, of course, that pro-war commentators will present an argument based on a simplistic algebra of death (using the "lowest" war-dead figures they can find) but the most logical way to deal with this is not by insisting upon a given (higher) figure, but by rejecting the crass premise that lives can be traded for others like so many casino chips. The recording and quantifying of war deaths is important for several reasons (see, for example, IBC's rationale), but if the effectiveness of opposition to war is believed to be determined by a simplistic factor such as the size of death toll reported by the media, then that opposition will never appear effective in the context of "smaller" wars, or in the run-up to hostilities.

A case in point is Afghanistan, where the war dead are measured "only" in the thousands, and where the "excess deaths" calculation can be interpreted as favouring the NATO invasion, if numbers are taken to be the sole criterion. For example, aJohns Hopkins University study (run by Gilbert Burnham, co-author of the 2006 Lancet Iraq survey) found lower infant and child mortality rates, due to improved medical care, following the invasion. The implication here is that the number of lives saved exceeds both tallied and estimated death tolls from the fighting.⁵

If a study indicates "benefits" from a war of aggression, does it follow that it provides "propaganda" for warmongers? The same question could be asked about studies which, due to practical limitations, present only partial or incomplete accounts of the destructive effects of a war. Media Lens doesn't address these questions in general logical terms, but it is quite specific in one of its conclusions regarding IBC: "We certainly agree that the IBC project is providing powerful propaganda for people responsible for horrendous war crimes." (Posted by Media Lens's editors to their message board, March 23, 2006). Ironically, this charge makes sense only from a pro-war framing of the issues, because in order to interpret research findings of this kind (eg tallies of war dead) as supporting the case for war, one must accept certain pro-war premises to begin with (see my above remark about trading lives like casino chips). Without such an interpretation findings are simply findings, not "propaganda".

The war and occupation of Iraq have involved the brutal violation of the human rights of the individual, multiplied so many times that one can't grasp the scale of it, except in an abstract way through numbers. The argument against the war, for me at least, has remained consistent since before it started in March 2003, and is about protecting basic human rights. Clearly it's important to quantify deaths from war, and this has a role in ending or preventing conflicts. But it's not about using a death toll to promote a case against (or for) a war in terms which reduce human lives to interchangeable units. Even in contexts which make it clear that this isn't the intention, we should pause for thought before referring to any given count of dead as "low".⁶

Footnotes

1. For example, separate tallies reported by the Los Angeles Times and Associated Press.
2. The IBC database showed 16,933-19,415 deaths for the period up to the end of September 2004 (approx 17,000-19,500). The maximum figure is more appropriate for comparison with the surveys since a lot of the IBC range is for civilian/combatant uncertainty, which is irrelevant to the surveys. Media Lens, however, seems to have taken the lower figure (17,000) as a comparison. A more rigorous approach would be to use the maximum figure, but to subtract deaths for Anbar (approx 2,000). I've used the precise figure of 17,687 as used by one academic source: http://tinyurl.com/4xsjtl [PDF document, p47]
3. ILCS estimated 24,000 war-related deaths of civilians and combatants; IBC's figure was approx 14,000 deaths of civilians for the ILCS coverage period: http://tinyurl.com/4xsjtl [PDF document, p47]
4. For example: "The damage that Iraq Body Count’s figures have done is huge, terrifying and shocking." (Gabriele Zamparini, April 14, 2006) 
5. A National Journal article claimed that Burnham's Afghanistan study shows that an "estimated 89,000 infants per year" are saved by medical improvements, and that this figure "far exceeds the estimates of people reported dead in the fighting between the government and the Taliban".
6. The kind of wording that often appeared on Media Lens's message board (describing IBC's count as "low") was regrettably reflected on a Johns Hopkins University web page ('Answers to Questions About Iraq Mortality Surveys', no longer available on the JHU website): "The low numbers of Iraqi deaths reported by IBC provide comfort to many." – until this wording was removed following complaints in early 2008.