Lead Poisoning, Crime, and the Media

A few days ago on Real Time with Bill Maher, Rick Lazzio was discussing crime trends in the United States, which appear to be declining in spite of the poor economy. Maher pointed to research linking violent crime to childhood lead exposure, and suggested that simple remedies like removing lead from paint and gasoline can sometimes have profound effects on certain kinds of social problems.

I think there is some truth to Maher’s comments, but I would also argue that it’s extremely difficult to measure these kinds of things accurately.

The study that Maher is referring to was popularized by a 2007 article in the Washington Post examining then-Republican presidential hopeful Rudy Guliani’s record of crime reduction in New York City. The author, Shankar Vedantam, cited evidence by economist Rick Nevin which seems to show that “lead poisoning accounts for much of the variation in violent crime in the United States.”

I’ve been thinking a lot lately about how research findings are reported in the media, and I’ve come to believe that journalists have a responsibility to discuss the potential limitations of various methodological approaches. This article is a case in point.

Nevin’s study relies on time-series regression models, which explain national crime rates as a function of the blood lead content rate in preschoolers. Looking at various industrialized countries, he shows that “the violent crime rate tracks lead poisoning levels two decades earlier” in virtually every instance.

I find this conclusion very appealing, but the research design strikes me as problematic for two reasons.

First, lots of factors are associated across time. In fact, the key problem with time series analysis is that you almost always find highly significant relationships with large coefficients of determination. When I was doing research on joblessness a few months back, for example, I came across this study showing that the (lagged) frequency of certain kinds of Google keyword searchers explains about 89 percent of the variation in German unemployment. That’s pretty astounding, but no one in their right mind would assert that keyword searchers actually cause unemployment. This relationship probably holds in almost every industrialized country, and we may even find similar lag times. None of this means that Google is killing jobs.

Second, even if Nevin relied on cross-sectional data, there are some important confounding factors that he probably couldn’t capture very well. In particular, it’s likely that some measure of social responsibility may explain at least part of the relationship between lead poisoning and crime rates. Heightened levels of social responsibility could lead to public calls for lead removal from various products. This same change in the social mindset could lead to more responsible parenting, which would likely lead to lower rates of crime about 20 years later when males are most likely to become violent offenders. Finding a proxy for “social responsibility” is pretty difficult, and Nevin doesn’t seem to control for this at all in his study.

The general point here is that inferring causality is difficult, and most studies have drawbacks that really need to be highlighted. I don’t expect every journalist to become an expert on research design, but I do think that it’s incumbent upon them to know something about the limitations of the specific research they’re summarizing for the public. Peer-review doesn’t guarantee a flawless study.

Maybe I'm putting too much blame on journalists, but I suspect the public might be a little less cynical about seemingly contradictory research findings if they had a better idea of how reserach is conducted, and at least some understanding of how we actually "know" the things we think we know.