Corporate Lobbying and Research Methods

I’m a big fan of NPR’s Planet Money blog, but this recent post on the rate of return from corporate lobbying is pretty indefensible.

The author, Alex Blumberg, cites a 2009 study from the University of Kansas, which found that corporate lobbying for the American Jobs Creation Act (AJCA) yielded a return on investment of approximately 22,000 percent. Among other things, the AJCA allowed companies to bring back profits from offshore divisions at a one-time reduced tax rate.

Naturally, many multinational corporations found AJCA exceedingly lucrative, but the important question is whether their lobbying efforts were actually successful. Is the fact that the law was passed evidence enough?

The key challenge in studying the impact of lobbying is trying to figure out what would’ve happened in the absence of these efforts. Would the outcome have been the same, or would lawmakers have come to a different conclusion?

Research methodologists call this kind of thinking “counterfactual causal analysis” because it first considers what would’ve happened without the intervention (no lobbying) and then compares this hypothetical to the present condition (lobbying). It’s complicated stuff, and it’s the kind of thing that makes real-world analysis so difficult.

So, how did University of Kansas researchers derive their counterfactual scenario? Well, they didn’t.

As Blumberg explains:

Raquel Alexander and Susan Scholz calculated the total amount the corporations saved from the lower tax rate. They compared the taxes saved to the amount the firms spent lobbying for the law. Their research showed the return on lobbying for those multinational corporations was 22,000 percent. That means for every dollar spent on lobbying, the companies got $220 in tax benefits.

The implicit assumption in Alexander and Scholz’s research is that the money spent on lobbying had a decisive impact on the ultimate passage of AJCA. In other words, the money was not squandered courting legislators who were already inclined to support the AJCA, or who failed to support the legislation in spite of lobbying efforts.

It’s certainly possible that this is true. Lobbying efforts from large corporations with deep pockets are likely to have some influence over lawmakers, given the nature of American electoral politics. And if these efforts were decisive in the case of AJCA, then the 22,000 percent return on investment is pretty astounding. That magnitude is worth considering as a pure thought experiment, if nothing else.

But whatever value the 22,000 percent figure may have in informing the debate over corporate lobbying, there’s an obvious problem with any research design that presumes what it’s trying to prove. As I’ve written before, not all research is good research. Well-respected news organizations like NPR should do a better job explaining the potential shortcomings of different research methodologies rather than simply regurgitating findings.

The bigger issue, of course, is that researchers must be more willing to acknowledge possible flaws in their methodology. While reporters may lack expertise in research design, there’s really no excuse for academics attempting to pass off weak studies as providing definitive results.

And, unfortunately, that is exactly what seems to be happening in this instance.