Legal Commentary

Environmental Substance Abuse

By Jonathan H. Adler

Speaking of errors by economists, environmental attorney Mark Atlas has authored a substantial tome dissecting over 500 social science studies of environmental policy, and it?s not pretty. The 700-plus-page paper, ?Environmental Substance Abuse: The Substantive Competence of Social Science Empirical Environmental Policy Research? is available on SSRN. Here is the abstract:

In a 2002 article, social science scholars criticized legal scholars for violating empirical analysis principles in law review articles. Their review of hundreds of empirical law review articles led to a pervasively grim assessment of these articles and their authors, concluding that empirical legal scholarship was deeply flawed, with serious problems of inference and methodology everywhere. In essence, the 2002 article argued that although legal scholars? articles might be substantively competent (i.e., knowledgeable about the law and facts), they were, at best, methodologically incompetent.

This Report reverses the 2002 article?s focus, assessing the substantive competence of social science empirical research articles, ignoring their methodological competence. This Report focuses on about 550 social science articles from peer-reviewed journals since the 1960?s that used quantitative research to study United States domestic environmental policies and practices. The 2002 article examined aspects of law review articles at which legal researchers might be deficient but at which social science researchers should be competent. This Report does the opposite by focusing on what legal researchers should be most expert ? determining the relevant laws, government policies, and facts. Consequently, just as the 2002 article evaluated whether law review articles violated empirical research rules, this Report evaluates whether social science environmental policy articles were incorrect or incomplete about the relevant laws, government policies, or facts.

Although the 2002 article concluded that every empirical law review article was fatally flawed methodologically, this Report does not conclude that every social science environmental policy article was fatally flawed substantively. However, the overwhelming majority of those articles were substantively uninformed, amateurish, shoddy, and/or deceptive. Anyone with a basic understanding of the environmental laws, policies, facts, and/or data relevant to any particular article would conclude after only a brief review that the article was seriously flawed. Unfortunately, social science journals publishing environmental policy articles have been like runaway trains of invalid research that keep picking up new passengers. This Report explains in detail the substantive problems with each of these articles.

I?ve certainly come across the occasional environmental economics paper that improperly characterizes a relevant law or regulation, but Atlas? attack suggests there is a more widespread problem within the literature, despite the use of peer review. If so, this would suggest that environmental economists need to understand more about environmental law, just as environmental law professors need to understand a bit about environmental economics ? and both need to understand something about environmental science.

The Atlas attack has caused a bit of a kerfuffle on a resource economics e-mail list, as David Zetland notes here and here.  Given his strong language, this is no surprise.  John Whitehead also offers a brief comment.

Full post as published by The Volokh Conspiracy on March 10, 2011 (boomark / email).