Online VRA symposium: Social science goes to court

The following contribution to our VRA symposium comes from Adam Cox, Professor of Law at NYU School of Law, and Thomas Miles, Professor of Law and Walter Mandel Research Scholar at the University of Chicago Law School.  Together they are authors of a number of empirical studies of voting rights litigation.

The Constitution need not be read to require that Congress amass social scientific evidence before exercising its power to enforce the Fourteenth Amendment.  Surprisingly, however, lower courts confronting a slew of recent challenges to the Voting Rights Act have come close to imposing such a requirement.  If the Supreme Court holds Congress to that standard, the Act is in serious jeopardy.

At issue is the constitutionality of Section 5 of the Voting Rights Act, a provision that singles out some parts of the country, but not others, for more intrusive federal oversight.  The idea behind Section 5 was to target those places where voting discrimination was worst.  The problem, according to Section 5’s challengers, is that Congress has never bothered to ask whether those “covered” jurisdictions have gotten better.  If they are no longer worse, why still single them out?

Confronted with just such a challenge by Shelby County, Alabama, the D.C. Circuit upheld Section 5 of the Voting Rights Act.  Perhaps ironically, however, Judge David Tatel’s thoughtful majority decision defending Section 5 may ultimately undermine it.  This is because his opinion implicitly accepts the idea that Congress needed to compile serious empirical evidence that voting discrimination against minorities remains worse today in jurisdictions covered by Section 5 than in uncovered jurisdictions.  He concluded that such evidence existed, and for that reason upheld the Act.  The problem is that, as is too often the case, the litigation process did a poor job of testing the validity of the empirical evidence that was before the court.  That evidence turns out to be deeply flawed, leaving the D.C. Circuit’s opinion vulnerable on its own terms.

The evidence that Judge Tatel called the “most concrete” support for the claim that discrimination in covered jurisdictions remains worse than in uncovered jurisdictions comes from a study of litigation under Section 2 of the Act – a provision that permits private parties to challenge state and local voting practices as discriminatory.  The central conclusion of that study, known as the Katz study, was that successful Section 2 cases were more common, and that Section 2 plaintiffs prevailed more frequently, in covered than uncovered jurisdictions.  This disparity in the success rates of litigants is taken by both the study and Judge Tatel as powerful evidence that “racial discrimination in voting remains concentrated in the jurisdictions singled out for preclearance.”

The problem is that the data underlying the Katz study do not support this conclusion.  They provide no support for Section 5, and no court seeking to support the constitutionality of Section 5 should rely on this study or the underlying data.  There are three principal reasons why this is so:

First, the data on Section 2 litigation contains an unmistakable time trend that the Katz study ignored in reaching its conclusions.  In concluding that covered jurisdictions remain worse, the study lumped together all of the Section 2 claims decided between 1982 and 2004, the entire period studied.  But lumping together cases spanning more than two decades masks an important trend: litigant success rates in covered jurisdictions have been declining steadily since 1982.  In fact, the decline is so sharp that over the second half of the study period, from 1994 and 2004, there is no difference in the rate of claimant success in covered versus uncovered jurisdictions.  We have documented this pattern here, here, and here, in a series of papers studying judicial behavior in voting rights litigation.

These patterns – a steady decline in success rates in covered jurisdictions, and a disappearance of any difference in success rates between covered and uncovered jurisdictions – undercut the claim that the data support the ongoing need for Section 5.  As Judge Tatel himself explained, the question is not whether covered jurisdictions were worse in the past.  What matters is whether they are worse today.  In fact, if one thought that we should really take the Section 2 litigation data at face value (which we don’t; more on that below), then the data on which the majority relied would actually support the dissent: the data would count as conclusive proof that singling out covered jurisdictions is no longer justified, because today there is no difference between covered and uncovered jurisdictions.

Second, even if one ignores this clear pattern and lumps together cases from 1982 to the present, the difference between covered and uncovered jurisdictions disappears once one controls for other factors that are correlated with liability.  The Katz study did not recognize this because it simply compared raw summary statistics, with no other controls.   When we attempted to replicate the study’s results using regression analysis to control for confounding factors, we discovered that being in a covered jurisdiction bears no relationship to how likely Section 2 plaintiffs are to succeed. (See here and here.)  In fact, in subsequent work the authors of the Katz study have actually replicated our finding that coverage is not correlated with liability.

Third, there is a deep methodological flaw in the Section 2 study.  It is not simply that there are problems with the way the data were analyzed; it is that the data themselves are unreliable as a measure of discrimination.  Litigation success rates tell us nothing about the level of discrimination in different parts of the country because those rates are driven by a host of factors (such as resources and attorneys’ strategies) that are largely unrelated to the underlying merits of the claims.  We have explained this methodological problem at length elsewhere, and it has been the consensus view among social scientists for decades.  It is also well understood in other areas of law.  For example, were we to find that personal injury plaintiffs had higher rates of success in Connecticut than California, no one would take that as evidence that torts are more prevalent in Connecticut.  Similarly, no one would think that higher criminal conviction rates in Westchester than Washington, D.C. are evidence that there is more crime in Westchester.  Yet this is precisely the inference the Katz study would have us draw from litigation success rates in voting rights suits.

Not surprisingly, Judge Tatel senses this problem with the Section 2 litigation data.  He tries his best to correct for it by weighting the data by state population, presumably in an effort to take the raw counts of successful cases and turn them into a measure of per capita discrimination.  But this weighting does not improve matters.  One problem is that the adjustment to the data is largely untethered to any clear theory about what that data measure: Why use total population rather than minority population, which would have undercut the majority’s interpretation of the data?  And why, if the interest is in per capita measures, use a state’s total population when so many policies are local and thus do not affect everyone in the state?  But while these problems with the weighting might in theory be overcome, they are dwarfed by the much larger methodological problem that plagues the entire Katz study.  Simply put, no adjustment to the data can correct for the fact that, at a fundamental level, the data provide no meaningful evidence about whether discrimination is worse in one part of the country than the other.

To be clear, this does not mean that the D.C. Circuit was wrong to uphold Section 5.  There is a powerful argument to be made that Congress’s enforcement authority under the Fourteenth Amendment should not rise or fall on its ability to produce systematic quantitative evidence that support the precise contours of its civil rights legislation.  The demand for such evidence creates an onerous burden for Congress – as well as a very tough task for courts, who must oversee a litigation process that is often not well designed to test the soundness of social scientific evidence.  But if the Supreme Court follows the lead of lower courts and requires Congress to have compiled such evidence, the outcome is likely to be different than it was in the D.C. Circuit.