Federal Judiciary

Public Policy Polling or Controlling? -- By: Jim Geraghty

For political junkies, the biggest names in polling, like Gallup and Mason-Dixon, never seem to offer new results frequently enough. But political reporters, like nature, abhor a vacuum, and so newer polling organizations have stepped in to fill the void, becoming widely cited and shaping the conventional wisdom about this year's competitive races.

Among these newer organizations are Rasmussen Reports, the prolific firm headed by Scott Rasmussen, and Research 2000, which is currently embroiled in a messy, high-profile legal fight with its best-known client, the liberal website Daily Kos. Then there's the Raleigh-based Public Policy Polling, founded in 2001 by businessman Dean Debnam.

#ad#In many ways, PPP has earned its status as a media favorite: It conducts lots of polls, provides detailed interpretations of what it thinks the results mean, asks timely and probing questions that hone in on the political world's interests (like whether the president's endorsement helps in high-profile Senate races), and even hosts online polls about which race it should poll next.

But it is a bit jarring to hear a pollster say, 'We're absolutely rooting in the race. We don't want Richard Burr to get reelected,' as PPP's Tom Jensen declared to Politico last year. (Jensen did go on to say, 'But our reputation is predicated on getting it right, and we're not going to cook the numbers just to tweak Richard Burr's nerves. They are what they are.') Burr is a first-term Republican North Carolina senator who has complained that PPP's partisan affiliation is largely ignored and that his numbers in PPP's polls seem strangely low compared to his numbers in other polls.

Indeed, the PPP client list reads like a who's who of North Carolina Democratic politics. Publications that cited the firm's polls without mentioning any Democratic affiliation include the Washington Post, Politico, the Chicago Sun-Times, the Milwaukee Journal Sentinel, the Dayton Daily News, the Austin American-Statesman, AOL's Politics Daily, The Atlantic, The Hill, the Pittsburgh Post-Gazette, and local North Carolina television stations. One GOP consultant in the state notes that the firm's name echoes those of the Public Policy Institute at Western Carolina University and the Carolina Institute for Public Policy at the University of North Carolina, two genuinely nonpartisan educational institutions. The consultant speculates that PPP benefits from the perceived association.

It's not just Republicans who question the firm's results and accuse it of playing favorites. In February, as the Democratic primary to decide who would run against Burr began heating up, candidate Ken Lewis complained that Debnam had donated $2,400 to one of his rivals, Cal Cunningham.

#page#In the previous cycle, Debnam made a pair of $2,300 contributions to Democrat Kay Hagan during her primary- and general-election bids for the U.S. Senate; she wound up defeating first-term incumbent Elizabeth Dole. (FEC records indicate that since 1995, Democratic candidates and committees have received $41,000 in donations from 'Dean Debnam,' 'C D Debnam,' 'Carey Dean Debnam,' and 'Carey Debnam'; the pollster has a daughter named Carey.) PPP has released polls on many races where Debnam has contributed to one of the candidates or relevant party committees, most notably those 2008 races involving Kay Hagan. The donations were not disclosed when the polls were released.

#ad#In addition, back in December, Tom Jensen told a North Carolina television station that PPP had conducted a poll for another of Lewis's rivals, Elaine Marshall, but he added that 'we're not actually part of any of the campaigns.' Lewis demanded that the firm include a disclaimer in future poll releases, a demand that was ignored.

In fact, the firm's May poll on the Democratic primary featured one past client and one recipient of the maximum legal donation from Dean Debnam. Neither fact was mentioned in the release. In June, the firm told the Raleigh News and Observer that it did not plan to release any polling data on the runoff between Cunningham and Marshall because it was working for a private client, but it declined to disclose the identity of the client.

Many pollsters work for partisan clients, and Debnam and his staff are free to donate to any candidates they like. But not disclosing that one candidate is a recent client is problematic.

Overall, PPP has a reputation for reliability; in 2008, the Wall Street Journal looked at various pollsters' results and ranked PPP among the best. Its last poll in the New Jersey governor's race had Republican Chris Christie winning by six points (he won by four), its last poll in the Virginia governor's race had Bob McDonnell winning by 14 (he won by 18), and its last poll in the Massachusetts special Senate election had Scott Brown by five (on the nose).

But this year in particular, PPP polls tend to have Democrats making up a larger share of the sample than one might expect in a midterm with strong headwinds against the incumbent party. In fact, in poll after poll, PPP samples have Democrats making up a larger share of the electorate than they did in 2008, at least according to exit polls conducted by CNN.  

In North Carolina, for example, PPP's latest poll splits 46 percent Democrat, 36 percent Republican, and 18 percent independent. That's a more heavily Democratic electorate than North Carolina had on Election Day 2008, when it split 42 percent Democrat, 31 percent Republican, and 27 percent independent.

#page#The same pattern prevails in several other states. Kentucky: PPP -- 52 percent Democrat, 37 percent Republican, 11 percent independent; 2008 exit polls -- 47 percent Democrat, 38 percent Republican, 15 percent independent. Ohio: PPP -- 44 percent Democrat, 38 percent Republican, 18 percent independent; 2008 exit polls -- 39 percent Democrat, 31 percent Republican, 30 percent independent. Pennsylvania: PPP -- 50 percent Democrat, 42 percent Republican, 8 percent independent; 2008 exit polls -- 44 percent Democrat, 37 percent Republican, 18 percent independent. Texas: PPP -- 37 percent Democrat, 43 percent Republican, 20 percent independent; 2008 exit polls -- 33 percent Democrat, 34 percent Republican, 33 percent independent.

#ad#Once in a while, PPP's sample will end up less Democratic than the 2008 breakdown; its Wisconsin sample splits 33 percent Democrat, 32 percent Republican, and 34 percent independent; on Election Day 2008, this state split 39 percent Democrat, 33 percent Republican, and 29 percent independent.

In all these cases, the PPP sample includes proportionally fewer independents than the most recent election. Independents are traditionally less active in midterm elections; on the other hand, most pollsters have found the number of Americans who self-identify as independents increasing since 2008.

Will Democrats really make up a larger share of the electorate in 2010 than they did in 2008, when the Obama campaign poured enormous resources into its get-out-the-vote effort, and droves of Democrats and first-time voters came out to elect the first African-American president?

Jensen told NRO that comparing exit polls to standard pre-election telephone polls 'is complete apples and oranges'; he says he never uses them as a guide for his polls. 'Yes, our [Kentucky] poll had 52 percent Democrats and the 2008 exit poll showed only 47 percent Democrats,' he said. 'But only 59 percent of the Democrats in our poll yesterday say they voted for Obama, whereas the exit poll showed 69 percent of Democrats voting for Obama. . . . If I call you up today and ask you who you're voting for, you might call yourself a Democrat, but if I ask you to fill something out right after you voted for John McCain you're less likely to call yourself a Democrat.' That is indeed possible, but it is worth noting that other pollsters find fewer Americans identifying themselves as Democrats since 2008, not more, and more voters identifying themselves as independents.

The partisan breakdown of each state's and district's electorates will shift from year to year, so pollsters are entitled to some leeway in their samples. But at some point, a poll sample just tilts too heavily to one party to be a reliable guide to the electorate as a whole. A poll of New York voters that had a sample in which half the respondents identified themselves as Republicans would be widely dismissed, and deservedly so; there just aren't that many GOP voters in the state. None of the samples from PPP are quite that outlandish, but they tend to presume much lower turnout among independents (plausible, considering the lower level of interest in non-presidential years), similar turnout among Republicans (unlikely, considering the indicators of high GOP enthusiasm), and higher turnout among Democrats (hard to imagine, barring some unexpected new factor that rejuvenates Democratic enthusiasm to its 2008 peak).

#page#Jensen told NRO that the firm weights its samples only for gender, race, and age. 'I think the 2008 vote is a better gauge of the representativeness of a poll than comparing the party ID numbers to the 2008 exit poll, but we don't weight for that either; we just put it out there for people to see,' he said. 'Last year, Democratic turnout was incredibly depressed in Virginia, but much less so in New Jersey. It would have been a mistake to weight those states to 2008 party ID or presidential horse-race numbers because the level of engagement is just different in different states.' Jensen can find lots of reasons why Democratic turnout may not be so bad in some states: 'In places like Texas, where Democrats have a great candidate in Bill White, and Kentucky, where Rand Paul has a polarizing impact, we're not seeing much falloff in Democratic interest from 2008. In Wisconsin, Pennsylvania, and Iowa, which all have very unpopular Democratic governors and where the president's popularity has really declined, we are seeing big dropoffs in Democratic interest, similar to what we saw in Virginia at this time last year. As a whole, Democratic turnout will certainly be down from 2008, but that's going to be true to different degrees in different states, and you can't take a one-size-fits-all approach to state polling.'

#ad#Jensen says the only polls from the past year that he is 'really unhappy with' were the ones for two special House elections -- in New York's 23rd Congressional District and Pennsyvania's 12th. Jensen says the firm 'seriously undermeasured Democratic interest and as a result overestimated Republican chances of winning. It's fine with me if your readers want to not believe our polls, but we have no history of cooking the numbers for Democrats. In 2009, we were out of the gate, well before even Rasmussen, showing that folks like Blanche Lincoln, Michael Bennet, Jim Doyle before he retired, and Bill Ritter before he retired were in serious trouble.'

All fair points, but the experience of Research 2000's impact in the Arkansas Democratic Senate primary indicates how much a poll or two can do to establish a powerful narrative. The polls, conducted for the Daily Kos, showed Bill Halter running ahead of incumbent Blanche Lincoln. This meant that he was treated as the favorite and she was covered as an embattled underdog, even though there is now no undisputed evidence that she ever trailed in the runoff.

Finally, a Republican consultant in North Carolina notes that PPP sometimes draws sweeping conclusions from tiny subsamples. For example, in August 2009, Jensen wrote about one of PPP's North Carolina polls, '9 percent of voters in the state disapprove of both Kay Hagan and Richard Burr's job performance. Those voters not surprisingly also give Barack Obama a 35 percent approval rating and only 12 percent like Bev Perdue. So these are the voters who basically don't like anybody. They're a Republican-leaning lot. 53 percent of them say they voted for John McCain last year to just 36 percent picking Barack Obama, and 45 percent are conservatives. . . . These folks are just mad in general and rather than taking it out on the party in control, they're taking it out on the person they can control, which in this case just happens to be Burr.' Of course, this is all discussing 9 percent of a sample of 749, which amounts to roughly 67 voters. The margin for error on a sample that small is roughly 11 percent, raising the question of whether these numbers provide any actual analytical value.

Back in February, when Democrat Ken Lewis complained about PPP's donations and past work for his rivals, Jensen responded that 'every statewide campaign that has ever publicly attacked us -- Richard Moore, Bob Orr, Elizabeth Dole -- has ended up losing badly.'

Perhaps the latest candidate to scoff at PPP results, Richard Burr, will end up losing badly as well. Of course, that scenario is just what Public Policy Polling has been rooting for all along.

-- Jim Geraghty writes the Campaign Spot on NRO.

Full post as published by Bench Memos - National Review Online on July 12, 2010 (boomark / email).