Friday, July 17, 2009

What's Wrong with 2012 Polls?

There has been an entertaining discussion on the utility of polling going on this week between Conor Clarke over The Atlantic and John Sides (among others) at The Monkey Cage. Clarke got the ball rolling in the Idea of the Day special section of The Atlantic's site by questioning the usefulness of polls -- mainly the overabundance of them and the effect that has on democratic governance and government. Sides, in a rejoinder to Clarke's response to his rebuttal, counters by arguing polling's positives are based on the accountability, evidence and representativeness they provide. I'm not going to rehash the arguments here (but do urge you to go check the discussion out), but I did want to take issue with Sides on one of his closing remarks.
"Lest I sound like a cheerleader: some polls bug me, much as they must bug Clarke. I hate the vastly premature 2012 presidential trial heats, for example."
First, I should say that I obviously come down on Sides', uh, side of the argument, but this hits a little close to home considering the ramped up 2012 gazing on FHQ of late (see here, here, here and here). Also, before I get in to this, I think that it is instructive to make a distinction between the trial heats and the GOP primary polling for 2012. I find the latter to be of greater use simply because of the position the Republican Party finds itself in in the aftermath of last November's elections: out of the White House, out of power in both chambers of Congress and leaderless.

On that last point is where the primary polling is of particular import; it is gauging how Republicans perceive the field of candidates/leaders to be shaping up. Now, it may be that at this point in the cycle what we are seeing in the cross-tabs is being driven by nothing more than name recognition, but that is providing us with a baseline for comparison as this race fully formalizes over the next couple of years. Stated differently, it is providing us with some evidence of a basic ordering of candidates. We could rely on pundits to tell us that it is all Sarah Palin and Mitt Romney and Mike Huckabee or we could poll it to find some evidence of this (Of course, the logical question that emerges here is whether the polling is influencing the pundit-speak or vice versa.). For a party that is leaderless on the national level, the 2012 nomination question is more than a mere proxy.

But what about these trial heats? I've tried to caution FHQ readers about the limitations of these polls. First of all, with the exception of the University of Texas poll last week, all of the national and statewide polls asking the "if the 2012 presidential election were held today" question have been conducted by Public Policy Polling. That, in addition to the relatively small sample sizes (only above 700 respondents) for their national polls, is a hit for these polls on the representativeness score. And I'll grant Sides that. This may not be the most representative series of polls and as a result may not be the best evidence of the true state of something (the 2012 GOP nomination) that may not be a race yet.*

Here's the kicker, though: You'd expect these trial heat numbers to track with either Obama's approval or favorability ratings. But they don't; not across the board at least. Let's look at those PPP trends (Obama v. Gingrich/Huckabee/Palin/Romney) on the national level versus the data on the president's approval and favorability:

Now, there's been a steady decline in both the approval and favorability trends since they peaked for Obama in December and January respectively. But that's not the trend we see across the Public Policy Polling surveys. Obama opened with good leads over all four prospective Republican candidates, widened those leads in May and came back a bit in June.

What's the deal with May? Why is it that those numbers go against the grain of what we might otherwise expect? It may be something as simple as sample size. The May poll was the only survey in which PPP had a national sample size approaching the 1000 (+/-) respondent mark that is standard. The others hover in the 600-700 range. Again, representativeness takes a hit in those circumstances. The interesting question from this is whether PPP's other numbers outside of May are over- or understating Obama's or the Republicans' positions relative to each other. If all the samples were 1000 respondents in size would we have witnessed a steady decay was we see in the national favorability and approval numbers for the president? That's really the question here.

So have I made the case for the 2012 polls here? No, I don't think so. There are some problems with these polls -- sample size and the fact that only one organization is doing the general election trial heats. But what this does provide is some context and that was one of Sides' underlying points in this discussion; that the interpretations of these polls be more than, "Here are the numbers" and that the folks consuming them do so with an eye to detail.

The thing about polling is that a snowball effect can build rather quickly. Once you start asking a question like "If the 2012 elections were held today who would you vote for?" all that does is trigger increased polling to find if there was any validity to the original results. And on and on it goes. Is it too early in the 2012 presidential cycle to begin asking this question? Maybe. McCain campaign (circa 2000) strategist, Mike Murphy just yesterday tweeted this: "New polling out on GOP '12 race. I say ignore it and heed the Milt Gwirtzman Rule; nothing matters till first contest." That dovetails with John Sides' statement and I partially agree with both of them.

But if the prospective candidates are seemingly acting as if they are running (There's no doubt in my mind that the not-so-behind-the-scenes campaigning and positioning that is currently going on is the opening salvo in the 2012 invisible primary.), then wouldn't it behoove us to have some data to back that up? If the candidates are active and fundraising (even if through their PACs), wouldn't make sense to have the other piece of the puzzle that is so often looked at in these candidate/nominee emergence models (Mayer, for example). And wouldn't that have at least some effect on the piece of the puzzle Cohen, et al. brought to the table in The Party Decides (endorsements)?

Perhaps. But it could also be that I just like the polls.

...even if they're too early and meaningless.

*Though, the actions of Huckabee, Palin, Pawlenty and Romney make it appear as if each is at least angling for a national run. Huckabee is making seemingly beneficial endorsements in Iowa (Chuck Grassley, Bob Vander Plaats) and Florida (Marco Rubio). Both are early states given the 2008 calendar (2012 is subject to change) and those endorsements are solid given that each would help among conservatives in primaries or caucuses closed to independents and Democrats. Palin's actions have been difficult to explain, but there seems to be some national ambition in there. Romney is building the organizational infrastructure necessary to be successful in 2012 given his fundraising and strategic disbursements to candidates and leaders within the party. And Pawlenty's decision not to seek a third term seems to have at least been somewhat politically driven. Again, there appears to be some upward ambition there.

Recent Posts:
State of the Race: New Jersey (7/16/09)

Romney Leads 2012 GOP Race (...and in more than just the Gallup Poll)

Revisiting Democratic Delegate Allocation (1976-2008)


Robert said...

I think the most important thing in the 2012 polling is how it could influence the operatives in Iowa and New Hampshire who can run their campaigns. Other than that it doesn't amount to much. The most important activities for 2012 candidates are to build a campaign organization and get as much face time on national and local media as possible.

Josh Putnam said...

Under normal conditions, when Iowa and New Hampshire's positions are not as secure, I would add the caveat that even operatives in those states might hesitate before jumping on board any campaign. But for once, it appears that Iowa and New Hampshire's positions are safe on both sides. Neither the Republican Temporary Delegate Selection Committee nor the Democratic Change Position are seriously looking into curbing the influence of either state.

So, I suspect you may be right. It is just too early for these polls to mean much. However, just like a moth to a flame, I will continue to look at these polls. But you probably already knew that.

Robert said...

I'm shocked!