Gary Langer, the director of polling at ABC News, takes a cut at both how seriously we need a reliable answer to the polling problems and how complex a good answer might eventually be.
---------
New Hampshire's Polling Fiasco
January 09, 2008 12:05 AM
There will be a serious, critical look at the final pre-election polls in the Democratic presidential primary in New Hampshire; that is essential. It is simply unprecedented for so many polls to have been so wrong. We need to know why.
But we need to know it through careful, empirically based analysis. There will be a lot of claims about what happened - about respondents who reputedly lied, about alleged difficulties polling in biracial contests. That may be so. It also may be a smokescreen - a convenient foil for pollsters who'd rather fault their respondents than own up to other possibilities - such as their own failings in sampling and likely voter modeling.
There have been previous races that misstated support for black candidates in biracial races. But most of those were long ago, and there have been plenty of polls in biracial races that were accurate. (For more on past problems with polls in biracial races, see this blog I wrote for Freakonomics last May.) And there was no overstatement of Obama in Iowa polls.
On the other hand, the pre-election polls in the New Hampshire Republican race were accurate. The problem was isolated to the Democratic side - where, it should be noted, we have not just one groundbreaking candidate in Barack Obama, but also another, in Hillary Clinton.
A starting point for this analysis will be to look at every significant Democratic subgroup in the New Hampshire pre-election polls, and see how those polls did in estimating the size of those groups and their vote choices. The polls' estimates of turnout overall will be relevant as well.
In the end there may be no smoking gun. Those polls may have been accurate, but done in by a superior get-out-the-vote effort, or by very late deciders whose motivations may or may not ever be known. They may have been inaccurate because of bad modeling, compromised sampling, or simply an overabundance of enthusiasm for Obama on the heels of his Iowa victory that led his would-be supporters to overstate their propensity to turn out. (A function, perhaps, of youth.)
Prof. Jon Krosnick of Stanford University has another argument: That the order of names on the New Hampshire ballot - in which, by random draw, Clinton was toward the top, Obama at the bottom - netted her about 3 percentage points more than she'd have gotten otherwise. That's not enough to explain the gap in some of the polls, which presumably randomized candidate names, but it might hold part of the answer.
The data may tell us; it may not. What's beyond question is that it is incumbent on us - and particularly on the producers of the New Hampshire pre-election polls - to look at the data, and to look closely, and to do it without prejudging.
blogs.abcnews.com |
