A Pollster explains why he is wrong. :>)
_______________________________________
THE POLLSTERS
Mark Mellman
Voter status and the pollsters
Pity the pollsters. We have an even tougher job than most survey researchers.
They have to obtain a random sample of a clearly defined universe — kids, seniors or cancer survivors. That is difficult enough.
But political pollsters need to survey a universe — voters — that does not even exist at the time we poll. People are either over 65 or under 18, or not. However, their status as voters is not fixed until the polls close.
Pollsters have long touted their skills at finding “likely voters.” Some even brag that they focus on “perfect” voters —those who vote in every election.
Much of this is hooey.
Many pollsters will use one or more “screening questions” to allegedly ascertain who is and is not a “likely voter.” The balance of the survey is then administered to those who are judged to be “likely voters,” while the rest get a polite hang-up.
Two sets of problems emerge. First, how good are the screening questions at ascertaining likelihood of voting? Not very. Of course, all pollsters say their screens are “well tested” and they “work.” But have they gone back, after the election, to determine how many of those who passed or failed their screening test did or did not actually vote? No.
Academic researchers have undertaken such studies in the fairly distant past, with less than encouraging results. Because it is a socially desirable behavior, the intent to vote is overreported.
But even if screening questions could really separate “likely” from “unlikely” voters, then what? Most pollsters get rid of “unlikely voters.” Why count those who won’t show up at the polls?
But “likely” and “unlikely” are statements about the probability of voting, not absolutes. A “likely” voter may have an 80 percent chance of going to the polls, and an “unlikely” voter only a 20 percent chance of showing up. That means out of every 100 “likely voters,” 20 will stay home. Out of every 100 “unlikely voters,” 20 will cast a ballot. Thus it is possible to have an electorate that consists significantly of “unlikely voters.”
That is not just a theoretical possibility; it is an objective reality. One of the most popular ways of defining “likely voters” is to ask whether the respondent voted in the last election. Those who say “yes” are “likely voters” and kept in the sample; those who say “no” are deemed “unlikely” and discarded.
Let’s ignore the first problem we discussed and assume the answers are correct. How well does past voting define the likely electorate? In 2000, 66 percent of California voters had voted in the presidential election four years before. But 34 percent of the electorate were “unlikely” voters by that criterion. Only 56 percent were “perfect” voters having participated in both ’96 and ’98. Perhaps more troubling for this method, nearly a quarter (23 percent) of Californians who cast a ballot in 2000 had not voted in either 1996 or 1998.
California is not unique. In Pennsylvania, 24 percent of 2000 voters had not participated in either 1996 or 1998. In Colorado, only 47 percent of 2000 voters had cast ballots in both ’96 and ’98, while 28 percent had not voted in either prior election. Could misreading the “likely electorate” have contributed to 2002 polls’ being off in Colorado?
Our real goal is not so much to determine who is a likely voter but to ascertain what the likely electorate will look like. Another reason to pity the pollsters — so much time and effort expended chasing after the wrong goal.
Of course, there are better ways to model the likely electorate, but I’m not going to give them away in a column that comes free with your subscription.
Mark S. Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982.
thehill.com |
