dougjn - re: "random" polls.
Polling today is a much more sophisticated than "randomly chosen Americans." Some of the trade secrets among "opinion research" firms center around their ability to rapidly acquire statistically valid results for their customers - it is a business and there are competitors.
You may not be aware of the term "profiling," but with today's systems, research firms can almost instantly customize a target base for a poll - and can even predict, within a few percentage points, the results of any poll their customer calls them for. It's the same rationale that some are using to suggest doing the census with appropriate sampling technique.
The two keys to a statistically valid poll outcome for the customer are: 1) What you ask; 2) Who you ask.
We have some information as to the former - usually we get to hear at least part of the question, but we rarely get to hear the whole body of interrogation and I think you, as well as anyone, would recognize how one can lead someone with questioning. As to the latter, profiling allows very in-depth pre-judgement of an outcome based on a number of data elements which, again, are considered highly protected trade secrets and take significant time, effort and money to collect. Such information includes, but isn't limited to: Age; sex; income; education; ethnicity; citizenship; legal judgements; religious affiliation; internet/e-mail us use; long distance service provider and calling patterns (i.e., local, state, national and international); cell phone ownership and usage; current employer & employment history; home & property ownership; last date/time polled and results; organization memberships; medical/health insurance status; age and status of parents; children - number and ages and schools they attend(ed); voter registration/party affiliation; magazine, newspaper and periodical subscriptions; the "purchasing power index" (an economic profile on it's own showing things like credit card activity and items purchased with credit, frequent flyer mileage, etc.); vehicle ownership/registration, etc., etc. The more data you have over a longer period of time, the better you are at the business of weeding out those you don't want to waste time calling.
Time is money, phone/agent time is money. They use predictive dialing so that the agent isn't sitting there waiting for someone to answer the phone. The list of targets is programmed into the system and the bank of agents has the script on the computer; when the call is answered by something other than an answering machine or voice mail, one of the agents is "handed the call." Often, these agents are in another company's call center, sitting in a room with hundreds of other people making things like sales and taking customer service calls. They get paid by the number of calls they attempt and the time they spend, so you give them sure-fire (the term is "hot targets" in the boiler room) contacts.
If the research organization has been in the business any length of time, the profile database has become very large, and they add to it every day - not just names, but filling in little details as they buy them from other data gathering entities to make it more and more accurate (and time-saving). It is all done in the name of accuracy, so that the poll is conducted to provide the customer with what they expect.
With product market research polling, the customer may not really know what the results are going to be, and it is done to improve a product or find out about a competitor's products or service. In political opinion and voter polls, the outcome is usually already known - the question is what is the percentage and has it shifted in a given time period.
There is a lot of room to give the customer what they want, and after about 1300-1500 "hits" the data is statistically valid. Assuming, of course, that you call the right target in the first place. Or, if you have enough data, you don't even have to make that many calls. A good rule of thumb is to use your natural sense and realize that if they only call a few hundred people, it's because they knew precisely who to call to get the results.
Now, to be fair - I have to say that a well-respected research firm is honest with it's customers, and after dealing with them for a long time, the customer can rely on the firm to give them accurate results quickly and economically. I think in most cases, if they took the time to call 100,000 people truly at random, the results would be different. Hence, why polls are usually off compared to voting results. (Lots of statistical and interviewing details behind that issue - won't bore you further.)
Can we all be so sure about "random" polls? No. It's like sausage - now that I know how it's made and what's in it, I don't eat it.
Mr. K.
|
