2% Among African-Americans?
Mystery Pollster
mysterypollster.com
By Mark Blumenthal on Polls in the News
Has George Bush's job approval among African Americans really fallen to just 2%? While MP was off atoning for his sins, several national pollsters released new surveys. More on the bigger picture later, but for now let's focus on that 2% number that comes from the latest Wall Street Journal/NBC poll. While this particular result appeared nowhere in either of the poll summary stories that appeared in the Journal or on MSNBC.com, Tim Russert hyped it with great enthusiasm to a considerably larger audience on the NBC Nightly News with Brian Williams on Wednesday night:
Brian, listen to this. Only 2 percent -- 2 percent! -- of African-Americans approve of George Bush's handling of the presidency -- the lowest we have ever seen in that particular measure.
WashingtonPost.com's Dan Froomkin (who also transcribed the above) contacted Democratic pollster Peter Hart, who regularly conducts the NBC/WSJ poll with Republican Bill McInturff . That support, according to Froomkin, registered at 19% "as recently as six months ago." Hart, Froomkin reports, had "never seen such a dramatic drop in presidential approval ratings, within any subgroup."
Now, the full results posted online by the Wall Street Journal do not include either a cross-tabulation of the job rating by race or the racial composition of the poll sample. However, Froomkin reports the following:
This latest poll included 807 people nationwide, and only 89 blacks. As a result, there is a considerable margin or error -- and the findings should not be considered definitive until or unless they are validated by other polls.
How big is that margin of error? Assuming simple random sampling, the standard 95% confidence level used in most public opinion surveys, the maximum margin of error on 89 interviews is 10%. However, those who took statistics may remember that the maximum error range applies for percentages near 50%, it gets smaller as percentages approach 0% or 100%. If one plugs 2% and n=89 into the standard formula for the margin of error, and I get +/- 3%. However, statisticians have long debated whether we should ignore the sharp reductions in the margin of error at very small percentages. Regardless, media pollsters are typically cautious about reporting on results from sample sizes of less than 100 interviews.
That is one reason why national media pollsters typically conduct special "oversamples" of African Americans when they want to investigate racial differences in attitudes. They weight those additional interviews back to match census estimates of the racial composition of American adults for their overall tabulations. However, the extra interviews give them more confidence in crosstabulations by race. Gallup, CBS/New York Times and ABC/Washington Post did just that with their post-Katrina polls last month.
MP also wonders about the 89 interviews of African Americans that Froomkin reports. Is that a weighted or unweighted total (and thanks to a loyal but anonymous MP reader that emailed with that question)? MP suspects it is weighted and that the actual number of African American interviews was a smaller number than 89.
Here's why: 89 interviews amounts to 11% of a sample of 807 adults. Eleven percent happens to be the percentage of African-Americans among US adults (as estimated by the 2000 Census). Most national pollsters weight their samples to match that estimate (see the percentage black in the last Washington Post poll). Response rates tend to be lower in urban areas, and as a result, unweighted national samples typically under-represent African Americans. As such, most national pollsters weight African-Americans up slightly to match the estimates provided by the US Census. Thus, the odds are good that the WSJ/NBC poll did not come in at 11% black unweighted.
But MP is burying the lede. Rather than speculate, we can simply check this result against other national polls conducted in the last week to see if the NBC/WSJ poll was a statistical outlier. Dan Froomkin found one:
Late Update: The Pew Research Center is just out with its latest poll, which has a larger sample, and it finds Bush's approval rating among blacks at 12 percent, down only slightly from 14 in July. Here are those results.
Note: Pew's sample included 135 African Americans and was fielded October 6 through 10. It should not be a difficult matter to obtain similar tabulations from some of the other public polls released this week. So stay tuned.
Meanwhile, a note to powerful pundits who present polling data on national television: A sample of less than 100 interviews can be a dangerous thing. Handle with caution, not exuberance. |
