January/February 2001

January/February 2001 | Contents

LOW MARKS FOR POLLS, MEDIA

BY ANDREW KOHUT

Oddly enough, given its extraordinary ending, American voters liked the conduct of Campaign 2000 better than that of the three previous presidential contests. Issues were discussed more, the debates were more helpful, and the candidates slung less mud, said the voters in a Pew Research Center poll conducted on the weekend after Election Day. A resounding 83 percent of respondents said they learned enough to make an informed choice, which was a larger percentage than felt that way in Pew surveys in 1988, 1992, and 1996.

But the positive reviews did not extend to the press. The media was given the same low grades that it received in the aftermath of past elections. Just 28 percent give the press top grades of A or B for its election coverage, while nearly four-in-ten (38 percent) graded the media D or F. In other words, the public felt better informed this time, yet still had little good to say about the source of campaign information -- the news media.

Frustration and anger about election night miscalls is an obvious explanation. This was a media moment that will linger in people's minds. It epitomized longstanding, but still growing, public criticisms of the media for inaccuracy, intrusiveness, and unfairness.

Seven-in-ten voters (69 percent) voiced anger or disappointment with the networks' premature calls that George W. Bush had won the presidency. More than half of voters (52 percent) said the networks' earlier mistake of calling Florida for Gore may have influenced the vote in other parts of the country. Most voters felt the bad calls were motivated by the networks' desire to be the first to declare a winner, rather than serving an audience need.

While the media got the same rotten grades as in past cycles, public judgments of polling worsened again. Only 29 percent of Pews' respondents gave pollsters a good grade -- down from 34 percent in 1996, and 46 percent in 1992. It would be easy to say this reflected frustration with the exit polls, and to some extent it probably did. But there is more to it than that.

Clearly, the good job done by the polls in predicting a too-close-to-call outcome this year did not absolve them of their sins in the eyes of their critics. (Indeed, the NCPP -- an association of pollsters that work for the news media -- found the prediction error in 2000 for the major polls half of what it was in 1996 and lower than it had been in many other presidential elections.)

Had the polls been wrong, there would have been a great hue and cry that surveys are no longer effective, that they are misleading, and so on. But being accurate in the end did not improve the image of polling in a campaign that was tarnished by too many horse-race polls that confused rather than clarified public opinion. The tracking polls and the instant polls were the principal culprits.

Tracking polls are market research tools that have been dragged out of the backroom and onto the center stage of political reporting. The idea is to gauge the impact of media campaigns on target audiences by conducting continuous polls that roll in new results daily. They are a way to watch the electorate make up its mind in the final days of a volatile campaign -- especially in tight primary races. But, in recent elections they have been used very early in the game by the media when there's little to track.

This year, CNN/USA Today started their tracking on September 4, and freshened each day's reporting with 300 new interviews that replaced 300 taken three days earlier. The polling -- conducted by Gallup, which does high quality interviewing, sampling, and turnout screening -- nonetheless produced loopy results that defied credibility. In one five-day stretch in mid-September, for example, when very little was happening in the campaign, the horse race ranged from a ten point Gore lead to a three point Bush lead. And so it went for much of their two-month daily reporting.

In part, the bouncing around reflected the smaller sample sizes of the tracking poll. In fact, to protect itself, Gallup re-released its results on a six-day cycle and its new figures were quite stable and similar to those of polling organizations doing traditional surveys. But daily tracking also failed conceptually. A new national poll every day might sound like a good way to both report and make news, but it made little sense when voter opinions were mired in indecision and the campaign activity was limited to the battleground states.

The tracking poll gave the impression of great voter volatility, when in fact other polls, conducted as discrete surveys, found only modest shifts in voter opinion during the general campaign. But Gallup's continuous survey and other tracking polls drowned out the stability evident in these discrete surveys.

The instant polls also poisoned the well of public confidence. One particularly bad moment for these on-the-spot polls came when they declared Gore the winner of the first presidential debate. This could not have been more wrong. Gore support waned in subsequent polls, which registered the degree to which voters disliked the vice president's performance, even though he had "won" the debate.

Despite these problems, tracking polls and instant polling will not disappear in 2004 given the fierce competition for the fragmented news audience. But the press and the pollsters should use these methods with more discretion. For example, CNN/USA Today could have beefed up the sample size of its tracking poll when it was obviously malfunctioning. Or at least affix clear warning labels on instant poll results.

Andrew Kohut, director of the Pew Research Center for the People and the Press, writes regularly for CJR about public attitudes toward the media.