Study: the likely voter model in that poll you’re reading about is probably garbage

Pew is out with a new study today comparing the relative merits of the various ways in which pollsters try to determine which of their respondents will actually cast ballots on Election Day.

The results don’t look good for most commercial pollsters. As in, the vast majority of the polls the media reports on, and are included in the polling averages that we use to gauge how the horse race is shaking out, rely on likely voter screens that fail to accurately portray the actual likely electorate. As Pew writes:

Vote via Shutterstock

Vote via Shutterstock

Public pollsters, such as Pew Research Center and the major news organizations that conduct election polls, typically have used random digit dial (RDD) samples to reach a random sampling of all Americans, then narrowing down to prospective voters by asking people a series of questions that gauge interest in the election, past voting behavior and intention to vote. Campaign pollsters tend to use samples from databases of registered voters and incorporate past vote history from those databases into their forecasting models, ensuring that they know whether the respondent has voted in the past. The sample employed in this study was originally obtained from an RDD survey and later matched to a voter file so that both the survey questions and the past vote history could be used in the analysis.

All of the methods examined here result in more-accurate forecasts than using either all those respondents who say they are registered to vote, or else all those who say they intend to vote, both of which include far too many people who ultimately will not cast a ballot.

The upshot here serves as confirmation for a point I’ve made before: non-voters are really bad at self-reporting their voter status, vote history and voting intentions. Voting is a democratic norm, and people who rarely, if ever, participate in the electoral process still feel social pressure to tell a pollster that they are a good democratic citizen. This being the case, asking a random sample of US adults if they are registered to vote, if they voted in x out of the last y elections and if they are planning to vote this coming November is bound to overestimate the eventual size of the electorate. But that’s exactly how most commercial pollsters operate.

They operate this way because keeping up to date voter files, and using those voter files’ voter histories to model the electorate, is time consuming and expensive. However, that’s time and resources that national party organizations and political campaigns are willing to spend, for a few reasons. For starters, they need to maintain up-to-date voter files for reasons that go beyond polling, so the data from which to pull is already available. But they also have a different set of incentives: Commercial pollsters have to make money, and they do that by churning out a large number of polls that (hopefully) turn out to be somewhat close to the end result. Campaigns, on the other hand, are infinitely more concerned with making their polls accurate than they are with making them profitable.

That said, the study also finds that any attempt to screen for likely voters is better than none at all — something that prior research has shown to be more true the closer to Election Day the poll is conducted. However, as commercial pollsters continue to produce popping headlines about shocking twists and turns in the presidential race, it’s always worth checking their sample, seeing if it screens for likely voters and checking whether that screen is based on self-reports or verified voter registration and history.

As one might have guessed, details like that matter.

Jon Green graduated from Kenyon College with a B.A. in Political Science and high honors in Political Cognition. He worked as a field organizer for Congressman Tom Perriello in 2010 and a Regional Field Director for President Obama's re-election campaign in 2012. Jon writes on a number of topics, but pays especially close attention to elections, religion and political cognition. Follow him on Twitter at @_Jon_Green, and on Google+. .

Share This Post

7 Responses to “Study: the likely voter model in that poll you’re reading about is probably garbage”

  1. Tarapcarter4 says:

    ❝my neighbor’s mate is getting 98$. HOURLY on the internet❞….

    A few days ago new McLaren F1 subsequent after earning 18,512$,,,this was my previous month’s paycheck ,and-a little over, 17k$ Last month ..3-5 h/r of work a day ..with extra open doors & weekly paychecks.. it’s realy the easiest work I have ever Do. I Joined This 7 months ago and now making over 87$, p/h.Learn More right Here
    ➤➤➤ http://GlobalSuperEmploymentVacanciesReportsMoney/GetPaid/98$hourly❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦.❦

  2. Essus says:

    Having worked in market research/polling in field offices and such, I always take polling info with a grain of salt. There are many ways in which data is skewed, one of which is landline only.

  3. Zorba says:

    Hey, wait just a minute there! While I agree that using only landline owners for polling most likely results in very inaccurate polls, some of us do not have landlines because of “the security blanket of antiquated technology,” and this may be hard for many of you to believe, some of us live in areas where there is no reliable cell phone service, and if we want to call people, or call for emergency services, we need to have a landline.
    People who live in very rural areas, people who live in mountainous regions, very frequently can’t get cell service. We live in a rural area, on top of a mountain, and we have to drive about 3/4 mile down the mountain before our cell phones work. Our landline is not a “security blanket,” it is a necessity.
    Not everyone in this country lives in cities, towns, and suburbs.

  4. koolaidyarn says:

    Same here in SC. Close to election time, we get easily 3-4 calls every damn day. We hang up on the robots, but if a live person happens to call, and I’m in the right mood, I’ll start out with “Sure I’ll answer your poll, but I have to tell you up front that I plan to lie on at least half of my answers. You can try to figure out which ones are the truth.” It never ceases to amaze me how many callers don’t seem to give a shit if I tell them flat-out that I’m deliberately providing them with bad data.

    I also can’t help but wonder if the decline in landlines is skewing phone polls to concentrate their efforts on just a handful of houses, because we get polling calls on the landline, but not on the individual cell phones. If the only people being asked are the select demographic who refuse to give up the security blanket of antiquated technology, what does that imply about the poll results?

  5. Opinionated Cat Lover says:

    Honestly, I don’t envy pollsters their job. They have to predict a capricious group’s behaviors based on past behaviors, in a situation where the group doesn’t even know itself what it’s going to do. No-win scenario….

    The best way to handle guessing what the public is going to do is pretend like your guy(/gal) is gonna lose and fight hard to make sure that doesn’t happen. That’s just my personal idea.

  6. BeccaM says:

    Polls began decoupling with real voting patterns when they failed to update their technology to keep up with the times. Ever since the 1990s, more and more people stopped having land-lines altogether, which is the pollsters preferred (and sometimes sole) means of polling. Websites, email, texting — all of these are also ignored.

    Basically, they need to come up with better methodologies all around, including accepting they really don’t know who ‘likely voters’ are these days.

  7. nicho says:

    I used to live in NH. For me, it was great sport to fuck with the pollsters’ minds. One poor woman went through her entire list of questions, and I savaged the GOP from beginning to end. Her last question — and even she started laughing as she read it — was whether I’d be willing to volunteer at the local GOP headquarters. I said, “Not if you held a freaking gun to my head.”

© 2020 AMERICAblog Media, LLC. All rights reserved. · Entries RSS