Polls Won't Tell You Who's Going To Win The Alabama Senate Race

Election polls often provide a pretty good sense of who’s about to win. But sometimes, what they tell us is simply that the race is too uncertain to make any bets before the votes are counted.

The surveys of Tuesday’s special election for Alabama’s next U.S. senator fall squarely in the latter category. Poll aggregation gives scandal-plagued Republican Roy Moore a negligible edge  as of Monday.  But that’s almost beside the point in a race where, on Monday alone, results ranged from a Fox News survey finding Moore trailing Democrat Doug Jones by 10 percentage points to an Emerson College poll  that gave Moore a 9-point lead.

The Alabama race is a case study in nearly everything that makes horserace polling tricky. Its mid-December date, and the fact that there’s nothing else on the ballot, means pollsters have relatively little data to guide their assumptions about who’ll turn out to vote. The flurry of sexual misconduct allegations surrounding Moore, and the shifting reactions of national and state Republicans the party’s national committee pulled funding for the race , but then reinstated it complicate things further.  

Add in variations in methodology, and the minor wrinkle of write-in votes (which will likely be scant, but which pollsters lack a good way of measuring), and you have a recipe for exactly the kind of volatile spread seen in the polls.

Some of Jones’ best numbers, it’s worth noting, come from pollsters like Fox News and The Washington Post who use live interviewers, rather than the automated calls used by many other survey-takers. The use of live interviewers also allows for direct calls to to cell phones, which automated polls can’t do . (Some pollsters using automated calls, though, attempt to supplement their samples by, for instance, using online panels.)

Fox News pollster Dana Blanton noted that the network’s survey found respondents interviewed by cell phone a group that tends to be younger and less white than landline users favored Jones by a 30-point margin.

“The fact that traditional, high-quality probability samples, like the Fox News poll, include both landline and cellphone numbers may be why these polls show Jones doing relatively well compared to automated or blended polls,” she wrote.

But it’s also worth noting that, if the 2016 election taught us anything, it’s that “traditional, high-quality” methods aren’t any guarantee of nailing the results . And the difference in mode (live-caller vs. automated calls) could also affect the surveys in other, unexpected ways. It’s not unthinkable that voters in this particular race could be more inclined to conceal their preferences from live interviewers.

And more than anything, as the hoary-but-true-political cliche goes, “it all comes down to turnout.”

If turnout looks like a typical off-year election in Alabama, Moore is the favorite to win. If disgruntled or conflicted Republicans end up staying home, or if fired-up Democrats flood to the polls, Jones has a better shot.

The problem is in figuring out which scenario is more plausible. Pollsters haven’t been consistent in finding whether Moore or Jones has an advantage among the smaller field of “likely voters” when compared to the entire electorate.

Several pollsters have driven home the uncertainty surrounding turnout by taking the still-unusual step of releasing multiple estimates of the race, and by publicly walking through the kinds of choices survey houses face when making sense of the data.

The online polling firm SurveyMonkey offered 10 different ways of looking at the same data using different weighting and assumptions about turnout. The results ranged from a 10-point lead for Moore to a 9-point lead for Jones.

“We know the actual electorate will be a fraction of all registered voters,” SurveyMonkey’s Mark Blumenthal said. ”[W]e just don’t know how small that fraction will be or who exactly will turn out.”

Using self-reported data on whether Alabamians felt they were probable or certain to vote gave Jones a healthy lead. Looking at self-reported data on their voting history favored Moore.

On top of all that, SurveyMonkey pollsters noted that in terms of how their sample group voted in last year’s presidential race, Donald Trump  only enjoyed an 11-point margin (he carried Alabama by about 28 points). Reweighting the results to match the outcome of 2016 boosted Moore’s numbers but it’s far from clear whether doing so is the right call.

The result “may indicate either a big underlying Democratic turnout advantage seeping into the response patterns of our respondents, or it may be a sign of sample bias in need of correction,” Blumenthal wrote. “Either way, when combined with the potential for purely random error inherent in all surveys, the findings make a projection of the outcome virtually impossible.”

A second turnout experiment, released Monday by Monmouth University , found a more modest spread, ranging from a 4-point lead for Moore when using a “historical midterm model, akin to Alabama’s 2014 turnout,” to a 3-point edge for Jones when using a “model with higher overall turnout, where voter demographics look more like the 2016 election.”

A third model, based on “patterns seen in last month’s Virginia gubernatorial race i.e. relatively higher turnout in Democratic strongholds,” found the race tied.

“In a typical year, we would probably default to the historical model, which shows Moore ahead,” Patrick Murray, the poll’s director, wrote. “It could still end up that way, but both 2016 and 2017 suggest that typical models may not apply.”

No matter what happens Tuesday, it shouldn’t come as a complete surprise. That doesn’t make the polling useless; when what the polls tell you is that a high level of uncertainty surrounds a race, they’re still telling you something important. And although many of the surveys will end up missing the mark, that’s not necessarily a blemish against the people fielding them. The Alabama race is, even above the normal challenges of polling, incredibly difficult to predict. The attempts to figure it will provide valuable data about the best ways to go about forecasting future elections.

WHAT OTHER PEOPLE  ARE SAYING ABOUT THE RACE:

Geoffrey Skelley, Sabato’s Crystal Ball : ”As the race enters its final days, there are reasons for optimism on both sides, though because of Alabama’s conservatism, we suspect the Moore team may have more reason for it. ...[W]hile we can’t say much about this race with certainty, we can at least say that the fact that the Crystal Ball can publish a serious article exploring a Democrat’s not-totally-implausible path to victory in a U.S. Senate race in Alabama is stunning.”

Darla Cameron, Dan Keating and Kim Soffen, Washington Post : “Moore is not a popular Republican. Popular enough to win but unpopular enough to lose a significant number of his party’s votes in the process. That pattern, in addition to the allegations against him, mean that (Jones) may have an opening to win. And where Moore underperforms the most — the northeast and areas near big cities — Jones may have the most potential.”

Harry Enten, FiveThirtyEight : “It’s a close race — close enough that the normal inexactness of polling gives either candidate a good chance of winning. Moore’s 3-point lead is smaller than the average error when there are only five to nine polls to average….Of course, Democrats shouldn’t go into Alabama assuming that a poll miss will necessarily benefit Jones. In fact, the favorite is just as likely to benefit as the underdog.”

Nate Silver, FiveThirtyEight : “What we’re seeing in Alabama goes beyond the usual warnings about minding the margin of error, however. There’s a massive spread in results from poll to poll ...and they reflect two highly different approaches to polling.”

David Byler, Weekly Standard : “It’s impossible to tell based just on one poll whether Jones or Moore will win or even which candidate is losing or gaining steam. But Fox News has a strong record in public polling , and this solid result for Jones reinforces the view that this race is truly a toss-up.”

 Justin McCarthy, Gallup : “According to Gallup data from the first half of 2017, only a handful of states have a greater Republican edge than Alabama consistent with what Gallup found in 2016 . Half of the state’s residents (50%) identify as Republicans or lean toward the Republican Party, while a third identify with or lean toward the Democratic Party (33%). Additionally, Republicans typically have a much higher turnout in non-presidential elections than do Democrats an advantage Moore could enjoy tomorrow.”

Philip Bump, Washington Post : “Whatever the outcome Tuesday, we will know two things about the polls that preceded it. First, that some polls, misfiring on turnout estimates, will have shown a very incorrect result. And, second, that this will be used to disparage polling in general as flawed and inaccurate.”

David Lauter, Los Angeles Times : ”Everyone wants to know who’s going to win, but forecasting the outcome isn’t the only reason — or even the main reason — for polling. Polls also tell us a lot about which voters are backing which candidate. ... Repeated polls can show how a race develops over time. ...[T]hey serve as a check on the self-serving spin that voters inevitably will hear starting Tuesday night, once the actual results are counted.”