The Upshot, FiveThirtyEight and Election Forecasting

Nate Silver. (Photo by Astrid Stawiarz/Getty Images for the 2014 Tribeca Film Festival)

Nate Silver. (Photo by Astrid Stawiarz/Getty Images for the 2014 Tribeca Film Festival)

The long-awaited midterm elections are almost upon us. When the polls close tomorrow night, analysts and pundits will begin to sort through the results looking for meaning, patterns and possible insight into 2016. Most of the big picture questions are already resolved. The Republican Party will retain control of the House of Representatives and are expected to win back control of the Senate. There is a slim possibility the Senate will be deadlocked at 50 seats for each party, leaving Vice President Joseph Biden with the tiebreaking vote, but the more probable outcome is that the Republicans will end up with between 52 and 54 seats.

Election Day will also make it possible to evaluate the analysts. Amateur and professional pundits are busy making their predictions for what tomorrow’s results are show. Arguably, the most well known are Nate Silver of FiveThirtyEight, and The Upshot, a part of The New York Times (Mr. Silver, of course, wrote for the New York Times before forming a partnership with ESPN). FiveThirtyEight and The Upshot are both fun and stimulating reads, but they also make claims about the Senate races that are about to be tested.

The Upshot has identified seven competitive Senate races: Alaska, Colorado, Georgia, Iowa, Kansas, New Hampshire, North Carolina. FiveThirtyEight adds Louisiana to its list. Among these eight races, there is little disagreement between the two models. Both expect the Democratic or independent candidate to win in Kansas, New Hampshire and North Carolina, and the Republicans to win the remaining five states. This would give the Republicans a total of 52 seats in the new Senate. Significantly, neither model makes definitive predictions about the chances of one party winning any specific race, but gives a their estimate of the likely percentage of that happening. For example, The Upshot gives Democrat Kay Hagan a 73% chance of winning in North Carolina and Republican David Perdue a 57% chance of winning in Georgia. Over at FiveThirtyEight, the numbers are very similar with Hagen given a 71% chance and Perdue a 68% chance of winning.

Mr. Silver achieved fame for predicting the electoral college results with great accuracy in both 2008 and 2012. This year, both he and The Upshot, using similar methodologies, have arrived at essentially the same conclusions. Interestingly, both of these models are relatively favorable for Democrats, as many expect the Republicans to win in Kansas and possibly North Carolina too.

Both of these models have some methodological quirks. The first is the curious decision to assign percentage chances to a binary outcome. Assigning percentages, rather than simply predicting a winner, does not add any real value other than a vague statement of confidence in the forecast. Mr. Perdue, for example, is given a 57 percent chance of winning by The Upshot and a 68 percent chance of winning by FiveThirtyEight. If he wins, as he probably will according to both models, they are both right; if Democrat Michelle Nunn wins, they are both wrong.

Additionally, it is also not clear to what extent these, particularly The Upshot, are actual models as opposed to a reflection of more subjective processes. The Upshot and FiveThirtyEight are not pollsters, but they aggregate public polls and base their forecasts on that data. This is a good, but incomplete methodology. The Upshot bolsters their methodology as they “tweak polls that count registered voters instead of likely ones,” and make “adjustments depending on who conducted the poll.” The problem with this methodology is words like tweak and adjustment suggest subjectivity. FiveThirtyEight provides much more detail on their model, suggesting that theirs might have a be somewhat more rigorous methodological approach.

In 2008, when Mr. Silver first began rigorously aggregating poll data, he made an important contribution to election forecasting, but six years later that contribution may have run its course. Today, to some extent because of the contributions of Mr. Silver and others, we have a better understanding of polls and the strengths and weaknesses of various polling methodologies than we did even in 2008. The result of that is that most reasonably astute observers of politics–if they are not burdened by a partisan agenda–can forecast most elections accurately, particularly with only a few days to go. Mr. Silver has helped make forecasting elections easier for everybody. That may be his most lasting legacy. Nonetheless, this year both Mr. Silver and The Upshot seem to have arrived at projections that are relatively positive for a beleaguered Democratic Party. If the Republicans end up with more than 52 seats this year, it will be evidence of a resounding defeat for President Obama and the Democrats, but also, at least for now, for Nate Silver and The Upshot.

Lincoln Mitchell is national political correspondent for The Observer. Follow him on Twitter @LincolnMitchell