In the fall of 1936, an American magazine mailed out ten million ballots, got 2.38 million of them back, and confidently announced that President Franklin D. Roosevelt was about to be thrown out of office.
He won 46 of the 48 states. The magazine was gone within 18 months, and its mistake is still taught in statistics classes today.
The poll that had never been wrong
The Literary Digest was a weekly general interest magazine founded in 1890. By 1927, it sold more than one million copies.
Starting in 1916, it ran a poll before every presidential election. And it had called the winner correctly every single time, five elections in a row.
In 1936, it went bigger than ever. Ten million questionnaires went out to its own readers, then to two other easy-to-find lists: registered car owners and telephone users. About 2.38 million people answered, an astronomical number for any opinion poll.
On October 31, the verdict was printed. Republican Alf Landon, governor of Kansas, would win with 57.08% of the vote and 370 electoral votes.
A landslide in the wrong direction
Three days later, Roosevelt took 60.8% of the popular vote. Landon got 36.56%.
Roosevelt carried every state except Maine and Vermont, for 523 of 531 electoral votes. Democratic Party chairman James Farley had predicted exactly those two losses, and he rewrote an old saying, “As Maine goes, so goes the nation,” into “As Maine goes, so goes Vermont.”
The Digest had missed the gap between the two candidates by 39.08 points. In 20 of the 30 states it had handed to Landon, the error was even larger.
The myth: a bigger sample is always a better sample
It feels obvious that asking more people gets you closer to the truth. With 2.38 million answers, the Digest should have been almost impossible to beat.
And yet it was beaten, by a much smaller poll. George Gallup, an advertising executive running a new polling outfit, used a carefully built quota sample of just 50,000 people. His final poll gave Roosevelt 55.7% of the vote and 481 electoral votes.
Gallup underestimated the landslide too. But he picked the right winner, with roughly one person for every 48 the Digest had heard from. In fact, every other poll at the time predicted a Roosevelt win.
Size only shrinks random noise, the kind of luck that evens out over many tries, as with the law of large numbers at a casino. It does nothing against a sample that leans one way from the start. Two million biased answers are just a very precise wrong answer.
The usual explanation is only half the story
The classic version goes like this: during the Great Depression, people who could afford a magazine subscription, a car or a phone were far wealthier than the average American, and wealthier voters leaned toward Landon.
That was part of it. But a 1976 article in The American Statistician pointed at something else: nobody was forced to answer. The people who mailed back their ballots, it argued, “represented only that subset of the population with a relatively intense interest in the subject at hand.”
Its conclusion was blunt: “it seems clear that the minority of anti-Roosevelt voters felt more strongly about the election than did the pro-Roosevelt majority.” In other words, the angry voters wrote back, and the satisfied ones mostly did not bother.
A more detailed study in 1988 found that both problems played a role. The wealthy mailing lists alone would not have been enough to predict a Landon victory. It took the silence of Roosevelt voters to turn a skewed poll into a historic blunder. Statisticians call this non-response bias.
Even the winner stumbled 12 years later
Gallup’s success made polling a fixture of every American election. His company would survive the Digest by decades.
But in 1948, Gallup predicted that Thomas Dewey would beat Harry Truman by 5 to 15 points. Truman won by 4.5. Gallup put the error down to stopping his polling three weeks before election day, which missed Truman’s late comeback.
That same night, the Chicago Tribune printed the headline “Dewey defeats Truman,” trusting a telephone survey. Phones were still not in every home, and the people who had one tended to be better off. The same trap as 1936, a dozen years later.
The lesson hiding in every survey
The Digest did not fail because it asked too few people. It failed because it never asked who was missing, a cousin of the survivorship bias that fools us when we only see the winners.
Every time a poll, a review page or a comment section claims to speak for “everyone,” the real question is the one the Digest forgot: who did not bother to answer, and what would they have said?

