Polls that focus on issues are typically less controversial than horse-race polls. I’ve spilled plenty of ink in this column warning against putting too much stock in horse-race polls this far out from the election, suggesting instead we focus on what people are telling pollsters about issues and their general mood. But last week, Nate Cohn of The New York Times shattered the relative quiet about issue polling and claimed that this type of polling is in crisis—and quite possibly worse than election polling.
Issue polling is not in crisis unless you think the purpose of every issue question is to explain election results. This seems to be the crux for Cohn: He says issue polling is in trouble based on the fact that in a lot of elections, voters’ responses to poll questions about the most important issues seem misaligned with what mattered most in the election results.
I agree that these particular questions are not useful for explaining election outcomes—at least, not by themselves. After all, since when does any single issue drive most people’s voting behavior in a coherent way they can state clearly and concisely, much less fit neatly into the answer options pollsters provide?
The biggest lie we tell ourselves with polling is that the responses we get are some immutable ground truth based on every individual’s carefully considered answers to each question, and that all of their answers tell a coherent story. We have more than 30 years of political-science and public-opinion research demonstrating that isn’t true. With this literature in mind, of course the most important issue in voters’ minds won’t always align with their choices.
For example, saying “the economy” is the most important issue doesn’t automatically mean a respondent thinks Republicans should be elected to handle it. Another example: Black voters are overwhelmingly likely to think crime is a problem; they are also overwhelmingly likely to say Democrats are better at handling it. If you assume that rating crime highly as a problem indicates voting for Republicans, you are missing significant parts of the story.
It’s also worth remembering that most elections are not close, and those that are hinge on a relatively small number of voters. So while most people might say the economy is the most important issue, odds are those people are mostly voting in noncompetitive elections. When the margin of victory is narrow, it could well be the few people who made the difference said abortion or democracy was the most important issue, even though they are the minority in the poll. We can’t determine cause and effect at this level of granularity from polling, or really any election data.
Moreover, most important issue questions are not the sum total of all issue polling, nor is most issue polling designed for the purpose of predicting or explaining election outcomes. Most issue polling is conducted to understand where voters are—at least approximately—on the issues queried, as well as policy alternatives voters prefer on the issues.
The well-respected Pew Research Center’s work is a comprehensive example of issue polling—the organization does not do horse-race polls but focuses on understanding life, politics, and society in the U.S. and around the world. The work they do on explaining elections uses many questions—not just one—to understand what happened and why voters cast their ballots in certain ways.
That’s not to say issue polls don’t have their problems. Certainly, issue polling suffers from the same problems that all polls face: People don’t want to answer their phones or take surveys; reaching the right mix of people to accurately reflect the population is incredibly difficult; people’s opinions often sway in the wind like a palm tree in a hurricane. Polls can be wrong.
And, as Cohn notes, we have survey techniques that allow us to evaluate how malleable people’s opinions are. Private and campaign pollsters often use message testing to assess shifts in opinion and vote choice after hearing specific information. To delve into fixing issue polling for public pollsters, Cohn and his colleagues conducted an experiment that is a very simple version of message testing: If respondents are given particular information about a candidate, does that shift how some people say they will vote? Absolutely.
There is nothing new about this finding—messaging experiments are just not often used in public polling, likely due to resource constraints. There are always many more questions to ask than there is budget for a project, and when the goal is to cover a variety of topics, it’s not always feasible to dig in on how concrete people’s views are on one particular issue. Plenty of public pollsters, like Pew, do this work, though.
Pollsters should not assume respondents think through issues and connect them to elections in the same way that political analysts do. Reading issue polling requires humility about how much the data can tell you and realism about respondents’ political sophistication. Shedding more light on that is good, but it’s not a crisis for the polls when analysts’ expectations and polling reality don’t align.
Weekly column written by Vice President Natalie Jackson for National Journal.
This column was originally published on November 28, 2023, on nationaljournal.com and is owned by and licensed from National Journal Group LLC.