Interview | Early poll can’t predict results, keep an open mind: Top US pollster

In a wide-ranging conversation, Lapinski spoke to HT on Friday about the challenges of polling in America and lessons from past polls, the uncertainty around the race between Donald Trump and Kamala Harris, the process of calling the results for the network and steps that can improve polling and preserve American election integrity. Edited excerpts:

Q: Let me begin with an open-ended question. All polls are in the margin of error. The election seems to boil down to seven swing states. From your vantage point, where is the race at the moment?

A: To be honest, we can’t really know exactly where the race is. We have mail voting, in-person voting, and it varies by state. And then we have election day voting. In 2020, we had over a 100 million people vote early. My guess is that this time roughly maybe half of the electorate will vote early, less than 2020 because we are not under pandemic conditions.

It could be one of the closest elections ever, but there’s a probability associated with that. So it may well lean one way or the other. I live with all three scenarios. This election could be called on Wednesday (November 6 eastern time), if it leans in one direction or the other. Or it could take a week if it is a tied race. We really have to look at these seven key states. Some of these states take longer to count the vote than others. Arizona and Nevada process their ballots a lot slower than the other five battleground states. Some of them are very efficient, a place like Georgia and North Carolina will have 99% of the vote counted sometime early Wednesday morning.

What people need to realise is that there are multiple scenarios that could play out. And if you grab a hold of one scenario in too strong a way, then you could be wrong. Everybody thought (Hillary) Clinton was going to win in 2016. She didn’t, right? What I have learned from my years of covering elections is that we have to just realise that there’s some uncertainty. These elections are usually close. We haven’t had a blowout election in a very long time in American politics. And the polls and the analysis of early vote are not going to be enough to fully know what’s going to happen until we actually land on Election Day.

In polls, knowing what you don’t know

Q: Reminds me of what an Indian politician had once said — you can only understand an election after an election is over.

A: I saw the clip of an Indian exit pollster on national TV crying. If you believe in one source of data too much, you might end up in tears. And I would want to make sure that that is not me ever. I obviously have a prominent role at NBC news leading their decision desk, but I am a professor at Penn and lead the data programme. What I have realised is that data is very powerful, but it’s much more important to realise what you don’t know instead of what you think. There are a lot of things that we don’t know. And you embrace that a little bit. It doesn’t mean that data is not powerful. It’s very powerful. But just don’t have too strong of a grip on your assumptions because your assumptions might be wrong.

Q: What don’t we know that makes this election so uncertain?

A: Okay, here’s a huge thing that we don’t know. People who are very attuned to politics and are most highly engaged, either Democrats or Republicans, habitually vote. We actually have the ability to know who these people are because of publicly available data in the US. Voter files provide past voting history on people. They are publicly available by states. And so those people are going to turn out most likely. If you are a Democrat, you are probably going to vote for Harris. And if you are a Republican, you are going to probably vote for Trump. There could be small percentages of people that don’t vote.

But what we don’t know are the mobilisation efforts of the campaigns and how effective they actually are going to be. So right now, polls are looking maybe just slightly good for Trump in some ways. But a lot of people think that Harris has a really good mobilisation effort. So for those more low-engaged voters, one of the big unknowns is how good is Trump’s effort? A lot of people think his get out the vote effort might not be great, but we don’t know that. What happens if the people who are more low-engaged voters on the Democrat side don’t turn out for Harris and maybe the ones that are low-engaged for Trump, maybe they will. It could go either way. There are certain segments of voters..we don’t know how they are going to behave.

Of all these early voters, we know that probably more Republicans will vote by mail than they did in 2020. Democrats will still have an advantage, but the big unknown is who are the people that actually turn out on election day. You could say that the Republicans have an advantage there. But it also might flip. We have talked to a lot of Democrats who have said that, well, they voted by mail before, but they feel it’s more secure to vote in person. In 2016, Clinton had this amazing advantage in the early vote, but she cannibalised all of her election day vote. And so she had a very weak election day vote. Everyone thought going in that she just had an insurmountable lead. But guess what? She didn’t.

Why past US polls got it wrong

Q: In 2016, polls predicted a Clinton win. In 2020, polls suggested that Biden would win with a far bigger margin than he eventually did. Polls have underestimated Trump’s support..

A: In 2012, they underestimated Obama support.

Q: So what are the learnings? How has polling methodology evolved? What are the weaknesses and what have you learned from it?

A: There are many weaknesses, and different things have caused different polls to have problems. In 2016, most polls weren’t weighted by education. Polling is both art and science, and there is a lot of art. And by art I mean assumptions. You make assumptions about the electorate. When you’re actually statistically weighting polling data, you have got to make some consequential decisions on how you weigh it. And so in 2016, there was definitely a weighting problem. There was a big divide on education, and most pollsters weren’t weighting by education.

In 2020, there was a little bit on the weighting side, but it was also on these likely voter models. You actually have a way that when you actually analyse the likely electorate, you make decisions on who you will include in your poll and who you will not. Everybody in 2020 thought that Trump had his base locked in, but he wouldn’t pick up new voters, but he picked up millions of new voters. So there’s a bunch of what we call low-propensity voters who were throwing out the model we could go through every election.

Polling is very difficult. Response rates are incredibly low. When you interview, you have got response rates of usually under 1%. And that’s crazy. That means that you are having to try to reach out to more than a 100 people to get one person to take your poll. And the question is, is that one person representative? When you start adding them up to the however many people are in your poll, can you make that representative?

In some ways, it’s surprising to me that polling has done so well given its challenges. What I would say on this is that even though these polls were sort of a little off, people have to stop using these polls to think that they can predict elections. They are not precise enough to do that. Everyone keeps asking from polls which they cannot give. They provide general direction. Most people sort of over-interpret these results. And my guess is that we are never going to make polling precise enough as people would like.

Calling the election right

Q: You are going to play a decisive role in calling the election on NBC on election night or whenever the final results come. Given how contested it was in 2020, at least from the Trump side, what is it that you will be looking at that will enable you to make that final call?

A: We have a lot of different sophisticated models, and we also have a huge data collection effort. The only thing that I care about in basically projecting elections is that we are right. We are in absolutely no hurry to project these elections. In a lot of instances for these close races, we will probably be waiting until almost all of the vote is counted. It just really depends upon how much spread there is. And I have a team of experts and a lot of those people are from the University of Pennsylvania. We are very careful and the only goal that we have in mind is getting it right.

People will just have to be patient because we are in no hurry to project these races. I think it would be extremely detrimental for American democracy if anybody that runs a decision desk calls any of these races wrong because you could create chaos. And I think we all realise there’s kind of a serious responsibility here. Everybody is always cautious, but we really can’t project a race until we are absolutely certain of the outcome.

Q: Given what happened in 2020, how worried are you about the outcome being contested by one side?

A: Obviously if anything is contested, we will empirically analyse it. If people say that some voters are ineligible, we will do a careful look at that. There has been no evidence in the past of that being true, but we always take every claim seriously and we have the ability to just sort of do that analysis.

Q: One of the scenarios you laid out in the beginning was a tie. What does that mean and how does that happen?

A: Well, when we say a tie, I mean a race that is just super close. The best example of that would be in 2020 in Arizona. NBC did not call Arizona. It was more than a week past the election. And Fox News and AP called it quickly. That race came down to 10,000 votes roughly. And we took a long time on that because again, we had to be certain that it was going to be an accurate call.

If a race really is coming down to 100s of votes or 1000s of votes, and it’s in a state that is pivotal, we wouldn’t project a race like that even if a candidate was just barely ahead. We have to make sure that there are no errors, even small errors. Under those circumstances, obviously candidates could potentially contest the vote. They could if it was really a tie. It varies by state, but states have provisions for recounts, and so there’s canvassing, there’s recounts. Under the situation of a tie, where it actually matters for the electoral college, we would find ourselves in a situation of just waiting.

Everybody has to realise that the networks have nothing to do with tabulating the votes or any of the vote counting process. That’s a government process. It’s a state and local government process. We just have to be patient and watch that play out and give election officials enough time to sort that stuff out.

The election churn

Q: Let me ask you to put your political science hat on. This country seems divided 47:47, and both sides will probably end up getting that vote share at a minimum. The battle is for the remaining few percentage point votes in limited geographies. And so the electorate looks calcified. But beneath that calcified look, there is a lot of churn happening. You talked about the number of voters who were new voters for Trump in 2020. Now polls seem to suggest there is a shift of traditionally Democratic voters towards Trump, maybe women voters towards Democrats. So how does one understand both the calcification as well as the churn when you are seeing the data?

A: From a political science perspective, the country is very divided. The intensity of that divide varies by voters. The thing that we will be carefully looking at is not just people’s attitudinal questions but also the behavioural ones. The whole game is really the get out the vote effort.

And I think the two groups that I am most interested in looking at is basically the African-American vote to see whether or not that splinters and whether or not people actually turn out. And then also the gender gap. Even though that the country is calcified in some ways, when we actually look at the gender gap, 47:47 isn’t true and that’s where there might be some movable and persuadable voters on both sides. It seems like men, even more Democrat, are moving a little bit more towards Trump, and there’s just a massive gender gap with women supporting Harris. There lots of different theories and ideas about there about why that is the case. And from a political science perspective, this election will be very interesting to analyse after the fact.

Results “hard, cold truth” for polling

Q: From the outside, looking in, the entire electoral process in America seems chaotic. You don’t have an Election Commission. The future of the world ends up depending on a few swing counties and a few swing states. There were issues with the transfer of power. Looking ahead, what is it that you think the American electoral system has to do to bring a degree of sanity and preserve its integrity?

A: We have an electoral college system that a lot of people don’t sort of understand, and a lot of people even in America don’t particularly like. That’s not going to change. The fact that we have decentralised elections is definitely not going to change. The thing that’s important for American democracy is just on the integrity questions that people can, regardless of our system, accept election results and that we have an orderly process of adjudicating that. And so I think this election will be sort of pivotal.

I know a lot of people think it’s going to be extremely chaotic, but it might not be. People in the media also focus on potentially the problems versus potentially the calm. We need to be a little bit more neutral what’s actually going to happen. A lot of people are trying to think that something’s going to happen that hasn’t happened yet. I am keeping a very open mind.

I am not saying that that’s not a possibility, but I think there are a range of possibilities and I think people are privileging some of the prospects of big problems versus maybe it’s not going to be as bad as people think. And so I think depending upon how that sort of happens, that’s going to have somewhat of a significant impact about how the world views how our system works.

Q: What is it that polling should pick from this experience and learn for the next cycle?

A: What polling is going to have to pick up and think about is trying to get these low-propensity voters, and how to properly capture them in our models. We may not be getting the right types of people — I don’t mean that in any judgment on the people, but just that whether the people that we actually get in our subgroups are representative of the groups. For example, for Republicans, we would want to make sure that we get kind of a representative sample of Republicans. And so the big question that we have right now is are we reaching the people that are much more MAGA or are we getting more traditional Republicans and is that proportion wrong? There are also issues that might have an impact on the Democratic vote share. Do we have a representative group for African-Americans and Latinos? Latinos and African-Americans are very hard to poll accurately for lots of different reasons.

And so I think there are a few things that we’ll have to focus in on, but we are not going to fully know until, unfortunately, after the election. But that’s why election polls are so valuable. It’s one of the few things that we poll on that we actually get to know the answer. How would you ever really know if you were wrong on presidential approval? There’s nothing to benchmark it to. But election results are hard, cold truth for polling.