Preet Bharara:              From Café, welcome to Stay Tuned. I’m Preet Bharara.

Ann Selzer:                   There’s not a lot for me to say firmly about what will happen as this primary plays out. It’s one of the most fascinating contests we’ve had. I say that every year, but it’s true, it’s got a lot of complications.

Preet Bharara:              That’s Ann Selzer. She’s the president and founder of the polling firm, Selzer & Company and one of the most accurate and admired pollsters in the country. A fixture in Iowa, Selzer has been running political polls for the Des Moines Register almost every year since the 1980s. and with the critical Iowa caucus approaching February 3rd, Selzer and I dive deep into the mechanics and methodology of polling.

Preet Bharara:              Is it an art or a science? What’s the appropriate sample size? How do we guard against bias? What should we expect in Iowa? and what distinguishes Selzer’s approach to polling from others? We get into all of that and more, but first let’s get to your questions. Stay tuned.

Preet Bharara:              This question comes from Twitter user [Sunpar Sand M 00:01:08] @PreetBharara, do judges often allow a reversal of a guilty plea? I understand that the judge has to sign off on Michael Flynn’s just announced move to change his plea #askpreet. Thank you.

Preet Bharara:              Well, the answer to your question as a lot of people have noted, and I think I’ve noted before when there were some indications that Michael Flynn would try to withdraw his guilty plea, because this has been swirling around for a while, you have to get permission from the judge. And when a guilty plea takes place, especially in federal court, it’s a somber proceeding and one of the most important parts of the guilty plea is an allocution, meaning statements made by the defendant himself or herself.

Preet Bharara:              They stand up and they engage in a colloquy, which is a questioning back and forth between the judge and the defendant and the judge makes a determination that the guilty plea is being entered into voluntarily, that there is no coercion, that the defendant’s mind is free and clear. Sometimes humorous things happen in court. When the question is asked, “Have you had anything to drink or taken any narcotics in the last 24 hours?”

Preet Bharara:              Further, the judge will always establish in what’s called a rule 11 allocution that the defendant and his or her lawyer have discussed the charges, have discussed the consequences of pleading guilty that include among other things, perhaps an effect on the right to vote and certainly an effect on the ability to possess a firearm. So, there is a long proceeding to determine voluntariness and understanding and communication with the lawyer so that that guilty plea and basically the entry of a conviction on a person’s record, with the only thing remaining being sentencing at that time, is serious and understood by everyone.

Preet Bharara:              And it is no longer within the power of the defendant to simply withdraw it because there’s been a change of mind or a change of lawyer or for the government also to make null and void the plea agreement. It’s an agreement between the government and the defendant and it’s basically signed off on and blessed by the judge.

Preet Bharara:              So it’s a serious undertaking, which is not lightly undone. Some basis on which a defendant might be able to withdraw a guilty plea is if it turns out the judge didn’t do a good job in making sure the plea was voluntary, if there’s a significant showing that the defendant was induced into pleading guilty based on false information by either the government or bad lawyering on the part of his defense attorney, but by and large it doesn’t work and can also anger the judge.

Preet Bharara:              Now, in this case with Michael Flynn, there’s been a lot of nonsense going back and forth for a while now. I remember he was in the middle of getting sentenced, so he was on the cusp of being sentenced, I think 12, 13, 14 months ago and it looked like Michael Flynn was starting to not accept responsibility for the crime to which he had pled guilty and the judge for that reason among others adjourned the sentencing. And here we are back on the cusp of sentencing again when there’s a lot of time to decide to make a motion to withdraw a guilty plea, it wasn’t done and he’s doing it now through a set of new lawyers.

Preet Bharara:              My prediction is, given how much exasperation the judge has shown with respect to Michael Flynn and how much accommodation he probably feels he has given to Michael Flynn, that the request to change the plea will fail. And by the way, the risks to somebody who decides to withdraw a guilty plea are not insubstantial. Presumably the government thinks that it has a really strong case against someone. And so, if there’s even permission granted to withdraw the plea, that just puts you back into the position you were in before the guilty plea, which is the charges still remain, the government may decide to file additional charges, the government may decide upon conviction at a trial to seek a much longer sentence.

Preet Bharara:              So, there’s a lot of risks and it’s especially odd given that in the initial instance, the sentencing guidelines called for merely zero to six months. And the government said explicitly that first time around that they were not advocating for a prison sentence. It’s about as good as you can get for a person like Michael Flynn.

Preet Bharara:              So, the particular arguments that the new lawyers for Michael Flynn are making include a claim that the government prosecutors have demanded testimony from Michael Flynn that would be false against a particular person they’re also prosecuting named Bijon Rafiekian. They’re also claiming that the government fell down on its obligations under the plea agreement, that they represented they wouldn’t call Michael Flynn a co-conspirator, and then in some other court documents they have done so.

Preet Bharara:              So there’s a mishmash of claims against the government. Maybe they’ll be convincing to the judge, maybe not, but given all the water under the bridge so far, I think it’s unlikely.

Preet Bharara:              This question comes in an email from Noemi in Las Vegas who writes @PreetBharara, what’s your reaction to the house managers that Pelosi picked? Anyone missing from the list or anyone that surprised you? So it was announced just minutes ago. I’m in the studio on Wednesday and Nancy Pelosi announced, I think in the last 75 minutes who the house managers were going to be.

Nancy Pelosi:                Today on the floor, we’ll pass a resolution naming the managers, as I mentioned, appropriating the funds for the trial and transmitting the articles of impeachment.

Preet Bharara:              I’m generally not surprised and I’m fairly impressed. She has by now, by the time you hear this, you’ll probably know this, she selected seven house managers, which is a pretty lean group compared to the house manager team in the Clinton impeachment trial, which had 13 house managers. As you might expect, it’s a pretty diverse group. There are three women, four men, geographically diverse.

Preet Bharara:              The least surprising thing about her selection is that chairman Adam Schiff, chairman of the Intel committee is the lead house manager. That makes eminent sense. It creates something of a chain of command. It means that the heart of the proceedings will be managed out of Adam Schiff’s office and brain and staff. And I think based on how quickly and efficiently and compellingly he brought the articles of impeachment to bear in the house after the impeachment inquiry hearings in rapid and streamlined fashion, I think it makes a lot of sense for Adam Schiff to carry the ball across the finish line at the trial in the Senate as well.

Preet Bharara:              Also, not a surprise Jerry Nadler who’s the chair of the Judiciary Committee, I don’t think there’s any way you couldn’t put him on the team of house managers. And you have a couple of other chairs as well. There’s Congresswoman Zoe Lofgren of California who’s the chair of the committee on house administration. There’s Hakeem Jeffries from around here, these parts in New York. He’s not the chair of a committee, but the chair of the house democratic caucus.

Preet Bharara:              What is also interesting to me is that with respect to each of these house managers selected by Nancy Pelosi, she was very careful to talk about their backgrounds and especially emphasize their backgrounds in military service, law enforcement service or litigation practice. So it looks like she was very keen to make sure that they were people who have an understanding of how trials operate, who have had some experience in the courtroom and if not in the courtroom, then in investigating and building cases and deeply understand national security. It looks like a good cohesive group.

Preet Bharara:              And as of the time of this recording, I’m not aware, maybe we’ll know by tomorrow by the time this drops who the president’s team will be. You may remember that I’ve mentioned this and I think Anne Milgram and I had mentioned it on The Insider podcast, that we were part of the group of folks who thought that a good house manager would be Justin Amash, conservative Republican recently, now an independent who’s had pretty good analysis on the impeachment proceedings and would have lent something of a bipartisan nature to the proceedings, I guess, in this one way.

Preet Bharara:              Speaker Pelosi has played it a little bit safe. You don’t want to have a wild card. She doesn’t really control Justin Amash in the same way that she has for want of a better word, some control over the members of her own caucus who elected her speaker. So I’m not going to say I’m disappointed. I think it would have been interesting and I think it would have been worthwhile to have somebody outside of the core group of Democrats as part of the house manager team, but I still think it’s a pretty solid group.

Preet Bharara:              This question comes in a tweet from Twitter user [Mandeal 00:08:14] who asks who within HPSCI, that’s the House Intelligence Committee would be reviewing the information on Lev Parnas’ devices? Staff, house members or both? #askpreet #letLevspeak.

Preet Bharara:              So of course that question relates to some bombshell news that happened yesterday when I was trying to watch the debate. A few minutes before the democratic debate began, there was a disclosure of a series of materials including text messages and handwritten notes of Lev Parnas. You’ll remember that Lev Parnas is a few things.

Preet Bharara:              One, he was a former associate of Rudy Giuliani, was gallivanting around Ukraine and doing a lot of things. Lev Parnas is also a defendant in a campaign finance criminal case brought by my old office to the Southern District of New York. And one of the things that’s been playing out over time is whether Lev Parnas would be able to speak with house investigators and maybe also now Senate investigators while the criminal case against him has been pending. And also whether or not the house would be able to get these materials from him and whether or not we would see them publicly.

Preet Bharara:              Well, guys we now know that we are able to see some of this material and Milgram and I will spend a lot more time on it next week. But among other things, these text messages show that Lev Parnas was engaged in communications with another gentleman, and I use that term advisedly, named Robert Hyde, who apparently is a candidate for Congress in Connecticut on the Republican side and it seems like they had it out for former Ukrainian ambassador, Yovanovitch whose safety has already been talked about in connection with the impeachment inquiry.

Preet Bharara:              And it looks like they are closely tracking, monitoring, depending on your verb of choice, stalking Ambassador Yovanovitch with respect to her specific whereabouts in Ukraine. And there’s some foul language that’s used by Hyde to talk about the ambassador. And you’ll remember that there has been testimony about how some folks felt including Rudy Giuliani that Yovanovitch was getting in the way of the thing that President Trump wanted. And that was an announcement by Ukraine and Zelensky, president of Ukraine in particular that there would be an investigation of Joe and Hunter Biden with respect to Burisma.

Preet Bharara:              And she was in the way and she was denigrated and she was character assassinated. And it now looks like from these texts, and I’m not going to go as far as some other people have gone. There was some kind of plot afoot to do her harm. So a couple of points on that. I actually spoke this morning before taping with Joe Bondi. Joe Bondi is the lawyer for Lev Parnas and he said to me and he said I could say this on the air that they dispute any suggestion that Lev Parnas himself, Joe Bondi’s client was involved in anything nefarious with respect to stalking or monitoring ambassador Yovanovitch.

Preet Bharara:              He said if you look at the texts, his client Lev Parnas is pretty mute and simply responds very tersely and that anything that was being hatched in the nature of doing harm to ambassador Yovanovitch was in the huddled mind of Robert Hyde. And so they’re distancing themselves from Robert Hyde in a great way. I think they think that he’s got issues.

Preet Bharara:              One side note on Mr. Hyde, if you go to his Twitter page, which notes that he’s running for Congress and it says hide for Congress, he interestingly spells Congress with three S’s. The other thing that Joe Bondi told me about Lev Parnas is that the quantity of material that was released yesterday is a tiny, tiny fraction of all the material that is in Lev Parnas’ possession that may be in the possession of the house and he will take some time to review and make determinations about sensitivity, but we’ve just seen the tip of the iceberg.

Preet Bharara:              And what that tells me and should tell you is that we will have a weird dynamic with the Senate impeachment trial starting on Tuesday, just a few days away, and possibly on a rolling basis. Other alarming and interesting things coming from the devices and the notes of Lev Parnas that bear directly on this issue of what was happening with respect to Ukraine.

Preet Bharara:              I mean, there’s a handwritten note that was released yesterday by Lev Parnas that essentially says, “Get Ukraine to announce investigation what the president wants.” There’s something else striking from the material that was released yesterday that bears on who can represent the president at the trial and also on the president’s own involvement in the Ukraine affair.

Preet Bharara:              And that is the president’s personal lawyer Rudy Giuliani has a letter that we saw yesterday to the leader of Ukraine asking for a meeting in which he says, “I’m requesting this meeting with the consent and permission of the president of the United States, and it relates to all this business about an investigation of Burisma/ Hunter Biden/ Joe Biden, which seems to go contrary to what the president of the United States had said that he wasn’t directing or ordering Rudy Giuliani to do anything back last spring or last summer.

Preet Bharara:              That letter seems to rebut that in a very serious way. So there is some shocking material that we have now become aware of, lots more shocking material potentially on the way, we’ll see how that plays out while the actual Senate trial is unfolding.

Preet Bharara:              It’s time for a short break, stay tuned. My guest this week is Ann Selzer. She’s a polling professional with an impressive track record, most notably predicting the then Senator Barack Obama would defeat then Senator Hillary Clinton in the 2008 Iowa caucuses. Selzer and I talk about her A political approach to polling when the science can get squishy, whether there’s such a thing as momentum and the best news a candidate can receive. Plus, we get into the purpose of the Iowa caucuses. Why some aspects of the polling process are kept secret and how Selzer’s method relies on her belief in people telling the truth. That’s coming up, stay tuned.

Preet Bharara:              Ann Selzer, so glad to have you on the show. Thank you.

Ann Selzer:                   Oh my pleasure.

Preet Bharara:              I’ve been looking forward to this, I will tell you, for weeks and months because I have a lot of questions and I think many of the listeners do about the art of polling. Let me begin with something that you once said. You said, “I was born to pole, which I think was the original version of the Springsteen song. I think he changed the lyrics. Tramps like us, baby. We were born to pole I think was the first one and then he changed away.” Can you explain to us what it means to be born to pole?

Ann Selzer:                   Well, it technically means that I did my first poll, if you will, when I was five years old. And I didn’t know what polling was, but I knew that I could gather the opinions of some important influencers in my neighborhood, which I determined to be the mothers of friends of mine. And asked them a very leading question. I was trying to get rid of a family nickname.

Ann Selzer:                   And so I said, “Don’t you think this is a good name for a witch?” They of course nodded because, I’m a four year old, five year old kid and I went home and reported to my mother that, “You know people think this is a good name for a witch and I don’t think I should be called that anymore.” And it worked. So, at a young age I understood about gathering data and thinking about who would be a meaningful universe. I didn’t know how to properly phrase a question because that wouldn’t have stood rigor.

Ann Selzer:                   But I was always science-y and I always liked the idea of gathering data about how people make the decisions that they make. So by the time I got to college and into graduate school, I mean that, and got the political bug, it just all came together.

Preet Bharara:              I see, but can we go back to when you were five for a moment, what was the name?

Ann Selzer:                   See, people ask that and you’re forgetting that it worked and I got rid of the name.

Preet Bharara:              Oh goodness. You really, you’re not going to tell us?

Ann Selzer:                   So why in front of your nationals? Why in front of your national audience would I suddenly decide?

Preet Bharara:              All right, I’m going to come back to that. So let me ask you a couple of dumb questions. One polling, and maybe you don’t like questions like this, but polling art or science.

Ann Selzer:                   Oh, mostly science. Mostly science. The art is knowing how to stay out of the way of the science, I guess, or not to overthink things. So there are probably plenty of other polling operations that apply what they think is a far more rigorous scientific method. For example, in how they decide what a likely voter is going to be. And they will live run regression equations and done all sorts of fancy math to say, “We know that if you have voted before and you know where your precinct is and you’re showing high interest, that you’re a more likely voter than people who don’t know those things and have not voted before.”

Ann Selzer:                   And that is what I call polling backwards. That is they look behind them to see what has predicted voting in the past and decide that that’s the way it will work in the future.

Preet Bharara:              But is that science?

Ann Selzer:                   So, it would look on paper like that’s a pretty good idea. Like that’s science. The best predictor of future behavior is past behavior, but the cells or caveat is unless there’s change, in which case you’re blinded. If you look at the past, you’re blinded from what’s happening ahead of you. So my method is to let my data tell me what’s going to happen at a future date.

Preet Bharara:              Let me ask you another dumb question. What is the point of political polling? In other words, why not have candidates go to states, whether it’s a presidential race or some other race, do their fundraising, do their campaigning and then have everyone vote without any preconceived notions or sense of the horse race? What’s the point of political polling?

Ann Selzer:                   So, what else is going to be happening if those candidates go to those states is that there’ll be a press corps covering their campaigns and when we had nearly two dozen candidates at one particular time, if the press corps doesn’t know who’s getting traction and who isn’t, how do they allocate their resources? And keep in mind my sponsors for the public polls that I do are almost always press.

Ann Selzer:                   So, it helps them understand what’s happening in that race and then they can convey to their readers or their viewers or their listeners with some accuracy about who’s getting traction, who’s on the move.

Preet Bharara:              Does that make full sense? Depending on when the polls are taken, sometimes people surge early and then they fade later because they have greater name recognition. Is it a good thing for the press to focus on a candidate who does better in the polls? Not because of good ideas necessarily or ultimate traction with the public, but because of name recognition based on polling?

Ann Selzer:                   Well, I think you’re wondering if there’s a self-fulfilling prophecy there.

Preet Bharara:              Yes.

Ann Selzer:                   And I think that we can show in our data that someone who leads early, this does not guarantee that they will win. And Joe Biden is a perfect example of that. We had a poll earlier last year that was basically a siren song to Joe Biden, “Get in, get in, Iowa loves you, Iowa wants to vote for you.” And then he did get in and then he was at the top of our next poll along with Bernie Sanders, who also had run before. And now Joe Biden is barely holding on to a place in the top four.

Ann Selzer:                   So, I understand the idea that it’s problematic, but in practice it doesn’t appear as though it works out to be that big a problem.

Preet Bharara:              Yeah. And sometimes it’s the case that a candidate pops in a poll and it doesn’t mean that they’ll win, but they get some notice that they might not have gotten before. So for example, in the last few days, and we’re taping this, I should tell folks on Monday afternoon, January 13th.

Preet Bharara:              Tom Steyer who has not done particularly well in the polls, I saw he searched to something like second or third place in South Carolina and in Nevada. Does that kind of a performance in a poll cause people to then take another look at him because they’re wondering what are these voters seeing in those states? And if so, is that a bad thing, a good thing, a neutral thing?

Ann Selzer:                   Well, I’ll speak in the generality that I would think, of course it would make you say, “What’s happening? What is he doing in these other states that has led people up?” He is one of the billionaires that’s in the race. And so, we know that what he’s doing is spending a lot of money on television advertising, probably on internet advertising.

Ann Selzer:                   I think probably it does cause people to take a look. Now, I will just mention in our most recent poll that came out over the weekend, we asked how important it is to them in deciding about what candidate to support, how they are doing in polls in other states? And that ranked the very lowest of anything that we offered them. And we offered about a dozen items.

Preet Bharara:              Right. Well, that might be a peculiarity of the folks in Iowa, no? We’re first we don’t care.

Ann Selzer:                   And that’s what I was going to say. That’s what I was going to say.

Preet Bharara:              If I were in Iowa I wouldn’t care much about other states’ poles either. I want to ask you a series of questions. I know you’ve been through this before, sort of basic mechanics and terminology and methodology of polling. And some of which I’ve been confused about, and so I’m assuming no knowledge on the part of listeners.

Preet Bharara:              So let’s start with the first thing. You do a poll, obviously the reason for a poll is you can’t actually ask everyone in a state or in the country a question. So you have to pick a sample. What’s the proper sample size?

Ann Selzer:                   Well, a sample size is determined by what level of precision you hope to achieve. So the general theory is the larger the sample size, the lower your margin of error. There’s a point of diminishing returns. So a sample of 800 is better than a sample of 400 by quite a bit, but a sample of 2000 is not that much better than a sample of 1000.

Ann Selzer:                   So it really comes down to how much precision you want. In a recent poll, we interviewed 701 likely democratic caucus goers that gave us a margin of error of plus or minus 3.7 percentage points. More important for us was that it gave us a bigger group of people who were supporters of the top candidates to poll them out for analysis purposes. So, the decision about the sample size is partly statistical, partly analytic, and then partly how much money you want to spend.

Preet Bharara:              Right. Right. So does it matter how large the electorate is that you’re sampling? So in other words, do you need a different sample size if you’re trying to get a national poll done, finding the views of all 300 some odd million Americans versus a state like Iowa, or does it not matter?

Ann Selzer:                   It does and it doesn’t matter. In the equation that we use to determine the margin of error, there is nothing in that equation that represents the size of the universe you’re sampling. So in that way, it doesn’t matter. Where it does matter is that you want to be sure you’ve got a well-balanced respondent pool after you’ve done the statistical work to get it weighted.

Ann Selzer:                   And so, you want to be sure that you have enough, that there’s some stability there and that you don’t have a lot of empty weight cells. That’s a very nerdish thing to say, but it’s typical that a national poll will be a thousand or more and that gives you enough to take a look at things by age, by race, by region of the country in order to get it balanced.

Preet Bharara:              If you’re a member of the public and you’re seeing what the sample size is, is there some number that should cause a member of the public not to trust the poll? A number that’s clearly too few, like 350 or 400? Is there some threshold number that it should always be above?

Ann Selzer:                   Well, as I said, it’s going to tell you what the margin of error is and that tells you a little bit about how stable the number is that is being reported out. We typically almost universally do not do any polling with under 400 respondents, but we’ll take a look at 400 especially if we don’t need to look at a lot of subgroup variation because a 400 is reasonably stabled.

Ann Selzer:                   In fact, it’s the turn in the curve about that more is better until it starts to diminish return. That curve where diminution starts is at 400. But sometimes you’re seeing people poll things out that are, you know, 230 out of a broader sample they want to report out as though this is a meaningful universe. That gets questionable. I will say the first time the Des Moines Register ever reported out likely caucus goers was before my time. This is ahead of the 1984 caucuses and they’d done a regular Iowa poll of 800 and from that they extracted those who said they were likely to go to caucus.

Ann Selzer:                   And I can never remember if it was 66 or 65 or 64 but it was, that was a very small group to show how it is that that the candidate preferences were going to go. That’s too small.

Preet Bharara:              So then what’s the definition of sample size? So let’s say you reach out to a thousand citizens and you ask them questions, but then you decide the poll is going to be on likely voters and it turns out you determined 600 of the thousand are likely voters. Is the sample the thousand or is the sample of the 600?

Ann Selzer:                   Well, the sample that is important is what number you’re going to use in your denominator when you’re going to create a percentage. So, what is the size of the group you’re reporting on? So very simply when you see a sentence that says, “65% of…” whatever is following that of is the denominator. So is that of likely voters or is that of the general population?

Ann Selzer:                   And if it’s 600 and it’s a meaningful statistic that you’re reporting out in that it makes some sense then that’s fine. But I’ll mention from our last poll that we had voice to voice contact with over 3000 active registered voters in order to find the 701 who were likely democratic caucus-goers.

Preet Bharara:              So how do you do that culling? Who did you eliminate once you contacted 3000 people?

Ann Selzer:                   Well, we did ask some questions of everybody that had to do with impeachment. But the way that you get into our poll of likely Democratic caucus-goers is we ask how likely it is that you will go and show up on caucus night. And you have to say either that you will definitely caucus or you will probably caucus, that if you might caucus, we’re done with you as far as that is concerned. We’re terribly polite about it. And of course if you say you’re probably not going to go to caucus again, we very politely terminate the call.

Preet Bharara:              And is that how everyone does it?

Ann Selzer:                   No.

Preet Bharara:              So why would you do it differently?

Ann Selzer:                   Well, let me rephrase the question. Why do I do it the way I do?

Preet Bharara:              Okay. A better way to put the question.

Ann Selzer:                   So the way I do it is for the respondents to tell me what it is that they’re planning to do and to trust that. Not taking a look at anything else that they might tell me about what’s happening, to trust that. And it sounds a little bit flimsy if you hear it, but I will tell you that this is the method in 2008 that told us in our final poll that 60% of the people who were going to show up on the democratic side, that would be their first caucus. There is no way to look at the history of Iowa caucuses and model the idea that there would be 60% first time caucus-goer.

Preet Bharara:              So you’re also asking people if they’ve caucused before, not just will they go caucus, but then you don’t necessarily weight them differently depending on how they’ve answered the past question.

Ann Selzer:                   Correct.

Preet Bharara:              But other pollsters may?

Ann Selzer:                   Other pollsters may. Other pollsters, especially in 2008 only looked at people who had caucused before or if they were registered as a Democrat. And so they were blinded that there was all of this new caucus activity. The Obama campaign looked at Iowa and said, if only the people who’ve caucused before show up on caucus night, we lose. So their campaign very deliberately went out and started talking to people who were registered as independence, started talking to younger people. They created a whole new cohort who went to caucus for the first time in early 2008.The entrance polls said, I think it was 57%. So, we were very close and other entities were blinded to the fact that that was happening.

Preet Bharara:              Do you have a way to test your hypothesis over time that by asking the question, are you definitely voting or likely to vote, are likely to go to caucus that that bears out or is it an article of faith?

Ann Selzer:                   Well, Nate Silver, who your listeners may or may not know, does a fantastic job of evaluating the final poll before an electoral event for over 300 pollsters and then does a rank order of how they rate. And every time that he has done this analysis, which I think has been three times, my company gets an A plus rating.

Preet Bharara:              That’s true. I guess I was asking a slightly different question and maybe there’s an ethical issue here and that’s why you can’t do it. You do a poll of 500 people and they tell you whether or not they’re likely to vote or will definitely vote or attend the caucus. There’s no follow-up later with that same group to test whether or not they actually did that or not.

Ann Selzer:                   There have been organizations who have done such things. There are political scientists around the country who love to do that kind of thing. And I think if memory serves me, what you find is that there’s a fair amount of squishiness. It’s not a perfect correlation. So are we polling people who say they’re going to caucus who end up not caucusing? Yes. Are we missing people who tell us they’re not going to go to caucus but then their neighbor pulls them in and says, “Get in the car, we’re going to go,” yes, we miss those. But for now, you can only look at the track record and say, “We do a decent job of it.”

Preet Bharara:              It’s interesting to me. You rightfully point to Nate Silver giving you an A plus. I think he gives very few A pluses. There are other folks who have said you have, and I love this phrase, quote near oracular status. So I will call you the oracle and Selzer. Lots of descriptions of your polling company in your polls as the gold standard.

Preet Bharara:              And if that’s all true, and we’re going to get into some of the great predictions that you have correctly made and other people got wrong, why don’t other polls move towards your forward looking methodology?

Ann Selzer:                   It’s an excellent question. I don’t know. And there will be people who get excited about what it is that I do. I think Politico had a headline that referred to my secret sauce and my reaction was, “Secret? We publish our methodology. There’s really no secret about it.”

Preet Bharara:              There’s no trade secret involved? You would open your books and everyone gets to see the methodology in all detail or do you keep some things back?

Ann Selzer:                   We keep nothing back.

Preet Bharara:              Oh, what do you keep back?

Ann Selzer:                   It is all published in the pages of the Des Moines Register and on the CNN website. Anybody who’s willing to dig around in the methodology statement, if you’re a pollster, you would know what to do to replicate what I do.

Preet Bharara:              And yet few people do that.

Ann Selzer:                   Some do.

Preet Bharara:              Politicians, people who are running for office. They want their own gauge and they might want to do it with more frequency. Do you have a sense of what those folks do, what methodology they use and if it’s different?

Ann Selzer:                   I would guess that it often is different. Keep in mind that what a campaign is trying to do is figure out how to move the number. So they may have some very specific needs to look at particular elements of the electorate. So they might want to know what the horse race looks like, but they also want to really zone in on movable voters, swing voters, persuadable voters.

Ann Selzer:                   So, I wouldn’t be surprised if they do some oversampling of particular types of population and other kinds of things like that as well. I read enough of the book about the Hillary Clinton campaign in 2016 by the name of Shattered to know that they were relying less on polling and more on analytics. That is taking a look at the electoral behavior at a micro level, state by state, by state, and they were getting all sorts of very positive indications that based on what had happened in past elections they didn’t need to spend much time in Wisconsin and Michigan and Pennsylvania.

Ann Selzer:                   Again, it’s the sort of thing that they were blinded from seeing what was happening in Wisconsin and Michigan and Pennsylvania because had they known, surely that campaign would have done something different.

Preet Bharara:              So how do you reach the sample? You call them. Some people call, some people worry about the proliferation of cell phones, whose numbers are harder to get. Some people use the web, some people use a hybrid. What do you use and why do you think it’s the best?

Ann Selzer:                   We use telephone and we work with the secretary of state’s voter list and work with a company that can provide assistance to enhance that list with phone numbers that would be more accurate.

Ann Selzer:                   And that includes cell phones as well as landlines. And that we make sure that every phone number we have has an equal chance to be contacted. We’re sort of paying attention about age cohort, so we’re calling in, I guess, there’s a little bit of technicality that isn’t necessarily published, but nothing that we would hide from what we do. And we call you on the phone.

Ann Selzer:                   The problem with online polling is that you’re polling people at some level who have volunteered to be polled. They basically agreed, “Poll me.” And so there’s a bit of a self-selection question that I wonder about. And there’s also, when you’re looking at a panel, there’s some very large panels and well-respected, but ultimately those people have all said, “I volunteer for you to contact me repeatedly and ask for my opinion.”

Ann Selzer:                   And the concern that I have is that something in social science we call the instrument effect. That is the mere fact of measuring something changes what you’re measuring. And if you think about taking your tire pressure, you release air from the tire as you measure it. You’re changing the tire pressure as you’re measuring it.

Ann Selzer:                   And so are you creating a group of people who, because they know they’re going to be polled in the future, are exposing themselves to political media differently, going to different events, are they behaving differently because they know they will be polled? So there’s some things to worry about there.

Preet Bharara:              Right. Now when you poll for the Iowa caucuses, do you only poll Democrats?

Ann Selzer:                   No. Whether you are registered as a Republican or an independent or a Democrat, we call you.

Preet Bharara:              And that’s because in Iowa, correct me if I’m wrong, Republicans can go to caucuses or not?

Ann Selzer:                   On caucus night you have to be a registered Democrat. You can change your party registration on caucus night, you can register to vote for the first time on caucus night, you can be 17 and register and participate on caucus night. So long as you’ll be 18 by the general election.

Preet Bharara:              So in years where there’s no competitive caucus for the other party, does that result in more crossovers or does it not really change?

Ann Selzer:                   I can tell you that there’s not a lot of crossover voting that we’re picking up, but we worry about and we made a very deliberate and expensive decision to leave no one out. So there’s a fair number of polls out there that will talk about registered Democrats, for example, and I’m always suspicious that surely that represents a big swath of the people who will show up on caucus night, but it’s leaving out some others who we think will show up as well.

Preet Bharara:              Who may have been disaffected by the president.

Ann Selzer:                   May have been.

Preet Bharara:              And may not decide until the last minute to walk into a caucus.

Ann Selzer:                   Yeah, I’ve sat next to some of those people at dinner parties.

Preet Bharara:              How do you control for bias, all sorts of bias? Does it matter the quality of the training of the person who’s making the phone call to the voter or is it something anybody can do reading from a script?

Ann Selzer:                   Oh, I think it’s quite a high level skill and we work with a phone bank we’ve worked with for decades and we could not have the success that we’ve had without a top flight phone bank.

Preet Bharara:              But why is that? If there’s… I mean, they don’t really ad lib, right? They read from a script and they have a decision tree and if the answer is yes to several questions, then you end the call. And if not, you continue asking questions that are written down in a script. So where does the skill come into play?

Ann Selzer:                   Well, your vision of this is all interviewer-centered and really the respondent brings a whole lot of stuff to the table. So, learning how to keep the respondent focused on what is the question at hand and how to, if the respondent gives you an answer other than what you’ve offered, what you do about that.

Ann Selzer:                   You could have a robot, and some do, doing the polling and they just read a question and pushed 2, if your answer is Joe Biden or whatever it is, there’s plenty of that out there. But having a living breathing interviewer who knows how to work the exchange with the respondent to get the data that the sponsor of the poll needs is, that’s a high-skill.

Preet Bharara:              I see. Should a pole last only a day or two days or three days? What’s the appropriate length of time so that the result doesn’t get skewed by outside events?

Ann Selzer:                   I would say that anytime you go into the field with a pole, there’s a danger that something will happen while you’re in the field that will change the results of the poll. It doesn’t matter if you’re only planning to be in the field two days or three days or seven days, there’s always the chance that something will happen in the real world that will affect how people think about things.

Ann Selzer:                   In terms of just the statistics of it, we need at least two days to dial through our entire sample. And once we, if you look at the first day, you know to don’t get married to those numbers. And really after the second day, then things start to settle in. But things can happen with campaigns. There are candidates that are out there spending millions of dollars and investing in hundreds of people canvassing door to door to change what’s happening in the pole. So of course things are going to happen.

Preet Bharara:              But then do you change course? Let’s say you’re doing a three day poll, and on day two one of the main candidates suffer some big public setbacks, some scandal emerges. Do you start from the beginning or do you extend the pole longer or how do you take into account those kinds of big external events?

Ann Selzer:                   Well, there would be a judgment call and I wouldn’t make that by myself. My sponsors would need to have some idea about what to do. But if we have enough data that we’ve collected ahead of the event and we’ll have enough after, we can look pre and post, how might that look? And then the decision is sometimes made about, well what do we report out? Because if you average the two together, you might not get something that really reflects what has happened and what the state of a race might be.

Preet Bharara:              Right. So, you get to the point where you have your likely caucus-goer and then they state their preference, who they’re planning to vote for. But then there’s all sorts of other questions you ask too that helped shape yours and the public’s understanding of who is likely to be the victor. Is that right?

Ann Selzer:                   That’s right.

Preet Bharara:              And what are those other questions?

Ann Selzer:                   Well, we want to know as much as we can about their preference. So we actually ask three questions, which is, who is your first choice? Who is your second choice? And there are reasons for that. Sometimes unique to Iowa. And then third, because we had such a big field and a hundred percentage points for the first choice is not a lot to go around when you have about two dozen candidates.

Ann Selzer:                   We also asked for each candidate if that candidate was not named as a first or second choice, well, are you actively considering this person? Yes or no? So, we could add those three percentages together and get a metric we’re now calling the footprint for each candidate. And my hope in creating that metric was that we could begin to see who has upside potential that might not be showing up in the horse race question. And so we were able to see that Kamala Harris, her footprint was more than 50%.

Ann Selzer:                   She was the only one not in the top, very top tier with that kind of thing. We were able to see early on that Elizabeth Warren’s footprint was equal to Joe Biden’s who was leading at that time. So there was upside potential. That’s been really helpful. Then we also want to understand why things are the way they are.

Ann Selzer:                   So, we’re curious about how people are making their decisions, what issues are important to them, how they rate the candidates on certain traits, how enthusiastically they’re supporting their candidate. We ask quite a few questions in order to better understand why things are the way they are.

Preet Bharara:              So let’s go to this big pole. I will tell you that maybe people don’t fully appreciate but there was a lot of anticipation for 6:00 PM last Friday when your poll comes out. And you did it jointly with, I think, CNN. And what everyone sees on the TV screens when the news breaks is the bottom line percentage numbers, Sanders at 20%, Warren at 17%, Buttigieg at 16%, Biden 15% and then a whole bunch of other folks in the single digits.

Ann Selzer:                   Correct.

Preet Bharara:              Now, further to what you were saying a second ago about footprint, what are the relative footprints for those folks?

Ann Selzer:                   So, let’s start with Bernie Sanders who is leading in our poll, his footprint was 55, Elizabeth Warren was 59, Joe Biden was 55, Pete Buttigieg 60.

Preet Bharara:              So what does that mean for Pete Buttigieg then?

Ann Selzer:                   Well, it says for Pete Buttigieg that there is additional goodwill, additional interest beyond his standing in the horse race question. And that there’s less of that. It’s a matter of a few percentage points, but sort of symbolically meaningful that there’s less of that for Bernie Sanders.

Preet Bharara:              And yet Pete Buttigieg had the largest drop from the November poll. I think nine points.

Ann Selzer:                   He’d also had the largest jump in the November poll.

Preet Bharara:              How much did his footprint change from November to now?

Ann Selzer:                   He dropped eight points.

Preet Bharara:              Is there such a thing as momentum?

Ann Selzer:                   Yes.

Preet Bharara:              A negative momentum?

Ann Selzer:                   How it plays out can vary widely. The Dean of the Iowa caucuses, Dave Nagle says, “Organize, organize, organize. Get hot at the end.” And in the past, what we’ve seen this getting hot at the end, Trump’s organization, because they’re in the week leading up to the caucus. In the past I felt you could almost feel the ground like a heartbeat on the ground. There was so much almost electricity in the air as everybody was excited in getting ready to go to caucus.

Preet Bharara:              Right. But you can’t measure that. Not withstanding your science. That’s impossible metric.

Ann Selzer:                   Well, we sort of see it in the level of enthusiasm that people are saying, the level of commitment of people saying they’re definitely rather than probably going to caucus. You can sense that. But sure, I’m giving you a subjective view of how that goes. And the crowd sizes at the events start picking up and people maybe are still shopping around and keeping an open mind, which the caucuses invite you to do and you see another candidate go, “Wow, okay.”

Ann Selzer:                   Or Cory Booker drops out. “Well, I was a Cory Booker so I need to go and check in on these other candidates.” And sometimes really most commonly in our final poll, if we’re in the field four days, five days, what we see is the lead changing from day one to the final day. That is not uncommon. So, is that momentum or is that just things solidifying? Could be anything, but we see it.

Preet Bharara:              So, if Bernie Sanders is sitting on top, this most recent poll of yours, is he the person who should be most comfortable and pleased?

Ann Selzer:                   Well, it was a good poll for Bernie Sanders. There’s no question about it, but it’s getting your people to show up and knowing what to do in the room on caucus night, which is no small feat. In the lead up to 2016 our poll said that Donald Trump was leading. And my comment at the time was, the caucuses are Donald Trump’s to lose and lose it he did. And the reports were that his organization had no idea what to do in the room on caucus night and the Ted Cruz organization did.

Ann Selzer:                   And so they swept in and started pulling people over from other candidates and said, “Look, your candidate doesn’t have a shot. Come stand with us. Come, write our name on the piece of paper because Republicans do things differently.” And the Donald Trump organization in many, many, many precincts, when it was their turn for the campaign to say, give a message to the people in the room, they had no one. They didn’t know how to organize in the room.

Ann Selzer:                   So there’s lots of things that can happen. My sense of the Sanders organization, especially since they’ve done it before, will be well organized and know what to do in the room

Preet Bharara:              Is the other thing in Sanders favor? The fact that more of the respondents who said they were planning to vote for him said that their mind was made up? In other words, the intensity of their feeling, that’s something you measure, right?

Ann Selzer:                   Yes, and that has been his calling card all the way through is that the support for Bernie Sanders has a very solid core and that is not so much the same for some of the other candidates.

Preet Bharara:              Any other things you were surprised about when you saw this poll? I believe you had a question on there asking people what was really important to them. And among the people who said that the most important thing for them was to defeat Trump, the percentage of people who said that’s the most important thing went down by a number of points. Is that fair?

Ann Selzer:                   Yes. It had been 63% in November and it’s now 55%.

Preet Bharara:              Do you think about why that might be?

Ann Selzer:                   I could speculate a little bit.

Preet Bharara:              I could speculate. I could speculate.

Ann Selzer:                   There’s some share of that that you might say corresponds with Bernie Sanders leading in this poll that all along his supporters have been more concerned about the values and the issues that a candidate represents than others. But I don’t think it accounts for the full gap.

Preet Bharara:              When you see shifts like that, do they ever seem so odd to you that you wonder about the rest of the poll because, and this is my own bias coming through for me, if I were a caucus-goer or a voter, the most important thing would be to defeat Donald Trump. And he hasn’t done anything in recent weeks to change, I think the general view of that, if that was your view going into 2020. And it just seems a little odd that there would be such a shift.

Ann Selzer:                   So, the way that we evaluate whether we feel like our poll is solid doesn’t look at questions like that and presume that we know what the real answer is. What we do is look at the demographics and how things fit and how those fit with the populations that we’ve been measuring.

Ann Selzer:                   But even there, we allow for some random variation that happens. But no, I would never look at a substantive issue question and say because that doesn’t fit with my expectation, my poll must be off.

Preet Bharara:              You stay out of it.

Ann Selzer:                   I let my data tell me.

Preet Bharara:              Right. Will there be another poll before the Iowa caucuses?

Ann Selzer:                   If history is any guide.

Preet Bharara:              But I ask that question in [inaudible 00:47:57]… But you don’t specify dates. That’s a secret.

Ann Selzer:                   And the reason for that is that in the olden days of your, it used to be that candidates would find out when the next poll was going to be, in say September, and then they would organize a big announcement or they’d hold a big event or they’d put a big media buy in in order to influence the poll. And we wanted no part in that. We were not interested in helping candidates game the poll.

Preet Bharara:              Although polling has played something of an outsize role in the democratic primaries because since there were so many candidates you were placed on the debate stage was governed by two things. One, the number of donors you have and the other where you stood in any number of polls. Do you think that was a fair debate criteria or do you stay out of that?

Ann Selzer:                   I stay out of it. That is the party’s decision to make and it ends up having consequences that we didn’t ask for.

Preet Bharara:              Did you make anyone in the polling industry feel additional responsibility?

Ann Selzer:                   Well, I think you see some polls that are happening on schedules that align with this. And in 2016 when they introduced this, I was polling for Bloomberg News in addition to my work with Dwayne Register and during the general election season.

Ann Selzer:                   Sure, we put a new poll on our schedule to match up so it would be considered as one of the polls that was taken into consideration. What is interesting at this point in time is that you now have a candidate in Michael Bloomberg who isn’t getting money from anybody except from his own wallet to pay for his campaign. And that’s one of the criteria for getting on the debate stage.

Ann Selzer:                   So it, even though he’s starting to get some traction in California and in some other states, and even nationally, it doesn’t matter because if he doesn’t have donors, he doesn’t get on the stage. So, I think that reveals that there’s nothing magical about having a certain number of polls or having a certain amount of money from certain number of donors. But it’s tricky when you have two dozen candidates who gets to be on the debate stage. I don’t envy the national party in trying to figure out how to figure that out.

Preet Bharara:              You said based on experience that what matters a lot is having momentum at the end. So we’re three weeks away. Is this the end or is Bernie Sanders potentially peaking too soon? What’s the perfect arc of when you want to peak, with respect to the Iowa caucuses?

Ann Selzer:                   I have no idea. What we saw in 2004 was that Howard Dean had been leading and that only in our final poll did John Kerry come up and surge and how Dean fell. So you needed two things to happen. That you needed Howard Dean’s support to, if I say collapse, that’s a bit strong, but you needed John Kerry to get his house in order and make a legitimate charge for the end.

Preet Bharara:              Is there any difference between Democrats and Republicans, or men and women, or older and younger voters that you poll? And I don’t want to get you in trouble with respect to how truthful they are or how well their own predictions about their own voting plays out when they’re answering polls.

Ann Selzer:                   Well, you’re trying to dissect things a little bit too finely to break it out by demographics. The point I would make is, of course, I think respondents are truthful. Of course, I do. And if they’re not, if say there’s one person who says, “I always lie,” my answer to them is, “Well, for every one of you, there’s a doppelganger who is doing just the opposite on the other side.” I couldn’t have the accuracy that I have if everybody is lying, unless they’re all lying in an orchestrated way.

Preet Bharara:              Right. So they cancel each other out. But so what do you make of these theories that I sometimes hear from pundits? Way back in ’08 people thought, “Well, there’s a certain amount of racism and maybe Barack Obama’s polling was overstated because people wanted to look like they were open to electing the first African American candidate. And some people I think are saying things like that in the opposite direction, that some people maybe are not excited to tell pollsters they’re voting for Trump, is all of that nonsense.

Ann Selzer:                   I would say there’s very little data to support that.

Preet Bharara:              That’s fair.

Ann Selzer:                   I can’t say it never happens, but there’s nothing in the aggregated data that we see that suggests that that’s a major dynamic at play.

Preet Bharara:              Now, the Iowa caucus is the most difficult kind of election that you poll for?

Ann Selzer:                   Yes.

Preet Bharara:              Why is that?

Ann Selzer:                   The proportion of the overall electorate who show up on caucus night is relatively small. In the past it’s been maybe 15%. So, you are, it’s a low incidence event and first off you need to be careful that you’re really getting people, you’re keeping a narrow enough definition of a likely caucus-goer so that it pretty much can align with what actually happens. But it’s also very expensive.

Ann Selzer:                   So we’re going to hang up on a large number of people who don’t meet our definition of a likely caucus-goer. So, that’s first. And that rules out a whole lot of would be pollsters who just can’t afford to do it. Secondly, as I said, people can show up on caucus night and register to vote for the first time. They can change their party on caucus night.

Ann Selzer:                   There are all of these ways that it’s very, very quirky. And then ultimately all a poll can do is a hope to estimate with some precision what people walk in the door intending to do. And by design on the democratic side, there’s an opportunity that is right in the process for people to change their minds.

Ann Selzer:                   So, there’s all sorts of ways. If you look at it on paper and you’ve never polled in the caucuses before, you would say can’t be done. It can’t be done.

Preet Bharara:              But you do it.

Ann Selzer:                   So what I say to my clients, I say, “Look, we’re just going to take our best shot. That’s my pledge is to take our best shot. But no guarantees ever.”

Preet Bharara:              Can you explain for some folks who have not been through this before and for some folks who have and still don’t really understand it, what happens on caucus night

Ann Selzer:                   And they’ve changed the rules a bit too. So what happens on caucus night is that everybody gets registered and gets in the room and maybe there are 500 people.

Preet Bharara:              Right. Unlike a polling place where in other states you go and use a machine, here you gather in a cafeteria at a school, for example.

Ann Selzer:                   Exactly, exactly. And then there’s some party business because this is a party event. Many things happen. There’s statements from me to the candidates organizations. And then I believe there will be an announcement of how many people have to support a candidate in order for it to be viable. That is to have 15% of the people in the room.

Ann Selzer:                   And so, let’s just say, “You’re going to need 87 people.” I’m making that number up. And so they say, “Now we’d like you to go and stand in your preference groups. Well, typically that’s this corner, this corner, this corner, this corner. They don’t have rooms with enough corners for the number of candidates we have.”

Preet Bharara:              [inaudible 00:55:20].

Ann Selzer:                   Yeah, exactly. So, part of the complication is you’ve got a precinct captain for most of those candidates who should be doing account as people are gathering in the group so they know if they’re approaching viability or whether they need to go as people are moving into other groups to look at those smaller groups and persuade them to come over and join them.

Ann Selzer:                   But once that initial alignment is done, those numbers will be reported out eventually. But the candidates who are viable will be declared and they can stay where they are, and the people who are standing with candidates who are not viable have a chance to move to one of the viable candidate.

Preet Bharara:              So they can leave or they can just go to their second choice?

Ann Selzer:                   They could leave, they could decide to go uncommitted. There are things that could happen there, but then there will be that second alignment and then that will be the final that is reported out. Those numbers will be converted into delegate equivalence and all of this gets reported into the state party headquarters for number crunching.

Preet Bharara:              That’s complicated.

Ann Selzer:                   It is complicated and it does take a couple of hours. And so, it requires a real investment on the part of the people who are going in terms of time and energy to get all of that done.

Preet Bharara:              Now, you never go correct?

Ann Selzer:                   I do not.

Preet Bharara:              And you are a very stridently either nonpartisan or hide whatever view you have. And that’s a deliberate decision, right?

Ann Selzer:                   Yes.

Preet Bharara:              Is it true what I read that famously you wouldn’t tell your boyfriend once upon a time if you were, if you favored one party with the other?

Ann Selzer:                   Well, I’ve never told any boyfriend, but there was one that really couldn’t quite believe it. So, took offense.

Preet Bharara:              And do you think that’s important for pollsters to maintain neutrality or at least the image of neutrality in that way?

Ann Selzer:                   You know, if I were honest with you, I would say probably my track record is good enough right now that people would not say, “Well, the reason that her numbers look this way is because she supports this candidate.” But I just don’t want to have that conversation with anybody about whether my numbers are reflecting a view on the one side or the other. And truly I believe, and this is my value, that the best news any candidate could get is the truth. So, there’s really nothing I would do differently than take my best shot at providing numbers that give an accurate view.

Preet Bharara:              Are there some polling outfits, you don’t have to identify them in this question, are there some polling outfits you think that are well known and whose names people would recognize that are perhaps biased in favor of one side or the other?

Ann Selzer:                   I don’t think that’s the way that I class them. You have the Wall Street Journal, NBC poll, which has two pollsters, one a Republican, one a Democrat at the helm of that pulse so that there would be balance that way. And there’s some other polling organizations that do the same thing.

Ann Selzer:                   I think the national party decided they only wanted polls from, they would wanted it to exclude campaign polls and they wanted to exclude pollsters who work for particular parties so that there’s not the appearance of impropriety there.

Preet Bharara:              Can we go back to 2008, because you made a reference to it. And what’s interesting about that is, you focus on your science, you focus on your own methodology and sometimes when you do that, you come up with a poll that is alarming to one or more folks who have a vested interest in the outcome. And that happened in 2008, right? You did a poll fairly near in time to the Iowa caucuses that showed Barack Obama leading by how much?

Ann Selzer:                   Like seven, eight points. It was a solid, substantial lead.

Preet Bharara:              And how close in time to the caucuses was that? Do you remember?

Ann Selzer:                   It was, we finished polling literally days before.

Preet Bharara:              So, that’s a point of interest for the public and for the press. But it’s a point of extreme annoyance to the Hillary folks. And they got mad at you, didn’t they?

Ann Selzer:                   They did.

Preet Bharara:              What did they say?

Ann Selzer:                   Well, their campaign manager wrote a memo within 30 minutes of the newspaper arriving on the doorsteps on New Year’s Eve to say she has done outrageous things in what she is assuming is going to happen on caucus night, she sang 60% are going to be first time caucus-goers. Nobody thinks that’s what’s going to happen. So ignore this poll. She’ll be fired. All will be well, Hillary will win.

Preet Bharara:              They predicted your firing, huh?

Ann Selzer:                   Well, maybe they didn’t go quite that far, but it was kind of the impression that I got.

Preet Bharara:              So what’s interesting going back to our earlier conversation is that it was important to them, perhaps because there is some self-fulfilling prophecy belief that the opposing campaign had. In other words, if it was so wrong, what’s the big deal for them? Why would they get so angry at you if they thought your polling was wrong?

Ann Selzer:                   Well, they were, this memo went out to the National Press Corps and they wanted them to spin the poll to downplay the Obama advantage. The next day I had a phone call from one of Hillary Clinton state chairs and he’s… if I called him a buddy of mine, maybe that’s a little too close, but we’re informal friends and he calls me from time to time.

Ann Selzer:                   He said, “Look, I’ve always trusted your numbers until now, but I’ve knocked on 99 doors and I don’t find this lurking Obama support.” And I said, “Tell me about the 99 doors.” Oh, we’re canvassing previous caucus-goers and registered Democrats.”

Ann Selzer:                   And I said, “That’s why you’re not seeing the lurking Obama support because he’s creating first time caucus-goers.” And this kind of feeds into what I was saying before, that the Clinton campaign was blind to what was happening because of the way that they had decided who the meaningful universe was that they would be polling.

Preet Bharara:              It was a parallel era to what some of the other pollsters do, which is relying on past behavior?

Ann Selzer:                   That’s right. And the Obama campaign said if we rely on people from the past, we lose and they went out and created new caucus scores.

Preet Bharara:              And is that a similar sort of thing that happened in 2016 with Trump finding new people who hadn’t voted before?

Ann Selzer:                   Perhaps it happened there more organically rather than as intentional kind of thing. He was a very different kind of candidate, and so it got people interested who might not have been interested before, but it was nothing on the, is not on the scale of the Obama phenomenon where the majority of people caucusing had never caucused before.

Preet Bharara:              Going forward in 2020, how should people think about polling? First with respect to the various primary states to determine who the democratic nominee is, but then also in the general, and I guess you can answer that a little bit by reference back to 2016 because there was a lot of hand wringing about polling and about the prediction of the election back in 2016. What lessons have you learned and more importantly, what lessons would you offer to other folks who are watching this year?

Ann Selzer:                   Okay, you’ve asked about 16 questions.

Preet Bharara:              I know. It’s a bad lawyers’ question since we’re not in court, I’m asking compound questions.

Ann Selzer:                   So the first thing, let me speak to the primary. What we know has happened in the past is that the candidate who is victorious in Iowa gets some buzz, get some media attention, has some momentum. There’s a very striking graph of the 2008 primary season in South Carolina. So first it’s Iowa, then it’s New Hampshire, and then it’s South Carolina. And the margin by which Hillary Clinton was leading Barack Obama in South Carolina ebbed and flowed, but that was the direction of the graph.

Ann Selzer:                   And after the Iowa caucuses, it’s like that graph turned upside down. And so the gap was Obama leading Hillary Clinton. And it’s the best example I’ve ever seen of the impact of a win in an early state on also an early state. So, that’s happening in a very short period of time. The other thing to think about though, is that you’ve got candidates playing a slightly different game, given that we have not only a super Tuesday, we have a super duper Tuesday with a lot of delegates in play, and you’ve got a candidate with billions of dollars who is saying that’s where he’s going to stake his claim. He wants to win on super Tuesday in as many states as he possibly can.

Ann Selzer:                   So, there’s not a lot for me to say firmly about what will happen as this primary plays out. It’s one of the most fascinating contest we’ve had. I say that every year, but it’s truly, it’s got a lot of complications.

Preet Bharara:              What’s the most interesting thing about your business to you?

Ann Selzer:                   I have always liked knowing something before other people do.

Preet Bharara:              Oh, you’re one of those?

Ann Selzer:                   I’m one of those. So, the first day we take a look at the data. In 2012 on the Republican side, what we saw that first day was Rick Santorum finally got above 5% and he was in double digits. And so you go, “Oh. Well, the Santorum campaign, wow, something’s going on.”

Ann Selzer:                   And then he ended up, during the four days we were in the field on that final day coming very close to challenging Mitt Romney. And of course, long story later, Santorum was declared the winner of that Republican caucus. But seeing those little things start to happen. And was it just one good day or is there a trend? That’s exciting.

Preet Bharara:              One question I have is you have this debate in campaigns and in the public square about candidates who get criticized for being poll-tested, whatever that means, and constantly running to a pollster to decide what opinions are popular or what is the right way to talk about something so that people will come to your side. And there’s a little bit of disdain that is introduced into public discourse about that as if you’re putting your finger in the wind instead of leading.

Preet Bharara:              And on the other hand, obviously data’s important. You’re talking about this great work you’ve done for advocacy groups to understand what the public wants. Do you have a thought about that debate?

Ann Selzer:                   Yes, I have complicated thoughts about that debate. What people are concerned about is whether a candidate is authentic, whether they are speaking from their heart and because if they are not, then there’s a trust issue. In my limited experience lately in dealing directly with campaigns and with candidates, what they were more looking for is if the way that they are putting words together to explain their position is somehow not matching where their electorate is, they need to know that.

Ann Selzer:                   So, I don’t know that there is a high proportion of candidates who decide, well, they don’t really care what issues they talk about so long as they’re talking about the issues in a way that people want to hear. Do you know? That would be very disingenuous and it’s hard to imagine that you can’t see through it.

Preet Bharara:              Right. I mean, look, there are various ways to use that kind of information. One is you believe in universal healthcare or some other policy and you want to know the best way to talk about it and be persuasive. Another is find out the things that people don’t agree with you on and then decide you’re not going to talk about them.

Preet Bharara:              And then maybe a third way, and there are many iterations of this would be, I’m not quite sure what I think about going to war in Iraq or some other policy question, and so I want to find out what my constituents, future constituents think, and then I’ll pick the most popular side. That’s the most disingenuous of all of the possibilities.

Ann Selzer:                   That’s right. And I think when people decide to run for president, it’s hard to imagine that they don’t know what their position is.

Preet Bharara:              Well, I’m not sure that’s true of everyone. I think there’s some folks on some issues that they may not have had a lot of experience with in what they’ve been doing so far on maybe not the most significant question is what they think.

Ann Selzer:                   And then if I were going to be gregarious toward these candidates I said, well, perhaps listening to actual people in a structured scientific way might help them decide what it is that they think. But I do understand what you’re saying and that people want candidates who will lead and not just follow the public whim, but I don’t know really on a percentage basis how common it is for candidates to abdicate in a decision-making power about what issues they’re going to talk about or what stand they’re going to take.

Preet Bharara:              Right. I mean, part of it is a question of strategy of micro-targeting. depending on where you go and who your audience is, you emphasize certain things more and emphasize other things last based on your understanding of what that particular audience is interested in.

Ann Selzer:                   But you know what’s interesting, candidates do that all the time. And I think it was 2012 that John Huntsman decided he wasn’t going to compete in Iowa because he didn’t favor ethanol subsidies. And so, I said to my client, the Des Moines Register, I go, “I wonder if that’s true. I wonder if Iowa Republicans are really hot for subsidies. I think maybe not.”

Ann Selzer:                   And so our next poll, we were able to ask that question. And no, a majority of Republicans did not favor ethanol subsidies. But John Huntsman made a decision on his own about what he presumed based on his own internal micro-targeting. You see what I’m saying?

Preet Bharara:              That was a mistake.

Ann Selzer:                   I think so. Possibly.

Preet Bharara:              The whole mechanism of polling is interesting to me. A stranger calls up someone, the stranger answers the phone, is talking to someone about their views of war, or agriculture, or the economy, or healthcare and who they might vote for, which is a very personal and important decision. It’s kind of a peculiar dynamic. What do we know about whether or not people like to do that or not? Or does it depend on the person?

Ann Selzer:                   Well, it’s a big question. The story I would tell you was from my days when I was on the staff of the De Moines Register. And our callers, we worked out of the classified advertising telephone stations and I had the ability to listen in on the calls as they were being made in real time. And this is, I’m brand new to the job, right ahead of the 88 caucuses. And I start working with my interviewers on politeness forms and teaching them how to say sir and ma’am in a way that sounded genuine.

Ann Selzer:                   And about how to say please and thank you. All of those things that we don’t necessarily put in the script but can be very helpful. And I remember the moment where I thought this was worth doing when a woman said, “I’ve been called 16 times by all kinds of polls and I’m talking to you and I turn all the others down. I don’t know why I’m talking to you.”

Ann Selzer:                   And I think that in a civil conversation about what you think, I think people are… they don’t mind sharing their opinion and their name isn’t going to be connected with it. So it is a little bit of a window that they can open to let out what they might be thinking.

Preet Bharara:              Yeah, I think that makes sense. This has been incredibly informative, educational, entertaining. Thank you for being an oracle, Ann Selzer.

Ann Selzer:                   Thank you.

Preet Bharara:              The conversation continues for members of the CAFE Insider Community. To hear this, stay tuned bonus with Ann Selzer and get the exclusive weekly CAFE Insider Podcast and other exclusive content head to cafe.com/insider. Right now you can try our CAFE Insider membership free for two weeks at cafe.com/insider.

Preet Bharara:              So as many of you know, representative Doug Collins, the ranking Republican member of the House Judiciary Committee made some remarks last week on Fox News that got me and many other people mad. Now, I don’t usually get as mad as I got last week because there’s a lot of nonsense that is uttered from Washington and you can’t get riled up about all of it otherwise you can’t live your life.

Preet Bharara:              But when he said the Democrats are in love with terrorists and that they mourn the death of Soleimani more than they’ve mourned the loss of gold star families in this country, that was a bit too much. And I went on a rant on Twitter and I cursed a little bit. And then the next day I wrote an open letter for cnn.com to Doug Collins. And then that evening I appeared on Anderson Cooper to talk about Doug Collins and this level of rhetoric and how hateful and disgraceful it was and predicted that he would not apologize because that doesn’t seem to be the fashion of the day.

Preet Bharara:              And people tend to double down in this Trump era rather than do the right thing and apologize, because he knew that was wrong. Democrats are not in love with terrorists and it’s ridiculous to say that. It says more about him than it says about anybody else. But then interestingly the next day he did on Twitter at least.

Preet Bharara:              And so you can complain that he didn’t do it in the same forum, that it came a little too late and have sincere doubts about his sincerity and the apology. But I think the apology is a good thing. But the reason I raised that is we got an email from a listener named Laila who makes reference to Doug Collins and suggest maybe lots of people’s voices made a difference in his decision to apologize. And then asks, I think a really important question, do ordinary messages from ordinary citizens have any impact?

Preet Bharara:              Do faxes or phone calls from out-of-state citizens make any difference? Effectiveness in comparison to phone calls or postcards? I’d be interested to know if there’s any data on this, thanks. So, I haven’t researched it. I don’t know what the data is, but I can speak a little bit from personal experience and common sense. As you remember, I’ve worked in the senate for four and a half years as chief counsel on the judiciary committee to senator Schumer. And sure, there are people who have big platforms and have big voices and those things make a difference, but I think it also matters if ordinary citizens in great number take a position on something also.

Preet Bharara:              When I was in the Senate from time to time, there would be a controversial issue. It could relate to a bill, it could relate to a nominee for a cabinet position or a Supreme Court spot. And Senator Schumer staff because he wanted to know this information would on some of these issues keep a running tally of phone calls coming to the front desk of how many people called in favor, how many people called against, and also where they’re from.

Preet Bharara:              So yes, typically the greatest impact will be felt if your own constituents in your congressional district or in your state, if you’re a Senator, make an impact. And it doesn’t always change the mind of the political leader, but at least in my experience in that one office, one of a hundred senators, it was something that was cared about, thought about, noted and monitored. My sense was, and this is maybe just peculiar to my own boss at the time, that phone calls where people were upset had the most impact because the phone is ringing in the office and some human being is taking the phone call.

Preet Bharara:              Although I think letters and emails make a difference too. Look, any outreach makes a difference. And if it’s a thousand people or 10,000 people as opposed to five people, that will make an impact and it’ll be discussed and have an impact on the member.

Preet Bharara:              And so, based on that experience, did Doug Collins decide to apologize because some people with big platforms called him out and said it was an obnoxious and disgraceful thing to say? Maybe. But I wouldn’t be surprised if he got a lot of calls to his offices in his district and not just from people outside his district, but from people who were otherwise reporters who might have thought, and maybe this is naive, but I wouldn’t be surprised. And they might have thought, “You know what, that was too far. You’ve done yourself a disservice and it’s harder for me to support you going forward.” And that had an effect on how he decided to change his tune on this disgraceful comment.

Preet Bharara:              So, the point is, in the same way that every vote makes a difference, I think every time you speak up it makes a difference. And if you can get your neighbors to speak up, it makes a difference. And if you can get your colleagues on the other side of the aisle to speak up, it makes a difference when it comes to matters, certainly, that should transcend politics, like coming together as a country to fight terrorism, which we’re all against.

Preet Bharara:              Well, that’s it for this episode of Stay Tuned. Thanks again to my guest, Ann Selzer. If you like what we do, rate and review the show on Apple Podcasts or wherever you listen. Every positive review helps new listeners find the show. Send me your questions about news, politics, and justice. Tweet them to me @PreetBharara with the hashtag askpreet, or you can call and leave me a message at (669) 247-7338. That’s (669) 24-Preet or you can send an email to staytuned@cafe.com.

Preet Bharara:              Stay Tuned is presented by CAFE, the executive producer is Tamara Sepper. The senior audio producer is David Tatsciore. And the CAFE team is Julia Doyle, Matthew Billy, David Kurlander, Calvin Lord, Sam Ozer Staton and Jeff Eisenman. Our music is by Andrew Dost. I’m Preet Bharara, stay tuned.