Trump, Biden and the problem with today’s polls

Are we traveling down the same well-worn path when it comes to reporting on polls? And when it come to some infamous 2016 predictions: where are they now?

Full transcript: 

Sharyl Attkisson  0:09  

Hi everybody Sharyl Attkisson here. Welcome to another edition of the Sharyl Attkisson Podcast on Just the News.com, a digital news site dedicated to facts, not spin, and reporting on underreported stories and views that cut across the grain or get ignored by other media. I hope you'll subscribe to all of our Just the News podcasts, John Solomon Reports and The Pod's Honest Truth and consider pre ordering my new book that's coming out in November ready to be pre ordered now. It's called Slanted: How the News Media Taught us to Love Censorship and Hate Journalism. Today, I'm going to take you on a polling trip down short term memory lane.

Sharyl Attkisson  1:00  

Polling, one of my favorite topics. Remember we asked after the media got caught by such surprise with a Trump win in 2016? We asked, would the media self correct next time? Look at its polling and punditry and change for 2020 improve? I guess we now have our answer. No. One popular thing that's often said now is that, well, the polls in 2016 weren't really wrong. They predicted Hillary Clinton would win the popular vote and she did. But that's not really the point is it? They mistakenly predicted she would win the election. And the reporting on the polls failed to point out that the elections are won and lost state by state with electoral votes, not the popular vote. So saying, hey, the polls were right, is a little like saying, I picked the correct lottery numbers just not the ones that won the prize. It matters. But here we are, again, with breathless reporting on polls, no caveats that the national polling may have little to do with the state results that will actually elect the next President or reelect President Trump.

Sharyl Attkisson  2:10  

All of this is part of what I call the transformation of polling from tools that attempt to sway public opinion rather than measure it. Not to mention the mysterious methodology and sampling. I studied calculus in college, and maybe somebody could eventually help me understand the weighting methods deployed and why. But they're so mysterious and complex, I tend to wonder if I'd ever understand them. Most of the polls are using samples that are so relatively small, sometimes 800 people or 1200 people, it's hard to imagine that that could be reflective of anything. One that I read the other day said the weighted sample is 1200. What's a weighted sample? Does that mean they interviewed more than 1200 people or less, but somehow deploy that complicated formula to hold the sample to precisely 1200? Why? Why not just use the sample you actually sampled? Then try reading one time the disclosures about the factors that they adjust the results for. They change the results based on demographics, age, gender, race, income, almost everything except political affiliation. So let's say they don't have enough Hispanic respondents in the sample to reflect what they decide is a reasonable proportion. They may give greater weight to the answers that they have from the Hispanics. They have maybe five Hispanic respondents become 10 for the purposes of the poll, but then they don't weight the political party aspect of the same poll. So are the Hispanic respondents doubled? But it still looks like in the end of the poll that the original number of Democrats and Republicans were sampled because that part wasn't weighted? Wouldn't that skew the appearance of the results.

Sharyl Attkisson  4:03  

It's almost like they interview 1200 people, and then make the results come out however, they kind of wanted to come out by changing the sampling to whatever they think it should be rather than what they actually sampled. Look deciding how many people they sample, the way they were the questions, who they asked what how they contact them, whether it's on the internet, cell phones, landlines, how the results are characterized and weighted. It's all determined by the polling company, and the news organization that pays them and we'll spin the results however they want to spin them. I urge you to look for yourself sometime. And really, there are easy ways to check some of these things out. When you see a poll being reported. Go to an article online. Usually there's a link somewhere within the story that links to the actual poll questions and answers and then somewhere on that document, you can find a link to what they may call the top line.data or other numbers, that's what you're looking for. 

Sharyl Attkisson  5:03  

When I get there, I like to look at a couple of things. First, I like to look at the number of Democrats versus Republicans interviewed. Sometimes for some reason they group together. People who say they're Democrats with them. People who say they lean Democrat. And sometimes they don't break down the separate numbers, Democrats versus Democrat leaners. They just group it together. Same with Republicans. Why? Who knows why, I mean, if you don't see the original numbers broken down, you can't really guess to see why they might be grouping them together sometimes for results, and sometimes they don't, but I will tell you, almost always they interview more Democrats. And I poked around about why that's the case back in 2016. I spoke to pollsters and polling companies. And the rationale is for interviewing more Democrats is that more Americans identify as Democrats than Republicans and they expected based on the most recent election that more Democrats would vote than would Republicans, although it was going to be hard to tell with a Trump factor, which was quite unusual. But the rationale that they should interview more Democrats and Republicans because more people identify as Democrats was not the case earlier this year. For the first time in a long time, more people identified as Republicans than Democrats in consistent polling, but I noted that the polls I checked at that time still interviewed substantially more Democrats than Republicans. In other words, even when that tilt changed, and more voters were identifying themselves as Republicans, the polls did not change. The ones that I looked at, did not change their sampling. Obviously, this makes a difference and should be included for context when the results are reported, but it almost never is. 

Sharyl Attkisson  6:54  

For the sake of simplicity by way of example, if you were to find Joe Biden, the Democrat is beating President Trump the republican 60% to 40%. That sounds awful for Trump. But what if you were to learn the poll interviewed 60% Democrats and 40%. Republicans? Well, that would be a different story. That would mean, the two candidates are pretty much running even, winning all of the support from their respective parties. Or what if you find, as I did in some polling between Hillary Clinton and Trump, and I'll oversimplify this for way of example, what if you were to find that Hillary Clinton was beating Trump 55% to 45% - 10 points, 10 percentage points? Well, that sounds bad for Trump, until you see that they interviewed 60% Democrats, which would mean if Hillary Clinton was only getting 55% of that support, that she's not even pulling in all the support from Democrats and Trump would actually have the edge even though the polling results would look like he's 10 percentage points behind. This polling is complicated and malleable stuff. certainly nothing wrong with measuring public opinion. But when the results are reported with zero context about the sampling, and the way the sample was changed, how helpful is that really in terms of accuracy?

Sharyl Attkisson  8:18  

Can you envision a scenario under which a poll polls a sample until it reaches the sort of response that might most please the news organization paying for the poll? Or maybe the pollsters work their magic to reduce certain answers and inflate others as part of the waiting process so that the right answers are achieved? Well, that wouldn't be ethical. And I'm sure that reputable polling organizations don't do that. Let me rephrase that. I hope they wouldn't do that. After a break, we will visit some notable predictions from analysts and prognosticators in 2016 and learn a little bit about where are they now.

Sharyl Attkisson  9:11  

We're back. In this election cycle, there was a poll from Quinnipiac in August of 2019. And it found that every top major Democrat, was beating President Trump by at least nine points. Some analysts press the Democrats hovering near the 1% mark at this time to hang it up. But the election in August of 2019 was still a lifetime away in political terms. Everyone seems to have forgotten that Trump was hovering near the bottom at the exact same time that the 2016 campaign. Again, one could extrapolate that those early polls should not be taken as hard indicators of what will come. So let's look back at may of 2015. The same polling company Quinnipiac University found that Donald Trump was at the top of the no way list among Republicans. 21% of Republican voters said they would definitely not support him, again, May of 2015. In June of 2015, there was an NBC Wall Street Journal poll that put Donald Trump at 1% behind 10 Republican candidates. Trump was behind in this NBC Wall Street Journal poll. Jeb Bush, Scott Walker, Marco Rubio, Ben Carson, Mike Huckabee, Rand Paul, Rick Perry, Ted Cruz, but you forgot all of these people had run, Chris Christie and Carly Fiorina. Now Politico's Daniel Straus tried to tamp down fears that Trump could actually be a winner. He wrote, quote, whispers of a Trump surge are making the rounds, it might be wise to take a deep breath nationally, Trump's polling has been on the decline.

Sharyl Attkisson  10:56  

Former New Hampshire republican party chairman Fergus Cohen declared that there was, quote, no visible grassroots movement for Trump in New Hampshire. And Patrick Murray, who is the director of the Monmouth University polling Institute. He said quote, at the end of the day, it's quite possible that Donald Trump will get 11% in New Hampshire, but that might be his cap. 11% he said, Well, Trump ended up winning the Republican primary in New Hampshire with just over 35% of the vote. That was more than triple Monmouth's prediction. In July of 2015, there was a USA Today Suffolk poll that found Trump trailed Clinton by 17 percentage points 51% to 34%. In September, and NBC Wall Street Journal poll still in 2015, found that quote, the only Republican whom Clinton led by a significant margin was businessman Donald Trump. She supposedly had a 10 point advantage at that time. Then moving a little closer to the primaries. Let's go to November 2015. Nate Silver of the polling site 538 concluded that Trump's odds of winning the presidency were quote higher than zero but considerably less than 20%. In December 2015, there was an NBC News Wall Street Journal poll that determined Hillary Clinton would beat Trump by 10 points. Another Quinnipiac University poll at that time found Clinton would thump Trump 47% to 40%. NBC Wall Street Journal said Clinton would smash Trump 50% to 40%. Fox News said Clinton would best Trump 49% to 38% deroy Murdoch of National Review predicted a Trump nomination would quote engineer a Hillary Clinton landslide. And he advised at the time that Marco Rubio would be, as he said a far more elusive target for Clinton slings and arrows.

Sharyl Attkisson  13:03  

Moving ahead to January 2016. David Wasserman wrote in 538, that a Donald Trump nomination would make Clinton's election very likely and raise the odds of a Democratic Senate. He wrote. In other words, if you're a member of the Republican party who wants to win in November, it's basically Rubio or bust. And in March 2016 a just the facts analysis of hard data by theconversation.com, determined that Trump would not win enough electoral votes to beat Clinton. They said that he would end up with just 236 electoral votes, which would be 34 fewer than needed: 270. Well, they were off by about 68 electoral votes, not a small number. Trump actually received 304 electoral votes, that was 34 more than he needed. Clinton actually ended up with even fewer electoral votes than the deficit that they predicted for Trump. All of this, as you can see, has contributed greatly to the media's declining credibility among the voting public, but there was no mountain of mainstream analysis or criticism of these erroneous polls and predictions. It was sort of just put down the memory hole like it never even happened. As promised, I looked up what a couple of these prognosticators have done since there are mistaken predictions in 2016. Here's a couple of findings. Remember, Politico's Daniel Straus, he was basically tamping down fears that Trump could possibly win in 2016. He is now listed as a senior political reporter at The Guardian newspaper.

Sharyl Attkisson  14:57  

What about Fergus Cullen the New Hampshire Republican party Chairman at the time, who declared there was no visible grassroots movement for Trump in New Hampshire. He is now the principal of the consulting firm Fergus Cullen Communications. Patrick Murray, director of the Monmouth University Polling Institute, he's the one who predicted that Trump could possibly get no more than 11% in New Hampshire. This is for the Republican primary when Trump ended up winning with more than triple that 35% of the vote. Well, Patrick Murray is still with Monmouth University polling Institute. Then Nate Silver of the polling site 538, who at one point said Trump's odds of winning are higher than zero but considerably less than 20%. He is still editor in chief of 538 and a special correspondent for ABC News. And David Wasserman, who wrote in 538 about a very likely Clinton when if Donald Trump were nominated on the Republican side, he also said that it would raise the odds of a Democratic senate wrong. And he said that really the best chance for Republicans to win was Rubio. He is house editor for the cook Political Report, where it says he is recognized as one of the nation's top election forecasters and he's a contributor to NBC News. His bio says in 2016, he drew praise for his accurate pre election analysis. Now, in fairness, all of these people I've mentioned, we're all over the map at different points on their predictions. In fact, Wasserman after initially saying ... seemingly that Trump couldn't win, or that if he were up against Clinton, Clinton would win. He ultimately did write an article two months before the election that said how Trump could win the white house while losing the popular vote, which is exactly what happened.

Sharyl Attkisson  17:02  

And one more mention about all of this and how it applies to Biden in 2020. You know, he survived his own bout with polling and prognosticators. Prior to his Super Tuesday surge. A fellow Democrat Van Jones of CNN wrote him off. You may remember this as dead man walking. And other Democrats at the time, you may remember were urging Biden to just give it up saying he didn't have a chance of winning the nomination or the presidency. The narrative was that he could not possibly win the nomination for Democrats. Just look at the polls. People said that the narrative and those polls turned out to be wrong. Food for thought, more than four months before the 2020 election. 

Sharyl Attkisson  17:56  

I hope you enjoy today's podcast check out Just the News.com. Don't forget to subscribe to the Sharyl Attkisson Podcast, my other podcast Full Measure After Hours and all the Just the News podcasts wherever you'd like to listen, they are free. And if you share and post reviews that helps get them distributed and get the word out. If you like the topics I talked about in my podcasts, I promise you're gonna love my new book that's coming out in November, and you can pre order it now anywhere. It's called Slanted: How the News Media Taught us to Love Censorship and Hate Journalism. I hope you'll consider supporting independent journalism and pre ordering Slanted anywhere. Do your own research. Make up your own mind. Think for yourself.