How did media get election wrong? A hurricane has the answer

This, FiveThirtyEight’s Nate Silver writes today, was a “ridiculous” and “dangerous” tweet from the Associated Press a couple of weeks ago when Hurricane Irma hit Florida.

AP deleted the tweet the next morning but Silver says it’s a fine example of the problem the media has covering hurricanes and any other story that involves the concept of probability.

Indeed, there’s a fairly widespread perception that meteorologists performed poorly with Irma, having overestimated the threat to some places and underestimated it elsewhere. Even President Trump chimed in to say the storm hadn’t been predicted well, tweeting that the devastation from Irma had been “far greater, at least in certain locations, than anyone thought.”

In fact, the Irma forecasts were pretty darn good: Meteorologists correctly anticipated days in advance that the storm would take a sharp right turn at some point while passing by Cuba. The places where Irma made landfall — in the Caribbean and then in Florida — were consistently within the cone of uncertainty.

The forecasts weren’t perfect: Irma’s eye wound up passing closer to Tampa than to St. Petersburg after all, for example. But they were about as good as advertised. And they undoubtedly saved a lot of lives by giving people time to evacuate in places like the Florida Keys.

The media keep misinterpreting data and then blame the data, says Silver.

It’s not just hurricanes. It was also Hillary Clinton.

Contrary to popular belief and the belief in the media, polls did not support the claim of a sure victory by the Democrat.

On the contrary, the more carefully one looked at the polling, the more reason there was to think that Clinton might not close the deal. In contrast to President Obama, who overperformed in the Electoral College relative to the popular vote in 2012, Clinton’s coalition (which relied heavily on urban, college-educated voters) was poorly configured for the Electoral College.

In contrast to 2012, when hardly any voters were undecided between Obama and Mitt Romney, about 14 percent of voters went into the final week of the 2016 campaign undecided about their vote or saying they planned to vote for a third-party candidate.

And in contrast to 2012, when polls were exceptionally stable, they were fairly volatile in 2016, with several swings back and forth between Clinton and Trump — including the final major swing of the campaign (after former FBI Director James Comey’s letter to Congress), which favored Trump.

The polls called for caution, rather than confidence? So why wasn’t it reported that way?

Confirmation bias is one reason.

“Journalists just didn’t believe that someone like Trump could become president, running a populist and at times also nationalist, racist and misogynistic campaign in a country that had twice elected Obama and whose demographics supposedly favored Democrats,” says Silver.

But there’s another: Journalists don’t understand probability and statistics.

And people associate numbers with precision.

A 70-percent chance of winning an election is interpreted as a candidate having a 70-to-30-percent lead in the polls. That’s not how it works.

“News organizations reporting under deadline pressure need to be more comfortable with a world in which our understanding of developing stories is provisional and probabilistic — and will frequently turn out to be wrong,” Silver concludes.

  • Gary F

    Or, the media just wanted Hillary to win. Its as simple as that.

    • lusophone

      A lot of people wanted Hillary to win.

      • X.A. Smith

        Most voters wanted her to win.

    • I think you just revealed that you didn’t read the post.

      • Gary F

        The public watches the media every day and sees their bias. Even NPR/MPR who claim to be above it all and “fair and balanced” Fox News.

        • Gary F

          And the Pollsters wanted Hillary to win, that is why their polls leaned Hillary. Polls mean less and less.

          • So you’re saying the AP wanted the hurricane to hit Tampa?

            Cruel, man. Cruel.

          • Jay T. Berken

            …and God must really not like how gays are coming out more and can now have same-sex marriage, so HE is making more 100 and 500 year storms in the past 10 years.

          • jon

            I’m just going to throw it out there, florida has only been hit by major hurricanes when the most recent FL presidential election went to republicans, going back to at least 1980. seems like a much better correlation than “the gays”.

          • Blasko

            You should read Silver’s article, if you haven’t, Gary. He seems to agree, but tweaks this with an important distinction: some folks in the media wanted Hillary to win, so they interpreted the polls in ways that agreed with them. In many cases, the polling data accurately reflected the electoral volatility and the late lean towards Trump. Polls might mean less and less if conducted poorly, sure, but it’s the interpretation of the data polls provide that should be scrutinized.

          • // You should read Silver’s article,

            Or even the one *I* wrote paraphrasing and quoting it since his conclusions were in it.

        • BJ

          “Journalists don’t understand probability and statistics.”

          I actually think that is a larger problem than most people realize.

          Journalists, mostly, study how to write. Most have not studied what they write about. Many, most, do a pretty good job of finding people who do know what they are writing about and interview them, and get those peoples conclusions.

          Pollsters do a pretty good job in their work. You will find very few that don’t care about it being as close to reality as they can make it.

          • The last time I studied how to write was senior year in high school.

            But we’re actually able to study and learn about more than one thing at a time.

            We’re kind of superhuman like that.

          • BJ

            So in agreeing with you, I guess we had a disagreement?

            I was more thinking that most Journalists haven’t got degrees in whatever the topic is so they rely on the people they interview to give them enough information so when they present it to others it makes sense. Sometimes the topics are different and are so involved that unless you have had some kind of formal training in it you probably don’t know what you don’t know. Like weather probability.

          • jon

            Bob flies airplanes…
            I’d think that would necessitate that he’d know at least a little something about the weather, though I’ll admit I don’t know how much formal training in weather related activities a pilot’s license calls for.

          • A fair amount although not as much as you get by going up and doing stupid things in it.

            http://rvnewsletter.blogspot.com/2014/10/the-trip-home.html

          • Actually, for decades now the advice for people who want to go into journalism is to get degrees in something else and an overwhelming number of people do. TV not so much.

            I think political reporters tend to know politics very well and to a degree they tend to know as much as political scientists.

            That’s NOT a good thing because then they get caught up in the politics of policy and not the humanity of policy.

          • QuietBlue

            Many people with journalism degrees end up going into PR, advertising, or marketing (I started with the third, but changed directions later on).

            Interestingly enough, my favorite J-school professors didn’t have journalism degrees themselves.

          • BJ

            Again, I think we are agreeing.

          • QuietBlue

            I have a B.A. in journalism, and at least in my program, we really didn’t spend much time studying how to write. When we did, it was to learn specific techniques and formats (i.e. a radio script, a press release, etc.). We were expected to already know how to write, and had to demonstrate it to be admitted to the program. By that point, the emphasis was on research and how to build the information you needed to be able to write or talk about something in the first place (interviewing, of course, was part of that), along with classes on subjects like media law, ethics, etc. that would round out our instruction in the field.

          • BJ

            >By that point, the emphasis was on research and how to build the information you needed to be able to write or talk about something in the first place (interviewing, of course, was part of that),

            I think we are talking the same things.

    • Lindsey

      Of course they wanted Hillary to win, so like everyone else, and as Bob said in the post, they interpreted the statistics wrong.

    • RBHolb

      Only the media. Sure, stick to that thought if it makes you feel better.

  • Al

    This, in a nutshell, is why Gary Schwitzer has a job. If you want to actually know what emerging health research says, and not what the local paper *thinks* it says, head on over to HealthNewsReview. Stellar stuff.

    https://www.healthnewsreview.org/blog/

  • jon

    Mad respect for Mr. Silver (most of the time…)
    But this message is unlikely to get through…

    Mr Silver is an excellent communicator of things statistical, but when his entire argument can be dismissed by one person who doesn’t understand data normalization by saying “cell phones cause the GOP to be over sampled!” or the hurricane model doesn’t take into account the positions of oil rigs in the gulf!

    When it takes pages of math to show that the position of oil rigs is insignificant compared to a hurricane, or that data normalization from cell phones vs home phones is not a difficult problem to overcome, people will tune out before you get finished proving it and just stick with the opinion that you are wrong, and math specifically statistics is useless…

  • Jerry

    Here’s another idea for political coverage: Quit gazing into a crystal ball and stick to reporting the news that *already* exists.

  • Guest

    Media does a very poor job of explaining stats. “Eating X cuts your chance of awful disease by 40%” means less if your original chance of awful disease was 1 of 1000 than a quarter of the population getting it.

    • “Media do…”

      But what you’re referring to are mostly research stories often based on how the scientific papers present it. But, yeah. Also, ignore any story with the word “may” in the headline.

  • Guest

    Reporting on the “Horse Race” is easier for the media than a neutral look at policies and possible effects.

    How many times has the “Those who vote for X are all Y” claims repeated in the media rather than Issue A is most important to those who plan to vote for X.

    What social issues matter to you the most and who do you think represents your views?

    Do you trust government to handle X issue the way you’d like?

    Get some REAL reporting by the media to understand the electorate and what is driving voters.

    This could also mean a 15 minute conversation with 20 totally different voters rather than boxes checked on a poll.

    It is all more complicated than a sound bite.

  • Guest

    Flip a coin 10 times, only 1 out of a thousand runs will give all heads or all tails.

    However if you record your exact list of 10 heads or tails, it will also be 1 out of a thousand run to duplicate your list exactly.

    SOOO every run of 10 flips actually is the same odds of 10 heads in a row.

    SOOO it is just as easy or just as hard to get what you hope for each time in 10 flips of a coin as getting all heads.

    A minor point but showing stats doesn’t say something is impossible as much as it says doing X is merely unlikely now, but bound to happen eventually.

    • Jerry

      Humans inability to understand probability is what makes casinos so very profitable.

      • QuietBlue

        Not necessarily. When I play at a casino (which, admittedly, is not very often), I know full well that the odds are very much against me. I do it because I enjoy it.

      • Ralphy

        Exactly.
        State sponsored gambling = a tax on those who can’t do math (or believe that a 1/1,000,000,000 doesn’t mean what it means).

      • Rob

        Magical thinking also plays a role.

  • Re: Tampa vs. St. Pete.

    Aren’t they “twin” cities? Isn’t that like predicting that the eye of a storm would hit Minneapolis rather than St. Paul?

    😉

    • Yep

      • Per what Silver wrote: “Irma’s eye wound up passing closer to Tampa than to St. Petersburg after all, for example.”

        Maybe I’m missing a significant detail, but I’m failing to understand what difference that makes – closer to Tampa than to St. Pete – especially wrt a hurricane’s eye, when all the separates the two cities is, what, 10 miles of water?

  • Postal Customer

    Nate loves probability. In fact, my takeaway message from his book was: “think probabilistically.”

    • Rob

      Probably good advice.