Tag Archives: statistics

Nate Silver falls off

In 2012, Nate Silver faced a conservative and media-led backlash for bringing rigor to election forecasting. His newly launched journalism project is now facing a backlash for failing to live up to its promise.

I am probably the ideal audience for Nate Silver’s new journalism project FiveThirtyEight.com. I am someone who values data in a frequently-substance-free world of reporting. Although I am an unabashed Sam Wang PEC partisan, I certainly appreciated Silver mainstreaming election forecasting based on factors other than wishful thinking and journalism biases towards “the horse race” and “momentum”. When Silver was attacked by know-nothings in the media and the conservative blogosphere, I cheered him on, and savored his election day vindication, anti-climatic as it was.

Rather than topping my “must read” list, however, the new FiveThirtyEight is something I won’t be reading. Here’s why:

I became aware of Silver’s imminent launch by his public Twitter announcement of two hires to cover science for his new venture: Emily Oster, a University of Chicago economist famous for counter-intuitive revelations (sound familiar?); and Roger Pielke Jr.

Image courtesy of Flickr user “ferdicam”, used under Creative Commons.

Now, I am not going to get into Roger’s pathological attacks on climate scientists. I am not going to get into his sweaty delusions of persecution. I am not going to get into Roger’s complete misunderstanding of elementary aspects of climate science. I am going to focus on just two things: what Nate Silver is known for, and what Roger Pielke Jr. is known for.

Nate Silver’s reputation is based on being a stats wiz. This is what his blogging was devoted to, what his best-selling book is about, and the one thing he has that his competition/peers like Ezra Klein or Matt Yglesias don’t. And one of Nate Silver’s very first, very public hires (Roger Pielke Jr.) sucks at statistics. Not “published something in need of minor correction once or twice” sucks. “Doesn’t understand how a t-test works” sucks. “Doesn’t understand basic probability” sucks. Sucks out loud. Sucks on ice.

Roger’s very first article for Silver’s new site is, unsurprisingly, about Roger’s hobbyhorse. The claim that disaster losses are not increasing due to climate change.

Let’s be clear about some things. Climate change is real. Humans are not just “contributing” to it, we are responsible for essentially all of it over the past several decades. Our perturbation of the climate through our emissions of greenhouse gases is fundamentally changing the Earth system. The biosphere and human systems are going to have to adapt to a rate of change as of now unseen anywhere else in the paleoclimatic record. In the absence of emissions stabilization, a difficult but decidedly achievable outcome, the threat to the biosphere and society is daunting. The amount of climate change we’ve already experienced, while extremely serious, is tiny compared to the impacts we will see in world of unchecked fossil fuel exploitation. In addition to changes in the average or mean state of the system, we have already begun to see changes in some types of extreme weather events, and changes to the drivers of yet other extreme events.

Ostensibly, Roger Pielke Jr. accepts all of the above. He just doesn’t want you to focus on this big picture. Instead Pielke wants you to believe and to focus on the claim that we’ve seen no increase in “normalized” damages due to climate change. The fundamental conceit of this claim is that even though disaster losses are unquestionably on the rise, once you account for changes in the value of infrastructure being built in areas affected by disaster (due to population growth, inflation, etc.), there is no “statistically significant increase”.

This claim rests on our ability to account for factors which might spuriously inflate the damages caused by disasters, but also our complete failure to account for factors that have allowed us to avoid even greater losses.

The case of 2012’s Superstorm Sandy is illustrative. While Roger spent the first few days of the disaster trying to play down the magnitude of the mounting carnage, Sandy ultimately ranked among the most costly storms on record, even using normalized losses. Preliminary estimates range from $50-65 billion USD.

And yet it could have been so much worse.

Hurricane Sandy uncharacteristically failed to recurve out to sea, and barreled back towards the East Coast of the US. Due to the amazing advances we’ve seen in our ability to model the global weather system, we knew well in advance that this unexpected turn by Sandy was a real possibility.

This possibility was taken into consideration by those trying to game out the impact of Sandy’s landfall. The impact of rising sea levels on the frequency and severity of storm surge flooding was as well. Looking to a future of warming-boosted surges, researchers identified huge vulnerabilities in the New York transportation infrastructure to previously rare events. Such considerations ultimately led the MTA to shutdown the subway system in order to avoid the corrosive impact of salt water if the tunnels were flooded. This decision, informed by modeling and meteorological sophistication unimaginable in the early 1900s, saved the subway system and prevented New York City from being paralyzed for weeks and nearly doubling the economic damages.

Image via Twitter

Roger Pielke Jr.’s “normalized” disaster loss fixation takes none of this into account. Nor does it account for the benefits of building code improvements. Or other disaster prevention measures like dikes.

Paul Krugman is among a growing list of knowledgeable folks who were hopeful about Silver’s new enterprise but are less than impressed. Krugman writes:

… data are never a substitute for hard thinking. If you think the data are speaking for themselves, what you’re really doing is implicit theorizing, which is a really bad idea (because you can’t test your assumptions if you don’t even know what you’re assuming.)

I feel bad about picking on a young staffer [Note: not Pielke Jr.], but I think this piece on corporate cash hoards — which is the site’s inaugural economic analysis — is a good example. The post tells us that the much-cited $2 trillion corporate cash hoard has been revised down by half a trillion dollars…

… what does this downward revision tell us? We’re told that the “whole narrative” is gone; which narrative? Is the notion that profits are high, but investment remains low, no longer borne out by the data? (I’m pretty sure it’s still true.) What is the model that has been refuted?

“Neener neener, people have been citing a number that was wrong” is just not helpful. Tell me something meaningful! Tell me why the data matter!

Though Krugman is referring to a different 538 article, he could easily be making the same criticism of Pielke’s. Why do Pielke’s data matter? Are disaster losses not increasing? They are. Does “normalizing” the loss data tell the whole, unbiased, story? No, it doesn’t. Are extreme events, and drivers of yet more extreme events, changing in response to GHG emissions? They are.

If Nate Silver’s mission is to bring statistical cachet to good journalism, he’s off to a terrible start. One of his first big hires is terrible at statistics. If Silver wants to tell us something meaningful instead of peddling freakonomics-lite contrarianism, he’s similarly off to a poor start. Pielke’s personal hobbyhorse obscures far more than it enlightens. It offers a cocktail party morsel of contra-conventional wisdom instead of intellectual nourishment.

There are probably a lot of people who would like to see Silver fail. I’m not one of them. I just won’t be one of his readers, either, unless he makes some big changes to his current model.

A test for establishment climate journalists

Image courtesy of Flickr user just.Luc

Over at Keith Kloor’s blog, I wrote:

Keith, respectfully- either you can acknowledge that [Bjorn Lomborg] engages in blatant misrepresentation of key indicators of climate change like [sea level rise] and [temperature] trends, or you cannot.

If you can’t, I’m not particularly interested in whether it’s a refusal to do so due to adherence to some imagined journalistic allegiance to neutrality or out of a lack of ability to understand that he’s doing it.

If climate journalists either can’t see what he’s doing or refuse to acknowledge it, then we’re in far worse trouble than I ever imagined.

I’ve submitted a similar question to Dot Earth. Can establishment climate journalists acknowledge what Lomborg does? If not, what hope is there that the general public can make an informed assessment of his credibility?

Untold gallons of figurative ink have been spilled over the efforts of climate “skeptics” to discredit Michael Mann and colleagues’ paleoclimatic reconstructions on the grounds of bad statistics. Republicans even went so far as to get “statistics expert” Ed Wegman to put an official seal on the supposed discrediting, and we can see how that’s all working out for them. Meanwhile, the overall conclusions of the Mann et al. papers have been upheld by independent reviews, other multiproxy reconstructions, and independent lines of paleoclimatic evidence, even though some of Mann’s initial statistical choices could have been better.

By contrast Lomborg takes a metric like temperature or sea level rise and then cherry-picks an interval to get the lowest possible trend out of it. If it’s an interval of two years at the time of press, so be it. If he needs to write another article and using the same interval no longer gives the lowest possible trend, he’ll use four. It’s inarguable that using his own intervals from previous claims completely contradicts his current ones, and that there is no physical, statistical, or logical justification for doing so. He is just cherry-picking. Period.

But you’d never know it reading Andy Revkin’s or Keith Kloor’s blogs. Why not? What good is climate journalism if it must slavishly attend to largely unfounded claims of “skeptics” but can’t identify clear-cut cases of misrepresentation by people like Bjorn Lomborg?