NOAA agrees to restore ‘scientific integrity’ in its influential $1 billion climate disaster tally

Every year, the National Oceanic and Atmospheric Administration tallies up all the disasters exceeding $1 billion in damages. The numbers get widespread media attention, which portray the costs of damages from climate-related natural disasters as increasing every year. The tally is often cited by climate policymakers and climate activists calling for regulations. 

In January, Dr. Roger Pielke Jr., retired professor of environmental studies at the University of Colorado at Boulder, released a preprint of his study, “Scientific integrity and U.S. ‘Billion Dollar Disasters,” finding that the methodology NOAA uses in the tally lacks scientific integrity. The preprint, which is a preliminary version of a scientific manuscript that researchers post online before peer review, has since been peer-reviewed and published. 

NOAA responded to the study on Thursday, saying it “will take actions to improve the documentation and transparency of the data set for greater compliance with NOAA’s Information Quality Guidelines.” 

“After many years of being stonewalled by the group inside of NOAA that produces the ‘billion dollar disaster’ tabulation, I am encouraged that NOAA leadership has now responded formally and positively to concerns about the tabulation's data and methods. Science should be self-correcting, and sometimes that takes a while, especially when science becomes wrapped up in politics,” Pielke told Just the News.

Normalized costs

Pielke has done extensive research spanning nearly three decades into the trends of disaster costs over time, which show the trends are actually declining.

His research normalizes the disaster costs, which means he adjusts for differences in wealth over time. Pielke explains why this is important in an article on his “The Honest Broker” Substack. If a category 3 hurricane hit Miami Beach in 1926, it would impact far less development than a storm of equal intensity hitting the beach today. Without controlling for these differences, Pielke writes, it’s impossible to reliably determine trends in damages.

Pielke says his study on NOAA’s billion-dollar disaster tally finds that the data NOAA publishes lacks transparency that would allow the sources of the data to be verified. This doesn’t follow the agency’s own guidelines for scientific integrity, the study notes. Likewise, the cost figures are not normalized, which produces misleading results. The agency is also, according to the study, incorrectly attributing the trends to changes in climate over time.

In January, a NOAA spokesperson defended the tally, arguing that it’s based on two decades of research and close collaboration with public- and private-sector partners. Now that the study is published, the agency has had a change of heart. 

The key points in the agency’s response to Pielke’s research, as Pielke lays out on his Substack, are that NOAA has failed to fully document its methods for calculating disaster losses. The agency admits that it doesn’t adequately define categories for losses. It also systematically failed to disclose its sources, and now provides 20 previously undisclosed sources. Also, the tabulation hasn’t been externally peer reviewed since 2015, which violates the agency’s own policies. NOAA is now initiating that process. 

The agency also reveals that historical versions of the tally go back to only 2020. “Incredibly, NOAA admits that they do not have 40 of 43+ years of original data that make up the tabulation,” Pielke said. 

Course correction

In its response, NOAA determined that no correction is needed for some points that Pielke’s research raises, including disclosure of insurance data it uses, which the agency said can’t be revealed due to privacy and intellectual property considerations. It will, however, evaluate the use of this data in the future. 

Pielke also requested changes to methodology, such as addition of events that don’t meet the billion-dollar threshold. The agency stated that it understands the value of these requested changes to its methodology, but stated they are outside the scope of its “request for correction” process. 

In the study, Pielke explains that the intent of his research is not to call into question that climate change is happening or that it’s a problem deserving of attention. The goal of the study is to question “whether the NOAA billion dollar disaster time series provides evidence of detection or attribution of changes in the climate of extreme weather events in the United States, as frequently claimed. Economic loss data is not suitable for detection and attribution of trends in extreme weather events because losses involve more than just climatic factors.” 

While the agency is reacting positively to the points raised in Pielke’s research, it remains to be seen if the study or the changes at NOAA will have any impact on climate reporters who often attribute trends in disaster costs to trends in climate over time. 

For example, The New York Times reported in January, “That rise in billion-dollar disasters over time reflects two long-term shifts, according to NOAA. One is the growing frequency and severity of extreme weather events as global temperatures rise.” Ill-informed climate reporters sometimes blame climate change for a wide range of problems. In October of 2022, The Washington Post declared that hate speech and aggressive anti-social behaviour was the result of climate change.

Unfortunately, the agency continues to promote the connection between climate change and disasters based on its less than scientific body of work. In an article posted Thursday on the National CPR Association’s website about a hailstorm in Denver, NOAA climatologist Adam Smith refers to the NOAA billion-dollar disaster data and claims “it shows that extreme disasters are increasing in frequency.” The association is primarily engaged in selling preparatory classes for certification in CPR and First Aid.

Pielke said in an interview that if NOAA follows through on the actions it says it will take, it will have a positive influence over policymakers. 

“President Biden, the U.S. National Climate Assessment, members of Congress, and federal agencies have all relied on the ‘billion dollar disaster’ tabulation as if it were science. Going forward, policy makers now can be clear that the tabulation is just clever marketing,” Pielke said.