Back in November, EID detailed why a study claiming fracking is causing a rise in premature births in shale regions of Pennsylvania actually fell spectacularly short of supporting that claim. Now, a top-tier biostatistics analysis expert has issued a sharp criticism of that study, offering even more insight into the numerous flaws the EID debunk pointed out.
Dr. Tony Cox — who is a clinical professor of biostatistics and informatics at the University of Colorado-Boulder, as well as President of Cox Associates, a Denver-based applied research company specializing in quantitative health risk analysis, causal modeling, advanced analytics and operations research — has penned a letter to the editor in the journal Epidemiology that essentially confirms that the study’s claims lack merit.
Cox takes particular issue with the authors’ claim that “This study adds to limited evidence that unconventional natural gas development adversely affects birth outcomes,” pointing out the study’s methodology was completely inappropriate to draw causal conclusions:
“First, ‘unconventional natural gas development adversely affects birth outcomes’ is an unwarranted causal interpretation of associational results: in general, ‘the associational or regression approach to inferring causal relations — on the basis of adjustment with observable confounders — is unreliable in many settings” (2). Without explicit causal analysis (e.g. Granger causality tests, causal graph models), claiming that associations provide evidence for a causal conclusion is unjustified.”
EID highlighted in its original debunk that the researchers also failed to analyze baseline data (despite the fact that it was readily available) and took no environmental samples. Instead, the authors chose to use modeling, which might have been somewhat acceptable — had they not botched their modeling completely, in Cox’s estimation:
“Exposure metrics were not validated and estimation errors were ignored. The authors constructed surrogate exposure metrics using well depths and distance, ignoring relevant geology and whether homes are up- or down-gradient from the well. No monitoring data were used. In addition, an unknown number of wells used imputed data for exposure characterization. It is uncertain whether the surrogate metrics accurately reflect true exposures. Yet, errors-in-variables describing such uncertainties are omitted from model specifications. This alone could invalidate reported estimates, confidence intervals and conclusions.”
“No model validation or diagnostics were conducted to determine whether assumed models appropriately describe the data; thus, conclusions may simply reflect unjustified modeling assumptions.”
Of particular note, despite using flawed modeling data when actual monitoring would have seemingly been a no-brainer, the study still showed premature birth weights in areas near shale wells was actually at or below the national average (11 percent), as EID highlighted in November.
EID also noted that the authors failed to factor in numerous other potential contributors to premature births, such as genetics, lifestyle choices and socioeconomic status. And Cox’s letter also points out that the study failed to use negative controls, such as locations with no unconventional development, that would have provided comparisons clearly indicating if there were increased premature birth rates in areas near shale wells.
It all adds up to a severely flawed study, in Cox’s estimation:
“In light of these limitations, positive associations observed in selected models between an unvalidated exposure index and two (of several) birth outcomes examined do not provide valid evidence of a causal relation between unconventional natural gas development and adverse health impacts in newborns.”
Which begs the question: Why would the researchers allow themselves to make so many fundamental flaws? Based on the authors’ affiliations, one can only speculate that the researchers might have been more interested in generating alarmist headlines than collecting accurate data that might have actually contributed to the scientific debate.
These are, after all, the same researchers responsible for the recent asthma study that yielded considerable media attention by concluding fracking is “associated” with increased asthma exacerbations in Pennsylvania — even though nine counties with no shale production from the study area had higher asthma-related hospitalization rates from 2009-2013 than the top five shale counties in the study area, according to state data.
One notable author of both studies, Brian Schwartz, is a fellow at the Post Carbon Institute, a group known for espousing discredited “peak-oil” views that, it just so happens, has called fracking a “virus” and states as its mission to eliminate fossil fuels altogether.
The studies’ authors also received funding from the Robert Wood Johnson Foundation: at least three of its board members are also on the board of World Wildlife Fund, which has made it clear that it is, “against the use of fracking to extract shale gas – or any other ‘unconventional’ fuels – from the ground.”
The premature birth and asthma studies were just two examples of loud but misleading additions to the anti-fracking echo chamber. Fortunately in this instance, a respected voice in Dr. Cox has emerged to (hopefully) limit the reverberations.