Posts Tagged ‘Richard Tol’

Richard Tol misrepresents consensus studies in order to falsely paint John Cook’s 97% as an outlier

September 24, 2015

John Cook warned me: if you attempt to quantify the level of scientific consensus on climate change, you will be fiercely criticized. Most of the counterarguments don’t stand up to scrutiny however. And so it happened.

The latest in this saga is a comment that Richard Tol submitted to ERL, as a response to John Cook’s study in which they found 97% agreement in the scientific literature that global warming is human caused. Tol tries to paint Cook’s 97% as an outlier, but in doing so misrepresents many other studies, including the survey that I undertook with colleagues in 2012. In his comment and his blogpost he shows the following graph:

Richard Tol misrepresenting existing consensus estimates

Richard Tol comes to very different conclusions regarding the level of scientific consensus than the authors of the respective articles themselves (Oreskes, 2004; Anderegg et al., 2010; Doran and Kendall Zimmerman, 2009; Stenhouse et al., 2013; Verheggen et al., 2014). On the one hand, he is using what he calls “complete sample” results, which in many cases are close to meaningless as an estimate of the actual level of agreement in the relevant scientific community (that counts most strongly for Oreskes and Anderegg et al). On the other hand he is using “subsample” results, which in some cases are even more meaningless (the most egregious example of which is the subsample of outspoken contrarians in Verheggen et al).

The type of reanalysis Tol has done, if applied to e.g. evolution, would look somewhat like this:

  • Of all evolutionary biology papers in the sample 75% explicitly or implicitly accept the consensus view on evolution. 25% did not take positon on whether evolution is accepted or not. None rejected evolution. Tol would conclude from this that the consensus on evolution is 75%. This number could easily be brought down to 0.5% if you sample all biology papers and count those that take an affirmative position in evolution as a fraction of the whole. This is analogous to how Tol misrepresented Oreskes (2004).
  • Let’s ask biologists what they think of evolution, but to get an idea of dissenting views let’s also ask some prominent creationists, e.g. from the Discovery Institute. Never mind that half of them aren’t actually biologists. Surprise, surprise, the level of agreement with evolution in this latter group is very low (the real surprise is that it’s not zero). Now let’s pretend that this is somehow representative of the scientific consensus on evolution, alongside subsamples of actual evolutionary biologists. That would be analogous to how Tol misrepresented the “unconvinced” subsample of Verheggen et al (2014).

Collin Maessen provide an detailed take-down of Richard Tol on his blog, quoting extensively from the scientists whose work was misrepresented by Tol (myself included). The only surveys which are not misrepresented are those by Bray and von Storch (2007; 2010). This is how I am quoted at Collin’s blog RealSkeptic:

Tol selectively quotes results from our survey. We provided results for different subsamples, based on different questions, and based on different types of calculating the level of agreement, in the Supporting Information with our article in ES&T. Because we cast a very wide net with our survey, we argued in our paper that subgroups based on a proxy for expertise (the number of climate related peer reviewed publications) provide the best estimate of the level of scientific consensus. Tol on the other hand presents all subsamples as representative of the scientific consensus, including those respondents who were tagged as “unconvinced”. This group consists to a large extent of signatories of public statements disapproving of mainstream climate science, many of whom are not publishing scientists. For example, some Heartland Institute staffers were also included. It is actually surprising that the level of consensus in this group is larger than 0%. To claim, as Richard Tol does, that the outcome for this subsample is somehow representative of the scientific consensus is entirely nonsensical.

Another issue is that Richard Tol bases the numbers he uses on just one of the two survey questions about the causes of recent climate change, i.e. a form of cherry picking. Moreover, we quantified the consensus as a fraction of those who actually answered the question by providing an estimate of the human greenhouse gas contribution. Tol on the other hand quantifies the consensus as a fraction of all those who were asked the question, including those who didn’t provide such an estimate. We provided a detailed argument for our interpretation in both the ES&T paper and in a recent blogpost.

Tol’s line of reasoning here is similar to his misrepresentation of Oreskes’ results, by taking the number of acceptance papers not just as a fraction of papers that take position, but rather as a fraction of all papers, including those that take no position on current anthropogenic climate change. Obviously, the latter should be excluded from the ratio, unless one is interested in producing an artificially low, but meaningless number.

Some quotes from the other scientists:


Obviously he is taking the 75% number below and misusing it. The point, which the original article made clear, is that we found no scientific dissent in the published literature.


This is by no means a correct or valid interpretation of our results.

Neil Stenhouse:

Tol’s description omits information in a way that seems designed to suggest—inaccurately—that the consensus among relevant experts is low.


To pull out a few of the less expert groups and give them the same weight as our most expert group is a completely irresponsible use of our data.

You can read their complete quotes at RealSkeptic.

See also this storify of my twitter discussion with Richard Tol.

To publish BS or not, that’s the question

November 11, 2011

Richard Tol levied a strong accusation at Judith Curry for highlighting two seriously flawed papers (via twitter):

Its wrong, but with @JudithCurry lending her authority it becomes disinformation

Judith defended herself in a post where she tries to shift the blame to the mainstream scientists:

 Here is a quiz for you.  How many of these disinformation tactics [a list containing a mix of logical fallacies and avoidance tactics] are used by:

  • JC (moi)
  • Public spokespersons for the IPCC
  • Joe Romm
  • Marc Morano

If that’s not a dog-whistle I don’t know what is. 

Keith has a nice rundown of the discussion, and the ensuing thread over there contains many good comments. He’s got a knack for hosting interesting discussions.

Richard has since laid out his argument as to what’s wrong with the papers in a guest post over at CE.  Basically they’re methodologically flawed:

Using “detrended” fluctuation analysis to study “trends” was a dead giveaway that something is not quite right with these papers.

Tol goes on to write: 

7. There is a substantial body of climate research that is credible — even if it reaches opposite conclusions — but there are also papers (left, right, and center) that are just flawed.
8. If flawed papers reach a certain prominence, they should be debunked. Prominent but flawed research does damage as it misinforms people about climate change. Publicly criticizing such research hardens the existing polarization.
9. If flawed papers linger in obscurity, they should be ignored. The papers are wrong but do no damage. Lifting a flawed paper out of obscurity only to debunk it, is no good to anybody.

Curry takes especially issue with the last statement:

Yours isn’t a statement about science, but about playing politics with science, and reinforces the gatekeeping mentality in climate science that was embarassingly revealed by the CRU emails. (…)

Of course scientists don’t want the public to be misinformed about the science! So If I’m concerned about public understanding of science, I’m automatically “playing politics with science”? Then I sure hope every scientist is.

Judith rightly says that “Of course there are flawed papers that get published.” But why shining the spotlight on them? What’s gained by doing so?

It’s true that these discussions don’t occur about science without policy relevance. Research on the mating behavior of fruit flies won’t result in argument whether a flawed paper should be promoted in the public sphere or not.

The differences are that 1) such research is not present in the public sphere, because the public isn’t interested, and 2) even though flawed papers exist in any field, the more its conclusions clash with ideologies, the more attempts will be made to reach opposite conclusions and thus the more deeply flawed/biased papers will be published. It’s not a coincidence that there’s no fruitflies-version of EIKE or Heartland. 


Most people don’t come to climate etc. to reinforce their prejudices (there are far too many echo chambers where this is much more satisfyingly accomplished). They come here to learn something by considering the various arguments.

The general tone of comments at CE makes me strongly doubt this last statement.


I would agree with you [no harm done by highlighting flawed studies] if climate blogs were exclusively read by well-intentioned, well-informed, and intelligent people.

Richard further shows his mastery in the tweet-universe with one-liners such as

I argue for self-censorship. It is what separates adults from children.

Over at CaS, Roger Pielke Jr makes the point that wrong or bad articles can be a useful teaching tool. And indeed they can. But as Stoat rightly says,

within a managed class structure with someone guiding the discussion, it is fine to discuss flawed texts, for the reason given: it encourages critical thinking. That wasn’t what Curry was doing.


Curry took two papers that almost nobody had read, and put them in the limelight.
The papers say 2+2=5.
There are a lot of people who would like to believe that. It is not true.
So now there is yet another dogfight about whether the answer is 3, 4, or 5. We can do without that.
There are plenty of real issues to argue over.

Jonathan Gilligan, consistently thoughtful, writes:

Pielke has said that he views blogs as more like the kind of discussions people conduct over beers at the neighborhood bar, and from that perspective Richard’s criticism makes no more sense than telling the crowd at the pub to leave sports commentary to the experts. 

Tol makes some valid points here, but Pielke is more persuasive. People will read these blogs or not as they choose, and when a blog repeatedly calls attention to crap, its credibility and its audience will adjust to reflect this. Climate Etc. is not The Wall Street Journal, so the greater danger in Curry’s gushing over crap is to Curry’s reputation, not to the public understanding of science.

I have also compared blogs to bar-discussions (quoting Bob Grumbine), but that comparison is about the presence (or lack) of quality control. As Tol rightly says, 

With academics blogging and tweeting, and journalists, and prime ministers, and institutes, departments, agencies and companies, I don’t think there is a one-rule-fits-all.

At CE, thousands of people are listening. Judith’s opinion and her writings make their way to the general public and politicians via mainstream media and Senate hearings as well. By the scale of those who are engaged in the conversation, that is orders of magnitude different from a discussion in a bar. That also means that the risk is twofold: Both to Curry’s reputation (her problem) and to the public understanding of science (everyone’s problem, even though Curry tries to belittle that).

Whereas Tol argued based on methodological flaws, Fred Moolten explains why the papers’ conclusions are unsupportable on physical grounds and I made a similar argument:

Conservation of energy precludes the climate to wonder off too far in any direction without being “forced” to by changing boundary conditions. Unless of course the energy is merely being redistributed within the system. Which it isn’t, since all other compartments of the climate system are gaining energy.

The paper’s conclusion that the observed warming is “predominantly a natural 100-year fluctuation” is at odds with conservation of energy.

All very reminiscent of the random walk saga and the Harry Potter theory of climate.


Get every new post delivered to your Inbox.

Join 150 other followers