Consensus on consensus: a synthesis of consensus estimates on human-caused global warming

by

Most scientists agree that current climate change is mainly caused by human activity. That has been repeatedly demonstrated on the basis of surveys of the scientific opinion as well as surveys of the scientific literature. In an article published today in the journal Environmental Research Letters (ERL) we provide a review of these different studies, which all arrive at a very similar conclusion using different methods. This shows the robustness of the scientific consensus on climate change.

This meta-study also shows that the level of agreement that the current warming is caused by human activity is greatest among researchers with the most expertise and/or the most publications in climate science. That explains why literature surveys generally find higher levels of consensus than opinion surveys. After all, experienced scientists who have published a lot about climate change have, generally speaking, a good understanding of the anthropogenic causes of global warming, and they often have more peer-reviewed publications than their contrarian colleagues.

Scientific consensus on human caused climate change vs expertise in climate scienceFigure: Level of consensus on human-induced climate change versus expertise in climate science. Black circles are data based on studies of the past 10 years. Green line is a fit through the data.

The initial reason for this review article was a specific comment by Richard Tol on John Cook’s literature survey as published in ERL in 2013. Cook found a 97% consensus on anthropogenic global warming in the scientific literature on climate change. This article has both been vilified and praised. Tol argued that Cook’s study is an outlier, but he did so by misrepresenting most other consensus studies, including the survey I undertook while at the Netherlands Environmental Assessment Agency (PBL). To get a gist of the discussion with Tol see e.g. this storify I made based on my twitter exchanges with him (warning: for climate nerds only). Suffice to say the authors of these other consensus studies were likewise not impressed by Tol’s caricature of their work. This is how the broad author team for the current meta-analysis arose, which shows that Cook’s literature survey fits well within the spectrum of other studies.

The video below provides a great overview of the context and conclusions of this study:

Surveys show that among the broad group of scientists who work on the topic of climate change the level of consensus is roughly between 83 and 97% (e.g. Doran, Anderegg, Verheggen, Rosenberg, Carlton, Bray, Stenhouse, Pew, Lichter, Vision Prize). If you zoom in on the subset of most actively publishing climate scientists you find a consensus of 97% (Doran, Anderegg). Analyses of the literature also indicate a level of consensus of 97% (Cook) or even 100% (Oreskes). The strength of literature surveys lies in the fact that they sample the prime locus of scientific evidence and thus they provide the most direct measure of the consilience of evidence. On the other hand, opinion surveys can achieve much more specificity about what exactly is agreed upon. The latter aspect – what exactly is agreed upon and how does that compare to the IPCC report- is something we investigated in detail in our ES&T article based on the PBL survey.

As evidenced by the many –unfounded- criticisms on consensus studies, this is still a hot topic in the public debate, despite the fact that study after study has confirmed that there is broad agreement among scientists about the big picture: our planet is getting warmer and that is (largely) due to human activity, primarily the burning of fossil fuels. A substantial fraction of the general public however is still confused even about the big picture. In politics, schools and media climate change is often not communicated in accordance with the current scientific understanding, even though the situation here in the Netherlands is not as extreme as e.g. in the US.

Whereas the presence of widespread agreement is obviously not proof of a theory being correct, it can’t be dismissed as irrelevant either: As the evidence accumulates and keeps pointing in the same general direction, the experts’ opinion will logically converge to reflect that, i.e. a consensus emerges. Typically, a theory either rises to the level of consensus or it is abandoned, though it may take considerable time for the scientific community to accept a theory, and even longer for the public at large.

Although science can never provide absolute certainty, it is the best method we have to understand complex systems and risks, such as climate change. If you value science it is wise not to brush aside broadly accepted scientific insights too easily, lest you have very good arguments for doing so (“extraordinary claims require extraordinary evidence”). I think it is important for proper democratic decision making that the public is well informed about what is scientifically known about important issues such as climate change.

More info/context/reflections:

Dutch version at sister-blog “klimaatverandering”

Column by first author John Cook in Bulletin of the Atomic Scientists

Stephan Lewandowsky on the psychology of consensus

Collin Maessen tells the backstory starting with Richard Tol’s nonsensus

Ken Rice at …And Then There’s Physics

Dana Nuccitelli in the Guardian

Sou at HotWhopper

Amsterdam University College (AUC) news item

 

Tags: , , , , , ,

19 Responses to “Consensus on consensus: a synthesis of consensus estimates on human-caused global warming”

  1. Brandon S? (@Corpus_no_Logos) Says:

    Bart, you say:

    As evidenced by the many –unfounded- criticisms on consensus studies, this is still a hot topic in the public debate, despite the fact that study after study has confirmed that there is broad agreement among scientists about the big picture: our planet is getting warmer and that is (largely) due to human activity, primarily the burning of fossil fuels.

    But one of the most famous studies on this topic would, according to its own author, find only a 1.6% consensus on the idea warming is (largely) due to human activity. I believe you’ve criticized this result in the past, saying it is misleading, but it is the very one proposed by the authors of the paper itself.

    I suspect you will always find the “consensus” a hot topic in the public debate when you base your claims there is a consensus on the consensus on studies which only seek to find a consensus on if humans have some influence on the planet’s temperature.

  2. Bart Verheggen Says:

    Brandon, we’ve gone over this before. If you were to apply your logic to estimate the level of consensus on another well established theory, such as e.g. evolution, you’d find a very low consensus, because most articles don’t state anything about endorsing evolution in the abstract. You would e.g. find something like “out of 10,000 articles on biology, only 0.5% endorsed the theory of evolution in their abstract”. It is utterly misleading to hold that up as an estimate of the level of consensus, if the vast majority of those abstracts didn’t actually state anything for or against the theory in question.

  3. ...and Then There's Physics Says:

    As Bart has already said, Brandon appears to be confusing a consensus studies with an attribution studies. Something that struck me was that if you applied something similar to Brandon’s logic to gravity, you would probably conclude that most reject the standard Newtonian gravity. This is because the only papers, at the moment, that are directly considering the actual form of gravity (rather than simply utilising it in some related research) are considered modified forms of gravity so as to see if there is an alternative to Dark Matter. However, most who work in the field probably regard these studies as unlikely to turn out to be correct, but still regard them as interesting possibilities.

  4. Brandon S? (@Corpus_no_Logos) Says:

    Bart, I’m afraid your claim my approach would find a low consensus for evolution doesn’t make sense. You say it is true:

    because most articles don’t state anything about endorsing evolution in the abstract

    But this is a reference to papers taking no position on a matter. There are about 8,000 such papers for the Cook et al study, and I have never said a word suggesting they do anything to disprove or weaken the Cook et al results. In fact, I have consistently told people who made such claims they are wrong.

    Additionally, the comparison I make is not one of my own invention. I explicitly constructed the comparison in the exact form proposed by Dana Nuccitelli when he created the categories for the rating system.

    You seem to have completely misunderstood what I’ve said, despite having referenced a previous discussion where I never said anything like what you discuss now. It’s a little weird.

  5. thomaswfuller2 Says:

    Well. I have stated my opinions about several of these studies with which I am acquainted.

    I believe Anderegg, Prall et al to be fatally flawed in almost every aspect of the research study. I feel the same about Oreskes’ Ivory Tower and Cook’s exercise in abstract rating.

    You did a very good survey, but I don’t agree with your emphasis on analysis. I thought Bray, Storch et al 2008 (which replicated your headline finding in advance, as it were) was also good.

    That there is a ‘consensus’ on consensus in this case means that the people who did the studies stand by them. Wotta surprise. Cook likes Cook–who knew?

    Finally, you show a rising curve and label that ‘expertise.’ It does not. It may measure tenure, other measurements of length of experience, number of publications. It does not measure expertise and you know it.

  6. ...and Then There's Physics Says:

    Tom,

    You did a very good survey, but I don’t agree with your emphasis on analysis.

    Then why publish something? If Richard can do it, anyone can.

    It does not measure expertise and you know it.

    It almost sounds as if you’re accusing Bart of lying. Also, your claim about expertise is a little bizarre. It may be true that experience, number of publications, etc, are not a perfect indicator of expertise, and it may be that someone who doesn’t satisfy these conditions does have relevant expertise. However, they are clearly indicators of expertise.

  7. ...and Then There's Physics Says:

    “Then why not publish something?”

  8. Brandon S? (@Corpus_no_Logos) Says:

    It’s been pointed out to me I should be using the value of ~45% instead of 1.6% to be in line with the approach to this comparison given by Dana Nuccitelli. I hadn’t really given it much thought as whether the result should be 1.6% or 45% doesn’t affect the point I was trying to make – that when the Cook et al study’s results are examined properly, they don’t show anywhere near a 97% consensus on the idea humans are the main cause of global warming.

    So yeah, the 1.6% number was probably the wrong one to give, but the point doesn’t change.

  9. thomaswfuller2 Says:

    ATTP, Bart knows what I’m referring to–an earlier thread where we went back and forth on this subject. Your attempt to stir up trouble is typical of you. Not welcome, just typical.

    And I don’t object to Bart’s analysis–if he wants to focus on his proxy for expertise, that’s his right. What I object to is burying the headline finding. But he and I have exchanged views on that repeatedly. We both have a clear idea of each other’s’ opinion and managed to arrive at it without your assistance. You may resume stalking Anthony Watts, if you need an adrenaline rush.

  10. ...and Then There's Physics Says:

    Tom,
    I’m not trying to stir up touble and I remember that thread. I’m making a serious point. The data is avaiable. You are free to publish something using that data and drawing a different conclusion to the conclusion drawn by the original authors.

  11. thomaswfuller2 Says:

    As for more info/context/reflections, here is my take:

    We Need A Census On Consensus

  12. Willard Says:

    > I’m afraid your claim my approach would find a low consensus for evolution doesn’t make sense.

    Chewbacca roars again:

    http://neverendingaudit.tumblr.com/tagged/chewbacca

  13. Willard Says:

    > Bart knows what I’m referring to […]

    There are alternative explanations to the failure of providing citations than that:

    I regret the error.

    The Mistake I Made When Criticizing Anderegg, Prall et al PNAS 2010

    Richard Tol misrepresents consensus studies in order to falsely paint John Cook’s 97% as an outlier

    No mention of that regret in “I believe Anderegg, Prall et al to be fatally flawed in almost every aspect of the research study.”

  14. Willard Says:

    > I hadn’t really given it much thought as whether the result should be 1.6% or 45% doesn’t affect the point I was trying to make – that when the Cook et al study’s results are examined properly, they don’t show anywhere near a 97% consensus on the idea humans are the main cause of global warming.

    As long as the results don’t show anywhere near a 97% consensus, they’re examined properly, no doubt.

  15. KR Says:

    Brandon, you continue to conflate direct attribution studies with climate papers that agree with the consensus. That’s a very basic error, one rather damning to your arguments.

  16. Brandon S? (@Corpus_no_Logos) Says:

    KR, I’ll repeat myself. The comparison I’m making is one planned by Dana Nuccitelli when he created the rating categories used in this paper. As he explained, he planned to look at the strength of the “consensus” on two issues:

    1) That man is contributing to global warming.
    2) That man is causing most of observed warming.

    Using his stated design for examining these two issues, one gets an incredibly high “consensus” value for point 1 but a less than 50% “consensus” value for point 2. The authors hid this fact by compressing categories 1-3 and categories 5-7 even though doing so does not result in any coherent or clear definition of the “consensus” position their results were for.

    All it would take to show me wrong is to provide a clear definition of the position Cook et al (2013) found a 97% “consensus” for. You won’t be able to. The authors themselves acknowledged they didn’t have one.

  17. ...and Then There's Physics Says:

    Given that Brandon S has explicitly accused the authors of the new consensus paper of being liars, I think Bart is being very generous in letting Brandon S comment here. I will simply point out that – IMO – Brandon’s accusations mean that his concerns should be ignored; we shouldn’t even thank him for them.

  18. Willard Says:

    > Using his stated design for examining these two issues, one gets an incredibly high “consensus” value for point 1 but a less than 50% “consensus” value for point 2.

    I’d replace “using his stated design” with “committing the same error” underlined three years ago, for instance here:

    I think the different categories are meant as a nominal scale, i.e. the fact that two different research abstracts, one estimating the human contribution to be 55 and the other 45% arrive in categories 7 and 1 resp is not necessarily a problem. It is a direct consequence of the description of the categories.

    This also means that category one is not necessarily a stronger endorsement than categories 2 or 3. It is clearer (since it is explicitly quantified), but that’s about it.

    In the survey of scientists that me and others did last year we used ordinal and interval scales, so more detail about the respondent’s opinion is gained. But that’s because we specifically asked. In the Cook et al survey, the data is the abstracts of a good part of the scientific literature. That is more directly related to scientific knowledge, but offers less detail (cf asking scientists what they think).

    Btw, the AR4 statement is written in terms of the anthropogenic GHG contribution; this is likely much higher than the net anthropogenic contribution (because of aerosol cooling masking part of the greenhouse warming). I’ll be beating that drum in the hopefully not too distant future.

    Consensus: Behind the numbers

    Audits never end.

  19. Polly Says:

    That’s a pretty solid consensus … we need to get past this debate and start taking some solid action of global warming.

Leave a comment