The IAC has released their report evaluating the IPCC procedures. They did not investigate the actual contents of the IPCC reports.
From the executive summary:
The Committee found that the IPCC assessment process has been successful overall. However, the world has changed considerably since the creation of the IPCC (…)
The IPCC must continue to adapt to these changing conditions in order to continue serving society well in the future.
Their key recommendations appear sensible and constructive:
The Committee’s main recommendations relate to governance and management, the review process, characterizing and communicating uncertainty, communications, and transparency in the assessment process.
Recommendation: The IPCC should establish an Executive Committee to act on its behalf between Plenary sessions. The membership of the Committee should include the IPCC Chair, the Working Group Co-chairs, the senior member of the Secretariat, and 3 independent members, including some from outside of the climate community. Members would be elected by the Plenary and serve until their successors are in place.
Recommendation: The IPCC should elect an Executive Director to lead the Secretariat and handle day-to-day operations of the organization. The term of this senior scientist should be limited to the timeframe of one assessment.
Recommendation: The IPCC should encourage Review Editors to fully exercise their authority to ensure that reviewers’ comments are adequately considered by the authors and that genuine controversies are adequately reflected in the report.
Recommendation: The IPCC should adopt a more targeted and effective process for responding to reviewer comments. In such a process, Review Editors would prepare a written summary of the most significant issues raised by reviewers shortly after review comments have been received. Authors would be required to provide detailed written responses to the most significant review issues identified by the Review Editors, abbreviated responses to all non-editorial comments, and no written responses to editorial comments.
Recommendation: All Working Groups should use the qualitative level-of-understanding scale in their Summary for Policy Makers and Technical Summary, as suggested in IPCC’s uncertainty guidance for the Fourth Assessment Report. This scale may be supplemented by a quantitative probability scale, if appropriate.
Recommendation: Quantitative probabilities (as in the likelihood scale) should be used to describe the probability of well-defined outcomes only when there is sufficient evidence. Authors should indicate the basis for assigning a probability to an outcome or event (e.g., based on measurement, expert judgment, and/or model runs).
(See e.g Gavin’s comment regarding this point in an in-line response at RC here. It is certainly important to be very clear in explaining what the basis is for assigning a certain level of confidence or probability, a point also made by Judith Curry.)
Recommendation: The IPCC should complete and implement a communications strategy that emphasizes transparency, rapid and thoughtful responses, and relevance to stakeholders, and which includes guidelines about who can speak on behalf of IPCC and how to represent the organization appropriately.
The Committee recommends that the IPCC establish criteria for selecting participants for the scoping meeting, where preliminary decisions about the scope and outline of the assessment reports are made; for selecting the IPCC Chair, the Working Group co-chairs, and other members of the Bureau; and for selecting the authors of the assessment reports. The Committee also recommends that Lead Authors document that they have considered the full range of thoughtful views, even if these views do not appear in the assessment report.
Climate Central offers the following translation “from report-ese into English”:
- The IPCC should create an Executive Committee to run the organization in between major conferences.
- Rather than have the IPCC director serve for two six-year terms, a new director should be appointed for each major assessment report (there have been four so far). Since the IPCC is well into the fifth assessment, it isn’t clear whether Dr. Pachauri will step down (he’s evidently said that any decision will have to wait for the next IPCC meeting, in Korea in October).
- The reviewers who decide what makes it into the final report and what doesn’t should work harder to address comments from authors, and to let dissenting views be reflected more fully in the finished product.
- Statements about certainties and uncertainties about climate science need to be more explicit, need to be based on a more uniform set of criteria, and need to be clearer about how they were calculated.
- The IPCC in general needs to be more open and transparent about how it goes about its business.
- The IPCC needs to improve the way it deals with so-called “grey literature” — that is, non-peer-reviewed reports that contain valuable information, but which haven’t already been subjected to strict scientific scrutiny.
Much of the media still uses this as a coat rack to rehash (often exaggerated) stories about alleged errors or misconduct.