1. It is ironic that the left-of-centre newspaper the Guardian has withdrawn from press regulation. They appear to think that companies are in the best position to judge their own behaviour. Only people who are rich enough to afford a lawsuit are protected against any falsehoods that this newspaper may decide to print.

    The Guardian has also long abandoned the journalistic principle that both sides of a story need to be heard.

    In its latest installment of a series of hatchet jobs, the Guardian published an article by Mr Robert ET Ward BSc, Lord Stern's PR man. The central claim of the article is, simply, false.

    The figure below is as it appears in the final, published version of Chapter 10 of the Contribution of Working Group II to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change.
    This figure is slightly different than the one that appears in the final draft, but both show that the initial impacts of climate change may be beneficial.

    In the final version, we replaced the previous, vague "may be beneficial" (which refers to a regression curve that is in the literature but not reproduced in the IPCC report) with the more precise "17 out of 20 are negative" (from which the intelligent reader would deduce that the remaining 3 are non-negative).

    The claim still stands.

    The Guardian also refers to "faulty data". Presumably, Mr Ward refers to his single contribution, noting a minor error: The draft chapter reported the estimate of Roson and van der Mensbrugghe as an economic loss of 4.6% for a global warming of 4.9K, whereas in fact it should be for 4.8K.

    There were other errors in the draft chapter, of course, but none of them materially affects the qualitative results. We still find that the initial impacts of climate change may be beneficial; and that the impacts of climate change are small relative to such things as the Euro-crisis.

    A striking conclusion, that unfortunately did not make it into the IPCC because the paper (open access pre-print) appeared after the deadline, is that the impacts of climate change do not deviate from zero, in a statistically significant way, until about 3K warming.

    Mr Ward also claims to have been a reviewer of the IPCC WG2 AR5. There must have been an error, because a search does not return any of his comments, while the list of reviewers omits Mr Ward.

    UPDATE: Spiegel has a more balanced story, noting that the bottom-line conclusion hasn't change, but highlighting the change of tone.

    Disturbingly, it cites Chris Field to say "When these numerical errors were corrected, the statistical relationship between warming and economic impacts had a different shape." Field, I hope, speaks in his personal capacity rather than on behalf of the IPCC. If not, he has violated IPCC protocol.

    Field is wrong. The numerical errors did not affect the shape of said relationship. The new estimates do. The new estimates are the red diamonds in the figure above.

    Here is the story. The old data (the blue circles) roughly fit a parabola: first up, then down, and ever faster down.

    The new data do not fit a parabola: The initial impacts are positive, but the progression to negative impacts is linear rather than quadratic.

    If you fit a parabola to this data, you will find that the mildly negative estimate at 5.5K dominates the positive estimate at 1.0K and the sharply negative estimate at 3.2K. The parabola become essentially a straight line through the origin and the right-most observation.

    I think the appropriate conclusion from this is to fit a bi-linear relationship to the data, rather than stick with a parabolic one. This was not yet in the peer-reviewed literature when the window for AR5 closed (it is now: paper open access pre-print), so we decided to just show the data.

    Chris Field instead cites an analysis that is just inappropriate.

    UPDATE2 In a piece at WattsUpWithThat, Brandon Shollenberger get their knickers in a twist over procedural errors. Brandon is great for finding stuff on computers that no one else can, but he sometimes falls short when humans are involved.

    Chapter 10 of IPCC WG2 AR5 was changed between the acceptance of the draft and the publication of the report. Contrary to what some people claim, that is perfectly in line with IPCC procedures.

    There are three routes through which changes can be made.

    First, there is trickle-back. Governments write the Summary for Policy Makers (SPM). Trickle-back ensures that the Summary is consistent with the report it supposedly summarizes. This is the wrong way around, of course. In the case of Chapter 10, the SPM cites summary statistics that were not in the chapter. They are now.

    Second, there are errata.

    Third, if errors are found between acceptance and publication (a six month period in this case), the errors are corrected and documented. That is what happened here. Unfortunately, the documentation has yet to be uploaded.
    1

    View comments

  2. Abstract
    While earlier research had exposed severe problems with the data quality and analysis of the 97% consensus paper (Cook et al, 2013, Environmental Research Letters), this note finds the authors have contradicted themselves and that the data gathering invalidates all results.


    The 97% consensus paper (Cook et al., 2013) was hailed as the best ERL paper of 2013 (Cook, 2014). Downloaded more than 228,000 times (ERL, 2013) and with an Altmetric score of almost 1500 (Altmetric, 2013), it definitely was the most visible.

    The core result of Cook et al. is unremarkable (Montford, 2013): They find that the academic literature says that human activity is one of causes of the observed global climate change.

    Some have claimed that Cook et al. found a consensus on the dangers of climate change (Kammen, 2013) or on the need for climate policy (Davey, 2013). They investigated neither. Even some of the authors of the paper misrepresent its findings (Nuccitelli, 2014, Friedman, 2014, Henderson, 2014).

    Cook et al. took a sample of the academic literature and rated its contents. The raters were recruited through a partisan website (Cook et al., 2013) and frequently communicated with each other (Duarte, 2014). Their sample is not representative of the literature (Tol, 2014a). The sample was padded with large numbers of irrelevant papers (Tol, 2014a). For example, a paper on photovoltaics in Kenya (Acker and Kammen, 1996) was taken as evidence that climate change is caused by humans as was a paper on the coverage of climate change on US TV (Boykoff, 2008). Three-quarters of the "endorsing" abstracts offer no evidence either way (Tol, 2014a). Their attempt to validate the data failed (Tol, 2014a). An attempt to replicate part of the data failed too (Legates et al., 2013). The data show inexplicable patterns (Tol, 2014a) while the consensus rate suffers from confirmation bias (Cook et al., 2014a, Tol, 2014b).

    The problems do not stop there. It appears - no survey protocol was released - that the research team (1) gathered data (19 February to 15 April 2014), (2) studied the results, (3) gathered more data (11 May to 1 June 2012), (4) studied the results again, (5) changed the classification system, and (6) gathered more data and reinterpreted the rest. The results from step (1) and (3) are different (raw sample chi-sq(df=6)=255, p<0.001; matched sample chi-sq(df=6)=393, p<0.001). The results from step (3) and (6) are different too: The dissensus rate changes by one-half (Tol, 2014a).

    Alterations of the sampling strategy or survey design during the course of data gathering, and revealing preliminary results to the data gatherers are typically frowned upon because of the risks of (inadvertently) skewing the results. In this case, the authors had a sharp prior on the results before the first data were collected (Andrew, 2013, Montford, 2014).

    Cook et al. were slow to release part of their raw data, hindering attempts to check their analysis in contravention of journal policy (IOP, 2013). Part of their data were never released.

    Time stamps were held back because they "were not collected" (Cook et al., 2014b) although they were referred to in an earlier exchange (Cook, 2013). Time stamps were part of an unofficial data release (Shollenberger, 2014). This allowed the above reconstruction of the data gathering process.
    Time stamps corroborate my earlier hypothesis (Tol, 2014a) that some of the strange patterns in the data are due to rater fatigue. This was contradicted by (Cook et al., 2014b) even though John Cook had earlier noted that "[e]veryone's suffering rater fatigue" (Cook, 2012). Indeed, one rater read and classified 765 abstracts in the course of 72 hours. Rater fatigue implies unreliable data.

    Rater IDs were also held back, referring to a confidentiality clause in an ethics approval (UQ, 2014) that does not cover this part of the data gathering (UQ, 2012). Rater IDs were unofficially released later. The officially released data show that different people rated the same paper differently in one-third of all cases (Cook et al., 2013). The unofficial data reveal more. The survey started with a team of 24 raters, but numbers fell quickly. There is a difference between the raters that stayed and those that left (t=-6.51, p<0.001). The group of raters also changed its composition after the data were inspected (chi-sq(df=23)=7265, p<0.001).

    In sum, one of the most visible climate papers of recent years is not sound. Whereas previous critique could be interpreted as a lack of competence (Tol, 2014a), the later data release suggests that Cook et al., perhaps inadvertently, worked towards a given answer. This reflects badly on the authors, referees, editors and publisher. It also weakens the activists and politicians who cite Cook et al. in support of their position.

    The data and statistical tests underlying this note can be found at http://www.sussex.ac.uk/Users/rt220/CookERL.xlsx Jose Duarte and Andrew Montford had excellent comments on an earlier version.


    REFERENCES
    ACKER, R. H. & KAMMEN, D. M. 1996. The quiet (energy) revolution: Analysing the dissemination of photovoltaic power systems in Kenya. Energy Policy, 24, 81-111.
    ALTMETRIC. 2013. Score in context [Online]. Available: http://www.altmetric.com/details.php?citation_id=1478869&src=bookmarklet [Accessed 5/9/2014.
    ANDREW. 2013. Cook's 97% consensus study game plan revealed. Popular Technology [Online]. Available from: http://www.populartechnology.net/2013/06/cooks-97-consensus-study-game-plan.html.
    BOYKOFF, M. 2008. Lost in translation? United States television news coverage of anthropogenic climate change, 1995-2004. Climatic Change, 86, 1-11.
    COOK, J. 2012. Fatigue. Available from: http://rankexploits.com/musings/2014/sks-tcp-front/#comment-130926.
    COOK, J. 2013. Query re request for Cook et al. data [Online]. University of Queensland. Available: http://www.sussex.ac.uk/Users/rt220/Cook31July.png
    COOK, J. 2014. Skeptical Science consensus paper voted ERL's best article of 2013. SkepticalScience [Online]. Available from: http://skepticalscience.com/SkS-consensus-paper-ERL-best-article-2013.html.
    COOK, J., NUCCITELLI, D., GREEN, S. A., RICHARDSON, M., WINKLER, B., PAINTING, R., WAY, R., JACOBS, P. & SKUCE, A. 2013. Quantifying the consensus on anthropogenic global warming in the scientific literature. Environmental Research Letters, 8.
    COOK, J., NUCCITELLI, D., SKUCE, A., JACOBS, P., PAINTING, R., HONEYCUTT, R., GREEN, S. A., LEWANDOWSKY, S., RICHARDSON, M. & WAY, R. G. 2014a. Reply to 'Quantifying the consensus on anthropogenic global warming in the scientific literature: A re-analysis'. Energy Policy, 73, 706-708.
    COOK, J., NUCCITELLI, D., SKUCE, A., WAY, R., JACOBS, P., PAINTING, R., HONEYCUTT, R., GREEN, S. A., LEWANDOWSKY, S. & COULTER, A. 2014b. 24 Critical Errors in Tol (2014) - Reaffirming the 97% consensus on anthropogenic global warming. Brisbane: SkepticalScience, University of Queensland.
    DAVEY, E. 2013. Climate change, acting on the science. Gov.uk [Online]. Available from: https://www.gov.uk/government/speeches/edward-davey-speech-climate-change-acting-on-the-science
    DUARTE, J. 2014. Cooking stove use, housing associations, white males, and the 97%. José Duarte [Online]. Available from: http://www.joseduarte.com/blog/cooking-stove-use-housing-associations-white-males-and-the-97
    ERL. 2013. Metrics [Online]. Institute of Physics. Available: http://iopscience.iop.org/1748-9326/8/2/024024/metrics [Accessed 5/9/2014.
    FRIEDMAN, D. 2014. A climate falsehood you can check for yourself. Ideas [Online]. Available from: http://daviddfriedman.blogspot.co.uk/2014/02/a-climate-falsehood-you-can-check-for.html.
    HENDERSON, D. R. 2014. David Friedman on the 97% Consensus on Global Warming. Library of Economics and Liberty [Online]. Available from: http://econlog.econlib.org/archives/2014/02/david_friedman_14.html.
    IOP. 2013. IOP Ethical Policy for Journals [Online]. Institute of Physics. Available: http://authors.iop.org/atom/help.nsf/0/F18C019D6808524380256F630037B3C2?OpenDocument
    KAMMEN, D. M. 2013. The story of a presidential tweet. The Berkeley Blog [Online]. Available from: http://blogs.berkeley.edu/2013/05/29/the-story-of-a-presidential-tweet/
    LEGATES, D. R., SOON, W., BRIGGS, W. M. & MONCKTON OF BRENCHLEY, C. 2013. Climate Consensus and 'Misinformation': A Rejoinder to Agnotology, Scientific Consensus, and the Teaching and Learning of Climate Change. Science and Education.
    MONTFORD, A. W. 2013. Consensus? What Consensus? London: Global Warming Policy Foundation.
    MONTFORD, A. W. 2014. Fraud, bias and public relations - The 97% consensus and its critics. London: Global Warming Policy Foundation.
    NUCCITELLI, D. 2014. Twitter profile [Online]. Available: https://twitter.com/dana1981 [Accessed 5/9/2014.
    SHOLLENBERGER, B. 2014. TCP Results! Izuru [Online]. Available from: http://www.hi-izuru.org/mirror/
    TOL, R. S. J. 2014a. Quantifying the consensus on anthropogenic global warming in the literature: A re-analysis. Energy Policy, 73, 701-705.
    TOL, R. S. J. 2014b. Quantifying the consensus on anthropogenic global warming in the literature: Rejoinder. Energy Policy, 73, 709.
    UQ. 2012. Notification of approval [Online]. Available: http://www.climateaudit.info/correspondence/foi/queensland/cook%20consensus%20Documents%20released%20under%20RTI.pdf.
    UQ. 2014. UG and climate change research. UQ News [Online]. Available from: http://www.uq.edu.au/news/article/2014/05/uq-and-climate-change-research.

    UPDATE: The comments by an editorial board member are below. I appealed this three times, but to an avail.

    This is an unsolicited Perspective on the Cook et al. paper from 2013 [1] by an author who wishes to reflect on the original study published in ERL.
    This submission follows a solicited Perspective [2] on the Cook et al. paper, which reflected on the study and the broader implications of the study internationally at the time of publication.

    This Perspective also follows another submission to ERL, by the same author, of comments on the Cook et al. study, which were rejected by ERL after peer review. These comments were subsequently published [3, 4] with a response from Cook et al. [5].

    ERL welcomes debate and, despite the fact that we have already published a Perspective on this study, the original paper continues to be highly popular and debated. In theory then, ERL could publish another Perspective, provided it contributes to healthy scientific debate, is original, timely, and advances the literature on this important theme.

    Overall, this current submission shows a different standpoint from the commissioned Perspective published in 2013, and thus could have had the potential to contribute. Unfortunately, in its current form, this piece is not of satisfactory scope or breadth for a Perspective, and is not sufficiently original. It also has problems of unsubstantiated assertion, lack of disinterested reflection, and polemic expression.

    I have the following detailed comments and suggestions for improvement:

    1. Scope: ERL guidelines suggest that a Perspective is a commentary “highlighting the impact and wider environmental implications of research appearing in ERL”. This submission does, briefly, look at the impact of the Cook et al paper internationally (particularly lines 29 to 38 on page 1 and lines 40 to 45 on page 2). Unfortunately, at present, this discussion of impacts forms a minor part of the submission, and the bulk of the discussion is a repeat assertive critique of the methodology of the original study, which is not the scope or purpose of a Perspective in ERL. Generally, there is also a missed opportunity here to discuss the one-year-on implications of the original research (particularly given its profile internationally). There is not even a reference to the previous ERL Perspective [2], which did look at implications of the work in 2013, and which I think could, very usefully, have been the basis for this follow up Perspective.

    2. Breadth: This might have been an interesting perspective had the author put this study in the context of the fairly prolific academic literature on the theme of scientific consensus around anthropogenic climate change (including the ERL commissioned Perspective[2]). In the context of this submission, it is odd that the author does not reflect on other previously published studies, including the original study, (on which Cook et al. base part of their design[6]), and other studies of climate scientists’ viewpoints on anthropogenic climate change. In fact it is remarkable, and would have been an interesting point for reflection, that the results of other studies published in peer reviewed journals in this field, conducted over different periods, and using a variety of methods (ranging from direct interviews of climate scientists to literature reviews of climate papers) all find the same narrow range of 94-98% scientific consensus on anthropogenic climate change [6-9]. In this context, it is not clear why the author focuses so much time and energy on this one study within the overall literature, as it was, essentially, simply a timely, interesting, original study, published by ERL as such, which corroborates results of other studies within this field. A good Perspective would look at the wider implications of this latest study, including how it reflects overall state of knowledge of the theme, and would discuss how this study fitted into the overall policy debate - this is a missed opportunity to do this.

    2. Originality: The author himself points us to three other published papers (two of which are his) discussing the Cook et al. paper[3-5]. The bulk of this submission goes over the same ground in less depth or justification, but with stronger and less considered assertion (Lines 42 to 61 on page 1 and lines 1 to 38 on page 2). The majority of this submission does not add any new information, and it is not clear why the author wishes to re-publish the same ideas contained in his already published methodological analysis of the Cook et al. study. At present then, the bulk of this submission is unoriginal. I would suggest that all this material is removed, as it adds nothing new, and the author would then have space to consider points more relevant to a Perspective (as discussed above).

    4. Tone of Piece: This is the most troubling aspect of this submission. I will begin by stating the obvious: there is an important space in science for debate – openness to doubt and debates about “truth”, are an essential part of science. This debate can be very lively, but is at its most healthy when it is conducted with respect, and with rigorous presentation of evidence to back a position. In the spirit of this, ERL hosts Perspectives to encourage reflection and discussion of the scientific papers that it publishes.

    Overall, the commentary could be made substantially more balanced and contemplative – for example, as proof of “truth” the author cites himself and a series of mostly social media sources, with little reference to the academic literature and with little evidence of neutrality in his selection of “evidence”.

    There is a more unfortunate and confrontational aspect to the tone of this submission when the author makes his final unsubstantiated reflections on the soundness of the original research and assertions about Cook et al.’s scientific conduct (as he does in line 42 on page 2 “Whereas previous critique could be interpreted as a lack of competence (Tol, 2014a), the later data release suggests that Cook et al., perhaps inadvertently, worked towards a given answer”). At it’s most basic, there is simply not sufficient evidence of this assertion of unsoundness: the authors of the original paper made their hypothesis clear, (and most studies would have had the same hypothesis, particularly in a context where previous studies have found overwhelmingly significant positive results looking at the same issue). The original paper also had a detailed section on study design and methodology and a discussion of interpretation issues and other study design possibilities. Cook et al have also been open to release raw data, and to discuss their study methodology in subsequent correspondence. There is nothing to suggest this is “unsound” – on the contrary, most scientists would think it suggests exemplary scientific conduct.

    Finally, it is not quite clear what the author of this submission aims to achieve when, much more problematically, he goes on to suggest that somehow the act of publication of the Cook et al paper by ERL “reflects badly” on the reviewers of the original paper, on ERL editors and on the publishers. What does the author imply with this statement and on what basis? The author has no evidence, and there is no evidence, that ERL and the IOP employed anything but normal state-of-the-art peer-review processes and publishing procedures when dealing with Cook et al.

    Overall, I welcomed this unsolicited Perspective in the interests of a constructive debate around an interesting study published in ERL. However, in its current form, I do not find it to fit ERL guidelines for a Perspective, nor to be original, nor to be of sufficient breadth or disinterested reflection to contribute to the literature, or to knowledge.

    If the author were able to do this, it would be interesting to see a substantially revised version of this perspective with these concerns addressed.

    1. Cook, J., et al., Quantifying the consensus on anthropogenic global warming in the scientific literature. Environmental Research Letters, 2013. 8(2): p. 024024.
    2. Reusswig, F., History and future of the scientific consensus on anthropogenic global warming. Environmental Research Letters, 2013. 8(3): p. 031003.
    3. Tol, R.S.J., Quantifying the consensus on anthropogenic global warming in the literature: A re-analysis. Energy Policy, 2014. 73(0): p. 701-705.
    4. Tol, R.S.J., Quantifying the consensus on anthropogenic global warming in the literature: Rejoinder. Energy Policy, 2014. 73(0): p. 709.
    5. Cook, J., et al., Reply to ‘Quantifying the consensus on anthropogenic global warming in the scientific literature: A re-analysis’. Energy Policy, 2014. 73(0): p. 706-708.
    6. Oreskes, N., Beyond the ivory tower. The scientific consensus on climate change. Science, 2004. 306(5702): p. 1686.
    7. Anderegg, W.R.L., et al., Expert credibility in climate change. Proceedings of the National Academy of Sciences, 2010. 107(27): p. 12107-12109.
    8. Doran, P.T. and M.K. Zimmerman, Examining the Scientific Consensus on Climate Change. Eos, Transactions American Geophysical Union, 2009. 90(3): p. 22-23.
    9. Farnsworth, S.J. and S.R. Lichter, The Structure of Scientific Opinion on Climate Change. International Journal of Public Opinion Research, 2011.
    1

    View comments

Blog roll
Blog roll
Translate
Translate
Blog Archive
About Me
About Me
Subscribe
Subscribe
Loading
Dynamic Views theme. Powered by Blogger. Report Abuse.