Nature Communications published a paper last week that argues that the media pay undue attention to people who do not worry much about climate change. The journal and authors are now in all sorts of trouble because they identified those people, smeared their name, and released personal data. The journal editor confirmed that she did not check whether the paper had IRB approval.**

UC Merced confirms that the paper had no such approval*** and argues that it did not need such approval because no data were collected on political views or academic qualifications.**** Yet, paper and press release refer to views and qualification, the paper obliquely as "political origins" and "non-scientific experts", the press release explicitly with its "false authority" and "lack [of] scientific training". If no data were collected on political views and academic qualifications, then how do the authors support these remarks?

Regardless of the above, the paper is bad. At its core, it compares a sample (A) of researchers who are well published and cited in climate research to another sample (B) who are skeptical of climate policy and prominent in the media. They conclude that sample (A) is better published and cited in climate research while sample (B) attracts more media attention.

It should be trivial to support a trivial hypothesis, but the authors managed to mess it up. Samples (A) and (B) overlapped and people were removed from sample (A). Rob and Ray Bradley were mixed up. Barbara and Harold Betts, realtors praising the climate of California, were seemingly mistaken for Richard Betts, a climate modeller.* Or maybe its Kelly Betts, a photographer for the local newspaper.* Judith Curry may have been mistaken for a dish of Indian origin.* Media attention for Marc Morano was mostly from Morano's blog. (A secondary conclusion in the paper is that non-conventional media pay more attention to people like Morano.) And the paper ignores those who, without academic credentials, argue for climate policy in media, such as Al Gore, Leonardo di Caprio and Greta Thunberg.

How did this paper get published? The authors are trained as natural scientists and, moonlighting in the social sciences, may not have been aware of the rules that apply to working with human subjects. For two years, the authors worked with human subjects and never paused to wonder about the ethics or consult with a social scientist. You cannot just go around and identify someone with a "lack [of] scientific training" or as a member of a "political movement" -- not if you collected data to prove your point and certainly not, as seems to have happened here, without such data.

The editors did not stop them either, nor did the referees.

The referees did not spot the basic flaws in research design and data collection, errors that were very quickly found post publication. The paper's acknowledgements refer to anonymous referees and James Painter, whose own research does not go beyond basic descriptive statistics.

Over the last few years, we have seen published a number of papers on the science-media interface that are very bad, so bad that the idea should have been killed over the first coffee. Those papers were challenged but never corrected or retracted. Editors therefore now have a pool of referees who do not know the first thing about research ethics or experimental design.

If the Nature Communications paper stands, Petersen, Vincent and Westerling will be asked to review similar papers in the future.


*Ruth Dixon argues that all references to the Laguna Beach Independent were removed in an intermediate step not described in the paper.
** De Raneiri, personal communication, 20 August 2019.
*** Eric Kalmin, personal communication, 23 August 2019.
**** Luanna Putney, personal communication, 28 August 2019.

Elsevier Weekblad plaatste een interview met ondergetekende op 23 Maart 2019. Het gesprek ging over van alles en nog wat. Op de titelpagina roep ik dat een "CO2-heffing [op] bedrijven het domste [is] wat je kunt doen". Dat was voor mij een bijzaak, maar de redactie gooit natuurlijk graag olie op het vuur.

Een CO2-heffing in een lidstaat van de EU op uitstoot dat al onder het Europese systeem van verhandelbare emissierechten (EU ETS) valt, verhoogt de kosten van klimaatbeleid zonder de uitstoot verder te beperken. De grenskosten zijn oneindig hoog dus dit is echt het domste wat je kunt doen. Dit is het zogenaamde waterbedeffect: De emissiedoelstelling is Europa-breed, en meer doen in een land betekent minder doen in een ander land.

Stephane Alonso van het NRC viel daar over. Milieuactivisten roepen graag dat ze in naam van de wetenschap handelen, maar hebben vaak weinig kaas gegeten van de klimaatmaterie. Alonso bestempelde mijn uitspraak als "grotendeels onwaar". Hij lijkt niet de relevante regelgeving bestudeerd te hebben, hij schijnt de literatuur niet geraadpleegd te hebben, hij heeft mij niet gesproken, en hij lijkt ook niet met de verantwoordelijke ambtenaren in Brussel en Den Haag gesproken te hebben. Hij heeft twee mensen gesproken die wel wat verstand van zaken hebben, maar laat die eigenlijk nauwelijks aan het woord.

Er is een overschot aan emissierechten in het EU ETS. Op de korte termijn treedt het waterbedeffect dus niet op, maar aangezien uitstootrechten voor altijd geldig zijn, is het waterbedeffect op de lange termijn onverkort geldig -- al kun je stellen dat de verschuiving meer door de tijd dan door de ruimte is. De EU heeft met aanvullend beleid de overschotten teruggedrongen, maar door het waterbedeffect op de korte termijn versterkt wordt -- precies het tegenovergestelde van wat Alonso beweert.

Het aanvullend beleid van de EU is onafhankelijk van flankerend beleid door de lidstaten, zodat het lange-termijn waterbedeffect gelijk blijft.

De reactie van het NRC was veelzeggend. Pas na lang aandringen erkenden ze mijn bestaan. Alonso schreef een hautaine email, waarin hij onderstreepte dat Pieter Boot en Sander de Bruyn* toch echt wereldvermaarde experts zijn, en ook zijn eigen expertise benadrukte: Hij is een Latijns-Amerikanist, die direct na zijn studie journalist werd.

Na herhaaldelijk aandringen heeft het NRC uiteindelijk een brief geplaatst. Mijn aanname was dat er fatsoenlijke en vakkundige mensen bij het NRC werken, maar die mening heb ik toch wel bijgesteld.



*Sander en ik werkten tegelijkertijd aan onze proefschriften in de economie aan de Vrije Universiteit.

Research used to be open access. To read a learned paper, you went to a university library. Academics had privileged access because that library was near their offices. Papers have now moved online, often behind a paywall. Search has much improved and faculty (or their research assistants) no longer have to walk to the library. Others are not so lucky. This is wrong in principle, but the proposed solution, Plan S (Coalition 2018, Measey 2018, Thornton 2018, Rabesandratana 2019), is wrongheaded.

Plan S, supported by research agencies across Europe, demands that all research funded by public grants should be published in open access journals. Anyone should be able to read any paper.

The vast majority of people outside academia never read a scientific journal. They can be given free access to, say, six papers per publisher per year. Few would take this offer, but the political demand is satisfied: Taxpayers can read the research they paid for. Newspapers and magazines have successfully implemented this freemium model (Kumar 2014).

The costs of publication need to be covered somehow. Open access therefore means that subscription fees are replaced by submission or publication fees. Submission fees are more lucrative for publishers as most academics overestimate their chances of getting accepted. Publication fees incentivize editors to accept subpar work.

A shift from payment-for-reading to payment-for-publishing is fine in principle although many universities will struggle with shifting budget from the library to the departments. Poor universities now have limited access to research, but in the future may have to ration publications. Emeriti have library privileges (at zero incremental cost) but universities may be less keen to pay their publication fees.

Even if all new research is open access, universities will need to pay subscription fees for older journal volumes. Subscription fees are typically paid for collections rather than individual journals. Publishers may extend those contracts to include block waivers for publication fees. If so, Plan S would not change much. Universities will continue to hand over large sums of money to publishers. The general public will still not read my latest paper.

Plan S does not address the key issue. Publishers have a monopoly on titles. Professors must read their journals to keep up to date and have no choice but to pay the asking price. Academic publishers make excessive profits, employ excessive numbers of people, and pay excessive wages. Innovation is slow. In short, publishers behave like textbook monopolists (Mankiw 2014).

Open access does not solve this. Researchers must publish in the best journals for promotion, grants, and prizes. Plan S replaces the monopoly of subscription fees by a monopsony of publication or submission fees. The best journals can and will charge excessive fees.

Monopolies and monopsonies are best controlled by new entrants, but it takes so long to build the reputation of a journal that this is not a realistic option. The next-best alternative is regulation. Road, rail, power grid, and water treatment are all natural monopolies. When in private ownership, regulation is and should be tight (Mankiw 2014). Academic publishers are like utilities. They provide an essential service and face no competition. Publishers should therefore be treated like utilities.

Utilities are regulated differently in different countries. The simplest solution is price regulation (Laffont and Tirole 1994): free access for the first six papers, and a flat, low fee for the rest. Aggressive price regulation may bankrupt publishers. Regulators may therefore instead put a cap on the return on capital, and perhaps regulate costs as well, so that prices are lowered rather than white elephants build (Liston 1993).

The details of regulating a monopoly need careful study. But we first need to recognize the reality. Privately owned companies are sponging vast amounts of money from the largely public higher education sector. Plan S takes a principled stand on a minor problem but ignores a bigger one.



Coalition, S. 2018. Plan S: Making full and immediate Open Access a reality. Science Europe.

Kumar, V. 2014. Making 'freemium' work. Harvard Business Review (May 2014).

Laffont, J. J., and J. Tirole. 1994. Access pricing and competition. European Economic Review 38 (9):1673-1710.

Liston, C. 1993. Price-cap versus rate-of-return regulation. Journal of Regulatory Economics 5 (1):25-48.

Mankiw, N. G. 2014. Principles of Microeconomics. Nashville: Southwestern College Publishing.

Measey, J. 2018. Europe's plan S could raise everyone else's publication paywall. Nature 562 (7728):494.

Rabesandratana, T. 2019. The world debates open-access mandates. Science 363 (6422):11-121.

Thornton, J. 2018. Transition to immediate open access publishing under Plan S will be smooth, promise backers. British Medical Journal 363:k5019.

Dear Professor van Dijck,

In November 2017, Professor Jeffrey Harvey of the Netherlands Institute of Ecology, a KNAW institute, was the lead author on a paper published in the journal BioScience.

I requested the data behind the paper, and was pointed to the data archive. Unfortunately, the data released was incomplete.

The paper classifies blogs as “yellow” or “blue”. This classification is hardwired into the R code used to analyse the data. The paper only vaguely describes the procedure for classification: The content of the blogs is compared to the reports of the Intergovernmental Panel on Climate Change (IPCC). Unfortunately, the data released only contain the home page of the blogs investigated, rather than the specific blog posts examined. Some of these blogs consists of hundreds if not thousands of posts. Similarly, the IPCC has published many reports running to thousands of pages, with positions that change over time and chapters that contradict one another. Agreement or disagreement with the IPCC is therefore a meaningless concept, unless chapter and verse are specified.

I have repeatedly asked Professor Harvey and his co-authors to be more explicit, but to no avail. Specifically, I have asked, in vain, for the list of blogposts examined, for the IPCC statements assessed, and for the score per post and statement.

This may appear to be a detail, but it is a key contribution of the paper. Professor Harvey argues that a blog’s position on the IPCC predicts its position on polar bears. However, not knowing how the IPCC position is derived, it cannot be excluded that Harvey fitted the data to match the conclusion.

In the same paper, Professor Harvey and colleagues analyse the contents of a list of published papers. The released data specify the list of papers, but replicating their query to the Web of Science obtains a different list of papers. I have asked Professor Harvey, in vain, to specify the criteria by which certain papers were excluded; and to specify the query through which papers were added.

This point may seem mundane, but it is the other main conclusion of the paper. Professor Harvey claims that blue blogs agree with “the literature”, and yellow blogs do not. Unfortunately, as Professor Harvey will not explain how “the literature” was delineated, the validity of this conclusion is unknown.

As indicated above, I have repeatedly asked Professor Harvey and his co-authors to release the full dataset. They have refused to do, suggesting that they have something to hide.

I have contacted Mr Melle de Vries, the head of information policy at the KNAW. KNAW data policy is clear: All data should be released, and exceptions specified. My experience with Mr de Vries was frustrating. He kept blandly repeating that all data had been released, which is not true as described above and repeatedly explained to Mr de Vries.

I apologize to have to bring this to your attention. KNAW data policy sets a good example: Scientific data should be open to outside scrutiny. Unfortunately, KNAW researchers do not follow this policy, and KNAW officers do not enforce it.

I would be grateful for your intervention in this specific case, and for your efforts to make sure that KNAW data policy is maintained.

I look forward to your reply.


Yours sincerely,



Richard Tol

In their eagerness to discredit a colleague[1] Harvey et al. (2017) got ahead of themselves. The write-up shows signs of haste – typographical errors (“principle component analysis”, “refereces cited”) and nonsensical statements (“95% normal probability”) escaped the attention of the 14 authors, 3 referees and editor – but so does the analysis. The paper does three things: It creates a database, it classifies subjects, and it conducts a principal component analysis. Details have not been shared on the database construction or the classification (Lewandowsky and Bishop 2016), so I focus on the principal component analysis.

Principal component analysis (PCA) aims to reduce the dimension of a dataset by a linear transformation of its variables into orthogonal components and limiting the attention to those components that are principal in explaining the variation in the variables. Harvey et al. (2017) reduce seven variables to two. One variable denotes citation of Susan Crockford. This is recorded as a binary variable, that is, no distinction is made between mentioning her work in passing, criticizing her work, and uncritically adopting her position. The remaining six variables denote agreement with the following statements:

1. sea-ice extent is on average declining rapidly in the Arctic;

2. sea-ice extent is decreasing only marginally, is not decreasing significantly, or is currently recovering in the Arctic;

3. changes in sea-ice extent in the Arctic are due to natural variability, and it is impossible to predict future conditions;

4. polar bears are threatened with extinction by present and future anthropogenic global warming;

5. polar bears are not threatened with extinction by present and future anthropogenic global warming; and

6. polar bears will adapt to any future changes in Arctic ice extent whether because of anthropogenic global warming or natural variability.

Agreement is measured on a binary scale, even though nuances are clearly possible. Taking the first statement, “extent” could refer to sea-ice area or volume, both of which vary over space and seasons so that “average” obtains different meanings, while “rapidly” could mean different things to different people. Similar objections can be raised against the other five statements.

The data released by Harvey et al. (2017) contains only zeroes and ones, suggesting that either agreement was recorded by a single coder or, less plausibly, that all coders agreed on all 6 statements by all 182 subjects.

Intriguingly, there are four subjects who appear to argue that sea-ice is neither shrinking nor stable or growing. One subject seems to argue that polar bears are both threatened by climate change and not threatened, and another is recorded as arguing that polar bears are neither threatened nor not threatened. This corroborates the above assertion that statements were coded by a single coder.

Statements 1 and 2, and 4 and 5 are mutually exclusive, and statements 3 and 6 are close to statements 2 and 4, respectively. A PCA is redundant in a case like this. The analysts artificially inflated the dimensionality of the data, before using PCA to reduce it again.

Figure 2 in Harvey et al. (2017) plays another statistical sleight of hand. The figure shows many observations taking many different positions. Seven binary observations can take at most 128 positions, rather than the 182 suggested in Figure 2. In fact, there are only 19 different positions in the underlying data. The jitter applied by Harvey et al. (2017) suggests that the “majority view papers” all take a slightly different position on sea-ice and polar bears – as you would expect had you not known about the binary coding. Actually, the 86 papers fully agree with each other, and with 27 of the “science-based blogs”. Figure 1 shows a more faithful depiction, with the first and second principal component on the axes and the size of the circles reflecting the number of observations.


Figure 2 in Harvey et al. (2017) shows that different classes of respondent[2] differ strongly on the first principal component, but that there are no significant difference on the second principal component. Table 1 confirms this.

Table 1. The average and standard deviation by observation class for the first and second principal component with seven or six variables per observation.

	P	majority	science	controversial	denier
N		86	45	6	45
PC1	7	-1.50	-1.25	1.40	3.92
		(0.16)	(0.41)	(1.39)	(0.91)
PC1	6	-1.40	-1.33	1.12	3.85
		(0.15)	(0.31)	(1.38)	(0.90)
PC2	7	-0.26	0.51	1.30	-0.18
		(0.03)	(1.04)	(1.06)	(1.20)
PC2	6	-0.05	-0.04	1.39	-0.05
		(0.01)	0.11)	(0.87)	(0.97)

The second principal component largely reflects statement 6, whether polar bears can adapt to future climate change. Polar bears appear to have survived the onset of two interglacials and one ice age (Lindqvist et al. 2010). Unfortunately, 114 of the 182 subjects do not take a position on this. Harvey et al. (2017) replaced these missing observations with zeroes (after standardization). Omitting this column makes the first principal component more important (explained variance rises from 80% to 89%) and the second principal component less important (explained variance falls from 11% to 5%). This does not affect the qualitative results: The first principal component explains the differences between the types of observations. The second principal component does not have discriminatory power. See Table 1.

Harvey et al. (2017) thus really show that there are people who worry about sea-ice and polar bears, and those who do not and cite Dr Crockford.

But Harvey et al. (2017) do not just show that there are two camps. They take sides. Unfortunately, they count noses and argue from authority, rather than assess the strength of the evidence. It is well-known that like-minded blogs often copy or paraphrase material from one another. Similarly, academic papers often repeat a salient conclusion from previous research. Counting noses is a poor method.

The argument from authority is weakened by examining the 92 learned papers. Of the 86 “majority view” papers, 39 were authored by Steven Amstrup, Rascha Nuijten or Ian Stirling, who are among the alii in Harvey et al. (2017). Another 13 were authored by Andrew Derocher, a frequent (n=10) co-author of Amstrup.

The paper does not specify how these 92 papers were identified, beyond a “broad keyword search” on the “ISI Web of Science”. The Web of Science returns 179 articles for a query on “polar bear” and “sea ice”. No information is given how the larger sample of relevant papers was reduced to the smaller one used by Harvey et al. (2017). Comparing the relative contribution of the ten most prolific authors according to the Web of Science to their relative presence in Harvey’s sample reveals that the latter is not representative of the former (chi^2(9)=17.4, p=0.04). Research by co-authors Amstrup and Stirling is overrepresented in Harvey et al. (2017), and work by Jon Aars and Oystein Wiig underrepresented.[3] The sample used by Harvey et al. (2017) appears to be a sample of convenience, and unrepresentative.

In sum, Harvey et al. (2017) play a statistical game of smoke and mirrors. They validate their data, collected by an unclear process, by comparing it to data of unknown provenance. They artificially inflate the dimensionality of their data only to reduce that dimensionality using a principal component analysis. They pretend their results are two dimensional where there is only one dimension. They suggest that there are many nuanced positions where there are only a few stark ones – at least, in their data. On a topic as complex as this, there are of course many nuanced positions; the jitter applied conceals the poor quality of Harvey’s data. They show that these is disagreement on the vulnerability of polar bears to climate change, but offer no new evidence who is right or wrong – apart from a fallacious argument from authority, with a "majority view" taken from an unrepresentative sample. Once the substandard statistical application to poor data is removed, what remains is a not-so-veiled attempt at a colleague’s reputation.