The conservative face of science and the role of consensus

[I realize that this has already been covered at Climate Progress and elsewhere, I am doing this more or less for archival purposes. -TB]

Image courtesy of Flickr user Scott Ableman, used under Creative Commons

The year 2011 started off with something of a surprise- George Will seemingly supporting science! Yes, this George Will. I wasn’t the only one taken aback.

Will’s journalistic colleague Andy Revkin was likewise surprised by this seeming about face from someone who all too readily attacked science when it conflicted with his conservative ideology, writing:

I think it’d make sense to devote at least as many column inches to this vital issue as you’ve expended trying to undercut decades of scientific study pointing to a growing human influence on the climate system.

This summer, Revkin again called upon Will to show how serious Will actually is about supporting science (and again at the end of August) by penning “a fresh column… building on [Will’s] January rebuke of Republican lawmakers seemingly seeking to lead a charge away from federal support for science.” At the time, Revkin pointedly noted that Will was preoccupied with other topics.

Well, it appears Revkin now has Will’s response. GOP Presidential candidate Jon Huntsman recently had the gall to side with the scientific community on the issues of climate change and evolution. Today’s Republican party is infamously unique in its rejection of the scientific reality of man-made global warming. That a top-tier Republican candidate like Jon Huntsman would unabashedly stand with the scientific community was a welcome surprise.

Such apostasy was apparently sufficient to rouse Will’s attention where Revkin’s pleas to stand up for science were not. Will took to the pages of Fred Hiatt’s Washington Post to join his fellow Republicans’ assault on science.

Will sneered:

For Jon Huntsman: You, who preen about having cornered the market on good manners, recently tweeted, “I believe in evolution and trust scientists on global warming. Call me crazy.” Call you sarcastic. In the 1970s, would you have trusted scientists predicting calamity from global cooling?

Gee, Will sure does love recycling!

Setting aside the fallacy of believing that because science got something wrong in the past it follows that it’s incorrect now, Will is actually engaging in revisionist history.

Despite repeated claims by Will and others to the contrary, there was no consensus predicting cooling in the 70s. Rather predictions of warming “even then dominated scientists’ thinking” (Peterson 2008):

Had Huntsman listened to the balance of the scientific evidence in the 1970s, he would be looking pretty good 30 plus years later. Contrast that with Will, who manages to still get what was said then wrong today, even with the benefit of hindsight!

Will continues:

Are scientists a cohort without a sociology — uniquely homogenous and unanimous

I will freely stipulate that true unanimity is seldom achieved on any subject, no matter how well-established scientifically. That being said, on the question of the reality of man-made warming of the climate, it’s pretty darn close. Surveys of the primary literature show virtually no opposition (Oreskes 2004). Survey data also show that 97-98% of scientists with relevant expertise/who are actively publishing in relevant fields likewise support the consensus (Doran 2009, Anderegg 2010).

without factions or interests

On the contrary, climate science is necessarily an interdisciplinary field. And it’s precisely this patchwork, factious nature of the field that makes the aforementioned consensus all the more striking.

That scientists whose life’s work is focused on solar influence on climate are broadly in agreement with those who focus on the ocean’s role, and with those who study climatic changes in the geologic past due to orbital variation, volcanism, or plate tectonics, etc. that anthropogenic warming is driving the present climatic change is quite amazing, especially if one is as cynically-minded as Will. Self-interest (which we will see Will believes is quite the powerful motivator) is poorly served by the various alternative drivers of warming being exonerated by the scientists that study them.

and impervious to peer pressures or the agendas of funding agencies?

This is a rather pathetic appeal to motive. And it fails for much the same reason that the previous comment does. If one were interested in prolonging and maximizing the amount of funding one could receive for one’s own corner of the scientific community, swiftly and virtually unanimously reaching consensus on something is probably the worst possible way to go about it.

But if Will is genuinely interested in how scientific consensus can be reached and trusted, he could always consult an expert on the subject. Naomi Oreskes literally wrote the book on this topic as it concerns the triumph of plate tectonics (Oreskes 2001). For the truly concerned like Will, she’s also written an accessible primer on the consensus on global warming (Oreskes 2007).

Alas, given Will’s track record (e.g. here, here, here, here, here, here, here, here, here, here), the likelihood of him bothering to actually educate himself on the subject appears to be about as slim as Huntsman’s chances for the Republican nomination.

As a parting shot, Will cannot resist twisting the knife in Huntsman over his science-affirming campaign’s poor reception by today’s GOP voters:

Your chief strategist, John Weaver, says the “simple reason” the GOP is “nowhere near being a national governing party” is that “no one wants to be around a bunch of cranks.” … Although you say the country is “crying out” for a “sensible middle ground,” you have campaigned for three months on what you say is that ground and, according to the most recent Gallup poll, your support among Republicans and Republican-leaning independents is 1 percent.

The folly of codifying anti-science beliefs into a technologically-rooted nation’s political platform would seem self-evident, a “no brainer” as it were. Will and his fellow conservative elites would do well to reconsider their present course,  which is a “no brainer” of an altogether different kind.

Image courtesy of Flickr user saucy_pan, used under Creative Commons

References:

  • Anderegg, W., et al. (2010): Expert credibility in climate change. Proceedings of the National Academies of Science (USA), 107, 27, 12107-12109, doi:10.1073/pnas.1003187107.
  • Doran, P.T., and M.K. Zimmerman (2009): Examining the Scientific Consensus on Climate Change, Eos Trans. AGU, 90(3), 22, doi:10.1029/2009EO030002.
  • Oreskes, N., ed. (2001): Plate Tectonics: An Insider’s History of the Modern Theory of the Earth. Boulder: Westview Press, with Homer E. Le Grand.
  • Oreskes, N. (2004): Beyond the Ivory Tower: The Scientific Consensus on Climate Change. Science, 306, 57021686, doi:10.1126/science.1103618.
  • Oreskes, N. (2007): The scientific consensus on climate change: How do we know we’re not wrong? Climate Change: What It Means for Us, Our Children, and Our Grandchildren, 65-99, DiMento and Doughman eds., MIT Press.
  • Peterson, T.C., et al. (2008): The Myth of the 1970s Global Cooling Scientific Consensus. Bulletin of the American Meteorological Society, 89, 9, 1325-1337, doi:10.1175/2008BAMS2370.

[UPDATE: I see Phil Plait was having similar thoughts today.]

About these ads

90 responses to “The conservative face of science and the role of consensus

  1. Will is apparently from Fuckham:
    Fuckem’s Razor and the solution to the climate question

    http://www.scholarsandrogues.com/2011/09/08/fuckems-razor-and-the-solution-to-the-climate-question/

  2. You write, “The folly of codifying anti-science beliefs into a technologically-rooted nation’s political platform…” I haven’t seen any of the candidates proposing codifying anti-science beliefs. What examples are you talking about?

  3. I think Capital Climate was referring to the way deniers learn of littany of skeptic arguments, which are basically urban legends. They then go about dropping these turds in comments sections of news media and mainstream climate science blogs. Show them the error in one argument and they roll out the next three or four in one breath, Invariably, they are complete nonsense, like the drivel that scientifically illiterate GOP politicians repeat smugly as they carry on witch hunts against climate scientists. Yes, this is exactly what has been happening.

  4. Are people being prevented from examining scientific evidence or else duplicitously biased against science by the mass media?

    How could the perpetration of so pervasive a silence with respect to science of human population dynamics, as well as the widely shared consensually validated broadcasts of pseudoscience, ever have been accomplished during the last four decades? What is going on? I cannot help but ask the questions. How did this willful denial of what could be real occur on our watch? How have the ubiquitous broadcasts of false hopes and promises been maintained regarding a benign, soon to occur, and somehow automatic end to human population growth by 2050? Has delusional thinking and ideological idiocy been allowed to rule the world during our lifetime because many too many experts chose to remain mute regarding evidence of human population growth or, even worse, to act as censors of the best available scientific evidence? Are powerful and influential experts colluding with “the powers that be” to deny science?

    Why would leaders and followers in a single generation choose not to speak out loudly, clearly and often in a time when a paradise is being turned into an inferno? Can malignant narcissism, pathological arrogance, extreme foolishness and outrageous avarice of a tiny dishonest and immoral minority of the human family be at least partially responsible for such an intolerable situation? How else can such a thing as a colossal human-driven extinction event occur so fast, before our eyes, with ‘the brightest and best’, “the smartest guys in the room” leading the way? Are self-proclaimed masters of the universe in possession of critical decision-making authority at the top of the global political economy leading all of us down a primrose path, come what may? Are power and greed mongers shouting everywhere “greed is good” and acting on what they have proclaimed to be their ‘inalienable rights’ to perpetrate some unimaginable sort of global ecological wreckage without a word being spoken for 40 years. Perhaps we are witnessing not only the sixth extinction event, the first such event to be precipitated by a species, but also the sight of something unthinkable: silence killing the world.

    Leaders and followers alike in the family of humanity can do better, and I trust we will soon enough awaken in a determinative critical mass of human beings with feet of clay to the need for behavior change. Support in the form of silent consent of the adamant advocacy and relentless pursuit of a morally disengaged and patently unsustainable way of life — one of endless population growth and unbridled economic expansion — simply cannot be tolerated any longer.

    It is never too late to stand up and be counted or to do the right thing, I suppose.

  5. And your first cite is Anderegg et al? Sure, you’re serious.

  6. If you would like to list the differences between your own good self and George Will, I would be happy to counter with a list of similarities.

    Returning to the same subjects, citing the same sources time after time, not listening to those with dissenting views, arriving at a conclusion very early in the information gathering process and setting it in stone…

    Hey, it’s worked for Will–maybe it’ll work for you, too. But citing Anderegg, et al? Tripe.

    • Returning to the same subjects, citing the same sources time after time, not listening to those with dissenting views, arriving at a conclusion very early in the information gathering process and setting it in stone

      “Returning to the same subjects?” This blog has a focus. If this claim refers to something else, elaborate.

      “Citing the same sources?” The problem isn’t that Will doesn’t cite a diversity of sources, it’s that he’s A) deliberately equivocating between what the science actually said and media reports, and B) cherry-picking even the latter.

      “Not listening to those with dissenting views?” I listen to the “dissenting views” all the time. The “best of the best” of “dissenting views” just doesn’t have the evidence necessary to change the big picture.

      “Arriving at a conclusion very early in the information gathering process…” Tom, it’s been more than 30 years since the Charney Report. The fundamentals haven’t changed all that much. Though I didn’t make up my mind decades ago, and my view on the subject is very much a dynamic one, to pretend that we’re “early in the process” is beyond absurd.

      It’s an act of willful denial of reality.

      But citing Anderegg, et al? Tripe.

      Argument from personal incredulity; boring.

  7. I’ll just address your final point as the others reinforce my previous post.

    “But citing Anderegg, et al? Tripe.”

    “Argument from personal incredulity; boring.”

    It is not argument from personal incredulity. It is argument based on 19 years of professional expertise in the subject matter covered by the paper.

    • It is not argument from personal incredulity.

      Really?

      It is argument based on 19 years of professional expertise in the subject matter covered by the paper.

      Oh, I see! It’s not argument from personal incredulity, it’s argument from self-knowing/personal experience. Much better!

  8. Any time you’d like to start making sense, it’s okay by me.

  9. I recognize that what I’ve written so far is not really contributing. Neither is it trolling. That post you want me to click on is sort of the mirror image of trolling, being a bait and switch offer to engage when in fact your post would have been recognized for pure trollery had it appeared in any other venue.

    What I’m doing here is trying to plant a stake in the ground regarding Anderegg et al. It was not good scholarship, it is not good science, and this is just my little protest movement with regards to its use.

    • I recognize that what I’ve written so far is not really contributing.

      What I’m doing here is trying to plant a stake in the ground regarding Anderegg et al. It was not good scholarship, it is not good science, and this is just my little protest movement with regards to its use.

      Tom, what I’m trying to point out is that you have provided zero evidence in support of your claims. It’s not contributing. I am more than happy to try to have positive, healthy debates about issues. (I find surrounding oneself with only the like-minded to be a repellent existence.)

      But if you’re going to bash a given paper, please provide some substance beyond your personal opinion. Obviously, you aren’t “required” to, but claims that are made without evidence can be dismissed without evidence.

  10. I provided all the evidence you would want to see. I did it at Real Climate (with the collaboration of Roman M). I did it at Tobis’s site. I did it at Kloor’s. I know you were there–you were commenting.

    Do some bloody work yourself. Find what I wrote. Read it. You obviously didn’t the first time around.

    • Tom, it might seem “normal” for you to act this way, but to me (and I can only assume others who aren’t privy to your internal monologue), you’re coming off more than a little needlessly combative.

      If you can’t be bothered to provide the evidence you claim you have, why should anyone else be bothered to listen to you?

      My recollection of the Anderegg publication response “in the blogs” was dominated by the absurd claims of Pielke Jr. and others about a) claiming the collation of publicly, voluntarily signed petitions and letters constituted a “blacklist”, and b) pretending that Pielke Sr.- who was writing multiple blog posts about “rejecting” the “IPCC hypothesis” didn’t qualify as a “skeptic”.

      I believe you when you claim to have written things at other blogs, but I sure as hell don’t remember them, and I have little interest in devoting time to something you can’t even be bothered to link to.

  11. Here is what what actually got published at Real Climate:

    Real Climate has published a letter in defense of an indefensible paper. I have attempted to respond on Real Climate’s website, but they have decided not to publish my response.

    [Response: Most of this comment and the previous one was simply a list of accusations and insults, and not appropriate in this forum. The one possibly substantive issue we have left in for people to respond to. – gavin]

    [edit]

    They used Google Scholar instead of any one of several academic databases. They searched only in English, despite the fact that many climate scientists publish in other languages (and have many more who are skeptical of the English and American mania for histrionic claims of disaster due to CO2).

    They got names, job titles and specializations wrong–they obviously did zero quality control checking.

    As for the publications they were counting, they got them incredibly wrong. They hugely inflated the publication counts for their ‘side’ and reduced the publication counts for the opposition.

    [edit]

    The paper Expert Credibility in Climate Change, published in PNAS by Anderegg, the late Stephen Schneider, James Prall and Jacob Harold attempts to measure the credibility of climate scientists by counting how many papers they have published and how often their work has been cited by others.

    This led to the creation of a blacklist that will be used to injure the careers of those who have signed letters or petitions that do not agree with the Al Gore/James Hansen position on climate change, and to intimidate future scientists, effectively silencing dissent.

    The paper is poorly done, as I’ve explained elsewhere. They used Google Scholar instead of an academic database. They searched only in English, despite the global nature of climate science. They got names wrong. They got job titles wrong. They got incorrect numbers of publications and citations.

    As I’ve mentioned, the highly respected Spencer Weart dismissed the paper as rubbish, saying it should not have been published.

    But the worst part of this is the violation of the rights of those they studied. Because Prall keeps lists of skeptical scientists on his weblog, obsessively trawling through online petitions and published lists of letters, and because those lists were used as part of the research, anyone now or in the future can have at their fingertips the names of those who now or in the past dared to disagree.

    [edit of likely libelous statement]

    It doesn’t matter that the nature of the letters and petitions they signed varied widely, from outright skepticism to really innocuous questioning of the state of the science.

    The paper is tagged ‘Climate Deniers.’ Now, so are they.

    This is an outright violation of every ethical code of conduct for research that has ever been published.

    They violate several sections of the American Sociological Association Ethical Guidelines:

    “Sociologists conduct research, teach, practice, and provide service only within the boundaries of their competence, based on their education, training, supervised experience, or appropriate professional experience.”

    The members of the research team were operating outside their areas of professional competence.

    “Sociologists refrain from undertaking an activity when their personal circumstances may interfere with their professional work or lead to harm for a student, supervisee, human subject, client, colleague, or other person to whom they have a scientific, teaching, consulting, or other professional obligation.” The subjects of their research–the scientists on the list–risk grave harm as a result of this paper.

    “11. Confidentiality
    Sociologists have an obligation to ensure that confidential information is protected. They do so to ensure the integrity of research and the open communication with research participants and to protect sensitive information obtained in research, teaching, practice, and service. When gathering confidential information, sociologists should take into account the long-term uses of the information, including its potential placement in public archives or the examination of the information by other researchers or
    practitioners.

    11.01 Maintaining Confidentiality

    (a) Sociologists take reasonable precautions to protect the confidentiality rights of research participants, students, employees, clients, or others.

    (b) Confidential information provided by research participants, students, employees, clients, or others is treated as such by sociologists even if there is no legal protection or privilege to do so. Sociologists have an obligation to protect confidential information and not allow information gained in confidence from
    being used in ways that would unfairly compromise research participants, students, employees, clients, or others.

    (c) Information provided under an understanding of confidentiality is treated as such even after the death of those providing that information.

    (d) Sociologists maintain the integrity of confidential deliberations, activities, or
    roles, including, where applicable, that of professional committees, review panels,
    or advisory groups (e.g., the ASA Committee on Professional Ethics).

    (e) Sociologists, to the extent possible, protect the confidentiality of student records,
    performance data, and personal information, whether verbal or written, given in the context of academic consultation, supervision, or advising.

    (f) The obligation to maintain confidentiality extends to members of research or training teams and collaborating organizations who have access to the information. To ensure that access to confidential information is restricted, it is the responsibility of researchers, administrators, and principal investigators to instruct staff to take the steps necessary to protect confidentiality.

    (g) When using private information about individuals collected by other persons or institutions, sociologists protect the confidentiality of individually identifiable information. Information is private when an individual can reasonably expect that the information will not be made public with personal identifiers (e.g., medical or employment records).”

    I think it is clear that the paper, wrong on the facts, is unethical in its intent and outcome. I call for the pape to be withdrawn and for Prall’s website to take down the Blacklist.

    [Response: hmm… A long cut and paste about the need for sociologists to maintain confidentiality. But perhaps you could point out what it is about an open letter in the New York Times or full page ad in the WSJ and google scholar that is at all confidential? – gavin]

    (Cont. on another post)

  12. (Cont.)

    Gavin re your note to 74

    Being public is not the same as being labeled as a climate denier by an IT administrator and a grad student doing what they call ‘research.’ Those… people… had a duty of care to their research subjects which they totally failed to observe.

    [Response: Rubbish. If Lindzen or Michaels or whoever sign a public letter, I owe them no duty of care whatsoever in mentioning it wherever I like. The same would be true of them in mentioning the fact I signed the Bali letter. The same is true for the authors of this paper. Had a comment of support been made in a private email or an interview in which confidentiality had been assured, it would be a completely different story. (PS. I can see that you are trying not be overtly insulting, but please try a little harder – such language is not conducive to discussion). – gavin]

    Signing a letter or a petition should not earn you what somebody else uses as an insult or a weapon to deny them future employment, tenure or a grant. And that is surely what will happen.

    Roberts, what part of “I have to admit that this paper should not have been published in the present form. I haven’t read any other posts on this; the defects are obvious on a quick reading of the paper itself.” is difficult to understand?

    [Response: Well, it’s not hard to understand, but it’s not the same as “this study is rubbish” as you implied at first. That I assume is his point. Given that you pretending to be a champion of ethical scholarship here, I suggest that you be more careful in your quotes. – gavin]

    Mr. Prall, I’m really shocked at your lackadaisacal attitude towards the implications faced by those named on your list as climate deniers. In this charged atmosphere it will serve as an impediment to their careers. As some of the letters and petitions you use as reference are fairly innocuous, there are many scientists who do not consider themselves climate deniers who have now been named as such by your paper.

    And you confuse your duty to them as a researcher. It is not to protect their privacy. It is to dissassociate their identities from your labels. You can say climate deniers publish 2 papers to climate consensus holders publisher 93 papers all you want. I have no problem with that.

    As Spencer Weart said, “The statistics are certainly interesting, but must be interpreted as “2-3% of people who have published 20 climate papers are willing to publicly attack the IPCC’s conclusions.” That is, to me, a surprisingly high fraction, although I think it can largely be attributed not to the scientific process but to the unfortunate extreme political polarization, which can induce blindness… on both sides.” (Gavin and Hank Roberts, note the use of quotation marks to indicate direct quotes.)

    But you harm these people when you make it possible for them to be identified as climate deniers. Even if some of them are.

    I actually think what you’ve done borders on being actionable. It unquestionably violates the UK’s and EU’s Data Privacy Acts. It violates every research code of conduct or code of ethics I have ever seen.

    I’ve been doing this type of research for 15 years and I have never–never–seen the privacy of research subjects treated in such a cavalier fashion.

    [Response: But no private information was sought or used. Therefore there is no privacy to protect. And I would be astonished if any research code anywhere forbade commentary or labeling of public figures and their public positions. No data privacy act outlaws commentary on public actions. Certainly in the US, it is common practice to comb public archives and cross tabulate different fields. And that isn’t even dealing with the fact that the paper does not say that the UE group are ‘deniers’ in the first place. The only quote in the text refers to how “[t]his group, often termed climate change skeptics, contrarians, or deniers, has received large amounts of media attention” – which is certainly true. They have often been called these things, and they have received disproportionate media attention. Frankly, your level of outrage over this paper is similarly disproportionate. – gavin]

    I see you didn’t have the courage to publish my later comment. And you continue to miss the point. It is not important that the names were public nor that the signatories performed a public action.

    The canons of research require, for very good reason, that you do not associate individuals with research outputs.

    To say otherwise is anti-science.

    [Response: Nonsense. If I am doing research on the impact of Gordon Brown on economic policy in Europe using his public speeches, I think it would be a little weird not to mention his name in the resulting paper. If I wanted to aggregate his impact along with all other heads of state in Europe for the same time period (which I am pretty sure is also in the public domain), would I be expected to say ‘a sampling of heads of state’ rather than just give a list of my ‘research subjects’? This is just bogus. (PS. continued whines about how you are being repressed will be met with persistent links to this youtube video. Either add something substantive (politely) or don’t bother). – gavin]

    Next post will feature RomanM’s critique

  13. Was this particular paper peer-reviewed?

    Unfortunately, there seem to be a number of technical shortcomings in the paper. Several of these induce biases in virtually every aspect of the analysis of the data.

    The samples were not a random selection from a larger population, but rather the selection included the AR4 working group (over 600 of the 903 CE group subjects) who themselves had been chosen for their prolific publication and citation records. The UE group were chosen from individuals who had expressed opinions regarding the evidence for global warming. the numbers in the two groups do not properly represent the relative numbers in the population.

    There was no control for the actual number of authors on each paper. Thus, if there were 10 authors on a particular publication with 100 citations, each author received credit for both the publication (total of 10) as well as the citations (total 1000) that the paper received. If the mean number of authors is higher for one of the groups, then this biases the results in favor of that group and exaggerates the extreme high values of the most prolific authors. Furthermore, because the counts are not independent, it puts into question the validity of using the Mann-Whitney test for analyzing that data.

    Figures 1 and 3 reflect the disparity in size of the two samples by graphing counts rather than percentages.

    However, there is also a major statistical error on page 2:

    We examined a subsample of the 50 most-published (highest expertise) researchers from each group. Such subsampling facilitates comparison of relative expertise between groups (normalizing differences between absolute numbers). This method reveals large differences in relative expertise between CE and UE groups (Fig. 2). Though the top-published researchers in the CE group have an average of 408 climate publications (median = 344), the top UE researchers average only 89 publications (median = 68; Mann–Whitney U test:W= 2,455; P < 10−15). Thus, this suggests that not all experts are equal, and top CE researchers have much stronger expertise in climate science than those in the top UE group.

    If one were to take two samples of sizes 903 and 472 randomly from the same population, order them by size and then take the largest fifty from each, it is a virtual certainty that the average (and/or median) of the 50 from the larger 903 subject group will be greater than that for the smaller sam0ple group. The exact amount (which may be very large) will depend on the distribution of the values from which the samples were selected. The analysis, the MW test statistic and p-value are meaningless here and the conclusion is unwarranted.

    Perhaps you don’t understand the objection to the part I quoted:

    We examined a subsample of the 50 most-published (highest expertise) researchers from each group. Such subsampling facilitates comparison of relative expertise between groups (normalizing differences between absolute numbers).

    This claims that that selecting in this fashion somehow “equalizes” the imbalance in sample sizes and that is patently false. The statistics and subsequent analysis based on this sub-selection are meaningless and wrong. By all means, let’s hear your technical complaints on this and other issues.

    Yes, I read the head post. Did you actually view the results of “repeated analysis” or are you simply repeating the arm waving?

    Didactylos, what do you think the sentence “Such subsampling facilitates comparison of relative expertise between groups (normalizing differences between absolute numbers)” means? I didn’t suggest doing that – the authors did.

    There is nothing to “fix” because this approach is wrong right from the start. You clearly still do not understand my initial explanation.

    To make it more obvious, if I have two populations of numbers with the same distribution and I select 500 from one group and 50 from the second, I should get about the same median in both. Now, if I take the largest 50 from the first group and the largest 50 from the second (i.e. all 50) and take the median from the sub-samples, what will happen? The first will have all values (probably considerably) above the old median so the new median will be larger. The second still has the same median as before. The difference I am seeing is due purely to selecting in this fashion from unequal size groups. It is NOT any sort of comparison between the values in the two identical groups in question. The same will hold true for the average of the sub-samples. The point is that as done by the authors, any differences are distorted and no longer meaningful.

    The sub-sampling merely removes any watering-down caused by a long tail. You just don’t like it because it widens the distance between the CE experts and the UE experts.

    In fact, it is just the opposite. This procedure extracts” the long tails” which are naturally longer in the larger group. It widens the difference, but I “don’t like” it because it is wrong. if you don’t believe what I say ask a real statistician.

    #82, Didactylos, #88, Maple Leaf.
    The intent of my example was to point out to you that the procedure used by the authors to quantitatively summarize the differences between the most prolific publishers in each contains a substantial bias in favor of the larger group when applied to two groups that have identical characteristics. It follows from this that in other cases, the effect will be the same when the groups differ. The results are distorted – some of the difference is actual difference between the two populations and some due to the sample size and you can’t tell how much is due to either source.

    To further apply a statistical test on the results as a purported evaluation of the difference in the populations is incorrect. My criticism of this does not fall in the “how the study could have been done differently” category. It is simply not the analytically correct thing to do. Using the test results to substantiate conclusions is inappropriate.

    “You haven’t explained how you propose to fix this unfixable problem. How can you make the groups “equal”?”

    Suppose you are comparing two populations in a case where the distributions are severely skewed and there are a substantial number of extreme values. For other practical reasons, the sample sizes you collected may be unequal, say 500 in one and 1000 in the other. If I wish to compare the details of the distribution of the more extreme values, just choosing the largest 50 from each creates the problem that we have been discussing. One approach to correcting the imbalance could be to first place the two samples on the same size footing. How? By sub-sampling 500 observations randomly from the larger group first and then comparing the top fifty from each. This eliminates the size imbalance effect, but does introduce an effect due to the random selection. The latter can be dealt with by repeating this multiple times (easy to do with computers).

    It is my opinion (from the first two sentences in the quoted passage) that this was the intent of the authors, but they skipped the very important sub-sampling step.

    “But, if you disagree, then by all means go hunting for more UE candidates that have published highly cited papers. Good luck….”

    There is no single “credo” of global warming of which you are must be either convinced or unconvinced. Rather there is a continuum of observations, predictions, projections, explanations, evidence, etc. It seems reasonable that within the climate science community there must be a range of views from “superconvinced” to “completely unconvinced”. Defining the two groups as the authors did based on the expression of an opinion of various individuals with respect to the setting of policy seems specious with regard to scientific credibility.

    However, given their definition, it is not at all surprising that the numbers in the CE group as defined for the paper should be considerably larger than those in the UE group. Over the roughly 35 year period of the AGW movement, there has been a concerted effort to stifle contrary opinions and public debate on the issues (the science is settled, consensus exists) and to vilify anyone who may disagree with any portion of the “evidence” (demeaning names, “deniers”, mental illness of denial conferences, throw skeptics in jail, shills for big oil, flat earthers… including the paper currently under discussion).

    Within the insular environment created by this consensus building, it is very unlikely that students in climate science will buck the system. Doing so would make it difficult to get research money and build a career. Only someone who has been in the system longer and is more established can afford to publicly go on record that certain public courses of action are not suitable given their personal view of state of current climate knowledge. So finding more UE’s is not so easy in today’s universities. Notice that in the paper, the authors state that UE individuals tend to be older than CE’s, a fact which is consistent with the above observation.

    How does this affect the data in the paper? More individuals in the CE population means more papers written in “official” climate journals means more citations for the earlier works of that group and for their peers. Some of the papers by highly visible scientists will end up with an inordinate number of cites. Do these cites indicate high quality work? In some cases, yes – there are some very capable individuals doing climate science. However one should not confuse the number of citations as incontrovertible evidence of that.

    # 122 Didactylos

    As for your proposed method, the final sample sizes would probably be too small to be meaningful, but feel free to try it and compare the results.

    #136 Maple Leaf

    And therein lies the rub, the UE group is simple too small to obtain a sufficiently large sample by applying that approach.

    I thought I explained it pretty well, but I must be slipping.

    Let me spell it out for this case. Take a random sample of size 472 from the CE group. Take ALL 472 as the sample from the UE group. Select the top 50 from each of these two “subsamples” and do the same comparison as was done in the paper. You still end up comparing 50 subjects to 50 subjects. This procedure can be repeated multiple times to overcome the effects of the random selection. Where does the “too small a sample“ come into play?

    I see what is going on here. The Anderegg et al. paper is important and the findings troublesome/inconvenient for the ’skeptics’/contrarians, so it has to be attacked.

    No, I don’t think that this paper is particularly important nor the ‘findings” inconvenient, but I do find them troublesome from the viewpoint of ethics and professionally. [edit – no sideswipes, stick to the point]

    From a scientific viewpoint, it is not very solid. The groups were chosen in a poorly defined manner, the methods for gathering data were haphazard, the measures of “credibility is not particularly robust and the statistical analysis has already been discussed here. I still have not seen a credible answer to the question:

    Was this paper peer reviewed?

    It is my understanding that papers can be submitted PNAS by members without such a review so I would think that given the author’s desire for discussion to take place in peer-reviewed literature, this is a legitimate question.

    It is interesting to apply the paper’s measures of “credibility” to the authors themselves:

    Publication and citation analyses are not perfect indicators of researcher credibility, but they have been widely used in the natural sciences for comparing research productivity, quality, and prominence (21–24). Furthermore, these methods tend to correlate highly with other estimates of research quality, expertise, and prominence (21–26).

    The first three authors do not have a particularly large numbers of publications or citations (not surprising for Mr. Anderegg since he is just beginning his career). Although Prof. Schneider was prolific in the climate science area, I do not believe that he has published much in the area of evaluation of professional expertise ( Recall that the paper considered ONLY publications related to climate regardless of the other expertise of the individual). So using their criteria, the “research quality, expertise, and prominence of the authors” with regard to this a publication on this particular subject must be low.

    I generally put much stock in the metrics in the quote from the paper, but it appears that some of the posters here do.

    #159 ML

    I didn’t bring up the sub-sampling. The authors claimed they had done that and then you threw it at me as some sort of challenge. I told you one way it could be done, but you brought up an arcane “small sample” remark which indicated that you missed the point. Rather than take a personally aggressive and confrontational stance (such as the one in your latest comment), I politely explained it again.

    I gave a reason for why I thought that the paper should not have been published on ethical grounds but the moderator chose to remove as is his right. Further explanation is not warranted here.

    “And, who are you again?” This coming from a person whose first name is Maple? I have been identified a number of times on other blogs, but I choose not use my full name generally because frankly I don’t trust some of the individuals on the internet who might decide to abuse my privacy. Threats were issued a while back from greenpeace and although officially withdrawn later, it remains a worry. The existence of the paper in question indicates that someone else may decide to create another list and I don’t wish at this point in time to be on it.

    #164 ML

    My final words as well.

    …think about it though, is it any wonder that I and others have little patience for people who are happy to critique but who are unwilling to make the effort to follow through and advance the science? You seem to think that you could do better or improve upon the paper, so yet again I invite your to do what any credible scientist would do and follow through and publish.

    It may interest you to know that I have spent forty years in a professional capacity “critiquing” and tearing apart research projects, theses and papers before they were written or published for the pure purpose of advancing the science in a university environment in many distinct disciplines. The reason was if the mistakes were corrected and I could find nothing wrong, the research was on a pretty solid footing. At the time, one generally got a thank you in the result but no name on the paper as seems to be more common today.

    Why would I publish anything on this topic? In my area, we don’t publish simple things of this sort, and for me to go through the exercise of collecting the data and re-doing something that I find unethical seems something not even a “credible” scientist would do. I would hope that if Mr. Anderegg decides that what I drew attention to was incorrectly done, he would at least learn that statistical analysis is important enough to consult a good statistician next time. To me, this is advancing the science.

    Funny how Inhofe, Limbaugh, Beck and Morano have not made apologies for their inflammatory rhetoric. I hope that you have an issue with their lack of ethics? I may have an opportunity to go into climate science field in the near future, but as for now I am not doing so. Not because of paranoia, but because of the very real threats made against climate scientists.

    I do not countenance ANY inflammatory rhetoric or violence on either side although I must admit I am more concerned with the ones that are aimed in my direction. Let’s leave the causes and blame for another discussion.

    I would suggest that you are over-reacting to the threat in your personal situation. Do what you think you should.

    I don’t see any point in replying to each individual separately since many of the criticisms of my comments are rants with no substantive content and it would be a waste of time to address such nonsense.

    However I will make a few remarks on several issues that have been raised.
    My criticism of the specific analysis of the top 50 in each is based on the fact that the same analysis applied to separate samples of sizes 472 and 903 from the same population will invariably produce larger means and medians in the 903 size sample. This is not an opinion or a guess. It follows from the mathematics behind the situation. The statistical test used in this case in the paper will produce a “significant” p-value that the larger sample size group comes from a population which produces larger values – this is false since the sample were selected from the same generating mechanism.

    You do not need to “try it out” on real data to see if that is the case. However, if you wish to, go ahead. To get something similar to the data in the paper, try generating exponential variables with mean 100 for each of the samples. If you know some probability, you could calculate what to expect: the theoretical mean of the top 50 will be about 388 for the larger group and 324 for the smaller. For the data which has a higher percentage of extreme values, the difference would be greater.

    I could suggest to you that you try it on the data from the paper (as you seem to be implying I should to convince you of something which can be demonstrated otherwise). However, maybe you haven’t noticed, but the data is NOT available.

    The only semi-cogent suggestion was from #172, stewart longman,

    RomanM: It’s a trivial issue, so ‘publish’ it on this blog – what happens when you compare the top 5% of each group? Do you have any evidence that they are from the same population? It’s your job to demonstrate, not argue without data, that the authors did it wrong. That’s science – reality trumps argument. If you prefer to argue, that’s not science.

    The cogent part is that it makes more sense to compare the top 5% rather than the top 50 individuals in each group. Yes, that would be a better comparison, but that is NOT what the authors did. As a psychologist, you should understand that you do statistical testing to demonstrate that they ARE different, not the other way around. The onus is on the authors to use valid statistical techniques to make their point. In this case, showing that their technique is not valid can be done simply by analyzing what happens when the technique is applied.

    As far as publishing this type of criticism, what journal would you suggest? Statistics journals do not publish papers showing that someone misused a statistical technique. Such errors do need to be corrected before someone else decides to copycat and make the same mistake again. The best way to do it is to publish a corrigendum from the authors in the original publication. Only in the case that the authors refuse to do this is there a need for someone else to step in. Given the “climate” at PNAS, I suspect that would be a difficult task for an “outsider” to accomplish.

    I suggest that you ask Mr. Anderegg for his opinion on the points I have raised. If he wishes to ask for clarification or to discuss this off the record, I don’t mind the moderators passing my email address on to him.

    As an aside, you might try counting the number of ad homs, name callings and disparaging remarks found in this thread… and look at who is generating these.

    #202 Bob (S)

    This is a shame, especially after the whole MM/PCA brouhaha, since people seem to be saying that professional statisticians should be more involved in climate science, while you seem to be saying that they can’t be bothered.

    Not at all. What I said is that “mistakes” do not merit publication in statistics journals because they should be fixed at the original point of publication.

    I fully agree that statisticians in conjunction with climate scientists should be developing new methodology for use in climate research. Such methodology would definitely be published in good statistics journals because interactions with other statisticians would help move the subject forward.

    Look at the whole picture. If the authors wish to summarize the top 50 in each group, they are completely within their rights to do – no problem. Do all the graphs and calculate all of the statistics you want. However, when you are interpreting those statistics, remember that the differences are distorted by the imbalance in the sample sizes by amounts which you do not know.

    However, the authors first stated they were doing a procedure which would allow them to do a particular statistical test as part of their “evidence”. They then selected the top fifty from each and followed it up with the test not realizing that the assumptions of the test were severely violated by the way the selections were made. The test result was used in the subsequent statement about the groups. This is supposed to be a scientific paper and what they did was incorrect. If the same approach is used in another situation where the difference may not be as large, the end result could be false. Scientists have the habit of using methods that someone else has used in previous papers so I would think that a correction is in order.

    #210 Gator. The relevant feature is that the samples are of a different size, not the population.

    #211 Bob (S) . If the authors had been members of my own academic environment, I might have tried approach 1.

    Producing my own study – spending how many weeks or months gathering data to do something that I strongly feel should never have been done in the first place – not very realistic.

    I posted this on a blog where the paper was being presented by the authors. I was not aggressive about it nor have I said anything that could be interpreted as an attempt to embarrass anyone. Everyone here seems to place an inordinate importance on credentials. I did not flash a badge nor did I feel it would be appropriate to stomp in announcing myself before posting my comment. Everything that I have written is on the record. If I am wrong, it is there to be pointed out by people who can understand the situation. In the end it can have as much of an effect on the science but that may depend on how the authors view it. I have offered to be part of the process away from the blog.

    I have commitments which will prevent my continuing this discussion for a day or so at least . As it is, the discussion of this topic seems to be pretty much exhausted.

    Perhaps a correction to the paper might be warranted.

  14. Here’s what I published at Examiner.com:

    Just when you think things can’t sink any lower, the National Academy of Sciences has now published a list compiled by a non-academic weblogger that attempts to rank scientists by expertise, credibility and (oh!) belief in the consensus position on global warming.

    It is a blacklist. It’s also hilariously wrong. As Roger Pielke Jr. notes on his weblog, his father, who firmly believes in man’s impact on the climate, is rated as a skeptic, while James Hansen, who has repeatedly criticized the IPCC consensus (albeit for being too conservative) is mentioned as a supporter of the IPCC.

    This will contribute to the feeding frenzy on climate change and distract (as it is meant to do) from real discussion of climate change issues.

    It is a black day for science and shows that there are people more stupid than Ken Cuccinelli.

    The worst of it all is the fact that Stephen Schneider lent his name to this travesty. It’s no longer enough to quote from the McCarthy hearings (At long last, Senator, have you no decency?). Does anyone here have any sense of shame?

    All of the elements of climate consensus confusion creation (henceforth to be known as the 4Cs), are contained within this piece of junk. Creation of jargon. Attribution of motive. Assignation to a list because someone else put a scientist’s name on it. Using opinion of where scientists publish and what they say as if it were gospel.

    Very much of a piece with the other junk coming out these days. Very much a symptom of a group that can no longer respond to the real arguments.

    It’s a travesty. Stephen Schneider, how could you lend your name to this garbage?

    Continue reading on Examiner.com Global warming: Black list a black day for science – National environmental policy | Examiner.com http://www.examiner.com/environmental-policy-in-national/global-warming-black-list-a-black-day-for-science#ixzz1Yvpd0xXn

    Dear Professor Schneider,

    I am writing in regards to your recently published paper in the Proceedings of the National Academy of Sciences, titled ‘Expert Credibility in Climate Change.’

    I would like to start by asking you if your previously held opinions on global temperatures, which you have since discarded, should be used to disqualify you from current or future work or discussion regarding climate change. If not, why do you libel Roger Pielke Sr. as a skeptic based on his signature to a petition in 1992?

    Second, I would like to know how you could associate your name and reputation with a paper with so many errors of data. Are you aware of the mistakes regarding the backgrounds, employment and specializations of the scientists on the lists used for your paper? What quality control measures did you use that could get Willam Happer’s field of specialization wrong? Do you stand by the integrity of the data used in your paper?

    Third, I would like to know how you validated your lists as fit for purpose. How were the petitions selected? What quality control checks and validation procedures were used? Surely, assuming you libeled Pielke Sr. unintentionally, you would have realised that his appearance on your list would call into question the list itself and not his character or beliefs.

    Fourth, are you aware that this list is already being used to dismiss scientists as unfit for participation in the debate merely because of their presence on this list? How could you have been unaware that this would be a blacklist used to demean those on it and threaten those who might wish to voice an unpopular opinion in the future? Joe Romm wrote today, “It is time for the media to stop listening to, quoting, and enabling the anti-scientist disinformers.”

    Roger Pielke Sr., to continue with the case example, recently wrote on his weblog of how a project he had requested funding for had been denied, despite stellar reviews from referees. How do you think his next project will fare now that he is officially mislabeled (but libeled) as a skeptic, something he has been adamant about denying (I know from personal experience after mistakenly referring to him as one)?

    Judith Curry writes, “My first comment about the paper is that I suspect it was not peer reviewed. Since the 4th author Steve Schneider is a member of the National Academy of Sciences, a paper submitted by a member is published without review, sort of a “vanity press” for national academy members.” Can you either confirm or deny that this paper was in fact peer-reviewed?

    I am not the only questioning the data selection, methodology and analysis of this paper. Eric Steig says he is appalled by it. Lucia Liljegren writes,

    “With respect to this focus on counting papers, there are all sorts of obvious questions one might ask like:

    Are some researchers working on large collaborative teams writing papers with over a dozen authors while other write papers with only 1 or 2? If yes, how should a person listed as one of 20 co-authors on 10 papers be weighted against someone listed as one of 2 authors on 10 papers be accounted for when assessing expertise?

    Does participating in the IPCC help people make connections and help grease the wheels when submitting papers and going through peer review? Does merely going along to get along help people get papers published? Does signing a letter criticizing the IPCC make it harder to get papers published? If yes, is the number of differences in paper counts due to this effect rather than any true expertise?

    As for your reading in the idea that this paper tells us we can’t find two viable camps, let’s first assume that someone proposes there are two viable camps. Did that someone propose that the dividing line separating the camps lies between the groups the authors of the paper call “CE” and “UE”. Couldn’t a fault line lie somewhere in the group they called “CE”?

    I note no reference in your paper to West and McIlwaine. How do you address their conclusion that there is no correlation between number of citations and expert ratings of quality? Or Callaham et al in their findings that journals were a greater predictor of citations than quality of the research? Where can we find how you allowed for this?

    Is this science you are proud of? Does damaging the reputation of some scientists by mistakenly (or vindictively) including them on a blacklist serve science well? Does establishing a climate of fear that will dissuade scientists from expressing their true opinion?

    Continue reading on Examiner.com Global warming: Open letter to Stephen Schneider – National environmental policy | Examiner.com http://www.examiner.com/environmental-policy-in-national/global-warming-open-letter-to-stephen-schneider#ixzz1YvprvNQa

  15. Here’s what I wrote on Tobis’ site:

    Regardless of my opinion about the motives and eventual use of the defacto list that has been created (and time will tell, certainly), this is garbage science created by an amateur blogger and a grad student with Schneider’s name tacked on top of it.

    In reverse order:

    The findings are incorrect. It incorrectly labels ACC experts as either CE or UE.

    The analysis scheme is incorrect. It fails to account for confounding factors such as change of opinion over time, venue and approach for presenting petitions, comparative content of petitions, etc.

    The data collection is incorrect. They have wrong names, wrong specializations, wrong counts of publications and wrong citation numbers.

    The methodology is inappropriate. They searched with only one database, did not search in other languages, did not crosscheck their data.

    Their hypothesis is flawed. There are many factors that could equally explain differences in publication and citation by UE and CE scientists, including publication bias, confirmation bias, fear of retribution or erosion of career potential, etc.

    Spencer Weart said it best–the paper should not have been published in its present form. It does not survive the first casual reading.

    And yet you’ve got it up there as if it’s the Magna Carta.


    Assuming that’s what you want, I’ll just start off by saying that in point one, it’s clear that some of the people they claim are UE are in fact CE.

    They didn’t sense check their output. With so few names, there’s no excuse.

    It’s the same thing that wrecked Oreskes, really.

    Vic, your request for references spurred my curiosity.

    While serving as custom research director at Kable, now a division of the Guardian, I bid for and won a project to do research for the UK National Health Service.

    I designed the research study and served as program manager. I actually ended up doing a fair bit of the research as well, to save money.

    As prime contractor, I selected and worked with Colin Drummond of St. George’s University, a rather noted scientist who served as technical specialist for the study. I learned a lot from him about the practical aspects of conducting relevant research that can be useful for science as an institution.

    I call your attention to the following:

    1. The utility of using a spectrum of research techniques to investigate core issues is far more robust than relying on a Google search.

    2. A true literature search can be a valuable addition to a research study, as shown within. However, a broad citation search is of little value as a standalone product.

    I see that some mention this type of trawling through Google as something universities frequently do during their hiring process. I’m sure some do. However, you might note that they still do such old-fashioned research in the form of job interviews and inspection of resumes.

    http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4122341

    If I find other published examples I will bring them to your attention. However most of the work I do is proprietary in nature and I have various signed agreements with my clients about disclosing the nature of the research, the results and the identity of the clients.

    …No, Michael, and I criticized Senator Inhofe. As Mike Morano is an aggregator, I tend not to go after him directly, although I have criticized him on other issues.

    So that’s the moral standard for Stephen Schneider? Setting the bar a bit low, don’t you think? Also, refresh my memory about Inhofe’s and Morano’s scientific qualifications–I know those are mighty important to you…

    Mr. Diesel, I didn’t do the literature search–undergrads did. It was extensive and comprehensive and should be a lesson to Schneider et al. I designed it, commissioned it and assisted Professor Drummond in the report writing.

    As for strict bibliometric searches such as the one Schneider et al did, I have done many–but sadly they are proprietary.

    You should certainly look at the methodology of the Alcohol Harm Reduction study to understand how science normally approaches this type of issue. It is very different to what the PNAS published last week.

    1) I see no evidence that he took care to avoid bias. At all. Using his choice of Google Scholar alone is sample bias. What did he cross check against? Where is his search in other languages?

    I have no problem with his publishing it. I have no problem with the PNAS risking their reputation by accepting it. However, the ethics of social research require researchers to insure the anonymity of participants. This is especially important in studies that could, in the opinion of respondents, pose a threat to them in future based on the information collected about them. That was not done.

    2) A funded study to examine the strength and depth of consensus views on climate change would obviously start with secondary research to identify prominent proponents and exponent views. This research would include published papers, media statements and even–OMG–posts and comments on weblogs.

    A selection of these would be screened (using publication records and citations) for qualifications and asked to participate in depth interviews designed to insure that they did indeed still hold views similar to what was published. The interviews would also be used as referral sources for further contacts and permission sought to use respondent names in contacting further sources. Qualitative impressions of percentages for consensus would be sought.

    But one key finding would be a verified list of published papers from respondents to use for keyword searches for a more thorough pub check later on. Another would be respondents’ impressions of journal quality and bias. Another would be ‘best of league opponents’ or who they consider the leading lights of the other side of opinion.

    As mentioned, a more thorough literature search with hands-on evaluation of abstracts of published papers would follow. Although some useful segmentation buckets would fall out of the results, one to examine from the beginning would be scientists who study how the climate works and scientists who study what the climate does–in other words, who would be contributing to WG1 (or its antithesis in the skeptic community) as opposed to who would be working on WG2 or 3.

    I have seen no defense of PNAS. I have just seen assertions that it is good.

    I find this surprising, actually. I’m going to be offline for a couple of days. Perhaps that will give some here a chance to actually do some real work on this.

    So this is what passes for serious discussion of the PNAS paper here?

    Michael Tobis wants to play the moral equivalency game, meaning he doesn’t have to defend the PNAS paper.
    Frank wants skeptics to do their own survey, meaning he doesnt’ need to defend the PNAS paper.
    Marion Delgado just wants to remind you all of how low a human being I am, probably because he has not read the PNAS paper.
    Steve Bloom wants to talk about McCarthy, so he doesn’t have to defend the PNAS paper.
    Deech wants to change the subject to Oreskes, so he doesn’t have to defend the PNAS paper.
    Dirk wants to defend Oreskes, so he doesn’t have to defend the PNAS paper.
    New York wants to talk about a survey I ran on Examiner, saying it was not a random sample, ignoring the fact that I said about three thousand times that it could not be considered a public opinion poll because it was not a random sample. But at least that gives him a reason not to defend the PNAS paper.
    Dirk still wants to split hairs on the papers Oreskes missed, but doesn’t want to defend the PNAS paper.

    But finally–at last! New York wants to defend the PNAS paper. Oh–no he doesn’t. He wants to ignore every criticism I have made and for me to do what he wants.

    And then dhogaza gets to make up more stuff about me.

    So far, you all are not impressing me very much at all.

    The type of defense I expected to see:

    The authors considered the issue of sample bias that could be introduced by the use of a single database from a commercial source that has no academic supervision and has published no quality standards. The authors rejected these issues for the following reasons:

    The authors considered the issue of regional bias that could have been introduced by conducting their search in English only. The authors rejected this issue for the following reasons:

    The following controls were used to verify number of publications and citations attributed to individual scientists:

    The following analysis was made of the various opinion surveys used to label scientists as either UE or CE:

    Instead, what I have received here boils down to:

    1. Fuller is a big ugly meanie.
    2. I disagree with Spencer Weart so that is enough of a defense.
    3. Side issues that have nothing to do with this should be explored at length.

    Oh, and Michael, you say “Nobody is saying this is a massively important paper, just that it’s interesting enough to merit publication.”

    But exactly how much ink and prominent top of fold exposure have you given it? Why is that? Don’t say it’s because of skeptical criticism–skeptics criticize many things you manage to ignore.

    What other scientific publications have you feature prominently recently?

    Yeah, I guess so. I raise a number of points where potential bias undermines the conclusions. Your response is ‘so what?’ You disagree with Weart–but don’t say where, why or how.

    That does conclude any conversation aimed at finding out something useful, certainly.

    I thought this was supposed to be about science, as opposed to just being ‘sciency.’

    …The presence of error doesn’t mean there is no bias. Indeed the presence of error means the survey’s authors should work twice as hard at assuring readers that the error did not in fact introduce or reinforce bias.

    They did about as much work as you and your commenters here. The moral equivalent of zip.

    But I guess that’s okay, as long as you can use the headline for two weeks and remind people that the list is out there somewhere, and they sure don’t want to get their names added to it.

    Michael, the next time you complain about the state of communications about science, please remember this episode. You are a scientist trying to communicate with a journalist in front of an audience.

    Were I in your position I would be embarrassed at the amount of time and attention you have spent on this, let alone the way it is concluding.

    Dirk, because nobody can answer basic questions about the validity of the methodology, the study does not contribute to the body of science and trying to tease information out of wrongly collected and bad data can be left to someone else.

    Someone recently said that journalists ‘were the ball.’ You all dropped it.

    Dirk, my response to your comment is pretty much a mirror of your sentiments. Talk is cheap, and there has been no defence of the methodological and analytical errors evident in the paper, just calls for me to spend my time working with data I don’t trust. Did you read the commenter’s comments on this paper?

    Marco:

    1. The standard method for dealing with datasets of uncertain provenance and/or reliability is duplication and control checks. Why didn’t they use more than one database? Why didn’t they look at what databases returned for scientists they could check with? In any event, where is their discussion of the decisions they made regarding this?

    2. “Very likely?” Is that your opinion, their opinion or is it printed in a paper somewhere? Again, where is their discussion of methodological choices?

    3. “There is no reason to assume Google Scholar has a bias…?” Again, is that Marco, the authors, a published paper somewhere? And again where’s the discussion of the choices they made.

    4. By ‘Supplementary information’ I presume you mean the Supporting Information listed along with the paper. If so, I did read it. I see a list of the petitions they used. I see the assertion that signing these petitions indicated that the scientist strongly dissented from the IPCC. I see no discussion of any homogeneity or heterogeneity amongst the various petitions. I see no description of how and when these petitions were presented, in what venues, using what methods. I see no discussion of how any of these potentially confounding factors could bias the sample of UE scientists (and the same is obviously true of CE scientists as well).

    It’s amateur hour stuff.

    Marco, sorry I can’t stay and play today, but note the last thing you said. You (I’m sure unintentionally) showed the potential evil of this. It doesn’t differentiate between, as you said,

    ” all the petitions had to do was to question the IPCC conclusion. Who cares whether one said “yes, plenty of warming, but there is no need to do anything”, whereas another said “the IPCC is a bunch of frauds”.

    So now you have people on the same list with widely divergent beliefs. That list will be used to hurt some who are not skeptics, don’t want to be called skeptics and won’t get jobs or funding because they are on that list.

  16. Regardless of my opinion about the motives and eventual use of the defacto list that has been created (and time will tell, certainly), this is garbage science created by an amateur blogger and a grad student with Schneider’s name tacked on top of it.

    In reverse order:

    The findings are incorrect. It incorrectly labels ACC experts as either CE or UE.

    The analysis scheme is incorrect. It fails to account for confounding factors such as change of opinion over time, venue and approach for presenting petitions, comparative content of petitions, etc.

    The data collection is incorrect. They have wrong names, wrong specializations, wrong counts of publications and wrong citation numbers.

    The methodology is inappropriate. They searched with only one database, did not search in other languages, did not crosscheck their data.

    Their hypothesis is flawed. There are many factors that could equally explain differences in publication and citation by UE and CE scientists, including publication bias, confirmation bias, fear of retribution or erosion of career potential, etc.

    Spencer Weart said it best–the paper should not have been published in its present form. It does not survive the first casual reading.

    And yet you’ve got it up there as if it’s the Magna Carta.

    Mr. Diesel, I dismiss this paper because I have performed about 30 similar studies in the past 6 years for PESTLE, SWOT and competitive analyses and for industry studies. I’m doing one now, as a matter of fact.

    They didn’t do it right. That’s all.

    Vic, your request for references spurred my curiosity.

    While serving as custom research director at Kable, now a division of the Guardian, I bid for and won a project to do research for the UK National Health Service.

    I designed the research study and served as program manager. I actually ended up doing a fair bit of the research as well, to save money.

    As prime contractor, I selected and worked with Colin Drummond of St. George’s University, a rather noted scientist who served as technical specialist for the study. I learned a lot from him about the practical aspects of conducting relevant research that can be useful for science as an institution.

    I call your attention to the following:

    1. The utility of using a spectrum of research techniques to investigate core issues is far more robust than relying on a Google search.

    2. A true literature search can be a valuable addition to a research study, as shown within. However, a broad citation search is of little value as a standalone product.

    I see that some mention this type of trawling through Google as something universities frequently do during their hiring process. I’m sure some do. However, you might note that they still do such old-fashioned research in the form of job interviews and inspection of resumes.

    http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4122341

    If I find other published examples I will bring them to your attention. However most of the work I do is proprietary in nature and I have various signed agreements with my clients about disclosing the nature of the research, the results and the identity of the clients.

    No, Michael, and I criticized Senator Inhofe. As Mike Morano is an aggregator, I tend not to go after him directly, although I have criticized him on other issues.

    So that’s the moral standard for Stephen Schneider? Setting the bar a bit low, don’t you think? Also, refresh my memory about Inhofe’s and Morano’s scientific qualifications–I know those are mighty important to you…

    Mr. Diesel, I didn’t do the literature search–undergrads did. It was extensive and comprehensive and should be a lesson to Schneider et al. I designed it, commissioned it and assisted Professor Drummond in the report writing.

    As for strict bibliometric searches such as the one Schneider et al did, I have done many–but sadly they are proprietary.

    You should certainly look at the methodology of the Alcohol Harm Reduction study to understand how science normally approaches this type of issue. It is very different to what the PNAS published last week.

    1) I see no evidence that he took care to avoid bias. At all. Using his choice of Google Scholar alone is sample bias. What did he cross check against? Where is his search in other languages?

    I have no problem with his publishing it. I have no problem with the PNAS risking their reputation by accepting it. However, the ethics of social research require researchers to insure the anonymity of participants. This is especially important in studies that could, in the opinion of respondents, pose a threat to them in future based on the information collected about them. That was not done.

    2) A funded study to examine the strength and depth of consensus views on climate change would obviously start with secondary research to identify prominent proponents and exponent views. This research would include published papers, media statements and even–OMG–posts and comments on weblogs.

    A selection of these would be screened (using publication records and citations) for qualifications and asked to participate in depth interviews designed to insure that they did indeed still hold views similar to what was published. The interviews would also be used as referral sources for further contacts and permission sought to use respondent names in contacting further sources. Qualitative impressions of percentages for consensus would be sought.

    But one key finding would be a verified list of published papers from respondents to use for keyword searches for a more thorough pub check later on. Another would be respondents’ impressions of journal quality and bias. Another would be ‘best of league opponents’ or who they consider the leading lights of the other side of opinion.

    As mentioned, a more thorough literature search with hands-on evaluation of abstracts of published papers would follow. Although some useful segmentation buckets would fall out of the results, one to examine from the beginning would be scientists who study how the climate works and scientists who study what the climate does–in other words, who would be contributing to WG1 (or its antithesis in the skeptic community) as opposed to who would be working on WG2 or 3.

    So this is what passes for serious discussion of the PNAS paper here?

    Michael Tobis wants to play the moral equivalency game, meaning he doesn’t have to defend the PNAS paper.
    Frank wants skeptics to do their own survey, meaning he doesnt’ need to defend the PNAS paper.
    Marion Delgado just wants to remind you all of how low a human being I am, probably because he has not read the PNAS paper.
    Steve Bloom wants to talk about McCarthy, so he doesn’t have to defend the PNAS paper.
    Deech wants to change the subject to Oreskes, so he doesn’t have to defend the PNAS paper.
    Dirk wants to defend Oreskes, so he doesn’t have to defend the PNAS paper.
    New York wants to talk about a survey I ran on Examiner, saying it was not a random sample, ignoring the fact that I said about three thousand times that it could not be considered a public opinion poll because it was not a random sample. But at least that gives him a reason not to defend the PNAS paper.
    Dirk still wants to split hairs on the papers Oreskes missed, but doesn’t want to defend the PNAS paper.

    But finally–at last! New York wants to defend the PNAS paper. Oh–no he doesn’t. He wants to ignore every criticism I have made and for me to do what he wants.

    And then dhogaza gets to make up more stuff about me.

    So far, you all are not impressing me very much at all.

    The type of defense I expected to see:

    The authors considered the issue of sample bias that could be introduced by the use of a single database from a commercial source that has no academic supervision and has published no quality standards. The authors rejected these issues for the following reasons:

    The authors considered the issue of regional bias that could have been introduced by conducting their search in English only. The authors rejected this issue for the following reasons:

    The following controls were used to verify number of publications and citations attributed to individual scientists:

    The following analysis was made of the various opinion surveys used to label scientists as either UE or CE:

    Instead, what I have received here boils down to:

    1. Fuller is a big ugly meanie.
    2. I disagree with Spencer Weart so that is enough of a defense.
    3. Side issues that have nothing to do with this should be explored at length.

    Oh, and Michael, you say “Nobody is saying this is a massively important paper, just that it’s interesting enough to merit publication.”

    But exactly how much ink and prominent top of fold exposure have you given it? Why is that? Don’t say it’s because of skeptical criticism–skeptics criticize many things you manage to ignore.

    What other scientific publications have you feature prominently recently?

    The presence of error doesn’t mean there is no bias. Indeed the presence of error means the survey’s authors should work twice as hard at assuring readers that the error did not in fact introduce or reinforce bias.

    They did about as much work as you and your commenters here. The moral equivalent of zip.

    But I guess that’s okay, as long as you can use the headline for two weeks and remind people that the list is out there somewhere, and they sure don’t want to get their names added to it.

    Michael, the next time you complain about the state of communications about science, please remember this episode. You are a scientist trying to communicate with a journalist in front of an audience.

    Were I in your position I would be embarrassed at the amount of time and attention you have spent on this, let alone the way it is concluding.

    1. The standard method for dealing with datasets of uncertain provenance and/or reliability is duplication and control checks. Why didn’t they use more than one database? Why didn’t they look at what databases returned for scientists they could check with? In any event, where is their discussion of the decisions they made regarding this?

    2. “Very likely?” Is that your opinion, their opinion or is it printed in a paper somewhere? Again, where is their discussion of methodological choices?

    3. “There is no reason to assume Google Scholar has a bias…?” Again, is that Marco, the authors, a published paper somewhere? And again where’s the discussion of the choices they made.

    4. By ‘Supplementary information’ I presume you mean the Supporting Information listed along with the paper. If so, I did read it. I see a list of the petitions they used. I see the assertion that signing these petitions indicated that the scientist strongly dissented from the IPCC. I see no discussion of any homogeneity or heterogeneity amongst the various petitions. I see no description of how and when these petitions were presented, in what venues, using what methods. I see no discussion of how any of these potentially confounding factors could bias the sample of UE scientists (and the same is obviously true of CE scientists as well).

    It’s amateur hour stuff.

  17. Tom, you claim “Being public is not the same as being labeled as a climate denier by an IT administrator and a grad student doing what they call ‘research.’ ”

    Tell me, where do Anderegg or Prall use the label “denier” ?

  18. Check the metatag for the paper, Marco. They chose it.

  19. They labelled the paper ‘climate deniers.’ And you don’t think they labelled the people they single out as having signed petitions as ‘climate deniers.’

    • Yes, I don’t think that. This is not how I read it, even though I have no problem calling people deniers.

      That it can be read different should tell you something.

  20. Marco, as you may perhaps recall from our discussion of this topic at Only In It For the Gold, where you raised the same point, the supplementary information for this hit job leads to Jim Prall’s website, where he lists “Note: My list includes all 37 people covered in Lawrence Solomon’s book The Deniers. I’ve noted “LSDeniers” beside these 37 names.” As I wrote then and repeat now, he has violated the anonymity of his research subjects and his paper, in addition to being unforgiveably sloppy in data collection and completely erroneous in its statistical analysis, is also unethical in its treatment of its research subjects. It is a hit list of people he wants to paint as deniers, despite the fact that many of the people listed are not in fact deniers.

    And Thingsbreak chooses this cesspool as his first cite for criticizing George Will? He’s making Will look positively respectable, which I doubt ws his original intention.

    • You do know who Lawrence Solomon is, right? He is one of those I would call, without any hesitation, a climate denier.

      And those people who signed petitions have already given up on their anonimity. The whole purpose of openly signing petitions is to let others know what you think! How can it be a problem they are linked to petitions they themselves have willingly and openly linked themselves to?!

  21. I call your attention to http://www.eecg.utoronto.ca/~prall/climate/climate_author_photos.html

    On JIm Prall’s website, linked to in the supplementary information of his ‘paper.’ It has photos of scientists with red brackets around those he feels are skeptics. I guess red brackets are easier than bullseyes.

    Among those he has picked out of the hat as deniers:

    Roger Pielke Sr., I guess for the thought crime of writing, “Although the natural causes of climate variations and changes are undoubtedly important, the human influences are significant and involve a diverse range of first- order climate forcings, including, but not limited to, the human input of carbon dioxide (CO2).”

    Also included is John Christy, who wrote of this paper, “I was one of only three scientists who made both the “good guy” and the “bad guy” lists. Quite an honor I suppose. However, I think the study was pathetic. It basically says, “Those of us who agree with each other like to cite the work of our friends and not the other guys.” Duh. (One of my fellow scientists calls this “tribalism” – an appropriately primitive description.) I think the more sinister motive was evident in that the paper chided the media, such at the SF Chronicle, to stop investigative-reporting and just “trust us” (the guys on the “good guys” list) when it comes to climate change. It really was an attempt to make a blacklist. In that sense, I guess I ended up being gray, which fits my hair color now.”

    These bad people wrote a paper relying on research they were not qualified to perform. They did it with the intent of further marginalizing those who disagree with their position. They did not collect data correctly. They did not analyze it properly. They came to the wrong conclusions. But they linked to a list of people that one of the authors had already decided before the research had been conducted were climate deniers. And they linked to his list.

    George Will has never done anything as bad as that.

  22. A few quick points-

    The “black list” nonsense, the complaints about using a book title as a category, the Pielke Sr. idiocy are all silly, for reasons I’ve listed many times before and am happy to so do so again when I have a bit more time. These are emotional complaints and have nothing to do with the actual paper. Anderegg is not my “first cite”, it’s listed first in the References because they’re arranged alpha.This is something that someone who actually read the post rather than skimmed it would realize, Tom.

    I am happy to look at the few substantive issues you’ve brought up, but so far none of them seem to be meaningfully challenging the conclusions of the paper.

  23. Every one of the issues challenges the conclusions of the paper. If you can actually write the opposite, fair readers will be forced to speculate on either your intellectual honesty or your intellectual capacity.

    If you write a paper based on incorrect data and you use improper means of analysis and have pre-determined the conclusions in advance of beginning the process, your conclusions are not valid.

    It used to be we liberals said the facts were on our side. I want that back. You’re kicking mud on it. Tell me in what way you are different from George Will.

    • Every one of the issues challenges the conclusions of the paper.

      From a glance, your complaints look like possible sources of error, not bias. Like I said, I will give them a read when I have a bit more time, though.

      It used to be we liberals said the facts were on our side. I want that back. You’re kicking mud on it.

      Tom, I couldn’t care less what you want in terms of your general political preferences.

      Tell me in what way you are different from George Will.

      George Will is deliberately ignoring the body of scientific evidence, the primary literature, etc. By contrast, I am relying on them. Now, it’s certainly true that you happen to strongly dislike a specific paper, but that has absolutely zero bearing on the distinction between my approach and Will’s.

      This is something that a bright elementary student could understand, Tom. Your false equivalences are boring.

  24. No, Thingsbreak, you are not relying on the body of scientific evidence. You are trying to substitute Lysenkoism for it. You are doing the same thing as George Will–undermining a century-old, well-functioning approach to explaining our universe with dogma.

    At some point, do more than have a glance at what is being written. Or is that beneath you?

  25. And note, I didn’t write about bias here. I am writing about junk science.

  26. The suggestion that the Anderegg et al paper breaches confidentiality is so utterly laughable that I stopped reading and started skimming. Just as well, because there were 50 more pages of rehash, including a very lengthy and utterly irrelevant section of an ethics manual. Next time you want to refer to blog comments, maybe you could do it by linking?
    As far as I could tell, all that deathless prose included a single valid point, about subsampling bias (the 50-from-472 vs 50-from-903 issue). Someone should probably revisit Anderegg et al to correct this. My impression from eyeballing some of the data is that it won’t make much difference to the conclusions.

  27. Nick Barnes, maybe you shouldn’t have skimmed. Anderegg et al does something far worse than breach confidentiality. It classifies signatories to public documents as climate change deniers and provides a mechanism whereby interested observers can find out who Jim Prall thinks is a denier 20 years after they signed a petition saying the science wasn’t settled.

    Unless you do social research you may not be aware, but that’s incredibly unethical.

    The fact that they made every mistake in the book regarding social research apparently doesn’t matter to you–obviously you find the answer they arrived at congenial.

    As for providing links, many consensus junkies love to do that. And then they love to lie about what’s past the link, knowing that few follow it. Any errors in what I wrote are right here for all to see.

    Or you can smugly ignore the crap your team shovels out the barn door and just skim.

    • No, Tom, it did NOT label signatories of certain public documents as deniers. YOU DID! They labeled them UE (Unconvinced Experts).

      And as some have noted, if Prall were to put Pielke Sr in the CE category, it would make the UE group even less prominent. But we also know Pielke Sr has repeatedly stated he does not agree with the IPCC and claims other processes are more important than greenhouse gas emissions. That makes him an UE in my book, and I think in many other people’s books, too.

      • Marco, I love the way you go back and infill the comments section with your drivel days later. I don’t care about your book. Pielke has been no more critical of the IPCC than James Hansen, who is not listed as a UE. Why is that?

    • The unspeakably tedious ethics manual section you posted concerned confidential information. Please identify a piece of confidential information contained in Anderegg et al’s paper. Just one will do.

  28. Marco, no matter how many times you deny it, the paper is tagged Climate Deniers. Prall’s website labels them climate deniers.

    Hansen has repeatedly stated he does not agree with the IPCC. Why is he not on the list?

    Puerile hand-waving.

    • Tom, no matter how much you pontificate, YOU called them deniers, Anderegg et al did not. And Hansen isn’t on the list because he has not signed any of the public statements that contradict the IPCC. For someone who has so much criticism on the paper, you sure make it sound you actually haven’t read it very well…

  29. Nick Barnes, apparently the ethics manual proved too tedious for you. Prior to Section 11, which indeed does deal with confidential information, the manual says,

    “Sociologists refrain from undertaking an activity when their personal circumstances may interfere with their professional work or lead to harm for a student, supervisee, human subject, client, colleague, or other person to whom they have a scientific, teaching, consulting, or other professional obligation.” That has nothing to do with confidential information.

    The subjects of their research–the scientists on the list–risk grave harm as a result of this paper. They are labeled as deniers in an environment where that is a prejudicial term.

    • So you admit that section 11, concerning confidentiality, was utterly irrelevant and that pasting it in here was a waste of everyone’s time?
      You’re left with a sentence about the “personal circumstances” of the researchers. To which personal circumstances of Anderegg et al are you objecting?
      It seems to me that the most you’ve got is a gripe about Jim Prall’s website. Not about the paper.

      • No, Nick Barnes. Prall’s website is beneath contempt. The survey was poorly designed, collected data inaccurately, analyzed incorrectly and reported unethically.

        Any questions?

    • Tom Fuller, please explain why signing a public statement that openly contradicts the prevailing consensus is much more benign than being used as a subject in a paper, where you have to go deep into the Supplementary Information to even find the names.

      Oh, and I am 100% certain that the howls of “hidden data” would have deafened us, if the names were not somehow available. And likely including a howl from you.

      • Again Marco, which of the papers contradict the IPCC?

        And having an enemies list constructed for you and linked to a hate-site isn’t data. But it’s convenient for Joe Romm, when he ‘instructs’ the media not to give any of them airtime.

  30. Nick, I have to admit, you’re recycling Gavin’s handwaving pretty well.

  31. This is just me procrastinating by wasting some time in blog comment threads. The ones which do microscopic entrail-reading on the tedious bowels of other comment threads are the absolute best for sucking up a spare hour or two. If Gavin got there first, I’m glad to be in his company. He’s a smart guy, and also an actual scientist – I just play one on the internet. Assuming you’re referring to Gavin Schmidt.

    Other readers can judge for themselves whether it’s “handwaving” to point out the total irrelevance of a large part of your argument. Or can observe you changing the subject (from confidentiality to “personal circumstances”).

    For anyone just joining this fascinating post-match analysis, the subject is George Will, and his repeated failure to engage with the science of climate change.

  32. Nick, you’re welcome to waste as much or as little time on what I pasted in here as suits your fancy. Thingsbreak asked me to provide backup for what I said and I did so for him.

    Gavin may be a great scientist. He sure as hell doesn’t know doodly about sociological research. Neither did Schneider (requiescat in pace). Neither does Prall. Neither does Anderegg.

    There is a duty of care with regards to research subjects. It is unethical to reveal their identities or to make it possible for their identities to be revealed.

    Quick example: Climate science denial is toxic to a career in science, unless you have reached the level of seniority that a Lindzen, Nobel Prize winner Ivar Giaevar, or Freeman Dyson has.

    It is equivalent to revealing HIV status of someone in a high profile profession.

    Were I to conduct a study as sloppily as Anderegg et al did, and screwed up the math in the analysis as thoroughly as they did, and came up with a result that incorrectly identified Nick Barnes as HIV positive, would it comfort you if they said in their defense that you published comments on the internet and hence were a public figure and fair game?

    I doubt it.

    Why is it that you folks think Spencer Weart is such a great go-to resource but cannot accept his very clear and correct dismissal of this garbage?

    • Tom, people like Lindzen, Spencer, Christy are repeatedly called deniers, by a large group of people. Somehow, however, in Tom Fuller’s world, the problem is that they are labelled UE (which TF equates to “denier”) in a scientific paper, and THEREFORE will have problems in their professional scientific career. Complete nonsense.

      I’ll make it even better: come with ONE example from the list who would or has had a problem because he has been labeled a “denier” (according to you) in Anderegg et al, and not because he is widely known as a “denier” (in the actual sense of many using that label).

      I’m expecting to hear loads of crickets rather than a direct response to that request.

      • Some day you’ll learn how to read, Marco. I specifically wrote that those scientists are not part of the threatened group because they have achieved adequate seniority and/or tenure. This crap paper is the written equivalent of the No Pressure video. It is aimed at the younger generation.

  33. So we’re changing the subject again, from “personal circumstances” to “duty of care”. All rather distant from George Will and his continuing refusal to acknowledge or engage with the science, but OK.

    If, by your reference to Weart, you mean this: http://www.skepticalscience.com/How-many-climate-scientists-are-climate-skeptics.html#16811 (look, a link!) then first of all I would point out that you are using a somewhat bogus appeal to authority. Weart is an authority on the history of climate science, but not on the sociology of climate research. Secondly, I agree with everything Weart says there. I have no doubt – and I don’t suppose Weart does – that the effects described by Anderegg are real: the great majority of climate researchers who express an opinion on AGW express substantial confidence in the IPCC reports (yes, including Jim Hansen), and those who dissent are, on average, less published. This is hardly surprising, given that the IPCC process is exactly intended to identify the current state of research, including broad consensus where it exists. But I think there are some significant flaws in the Anderegg paper, and “more research is required”.

    You, on the other hand, continually attack the paper, largely on the basis of issues *which are not present in the paper* (and partly on the basis of issues, such as confidentiality, which are entirely irrelevant to the research). Specifically, *the paper* does not identify any of its subjects.
    So your main criticism is actually of Jim Prall’s website.

    Secondly, HIV status is a confidential matter, entirely different from published opinions. However, if I had signed a letter published in the New York Times saying “we, the undersigned, are HIV positive”, then I would have few grounds for complaint if someone later counted the people who had signed such letters and reported the total number in a paper (or even if they published my name on a website in an “HIV Positive column”). Someone signing a letter saying that the consensus is mistaken can have even less grounds for complaint when they are counted as “Unconvinced by the Evidence”.

    Lastly, you say “Climate science denial is toxic to a career in science”. That may be true, at least of careers in climate science (by analogy, I would expect evolution denial to be toxic to a career in biology or geology, but probably not a problem in chemistry or, for that matter, climate science). But can you demonstrate it? Can you identify the promising young careers blighted by the scourge of Jim Prall’s website?

    • By “more research required”, I would specifically be more interested in statistical studies of consensus/contrarian science publications. These petitions and open letters and so forth don’t interest me very much, because anyone can drum up a few hundred cranks to sign any damn thing.

  34. Nick, you should consider running for the U.S. presidency as a Republican. Your staunch refusal to even look at the science part of this is laughable. How do you feel about vaccines and evolution?

    At some point someone will lead you aside and gently explain to you that citing authorities is not the same as appealing to them.

    Your own counter-example displays your inability to grasp the implications of the idiocy of Anderegg, Prall, et al. A correct analogy would have been you, not HIV positive, signing a public letter against discrimination against those who were HIV positive and being labeled as HIV positive as a result. You still don’t seem to grasp the magnitude of their error. They published a paper with a metatag of climate deniers that include the names of scientists who are not deniers.

    Perhaps someone who understands mathematics will interpret RomanM’s damning analysis of their analytical errors. Perhaps someone with an 8th grade education will explain how their search parameters were poorly constructed. And someone with a bit of college might be needed to show you why Weart’s criticism is utterly damning, rather than just an aside in the comments of a weblog. Or perhaps you will grumble along thinking you actually understand what is going on here.

    You like the conclusion. Hence, you defend junk science. You don’t realize the implications of your act–not surprising, given the many things you have shown you don’t understand.

    If this is the standard of science to which the climate consensus aspires, it is no wonder that there are so many skeptics–indeed it is surprising there are not more. Perhaps most surprising, I still am not one.

    Barnes, Anderegg, Prall et al is not your friend. I am not your enemy. I do not think you a fool. But you are certainly acting like one. Grow up, accept reality and move on.

    It would be amazing if climate scientists who support the consensus did not have more publications and a wider base of citations. It’s a pity that Anderegg, Prall et al did such a shoddy job that they did not show it. But no-one–not skeptic, not lukewarmer not consensus acolytes such as yourself–disputes that.

    Had these bozos done their job we could have moved on to the more interesting conversation about why and what it means. Sadly, these fools were just as wedded to the consensus as you seem to be and they made the same fatal mistakes that you seem to be making.

    You do your cause no favors when you defend this tripe.

    • Tom, Anderegg et al explain at length several potential issues with their search terminology. You, on the other hand, have come with loads of handwaving, such as “they used google scholar, rather than other databases”. Anderegg et al actually already comment on that in their paper!

      You also claim that publications published in languages other than English are missed. Sure, that is the case, but is there a reason to assume that this will be markedly different for UE vs CE? You claim it does, but provide zero evidence for that claim. To me, that sounds like puerile handwaving.

      Finally, it now seems you also complain about the conclusion being logical. This may be true (indeed, I would not have expected otherwise), but here’s where the “climate deniers” label comes in: there is a group of people with significant political influence who cite a certain subgroup of scientists (the UE’s!) as evidence there is no consensus. Anderegg et al shows that this impression is due to very, very selective ‘citation’ of a very, very small group.

      • Their choice of search engine is actually the least of their mistakes. Seems to be the one you want to refer to. Perhaps you are incapable of understanding the others.

      • Their use of search engine is just one of many complaints you had, but which you failed to substantiate (as in “it matters, because [insert your own analysis]”). In the scientific world this is known as unscientific behavior. By you, just to make that clear.

      • Marco, since you are such a passionate defender of this pathetic hit job, I’m interested in your reactions to RomanM’s objections to their analysis scheme. Do you think Anderegg et al, used the correct approach statistically? If so, why?

        Similarly, please give us all your opinion on why this paper represents the state of science about the science for the world. They did not search in any language but English. Why do you think the results would be accurate?

        Tell me, Marco, why did they get the number of papers and citations wrong for one of the paper’s co-authors? Why did they fail in describing the career specialties of some of the biggest names in science?

        Would love your answers to these questions. I have others.

    • Oh, and Tom, you fall squarely in the climate obfuscator and tone troll category. Just see how you still have not answered TB’s questions:

      http://thingsbreak.wordpress.com/2011/09/08/the-conservative-face-of-science-and-the-role-of-consensus/#comment-3429

      But since you are so willing to pontificate about Anderegg et al, I guess that failure to respond is just a tacit admission that you actually uttered a load of nonsense on WUWT, for which there is no defence.

      • Read the thread, fool. I explained to thingsbreak why I did not answer those questions. Oh, yeah–sorry. It’s written in plain English.

      • Tom, your dismissal was just an admission of failure to substantiate your claims. On Anderegg et al you essentially do the same: throw lots of things in the air, but hardly substantiate most of them, and get all upset when it is pointed out to you. Obfuscation and tone troll into one.

    • “They published a paper with a metatag of climate deniers that include the names of scientists who are not deniers.”
      No they didn’t. Their paper includes the names of none of their subjects. You’re just lying now, so I’m going to stop.

      • You could have stopped any time. Their supplementary information links to Prall’s garbage dump where there are not only names but pictures.

  35. Marco, you were doing this handwaving a year ago. Now it is not only silly and wrong, it is stale. Barnes is acting like a fool. You are not acting.

    • So, asking for substantiation is “handwaving”? Gee, in the world of SCIENCE, substantiating your claims is actually the most important aspect. If you can’t, you add qualifiers (like Anderegg et al did).

      It is clear you are the one doing the handwaving. No surprise you consider me a fool, in the alternative universe you populate, people like me must be a real problem.

      • No, Marco, you are not a problem. You are just wasted time for me when I respond and for everyone else when they read.

      • For example, Marco, one year later you still have not addressed my comments to you on Tobis’ blog:

        Marco:

        1. The standard method for dealing with datasets of uncertain provenance and/or reliability is duplication and control checks. Why didn’t they use more than one database? Why didn’t they look at what databases returned for scientists they could check with? In any event, where is their discussion of the decisions they made regarding this?

        2. “Very likely?” Is that your opinion, their opinion or is it printed in a paper somewhere? Again, where is their discussion of methodological choices?

        3. “There is no reason to assume Google Scholar has a bias…?” Again, is that Marco, the authors, a published paper somewhere? And again where’s the discussion of the choices they made.

        4. By ‘Supplementary information’ I presume you mean the Supporting Information listed along with the paper. If so, I did read it. I see a list of the petitions they used. I see the assertion that signing these petitions indicated that the scientist strongly dissented from the IPCC. I see no discussion of any homogeneity or heterogeneity amongst the various petitions. I see no description of how and when these petitions were presented, in what venues, using what methods. I see no discussion of how any of these potentially confounding factors could bias the sample of UE scientists (and the same is obviously true of CE scientists as well).

        It’s amateur hour stuff.

        Marco, sorry I can’t stay and play today, but note the last thing you said. You (I’m sure unintentionally) showed the potential evil of this. It doesn’t differentiate between, as you said,

        ” all the petitions had to do was to question the IPCC conclusion. Who cares whether one said “yes, plenty of warming, but there is no need to do anything”, whereas another said “the IPCC is a bunch of frauds”.

        So now you have people on the same list with widely divergent beliefs. That list will be used to hurt some who are not skeptics, don’t want to be called skeptics and won’t get jobs or funding because they are on that list.

  36. “(g) When using private information about individuals collected by other persons or institutions, sociologists protect the confidentiality of individually identifiable information. Information is private when an individual can reasonably expect that the information will not be made public with personal identifiers”

    Did Tom “publish the confidential emails of climate scientists” Fuller really post this paragraph?

    “I’ve been doing this type of research for 15 years and I have never–never–seen the privacy of research subjects treated in such a cavalier fashion.”

    Tom, you wrote the book on privacy violation!

  37. Quite Waters, your liquids may be quiet but they are not deep. As I have mentioned here and elsewhere, although I was the first reporter to receive the leaked emails, I refused to publish them and explained my refusal in print.

    I only began to write about them after Gavin Schmidt did. He quoted them, he pointed to where they were on the internet, he referred to them while trying to defend them. So I did too.

    Sorry if that offends your sensibilities, but I have nothing to apologize for in terms of my actions.

    • Tom, you discussed those e-mails in a book with the express intent to malign the authors of those e-mails. Clearly, the book was meant as a blacklist.

      • Tell me where in the book I maligned the authors.

      • C’mon Marco,

        Tell me where in the book I maligned the authors. Or alternatively STFU.

        Do you always make stuff up or is that just reserved for people you disagree with?

  38. Tom, you can either bleat on about ethics or you can write books using stolen emails. You can’t do both. Your words mean nothing when your actions prove you really think ethics is the county north of suthics. The fact you continually claim you have nothing to apologise for “’cause Gav done it first” merely highlights your lack of moral fibre.

  39. Blah, blah. So Gavin can quote, refer and point to the emails but I cannot comment on them after they have been published on the internet?

    At some point, I’m sure you’ll try and connect this to what Anderegg, Prall et all did. I don’t know when, but I’m sure you will.

    Moral fibre! That’s what we need. Nature’s broom!

    • “So Gavin can quote, refer and point to the emails but I cannot comment on them after they have been published on the internet?”

      So Durkin, NIPCC, Solomon, Cato et al. can state the position of scientists regarding climate and publish them on the internet but Anderegg et al. can’t use those data for a paper?

      At some point, I’m sure you’ll try and connect what Anderegg, Prall et al. did to the OP. I don’t know when, but I’m sure you will.

      • Gee, I’m sorry Quiet Waters. You for some reason didn’t answer my question. Amazingly enough, you attempt to create a false equivalency in your reply. Wotta surprise.

        Durkin, the NIPCC, Solomon, Cato et al failed to do what Anderegg, Prall et al did. They did not create a website with a section with pictures of scientists mislabeled as deniers, and then write a paper that would not have made it through a high school science class that had as it goal leading readers to that website.

      • Quiet Waters

        “Durkin, the NIPCC, Solomon, Cato et al failed to do what Anderegg, Prall et al did.”

        And Gavin failed to do what you did – publish a book using other peoples personal correspondence.

        Tom, did you ask the permission of Mann, Jones, Briffa etc. to before publishing their private emails? Did you interview any of the correspondents to check the context before writing commentary purporting to expose the thinking & reasoning behind their words?

        If you can’t answer yes to either of these questions then your complaints about the ethics of Anderegg et al. smack of hypocricy.

      • Quiet Waters, I don’t know if you are incapable of understanding English or just unwilling to let go of this bone.

        As I wrote above, I did not publish the emails. I received them before anyone else and did not publish them. I wrote a column on Examiner.com explaining why I would not publish them. In fact, I never published them. When they were published by someone else online and Gavin began quoting them, referring to them by the number that had been assigned to the email by the person who published them and began discussing them, I began discussing them too.

        But if it comforts you to retain your personal boogeyman under your bed, just go ahead and pretend you never read this.

        As for the book Mosher and I wrote, we interviewed quite a few scientists off the record. None of the principals involved responded to our invitation to be interviewed. Imagine that.

        You can retain whatever image of me suits your paranoid fantasies, but Stephen Schneider said I gave a very fair and honest interview with him. But that doesn’t fit your worldview either, does it?

  40. Tom, obviously you feel very strongly about the ethical lapses of the researchers involved. As a responsible professional, you have of course behaved responsibly, to ensure that you did not misperceive this, and that you do not idly stand by when a wrong was committed. Of course, you consulted with a sociologist with a background in research ethics, to ensure that you were not making accusations falsely. What advice did you get? Then, of course, you contacted the American Sociological Association Committee on Professional Ethics, (http://www.asanet.org/about/ethics/COPE.cfm) to lay your complaint. What response did they provide? As the authors are/were not sociologists, and don’t seem, to my untutored eye, to be committing sociology, why are you giving the ASA code of ethics? Again, what feedback did you get from your contact on research ethics? Because, of course, it would be irresponsible jailhouse lawyering to make such statements without strong grounds.
    A link for those who aren’t Tom.

    http://www.patheos.com/community/slacktivist/2011/09/23/they-dont-really-believe-it/

  41. Stewart,

    I do feel very strongly about the ethical lapses of the researchers involved. I am a responsible professional.

    But you’re being a weasel about the rest of it. It was the responsibility of Anderegg, Prall et al to insure that their actions were ethical, that their paper documented their efforts at protecting the identities of their research subjects and the steps they took to insure quality control.

    As a responsible professional, I did not need to consult with other professionals in the field (although I did–their reaction was as incredulous as mine at the quality of the work). I am equally grateful for the like-minded opinions of professionals such as Spencer Weart and RomanM.

    And thank you for making my point for me. They were not professional sociologists, and they clearly didn’t consult any. That doesn’t excuse them for their behaviour.

    As for jailhouse lawyering, it is your tin cup beating on the bars, not mine.

    Still waiting for someone–anyone–to address any of the substance of the criticism offered here, instead of 8th grade insinuations about my motivation.

    Stewart–you’re here, you’re happy, you’re healthy. Take a stab at it. What is incorrect about RomanM’s criticism of their sampling techniques. What is wrong about Spencer Weart’s dismissal of the paper as unworthy of publication?

    Where in Anderegg et al do they address areas of their research that could have introduced or reinforced bias, ranging from single language search to using a commercial database without published quality control standards, to not checking results against other freely available databases, to simple error checking of results, such as the number of papers published by one of the co-authors?

  42. Tom:
    Thank you for your interest in increasing my publications – but I’m not interested, I have quite enough work to do that I am interested in, and that I can do better than others. The good thing about science is that, if someone is interested (that is, they think they can do substantially better, to challenge or extend ANY findings), they can do the work (especially on this paper), taking any criticisms or suggestions into account and see what happens. Thus we can have climate science overturned every week, if you get caught up in the narrative and don’t look at the actual papers. Of course, the evidence is that our understanding is improving, and the details are getting filled in, but the outline is not substantially changing.
    The narrative (which is all about gossip, group conflicts, etc) is the enemy of understanding, because it stops us thinking about the evidence. Will stepped out of the narrative for a few minutes, but went back in. Griping about Anderegg is fine, and you can complain about the weather too, if you like – but science is about challenging or extending it in the literature. No-one is being stopped from doing it better if they want to, and the best people to do it are those with strong feelings about it. In science, your worst professional enemy is your best ally.

  43. Stewart, I endorse your description of the theory of scientific publication and its role entirely. I wish the practice lived up to it. On a longer term scale, I think it is still valid. However, on a decade-length point of view, I think fashion and the follies of group think conspire against your Platonic idea.

  44. Sorry, Tom.
    As a practicing scientist, who reviews, submits, and gets frustrated at times, I strongly have to disagree. From the reviews I see, you are mistaking noise for signal (with a lot of folks out there trying to pump up the noise), focusing on a few complaints from partisans, and ignoring the big picture. Get away from the narrative, and look at the data.
    What has been the evidence over climate change, drivers, and effects in the past decade? The earth is not cooling down, the effects are consistent with CO2 as the forcing, the sun has not been the driver, and the effects on agriculture and sea ice are as predicted, except actually slightly worse/faster. Parameter estimates are getting more precise, paleodata is being refined, and sinks for heat and carbon are being identified. This doesn’t discredit the process to me – it does discredit those who seem to think it’s all a conspiracy by a group of scientists to – do what exactly? Science is a full contact sport – but the arena is journals (including electronic journals) and conferences, not blog posts or other media.

    Do you follow the XMRV/CFS saga? Do you not see how science works? Of science reporters, Brian Deer is a hero of mine – he understands the process, sought expert opinion from mainstream science, found out why it was correct, and helped expose a liar and fraud. I’d like to see more of that. If you’re a journalist, perhaps you might contact the leadership of the NAS and ask what evidence they looked at as the basis for their statement on athropogenic climate change, and how they would revise that on the basis of new evidence. You could even ask your questions about group think, ideal vs actual science, etc. You might be interested in the answers, as would your readers.

  45. Hi Stewart,

    Thanks for your reply. (I’m an ex-journalist, btw–I am actually doing honest work for a living these days, if I can get away with feeble humor).

    I agree with you that the earth is not cooling down and that what is happening is consistent with climate change theory and CO2 as at least a strong component–I like Pielke Sr.’s qualification about other primary forcings, but they certainly don’t minimize CO2, despite what some say.

    I also agree that the focus and funding for climate science is narrowing bands of uncertainty, although I think there’s a lot of work to be done still.

    I don’t actually agree that the horrible weather events of the past two years is in any way linked to climate change, or if it is, that it is in any way something we are equipped to measure or quantify. Sorry–I know your mileage may vary on this.

    Perhaps most importantly, I think scientific papers such as the one under discussion in this thread provides ammunition for those who do believe there is some sort of a conspiracy out there, and serves to discredit the huge majority of climate scientists who are laboring mightily to find answers to really important questions.

    To be clear, I don’t think Anderegg, Prall et al is the work of a ‘conspiracy’, anymore than I think the very real errors in judgement and practice uncovered by the Climategate emails were evidence of a conspiracy. Both clearly were not, a point I have made repeatedly, but which gets ignored by the rabid partisans at both ends of the spectrum.

    Anderegg, Prall et al is just bad science captured in a bad paper that was seized on by people with a political point to make. One of the authors, Prall, was one of those people with a political point that he had already made on a website.

    People like Thingsbreak who tend to try and use scientific publications in support of their position do themselves no favors at all by citing junk science. This is especially true for people like Thingsbreak who actually sometimes have a point to make, unlike the troll/terrorist brigade so beautifully personified by some other commenters here.

    Anderegg, Prall et al do not call into question the work of good science. That does not mean that Anderegg, Prall et al should be tolerated, or that citing the paper should go unremarked.

  46. Pingback: Moving slowly and avoiding whiplash: John Huntsman and Climate Edition | The Way Things Break

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s