[dantheman]

I also think a big part is how forthright a scientist is. Most scientists claim way more than can be supported by their evidence, and it's not always clear what assumptions are being made. There's a great talk, I can try to find it if there is interest, that discuss the hubris that most climate scientists have. Observing a system for a tiny portion of a cycle can in no way tell you the entire cycle. If you then admit that "hey we don't know, but here's our best guess, and here's why we think it" then you've got something. Once something is established like like copper conducts electricity or the speed of light then you can say "hey we're pretty sure about this". It has a lot to do with the maturity of the science and the tools and techniques available to the scientists. Complex systems that cannot be decomposed and upon which experiments can not be made have should have a much lower confidence value than repeatable tried and true experiments. To conflate them is to cause confusion and to weaken the trust in more established science. Peer review is more about how to position your work in relation to others for ease of understanding than to verify the results independently.

[thaumaturgy]

> Most scientists claim way more than can be supported by their evidence...

Ironically, you've made a claim here which is unsupported by any evidence at all. ;-)

Seriously though, the thing about this is that research and experimental scientists are working in environments that you and I know very little about. Their claims are an extension of their experience in that environment; they spend a great deal of time for example designing experiments or attempting to reconcile large volumes of data, and when they've spent months -- or, more likely, years -- doing that, they might publish a paper.

Then the public occasionally gets ahold of this brief summary of the scientist's work over that period, and they object that the claims are specious or sparse or incomplete.

This is not to say that a scientist's claims should be trusted just because they're a scientist, but rather that the only acceptable challenge to a shortcoming in a scientific presentation is ... more science, not argument or conjecture.

To put it another way: it is extremely unlikely that a layman will find a consequential error in a peer-reviewed scientific paper published in a reasonable journal, especially without any domain knowledge in that particular field.

[dantheman]

Errors in science are well known and are part of the process, I didn't think I needed to provide evidence of that. Unfortunately science that deals with complex systems is much more difficult than those that don't. For instance, just look at nutrition research, look at the history of the food pyramid and you see poorly understood science having a substantial impact on those who are not experts because the nutrition experts were scientists, and physics works well so nutrition must also be well understood.

I just want people to bend over backwards to tell me the ways in which they might be wrong, this is useful because it helps you understand the limits of understanding and can allow people to make informed decisions.

[thaumaturgy]

I wasn't being hostile (this time), I was just having a little fun.

What you're grappling with here is basically the "I don't know what I don't know" problem. Other people on HN have talked about this before, but basically, there's the knowledge you know, the knowledge you know you don't know, and the knowledge that you don't know you don't know. How can you possibly begin to challenge an assertion in a scientific paper if you don't even know where the mistakes might be?

(I'm going through something similar right now in a criminal case. I have no background in law. I do not trust the attorneys involved. It's maddening.)

I agree that it would be nice if as a matter of habit there were a brief statement in papers and presentations outlining incomplete areas of the research or areas justifying further study. However, I would not want that to evolve into an onus on the part of the working scientists to educate every layman with a passing interest in their research.

If you really want to challenge something in a specific field, the best thing you can do is immerse yourself in that field for as long as it takes to develop an understanding of the basic principles. Demanding much else is merely intellectual laziness.

[dantheman]

I didn't think you were being hostile, and I hope you don't detect any hostility from me, none is intended.

There are many incentives to overstate the validness of ones claims and unfortunately it weakens the overall strength of the field. I'm not asking that they make it understandable to the layman, but perhaps on any recommendation they could put confidence intervals on predictions.

Here's a presentation by Carl Wunsch, and here are the slides: http://web.mit.edu/esi/symposia/symposium-2009/2009-symposiu...

http://cdn.static.viddler.com/flash/simple_publisher.swf?key...

He's one of the worlds experts on the ocean, and is saying the same thing i am.

Here's a feynman quote on it:

I would like to add something that's not essential to the science, but something I kind of believe, which is that you should not fool the layman when you're talking as a scientist. I am not trying to tell you what to do about cheating on your wife, or fooling your girlfriend, or something like that, when you're not trying to be a scientist, but just trying to be an ordinary human being. We'll leave those problems up to you and your rabbi. I'm talking about a specific, extra type of integrity that is not lying, but bending over backwards to show how you are maybe wrong, that you ought to have when acting as a scientist. And this is our responsibility as scientists, certainly to other scientists, and I think to laymen. -- Richard Feynman, CARGO CULT SCIENCE

Also to be clear, I'm optimistic and hope we can solve all sorts of problems and discover how the world works -- but I think to we need to have integrity and maintain to foster and maintain that trust. The medical field is ripe with wild extrapolation from small studies, it's a sign of exhuberance -- "Hey, here's the answer to the problem you having!" When it should be, "It seems that in some situations this may help, we still have to do more studies and don't know why it works but it seems to." It's more honest and when things go wrong, which they almost always do the person understood that it wasn't well understood.

I encourage you to watch the video - it's by an expert saying that we don't have enough information to be sure about anything and he proposes the following advice:

In the meantime, study the system; study the control options; but----take precautions (mitigation), prepare for adaptation, and do nothing that is even possibly irreversible (which may be almost anything, including of course, the ongoing injection of greenhouse gases).

Also his example of gravity waves, I think is extremely enlightening.

[thaumaturgy]

I apologize for taking so long to respond -- I needed to get to a point where I had the time to properly listen to and consider the video. (So I waited until it was time to shave. Laptops are wonderful. ;-)

I went into the video with an open but critical mind, and attempted to stay that way even as I realized that he was presenting a point of view which I would not usually agree with. So, while I think he presented some casually interesting points, I also find it curious that he seemed to be committing some of the very same errors which you are criticizing others in the scientific community for committing, as well as other errors which even I -- as a layman -- was able to catch.

It's entirely possible that his presentation is merely incomplete, and that he has good reasons for drawing the conclusions that he was presenting, and he merely couldn't present those reasons to a non-scientific audience. (Which would be pretty much my side of the argument in this thread, so I'm totally willing to accept that possibility.)

I won't devote the time to a point-by-point analysis of his presentation, unless you express enough interest in continuing this to make it worthwhile, but in short:

+ He completely -- and carefully -- ignores various geological records which go back many hundreds of thousands of years, and which have a high confidence of accuracy in the climatological community;

+ Those records do present a regular semi-chaotic rhythm reminiscent of a strange attractor, and our era appears to currently be the zenith of that rhythm;

+ And if those records are accurate, then current measurements exceed any previous maxima of the last many hundreds of thousands of years;

+ He seems for some reason to draw the analogy that the climate is correctly modeled as a chaotic system of some sort -- i.e., one which is extremely sensitive to initial conditions -- even though this is still a matter of debate within the climatological community;

+ While the Titanic may arguably be one example where taking no action may have resulted in a more favorable outcome, coming up with or inventing countless counter-examples is trivial, so this does not support his case at all;

+ His "gambling models" proposal is quite silly, if for no other reason than that one of the biggest unpredictable parts of the climate are related to human activity.

A lot of the rest of his presentation seems to be an argument-from-ignorance. I think he is framing the discussion in such a way as to suggest we have less information and less knowledge than we actually do, and then within that frame, he is arguing that we haven't the knowledge to decide what to do.

But I'll think about it some more.

EDIT: Funny, I pulled up a random-ish TED video (searched for "climate"), and ended up watching a 4-minute presentation which is pretty much exactly the point I've been trying to make: http://www.ted.com/talks/lang/eng/rachel_pike_the_science_be...