[eemax]

Of course science is dangerous. The study of atomic physics very nearly led to nuclear war.

> Dangers and ethical issues only arise when science is applied as technology.

I mostly disagree with this sentiment. If you're a nuclear physicist, and you publish a paper about the theoretical aspects of fission chain reactions, and realize that they may allow for good things, like nuclear power, and bad things, like nuclear weapons, you have an obligation to consider the consequences of how your research might be applied by others, do the cost-benefit calculation, and decide to publish, or keep your knowledge a secret[0].

Of course, in the vast majority of cases (GMOs, medical research, human intelligence, etc., knock yourself out) it's probably better to publish - the benefits far outweigh any costs. But I see no reason why this has to be universally, fundamentally true - maybe there are secrets of the universe too dangerous for publication - knowledge which, if made public, would predictably lead to great destruction.

[0]: http://arxiv.org/html/physics/0207094 Leo Slizard is usually credited as the first to come up with the idea of weaponizing nuclear fission. Slizard was in favor of secrecy, Fermi was in favor of publication, and for a while, they kept some of their discoveries a secret. In the hindsight of history, this was probably not necessary, but at least they weighed the consequences and had the debate.

[jorleif]

I feel this solves a very small part of the problem. This is only about the morality of the individual scientist, while the problem is at the level of the collective process of science. I mean, it is of course good if the individual scientist who discovers X way to kill off humanity keeps it secret, but eventually someone else will also discover X, and then someone else, and so on. Do we require of scientists that they all keep these things secret?

Of course, we may not want to have terrorists building atomic bombs, so some secrecy is prudent, but secrecy does not generally hinder: 1) Some bad people from accessing said information, now having monopoly, which may make matters even worse 2) Discovering the same thing independently - Yes, we may keep it secret, but that does not change the laws of physics, the exploitation potential is still there.

[harperlee]

It is a complicated matter... as long a science advances and is accesible to all, we all gain more power. That enables everybody to be dangerous. I can go anywhere in the world, access a huge amount of information, purchase and have sent home lots of things very cheaply, etc.

We crave power, both in the sense of ability and energy, and it seems to grow monotonically (Freeman Dyson proposed searching for efficient star-energy-capturing devices as a way to find ET life: http://en.wikipedia.org/wiki/Dyson_sphere).

But I agree, it seems that it is not very easy to prevent new science to be discovered. We are a curious species, and even if we go back to the dark ages, even if we move up or down the knowledge scale randomly, this brownian movement will end up jumping too high, and giving too much power to someone.

So at least we should incorporate an element of respect to such power in our education, our ethics, and try to prevent it from being lost. In my opinion, this kind of thing is most efficiently managed through religion, but current religions do not advance very quickly nor are amenable to "improvements", whereas the idea of scientific religion, to us, now, seems abhorrent and oxymoronic. I believe this might change in the future, though. If science "just works" (and it is indistinguishable from magic), I could picture someone stripping out the "scientific inquiry and skepticism" part, and building a religion on top of the huge building of true scientific facts that we faithfully take for granted.

So in my opinion we have to focus on education. Sorry for the rambling :)

[daniel-cussen]

Szilard, not Slizard. (Technically Szilárd, but definitely not Slizard...though I like that name for maybe a supervillain.)