[adamsmith]

Congrats on the fundraise Greg and team!

Does this mean that OpenAI may not disclose progress, papers with details, and/or open source code as much as in the past? In other words, what proprietary advantage will Microsoft gain when licensing new tech from OpenAI?

I understand that keeping some innovations private may help commercialization, which may help raise more funds for OpenAI, getting us to AGI faster, so my opinion is that could plausibly make sense.

[gdb]

We'll still release quite a lot, and those releases won't look any different from the past.

> I understand that keeping some innovations private may help commercialization, which may help raise more funds for OpenAI, getting us to AGI faster, so my opinion is that could plausibly make sense.

That's exactly how we think about it. We're interested in licensing some technologies in order to fund our AGI efforts. But even if we keep technology private for this reason, we still might be able to eventually publish it.

[DSingularity]

Eventually open ai?

[marvin]

I thought from day one that the name «OpenAI» would at best be a slight misnomer, and at worst indicative of a misguided approach. If AGI is close to being achieved, sharing key details of the approach to any actors at all could trigger a Manhattan Project-type global arms race where safety was compromised and the whole thing became insanely risky for the future of humanity.

Glad to see that the team is taking a pragmatic safety-first approach here, as well as towards the near-term economical realities of funding a very expensive project to ensure the fastest possible progress.

In the early days of OpenAI, my thoughts were that the project had good intentions, but a misguided focus. The last year has changed that, though. They absolutely seem to be on the right track. Very excited to see their progress over the next years.

[nradov]

Not to worry. No one is anywhere close to achieving true AGI so any safety concerns are a moot issue. It's akin to worrying about an alien invasion.

[DuskStar]

> No one is anywhere close to achieving true AGI

No one knows how far off true AGI is, just like no one in 1940 (or 1910) knew how far off fission weapons were.

EDIT: I quite liked this article from a few years back [0], and the fission weapon prediction example is stolen from there.

0: https://intelligence.org/2017/10/13/fire-alarm/

[umvi]

Really? I thought by 1940 physicists generally understood fission and theoretically understood how to build a bomb - they just needed to find enough distilled fissile material (which was hard to do). And indeed, once they had enough U235, they had such a high degree of confidence in the theory, that they built a functioning U235 bomb without ever having previously tested one.

[idlewords]

This is a flawed analogy. The conceptual basis of nuclear weapons was well understood as soon as it was learned that the atom has a compact nucleus. The energy needed to bind that nucleus together gives a rough idea of the power of a fission weapon. If that energy could be liberated all at once, it would make an explosive orders of magnitude more powerful than anything known.

It was hard to predict when or if such a thing could be made, but everyone knew what was under discussion.

Compare this to AGI, some vaguely emergent property of a complex computer system that no one can define to anyone else's satisfaction. Attempts to be more precise what AGI is, how it would first manifest itself, and why on earth we should be afraid of it, rapidly devolve into nerd ghost stories.

[marvin]

The Manhattan Project is a very apt analogy. Even if you believe that AGI is impossible, it should be possible to appreciate that many billions would quickly be invested in its development if somehow a viable pathway to it became clear. Even if just to a few well-connected experts.

This is what happened when it became known nuclear weapons were a viable concept. The technology shifted power to such an extreme degree that it was impossible not to invest in it, and the delay from «likely impossible» to «done» happened too fast for most observers to notice.

[drinane]

AGI = Alien invasion

[hoseja]

I wouldn't be so sure the Manhattan Project-type global arms race isn't already happening.

[z3t4]

The atomic bomb was based on science theory. A computer can run many programs and do a great many things, but it will never be able to think by itself.

[throwawaywego]

> The atomic bomb was based on science theory.

Our study of (automated) intelligence is based on science too.

> A computer ... will never be able to think by itself.

Turing wrote an entire paper about this (Computing Machinery and Intelligence), where he rephrases your statement (because he finds it to be meaningless) and devises a test to answer it. He also directly attacks your phrasing of "but it will never":

> I believe they are mostly founded on the principle of scientific induction. A man has seen thousands of machines in his lifetime. From what he sees of them he draws a number of general conclusions. They are ugly, each is designed for a very limited purpose, when required for a minutely different purpose they are useless, the variety of behaviour of any one of them is very small, etc., etc. Naturally he concludes that these are necessary properties of machines in general.

> A better variant of the objection says that a machine can never "take us by surprise." This statement is a more direct challenge and can be met directly. Machines take me by surprise with great frequency. This is largely because I do not do sufficient calculation to decide what to expect them to do, or rather because, although I do a calculation, I do it in a hurried, slipshod fashion, taking risks.

[MetalGuru]

> A better variant of the objection says that a machine can never "take us by surprise." This statement is a more direct challenge and can be met directly. Machines take me by surprise with great frequency. This is largely because I do not do sufficient calculation to decide what to expect them to do, or rather because, although I do a calculation, I do it in a hurried, slipshod fashion, taking risks.

This seems like a cop out. Sure, if you do your calculations wrong, it doesn’t behave as you expect. But it’s still doing exactly what you wrote it to do. The surprise is in realizing your expectations were wrong, not that the machine decided to behave differently.

[z3t4]

Investing into a bubble only to make sure the money go to yourself. Seems like a economic loophole. You think computers will start to have dreams and desires? Abusing such a machine would be unethical. Go ahead and build a better OCR, just don't fall to the AGI hype.

[ekianjo]

> Our study of (automated) intelligence is based on science too.

Can you elaborate which part of sciences you are talking about here?

[streetcat1]

It does not need to. It just need to get complex enough. This is from an 1965 article:

"If the machines are permitted to make all their own decisions, we can’t make any conjectures as to the results, because it is impossible to guess how such machines might behave. We only point out that the fate of the human race would be at the mercy of the machines. It might be argued that the human race would never be foolish enough to hand over all power to the machines. But we are suggesting neither that the human race would voluntarily turn power over to the machines nor that the machines would willfully seize power. What we do suggest is that the human race might easily permit itself to drift into a position of such dependence on the machines that it would have no practical choice but to accept all of the machines’ decisions. As society and the problems that face it become more and more complex and as machines become more and more intelligent, people will let machines make more and more of their decisions for them, simply because machine-made decisions will bring better results than man-made ones. Eventually a stage may be reached at which the decisions necessary to keep the system running will be so complex that human beings will be incapable of making them intelligently. At that stage the machines will be in effective control. People won’t be able to just turn the machine off, because they will be so dependent on them that turning them off would amount to suicide."

[dane-pgp]

I agree with the above, but imagine the same argument where "the machines" is replaced with "subject-matter experts", or "politicians acting on the advice of subject-matter experts".

The accumulated knowledge and skills of not just specialised individuals but entire institutions, working on highly technical and abstract areas of society, seems like it has created a kind of empathy gap between the people ostensibly wielding power and those who are experiencing the effects of that power (or the limits of that power).

> "... turning them off would amount to suicide."

Although this conclusion appears equally valid in the replacement argument, it sadly doesn't come with the wanted guarantee of "therefore that wouldn't happen".

[DuskStar]

> A computer can run many programs and do a great many things, but it will never be able to think by itself.

A computer being able to simulate a brain that thinks for itself is the logical extrapolation of current brain-simulation efforts. Many people think there are far less computationally intensive ways to make an AI, but "physics sim of a human brain" is a good thought experiment.

Unless you think there's something magic about human brains? Using "magic" here to mean incomprehensible, unobservable, and incomputable.

[ekianjo]

> A computer being able to simulate a brain that thinks for itself is the logical extrapolation of current brain-simulation efforts

Except that our current neural networks have nothing to do with the actual neurons in our brain and how they work.

[JabavuAdams]

That's quite a claim, considering that we don't know what the word "think" means.

[jjeaff]

"maybe" eventually openAI

[oaeide]

Maybe eventually openAI

[apetrov]

maybe eventually open maybe ai

[ru999gol]

does anyone serious (non-encumbered) actually believe that?

[partisan]

> We'll still release quite a lot, and those releases won't look any different from the past.

I don’t mean to parse your words, but will you continue to publish using the same exact criteria as before or will there be a new editorial filter?