[neilk]

Your viewpoint is fascinating. So the inventor of backpropagation, Turing award winner, Google researcher, mentor to the CTO of OpenAI, doesn’t have any special insights about AI and the tech industry that’s forming around it? He might as well be some guy off the street?

Who, in your opinion, _does_ have enough context to be worth our attention?

Because if you’re waiting for Sam Altman or the entire OpenAI team to say “guys, I think we made a mistake here” we’re going to be knee-deep in paperclips.

[meh8881]

Someone who is actually doing it would be a lot more authoritative in my opinion. Hinton has been wrong on most of his big ideas in the past decade. He hasn’t actually been involved in the important advances of anything recent. Inventing backprop is great. No discredit to him there. But that’s not a free pass to be seen as someone who is on the cutting edge.

But beyond all of that, what are we really asking? Are we asking about social ramifications? Because I don’t think the OpenAI devs are particularly noteworthy in their ability to divine those either. It’s more of a business question if anything. Are we talking about where the tech goes next? Because then it’s probably the devs or at least indie folks playing with the models themselves.

None of that means Hinton’s opinions are wrong. Form your own opinions. Don’t delegate your thinking.

[mitthrowaway2]

I'm surprised you'd consider Hinton as not being "someone who is actually doing it".

Are you basically saying that you only trust warnings about AI from people who have pushed the most recent update to the latest headline-grabbing AI system at the latest AI darling unicorn? If so, aren't those people strongly self-selected to be optimistic about AI's impacts, else they might not be so keen on actively building it? And that's even setting aside they would also be financially incentivized against publicly expressing whatever doubts they do hold.

Isn't this is kind of like asking for authoritative opinions on carbon emissions from the people who are actually pumping the oil?

[meh8881]

No, that’s the opposite of what I’m saying. Asking Hinton for his opinions on the societal impact of new AI tech is like asking the people who used to pump oil 20 years ago. It’s both out of date and not really relevant to their skill set even if it’s adjacent.

[mitthrowaway2]

Let me clarify: who does qualify to offer an authoritative opinion, in your view? If, say, only Ilya Sutskever qualifies, then isn't that like asking someone actively pumping oil today about the danger of carbon emissions? If only Sam Altman, then isn't that like asking an oil executive?

If not Geoff Hinton, then, who?

Ultimately the harm is either real or not. If it is real, then the people with the most accurate beliefs and principles will be the ones who never joined the industry in the first place because they anticipated where it would lead, and didn't want to contribute. If it is not real, then the people with the most accurate beliefs will be the ones leading the charge to accelerate the industry. But neither group's opinions carry much credibility as opinions, because it's obvious in advance what opinions each group would self-select to have. (So they can only hope to persuade by offering logical arguments and data, not by the weight of their authoritative opinions.)

In my view, someone who makes landmark contributions to the oil industry for 20 years and then quits in order to speak frankly about their concerns with the societal impacts of their industry... is probably the most credible voice you could ever expect to find expressing a concern, if your measure of credibility involves experience pumping oil.

[meh8881]

If you want an authoritative opinion on the societal impact of something I would want the opinion of someone who studies the societal impact of things.

[mitthrowaway2]

So that seems to me like someone like Stuart Russel or Nick Bostrom? But what Geoff Hinton is saying seems to be vaguely in general agreement with what those people are saying.

[Mike_12345]

LOL. Hinton won the f**ing Turing Award for his research in deep learning / neural networks, and you're telling us his knowledge is irrelevant.

[neilk]

Nobody was arguing that Hinton should be listened to uncritically. You were the one asserting that he should not be listened to at all.

With respect, you seem to be shifting goalposts, from the indefensible (Hinton doesn't know what he's talking about) to the irrelevant (Hinton doesn't have perfect and complete knowledge of the future).

[meh8881]

I didn’t say anything you’re suggesting.

[eternalban]

Authority figures will not matter. This technology, like nuclear weapons, will be pursued to the utmost by all actors capable of marshalling the resources, in secret if necessary. (After all, the 'Hydrogen bomb' was debated pro/con by established authorities, including Oppenheimer and Teller. Did that stop their development?)

https://en.wikipedia.org/wiki/Thermonuclear_weapon

https://www.simonandschuster.com/books/Dark-Sun/Richard-Rhod...

Today:

Germany's relevant minister has already declared at G7 that Germany can not follow Italy's example. "Banning generative AI is not an option".

https://asia.nikkei.com/Spotlight/G-7-in-Japan/Banning-gener...

US Senate has a bill drawing the line on AI launching nuclear weapons but to think US military, intelligence, and industry will sit out the AI arms race is not realistic.

https://www.markey.senate.gov/imo/media/doc/block_nuclear_la...

China's CPC's future existence (imo) depends on AI based surveillance, propaganda, and realtime behavior conditioning. (re RT conditioning: We've already experienced this outselves via interacting with the recent chatbots to some extent. I certainly modulated my interactions to avoid the AI mommy retors.)

https://d3.harvard.edu/platform-rctom/submission/using-machi...