[evanevan]

Who should decide seems important. All of this is at the end of the day controlled by OpenAI right?

I wonder is there a way the AGI could be instructed to weight all of our preferences equally?

Seems vital so we end up with something in the interest of all of us and not something we just have to hope the few entities controlling AGI do the right thing, given historical precedent of the risks of entrusting systems that are supposed to benefit all of us to small groups too.

[ctoth]

The concept you are describing is called Coherent Extrapolated Volition.

> In poetic terms, our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.

[evanevan]

Thanks - yeah I think CEV hits the mark. Would be awesome if OpenAI (and others) committed to CEV then (with I'd add the qualification, equal weighting on each person).

Would be curious how this would play out for LLMs too. What do people actually want? I would guess most people think OpenAI is being too restrictive on what ChatGPT does, but would be curious to actually see that play out and how people are thinking about it.

[hprotagonist]

no entities control AGI because it doesn't exist.

[echelon]

Shortbets five years.

Even LLM weaknesses are just snafus. We've already "replaced" or made inroads at automating the jobs of artists, voice actors, copy editors, Go/chess/video game players, and more. I'm failing to communicate the breadth here.

Real actors, singers, manufacturing jobs, software developers, and a whole host of other types of tasks are shortly to come. As capital plunges in, there will be rapid maturation.

Each of these advances shows a comprehensive understanding of the complexity and nature of the task AI is applied to. If we keep piling these victories on, we'll have an approach for agency and consciousness in short order. Everyone is trying to solve this now, and we've made so much progress.

The pace of innovation is staggering and should shake the foundations of the models through which you understand and predict the world and the future. We're looking at a fundamentally different set of possible outcomes for 2030. It's almost impossible to predict, as our entire economic system may lurch forward in a next "industrial revolution".

[evanevan]

Not yet... but one day soon, no? Seems good to be starting to plan