Hacker News new | ask | show | jobs
by whoisthis12 1174 days ago
People keep mentioning we will have agi in our lifetimes but do we have any proper definition of agi? Are there any benchmarks to prove it?
6 comments

The resolution criteria for a Metaculus question[1] attempts to explain what a "weakly" general AI might entail. The question will be obsolete and the definition of AGI might need shifting if some things turn out more or less difficult than previously assumed though.

"For these purposes we will thus define "AI system" as a single unified software system that can satisfy the following criteria, all easily completable by a typical college-educated human.

    Able to reliably pass a Turing test of the type that would win the Loebner Silver Prize.

    Able to score 90% or more on a robust version of the Winograd Schema Challenge, e.g. the "Winogrande" challenge or comparable data set for which human performance is at 90+%

    Be able to score 75th percentile (as compared to the corresponding year's human students; this was a score of 600 in 2016) on all the full mathematics section of a circa-2015-2020 standard SAT exam, using just images of the exam pages and having less than ten SAT exams as part of the training data. (Training on other corpuses of math problems is fair game as long as they are arguably distinct from SAT exams.)

    Be able to learn the classic Atari game "Montezuma's revenge" (based on just visual inputs and standard controls) and explore all 24 rooms based on the equivalent of less than 100 hours of real-time play (see closely-related question.)"
[1]https://www.metaculus.com/questions/3479/date-weakly-general...
This is just my opinion.

The thing is - our ability is limited to our understanding. AI maybe already doing things we do not understand (which could be classified as AGI). Thinking of it like how dogs do not have the cognitive ability to understand the concept of the "future" or tomorrow - There is a good chance AI would already be doing things which are beyond our cognitive ability (including the smartest people working on the tech.)

But I am sure if we let 2 fairly good LLMs talk to each other - They'd shortly start talking things we feel are hallucinations but the 2 LLMs would understand and take it further.

Again, this is just my opinion and I have never worked on any LLMs. So an outsider.

> but the 2 LLMs would understand and take it further.

Some kind of future AI maybe but LLMs are not designed for that and structurally incapable of that

AGI can solve problems in any general context using its capabilities, including problems never seen before.

There's weak and strong AGI. Right now I'd argue we have weak AGI with ChatGPT and its plug-ins. The GPT-tech allows for general purpose problem solving, but it often makes mistakes and gets things wrong. We are on the path towards strong AGI, and I have no idea how long it will take to get there.

OpenAI and others are developing benchmarks to test this tech against various problems.

I wouldn't consider chatGPT by itself or with plugins AGI, however the langchain+long-term memory + automated agents, yeah that's f'n AGI at least as it's aptly called: Baby AGI.
It might be baby agi but as it stands I would argue it is not.
you realize, babies aren't as impressive as a PhD. right? I mean they're cute and impressive in how they learn and pick things up, baby AGI is kinda like that. the hype is basically going gaga around a baby.

but it will grow up and before you know it.

How could I not realize those things? Do you think I'm a moron?
I think personally we should also classify human level AGI and smart AGI. After all, the goal is not to make fake humans but to solve problems like cancer.
After watching how people are applying GPT3.5 and GPT4, I'd say both goals those types of goals will be attempted, and anything else that could be of use.
I’m not sure we have a proper definition of consciousness, so probably no.

But I suspect we’re five years from being able to create a self-directed simulation of a person that could behave in ways we consider autonomous. That is to say it will be able to fool most people observing from the outside.

I have some ideas on how I’d go about it so I imagine better minds than mine are hard at work on the problem.

I’m not sure we should be building that, but I’m sure we are.

An AGI will be able to improve itself - i.e. all the AI dev team at OpenAI will get fired, and only people policy/alignment people will remain (hopefully).
The simple definition is “its faster for a manager with medium computer skills to spin up an instance than to train a human for almost all desk jobs.” We look for other definitions because if an AI satisfying this definition showed up tomorrow, we’d all starve.