|
|
|
|
|
|
by startupsfail
1180 days ago
|
|
|
You can only hold around 7 to 10 numbers in your mind well, in your working memory. Let me give you a few: 6398 5385 3854 8577 You have 1 second, close your eyes and add them together. Write down the result. I’m pretty sure that GPT-4 at its 4k setting would outperform you. [The point being, we have not seen what even GPT-4 can do in its optimal environment. Humans use paper, computers, google, etc. to organize their thoughts and work efficiently. They don’t just sit in empty space and then put everything into the working memory and magically produce the results. So imagine now that you do have a similar level of tooling and sophistication around GPT-4, like there is present around humans. I’m considering that and it is difficult to extrapolate what even GPT-4 can do, in its optimal environment.] |
|
|
I'll point out that chatgpt needs to be paying attention to the numbers to remember them in the way I'm taking about. You will need to fine tune it or something to get it to remember them blind. I suppose that's not what you're talking about?
There is a strong chance that I'll remember where to find these numbers in a decade, after seeing and hearing untold trillions of "tokens" of input. The topic (Auto-GPT, which is revolutionary), my arguments about biological complexity (I'll continue to refine them but the rendition here was particularly fun to write) or any of these things will key me back to look up the precise details (here: these high entropy numbers). Attention is perhaps all you need... But in the world it's not quite arranged the same way as in machines. They're going to need some serious augmentation and extension to have these capabilities over the scales than we find trivial.
edit: you expanded your comment. Yes. We are augmented. Just dealing with all those augmented features requires precisely the long range correlation tracking I'm taking about. I don't doubt these systems will become ever more powerful, and will be adapted into a wider environment until their capabilities become truly human like. I am suggesting that the long range correlation issue is key. It's precisely what uniques humans from other beings on this planet. We have crazy endurance and our brains both cause and support that capability. All those connections are what let's us chase down large game, farm a piece of land for decades, write encyclopedias, and build complex cultures and relationships with hundreds and thousands of others. I'll be happy to be wrong, but it looks hard-as-in-quadratic to get this kind of general intelligence out of machines. Which scales badly.