Hacker News new | ask | show | jobs
by NitpickLawyer 21 days ago
There's 0 chance of that. You can maybe say that for things like material science, nuclear stuff, weird physics, and so on (basically anything relating to making big booms, delivering big booms, or ensuring others can't make big booms). But having LLMs as we understand them now in the 80s would be impossible. They simply didn't have the compute necessary. The entire world combined didn't have it.

For reference, a single 4090 GPU has more FP8 flops than the top supercomputer in 2007. A 4x 4090 computer (something you can buy today for ~10k) would be better than to top supercomputer in 2010 [1]. There's a reason deep learning only started to really work in the past few decades. We had the "theory" for a while, but no compute to actually put it in practice. And the current models are being trained on 10s to 100s of thousand of enterprise GPUs, that make the 4090 look like a toy.

[1] - https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...

2 comments

Another axis would be power consumption. Pretty sure if you want LLM training in 1980 you'd also need fusion power in 1980. And it would be very, very hard to hide the cooling apparatus of your datacenter.

All that aside, it would also likely mean that the secret chip fabs eclipse the volume of the non-secret chip fabs by orders of magnitude... covertly? Or are we imagining chips that are 30yr ahead of their time? Either way it's just absurdities upon absurdities..

Why are you so certain you know what tech existed in the 80's?

NASA used Commodore 64's as part of the constellation of computers needed to launch the space shuttle. That whole program was pretty hokey. That silliness doesn't preclude some radically advanced compute tech from existing elsewhere at the same time. Of course something like that would be ultra secret and you and I wouldn't be told about it.

You have to be told something before you can believe it, right? You also get to choose whether you believe that something you're told is 100% god's honest truth or a load of bullshit as well. It's kindof sad to see people try to gatekeep thought here. Lot of rigid thinkers.

I recall Intel in the 90's trying to build clockless pentiums. They couldn't figure it out or maybe it had integration problems with clockful peripherals and it never came to fruition. But I wondered if that was some furtive attempt to seed some tech into industry. I just try to keep an open mind to these sorts of things.

Far-out theories need solid evidence to be believable. However, there is not even circumstantial evidence to suggest that anybody was five decades ahead of public technology. You might rather claim that the military has super advanced aircraft, at least we have lots of sightings of UFOs doing apparently miraculous aerial acrobatics.
50 years for large models is a stretch, though TI did have smaller, real-time field-trainable neural nets almost 40 years ago:

https://en.wikipedia.org/wiki/FGM-148_Javelin#Seeker

It was featured in one of the Modern Warfare games from around 20 years ago, and I looked it up because back then, I called bs on the fire & forget aspect.

I just try to keep an open mind to these sorts of things.

An open mind doesn't mean "a mind full of open circuits."

Why are you so certain you know what tech existed in the 80's?

Useful LLMs could not have been built without deep-UV semiconductor fabrication at scale, which in turn can't exist without a massive, complex supply chain that envelops the globe like a spiderweb.

Such technology could not have been built speculatively, or without people noticing, or with the physics and material science of the day. We didn't even know how to make blue LEDs then!

Because an LLM is the only way to express intelligence with a computer. So your conception of ram/cpu/storage/networks and learning (training on huge corpus of data) is the only way to do artificial intelligence?

LLM's are the perfect consumer-grade AI, safe to release to the public. It lacks creativity so little chance for a runaway super-intelligence, and it gives the people real tools. It's the best case. Isn't it tidy how reality is? Almost seems planned.

“Keeping an open mind is a virtue—but, as the space engineer James Oberg once said, not so open that your brains fall out.” ― Carl Sagan, The Demon-Haunted World: Science as a Candle in the Dark
"Military grade" means overpriced and 5-10 years late.