Hacker News new | ask | show | jobs
by tim333 260 days ago
On the same topic of AI helping scientific discovery there was this tweet yesterday

https://x.com/DeryaTR_/status/1972115494787338484

>...noticed an email from one of my PhD students sent more than eight years ago, outlining a highly complex immune cell experiment that would run for several weeks and asking me to make corrections

>...Incredibly, GPT-5 Pro would have been as good as, if not better than, me at making these corrections, interpretations, analyses, and follow-up experiment suggestions! The experiment would also have yielded better results thanks to more precise planning...

Maybe the era of AI speeding things is upon us. Maybe not so long till AIs are helping make better AIs?
1 comments
_heimdall 260 days ago
If and when these tools become self-improving we have absolutely no idea what comes next.

Maybe its utopia, maybe its akin to an Eliezer Yudkowsky prediction, who knows. Regardless of the specific outcome, its a huge gamble with effectively unlimited risk.

link
cwmma 260 days ago
The architecture of the current models where learning is a separate and very expensive process make runaway self improvement seem like something that will require a bunch of breakthroughs in order to happen.
link
_heimdall 260 days ago
That seems reasonable, but I'm not sure if we really know yet. If current models are given direction and control to change how the next model is trained or architected it seems plausible that they could stumble into such a breakthrough.

The current LLM approach makes huge assumptions, including that training only on text prediction is enough to simulate true intelligence. That may or may not be a valid assumption, but it could be enough for the LLM to make one seemingly small change that ends up running away from us faster than we would realize.

link
tim333 259 days ago
I doubt there will be a sudden run away. Probably more of a gradual shift where humans and AI combine to improve the AI and gradually the AI does more of the work.
link
_heimdall 259 days ago
I'd agree that seems more likely. Though I can only make that expectation with public knowledge of what these AI companies say they're working on. I don't know what research projects or yet-to-be-released architectures may currently exist - my confidence in that assumption of likelihood isn't very high.

More importantly, I think we agree a sudden run away is possible, we don't have safeguards in place, and the risks are extremely high should that happen.

link