[fzeroracer]

It seems like every now and then we get someone doing dangerous science experiments in their backyards that threaten not only themselves but their neighbors as well. Brings to mind David Hahn, only worse since with AI the odds of it hallucinating a step and causing a serious problem is much higher.

[pololeono]

I don't know, I could imagine that this changes when AI becomes more sensible than the average internet advice.

Also I would like to see some evidence how dangerous the experiment with the AI inspired Fusor actually was. I recently read here "hiking in jeans" is dangerous.

[RansomStark]

I'd be interested in how an LLM could become more sensible than the average internet, they are by definition the average of the internet. I'm waiting for the next major innovation, and given AI's history I might be waiting a long time.

Fusors are somewhat dangerous, they use extremely high voltage, in the thousands to hundred thousand volt range. x-rays become an issue above around 30,000 volts, but they are frequently made by high school students, and I'm not aware of any deaths.

lots of details available here: https://fusor.net/board/viewtopic.php?t=4843

[lupusreal]

> they are frequently made by high school students, and I'm not aware of any deaths.

That's been done no more than a few dozen times I think? Maybe less than that. I think it's a rare enough activity that the accident rate simply hasn't been probed enough.

Wood burning with microwave transformers is notorious for getting people killed, but how many people does it kill relative to how many people have tried it? Maybe a handful few out of a hundred? On the other hand, kids building fusors are probably smarter than the average public, to whom wood burning with transformers is frighteningly accessible. I don't think teenagers building fusors is quite that dangerous, but I don't think we have enough data to call it a statistically safe activity.

[exe34]

> they are by definition the average of the internet.

Are you referring to base models?

Nowadays they also train on stolen books and are further "aligned" based on feedback. I imagine they are already learning to teach based on feedback from users.

[RansomStark]

To be honest, I was using internet as shorthand for average of human knowledge, on the basis that most books, peer reviewed articles, and everything else is already on the internet, even if they exist only in the more unsavory corners (I've seen nothing to suggest the FM producers were / are much bothered about where the data is from).

But yes referring to base models. I'm also not convinced that the average book is any more trustworthy than the average webpage, whether that be a purely technical book, where you really need to webpage of errata to be able to use the examples. Or the more pop-sci books that cherry pick data and jump to completely unfounded conclusions (I'm thinking of the ancient engineers - aliens built the pyramids books).

The feedback is great and might work in some areas, technical knowledge. But once you step outside of the physical sciences and engineering, you don't so much end up with better quality information, just a curated experience that aligns with the model owners (think DeepSeek and Tiananmen square)

[exe34]

the nice thing about books, especially STEM ones, is that you can tell if there's a problem because there will be inconsistencies. so even without the errata, you can fuzz until it all matches.