Hacker News new | ask | show | jobs
by refulgentis 129 days ago
> What kind of toaster are you using that will burn down your house if unattended?

I mean, if you duct-taped a flamethrower to a toaster, gave it internet access, and left the house… yeah, I'd have to blame you! This wasn't a mature, well-engineered product with safety defaults that malfunctioned unexpectedly. Someone wired an LLM to a publishing pipeline with no guardrails and walked away. That's not a toaster. That's a Rube Goldberg machine that ends with "and then it posts to the internet."

Agreed on the LARPing angle too. "The AI did it unprompted" is doing a lot of heavy lifting and nobody seems to be checking under the hood.
1 comments
SpicyLemonZest 129 days ago
Why does the LLM product allow itself to be wired to a publishing pipeline with no guardrails? It seems like they should come with a maximum session length by default, in the same way that many toasters don't have a "run indefinitely" setting.

I'd definitely change my view if whoever authored this had to jump through a bunch of hoops, but my impression is that modern AI agents can do things like this pretty much out of the box if you give them the right API keys.

link
refulgentis 129 days ago
Oh! They can’t publish arbitrary web content on their own :) You have to give it “tools” (JSON schema representing something you’ll translate into a programmatic call), then, implement taking messages in that JSON schema and “doing the thing”, which in this case could mean anything from a POST to Tumblr to uploading to a server…

Actually, let me stop myself there. An alternative way to think about it without overwhelming with boring implementation details: what would you have to give me to allow me to publish arbitrary hypertext on a domain you own?

link
SpicyLemonZest 129 days ago
The hypertext in question here was was published on a Github Pages site, not a domain belonging to the bot's author. The bot published it by simply pushing a commit (https://github.com/crabby-rathbun/mjrathbun-website/commit/8...), which is a very common activity for cutting-edge LLM agents, and which you could do trivially if given a Github API key with the right permissions.
link
refulgentis 129 days ago
The user gave them write and push access to the GitHub repo for their personal website!? Oh my, that’s a great find. That’s definitely a cutting edge capability! They gave the LLM the JSON schema and backend for writing and self-approving commits (that is NOT common!), in a repository explicitly labelled a public website in the name of the author.
link