Hacker News new | ask | show | jobs
by GolfPopper 19 days ago
It's interesting that you use the phrase "captive demon" because in fairy stories, the captive demon is always working to subvert the protagonist and work evil, whether by maliciously misinterpreting commands, or simply allowing the protagonist to damn themselves by enabling their worst impulses and poor choices.
1 comments

I was thinking along those lines when I wrote it. I do keep the thought in mind when interacting with an LLM that it could mislead me because they always sound so confident and yet get many things wrong, allowing me to damn myself if I'm not double checking things. I wouldn't put it past their creators to create them such that they handle certain prompts in a way that might benefit them, though, by maliciously misinterpreting prompts.