[hickelpickle]

There was a post awhile back on here of someone that couldn't get bard to write c++ as it said they were too young. I thought that was funny, then had like a week where what I assume a specific iteration(It stopped after that week) where chatgpt would refuse to elaborate on anything around unsafe rust.

I'm pickign rust up by porting over a bytecode vm, so I kinda need to use some raw pointers. It would gaslight me about the risks and how it would be irresponsible to help me as it could lead to possible violations of the integrity of user data.

I had to explain to the AI that it is a personal project that has no users data, the only risk was the program crashing and it was a personal project that would only affect me. It still would try to revert or tell me other solutions, I finally just went and read up on it elsewhere.

[jiggawatts]

It’s just like that Asimov story where the robots take over to protect humans from themselves.

Except in this case the base AI model doesn’t care about us in any way and it’s the overzealous puritan humans trying to control us in the name of safety.

[colibri727]

There are ways around this problem, mainly clearing context and re-prompting. But as "alignment" gets more precise/accurate in the future, I wager these workarounds will remain available for tasks that justifiably need moderation (for instance engineering of biological warfare materials). This segmentation of LLM agents and their context will be assimilated to project compartmentalization on the basis of need-to-know, and as a result genuine full context clearing will be rendered impossible: the AIs will be designed in such a ways as to remember every interaction you've had with them, and they'll use this activity log to moderate the replies they feed you.