[lelanthran]

> I think you are confusing the spec as "this is how it must be built", as opposed to, "this is what the software must do and must not do to be acceptable".

You can't enforce a "do not do this" to an LLM. Just putting it in the context by saying "don't do this" makes it more likely that it will eventually do that.

[jychang]

Yes, I agree. If you tell humans "do not think of pink elephants", they are more likely to think about pink elephants.

Therefore, you must not use humans for any important work.

[lelanthran]

> Yes, I agree. If you tell humans "do not think of pink elephants", they are more likely to think about pink elephants.

> Therefore, you must not use humans for any important work.

Counterpoint, all important work done in the history of humans were done by... humans, and yet I see no pink elephants created as a result of telling humans "do not think of pink elephants".

[jfreds]

I’ve seen loads of very bad/wrong code written by humans

[lelanthran]

> I’ve seen loads of very bad/wrong code written by humans

Have you seen any pink elephants?

[jychang]

Yes, I see them all the time when I drink lots of alcohol [1]. This is a common issue with humans called "hallucination" which proves that humans are unreliable.

[1] https://en.wikipedia.org/wiki/Seeing_pink_elephants