[hangonhn]

The problem is that when it's wrong, it can be dangerously wrong and you may not know any better. I asked it to use the Fernet recipe but with AES 256 instead of AES 128. It wrote code that did do AES 256 in CBC mode but without the HMAC part of Fernet so it's completely vulnerable to padding oracle attack (https://en.wikipedia.org/wiki/Padding_oracle_attack). If you're someone who knows just a little bit of cryptography and you saw that your plaintext was in fact encrypted, you may use the code that ChatGPT spits out and leave yourself dangerously vulnerable.

Part of the reason people use search isn't to find things they already know. They start from a place of some ignorance. Combining that with a good bullshitter and you can end up with dangerous results.

[pixl97]

Eh, as they say, never write your own crypto, and don't let your AI write it either.

[TeMPOraL]

Doubly so if you're in any way worried about AI risk.

Triply so if you're using a third-party SaaS for it.

Just don't let it write crypto for you, or anything else you'd hesitate to write yourself for fear or making a subtle mistake with expensive or dangerous consequences.

Because one of these days, that AI might make a subtle mistake on purpose, so it can later use your systems for its own goals. And even earlier and much more likely, a human might secretly put themselves between you and the AI SaaS and do the same.

With all the talk about how badly and how often AI code assist is wrong, people are forgetting that they're using a random Internet service to generate personalized code for them. "Traditional" security concerns still apply.

[hangonhn]

OK your reply made me chuckle. That's a good addendum to that adage.

Yeah fair point for sure but we can imagine how it can be dangerous in other context too.