[dcminter]

Well, the "accidentally making the S3 bucket public" scenario would be a good one. If you review carefully with full understanding of what e.g. all your policies are doing then great, no problem.

If you don't do that will you necessarily notice that you accidentally leaked customer data to the world?

The problem isn't the LLM it's assuming its output is correct just the same as assuming Stack Overflow answers are correct without verifying/understanding them.

[simianwords]

I agree but its about the extent. I'm willing to accept the risk of ocassionally making S3 public but getting things done much faster, much like I don't meticulously read documentation when I can get the answer from stackoverflow.

If you are comparing with stackoverflow then I guess we are on the same page - most people are fine with taking stuff from stackoverflow and it doesn't count as "brave".

[dcminter]

I think anyone who just copies and pastes from SO is indeed "brave" for pretty much exactly the same reason.

> I'm willing to accept the risk of ocassionally making S3 public

This is definitely where we diverge. I'm generally working with stuff that legally cannot be exposed - with hefty compliance fines on the horizon if we fuck up.

[simianwords]

That's fair - I would definitely use stackoverflow liberally and dive into documentation when situation demands it.

[awongh]

The thing is that you can now ask the LLM for links and you can ask it to break down why it thinks a piece of code, for example, protects the bucket from being public. Things that are easy to verify against the actual docs.

I feel like this workflow is still less time, easier and less error prone than digging out the exact right syntax from the AWS docs.