[neochris]

Full disclosure: the results are still worse than what you would get from a manually built lab in a live AWS environment.

Tried a bunch of SOTA prompt engineering techniques. I found that explicitly defining AWS permissions and identities in the context window (we used the .tf scripts from CloudGoat) helped make the responses a lot more grounded.

The default behavior for AI agents when simulating CloudTrail logs to "plagiarize" log samples from the AWS CloudTrail docs. To avoid that, you must use some prompt engineering tricks to make the agent focus really hard on resources defined using IaC.