Show HN: Generate malicious CloudTrail logs with AI agents

[neochris]

Hi HN,

I used AI agents to simulate malicious and normal CloudTrail logs. Was playing around with automated Cloud pentesting tools (stratus-red-team) to generate attacker behavior, but there wasn't a good way to generate "noise" AKA false positives. I needed both good and bad CloudTrail logs to fine-tune my AWS anomaly detection model.

Decided to throw together a few AI agents to generate it for me. You can play around with the attack builder here: https://simulation.tracecat.com

Anybody here try using LLM agents to generate synthetic data for security analysis?

If there's interest, happy to open source the prompts and code.

[neochris]

Full disclosure: the results are still worse than what you would get from a manually built lab in a live AWS environment.

Tried a bunch of SOTA prompt engineering techniques. I found that explicitly defining AWS permissions and identities in the context window (we used the .tf scripts from CloudGoat) helped make the responses a lot more grounded.

The default behavior for AI agents when simulating CloudTrail logs to "plagiarize" log samples from the AWS CloudTrail docs. To avoid that, you must use some prompt engineering tricks to make the agent focus really hard on resources defined using IaC.