[oxidant]

Your post suggests authorization as a feature:

> For each replay that we run, Roark checks if the agent follows key flows (e.g. verifying identity before sharing account details)

I don't know if AI will be more susceptible or less susceptible to phishing than humans, but this feels like a bad practice.

[zammitjames]

Appreciate the feedback! To clarify, Roark isn’t handling authentication itself - it’s a testing and observability tool to help teams catch when their AI fails to follow expected security protocols (like verifying identity before sharing sensitive info).

That said, totally fair point that this example could be clearer—we’ll keep that in mind for future demos. Thanks for calling it out!

[Closi]

Again though, verifying identity before sharing sensitive info shouldn’t be down to the LLM following its prompt - it should be enforced by design.