[samtp]

What type of software are you building with this workflow? Does it handle PII, need data to be exact, or have any security implications?

Because I might just not have a great imagination, but it's very hard for me to see how you basically automate the review process on anything that is business critical or has legal risks.

[paulhodge]

Mainly working on a dev tool / SaaS app right now. The PII is user names & email.

On the security layer, I wrote that code mostly by hand, with some 'pair programming' with Claude to get the Oauth handling working.

When I have the agent working on tasks independently, it's usually working on feature-specific business logic in the API and frontend. For that work it has a lot of standard helper functions to read/write data for the current authenticated user. With that scaffolding it's harder (not impossible) for the bot to mess up.

It's definitely a concern though, I've been brainstorming some creative ways to add extra tests and more auditing to look out for security issues. Overall I think the key for extremely fast development is to have an extremely good testing strategy.

[samtp]

I appreciate the helpful reply, honestly. One other question - are people currently using the app?

I think where I've become very hesitant is a lot of the programs that I touch has customer data belonging to clients with pretty hard-nosed legal teams. So it's quite difficult for me to imagine not reviewing the production code by hand.

[paulhodge]

No this app isn't launched yet. And yeah, customer data is definitely a very valid thing to be concerned about.