[sa-code]

> every time you try something innovative the "policy people" will climb out of their holes and put random roadblocks in your way

This is so relatable. I remember trying to set up an LLM gateway back in 2023. There were at least 3 different teams that blocked our rollout for months until they worked through their backlog. "We're blocking you, but you’ll have to chase and nag us for us to even consider unblocking you"

At the end of all that waiting, nothing changed. Each of those teams wrote a document saying they had a look and were presumably just happy to be involved somehow?

[miki123211]

I think you should read "the Phoenix project."

One of the lessons in that book is that the main reasons things in IT are slow isn't because tickets take a long time to complete, but that they spend a long time waiting in a queue. The busier a resource is, the longer the queue gets, eventually leading to ~2% of the ticket's time spent with somebody doing actual work on it. The rest is just the ticket waiting for somebody to get through the backlog, do their part and then push the rest into somebody else's backlog, which is just as long.

I'm surprised FAANGs don't have that part figured out yet.

[embedding-shape]

To be fair, the alternative is them having to maintain and continuously check N services that various devs deployed because it felt appropriate in the moment, and then there is a 50/50 chance the service will just sit there unused and introduce new vulnerability vectors.

I do know the feeling you're talking about though, and probably a better balance is somewhere in the middle. Just wanted to add that the solution probably isn't "Let devs deploy their own services without review", just as the solution probably also isn't "Stop devs for 6 months to deploy services they need".

[regularfry]

The trick is to make the class of pre-approved service types as wide as possible, and make the tools to build them correctly the default. That minimises the number of things that need review in the first place.

[throwaway_z0om]

Yes providing paved paths that let people build quickly without approvals is really important, while also having inspection to find things that are potential issues.

[pvtmert]

From my experience, it depends on how you frame your "service" to the reviewers. Obviously 2023 was the very early stage of LLMs, where the security aspects were quite murky at best. They (reviewers) probably did not had any runbook or review criteria at that time.

If you had advertised this as a "regular service which happens to use LLM for some specific functions" and the "output is rigorously validated and logged", I am pretty sure you would get a green-light.

This is because their concern is data-privacy and security. Not because they care or the company actually cares, but because fines of non-compliance are quite high and have greater visibility if things go wrong.