Hacker News new | ask | show | jobs
by embedding-shape 141 days ago
Right, but in these environments (PS, I'm also in the EU, also work in the ecosystem) we don't just deploy 3rd party data willy nilly, you take some sort of ownership of the data, review+polish and then you deploy that. Since security and compliance is important for you, I'm assuming you're doing the same?

And when you're doing that, you have plenty of opportunity to turn Pickle into whatever format you want, since you're holding and owning the data anyways.
1 comments
arseniibr 141 days ago
Don't you suppose that in a large company with teams of 50+ devs/DS pulling models for experiments, enforcing a manual "review+polish+convert" workflow for every single artifact can create a massive bottleneck and, as a result, shadow IT? Doesn't it make sense to automate the "review" part?
link
embedding-shape 140 days ago
If you run teams with 50+ devs then you MUST ensure the pipelines actually work, for every single project they work on, you don't PATCH validation on top of what seems to already be brittle in your infrastructure.

But I don't manage the infrastructure where you work, I don't have the full picture. But it sounds to me like there is a different issue going on, the issue isn't "Some HF repos use Git LFS so we need a tool to flag those".

link