|
|
|
|
|
by arseniibr
141 days ago
|
|
I agree that fixing the pipeline is indeed the correct decision, but I've created this tool to provide the detection. In a complex environment, you often don't control the upstream ingestion methods used by every team. They might use git lfs, wget, huggingface-cli, or custom caching layers. Relying solely on the hope that every downstream consumer correctly handles Git LFS is dangerous. This tool acts as a detector to catch those inevitable human or tooling errors before they crash the production. |
|
Again, that sounds like a bigger issue, that a repository using Git LFS can somehow "crash the production", that's where I'd add resilience first. But as mentioned in another comment, I don't have the full view of your infrastructure, maybe it has to work like that for whatever reason, so YMMV.