[danielgafni]

Hi HN, I'm Daniel, an ML Ops engineer.

I work at Anam and am responsible for the preparation of our training data.

I built Metaxy to avoid re-running expensive data processing steps on irrelevant upstream changes.

It's an open-source Python framework that can track granular sample versions and is infrastructure-agnostic.

See the full blog post here: https://anam.ai/blog/metaxy.

Happy to answer questions.

P.S. While Claude Code helped me a lot, this is *not* a vibe-coded project: I treated AI code like any line could be wrong, created a massive test suite, and Metaxy has been running in our production for about 2.5 months now.