Hacker News new | ask | show | jobs
by blagie 740 days ago
Yes.

A numpy program will work tomorrow.

ALL of the machine learning frameworks have incredible churn. I have code from two years ago which I can't make work reliably anymore -- not for lack of trying -- due to all the breaking changes and dependency issues. There are systems where each model runs in its own docker, with its own set of pinned library versions (many with security issues now). It's a complete and utter trainwreck. Don't even get me started on CUDA versions (or Intel/AMD compatibility, or older / deprecated GPUs).

For comparison, virtually all of my non-machine-learning Python code from the year 2010 all still works in 2024.

There are good reasons for this. Those breaking changes aren't just for fun; they're representative of the very rapid rate of progress in a rapidly-changing field. In contrast, Python or numpy are mature systems. Still, it makes many machine learning models insanely expensive to maintain in production environments.

If you're a machine learning researcher, it's fine, but if you have a system like an ecommerce web site or a compiler or whatever, where you'd like to be able to plug in a task-specific ML model, your downpayment is a weekend of hacking to make it work, but your ongoing rent of maintenance costs might be a few weeks each year for each model you use. I have a million places I'd love to plug in a little bit of ML. However, I'm very judicious with it, not because it's hard to do, but because it's expensive to maintain.

A pure Python + numpy implementation would mean that you can avoid all of that.

2 comments

That makes sense. I imagine that there are significant performance tradeoffs but those are probably worth it in many cases. I would be somewhat surprised if Mamba can be made usefully fast with NumPy, but it would be a pleasant surprise.
For me, there are plenty of old-school models which are still plenty useful and which run fine even on a modern, fast CPU and would do well with numpy, and even more so with cupy. Step through:

https://huggingface.co/learn/nlp-course/chapter1/1

And there's a pile of awesome. It feels pretty lame compared to GPT4o, ChatGPT, or even GPT3, but it's still super-useful a lot of the time, and not too resource-intensive.

(Disclaimer: That's the original Hugging Face course and is politely structured to work on reasonable machines too. They have other courses which require moderate GPU, and plenty of models which require crazy hardware.)

This was a very helpful explanation. I appreciate it.