|
|
|
|
|
|
by penagwin
2536 days ago
|
|
|
I know it's going to become an even bigger bottleneck moving forward, but it really raises the barrier for us hobbyists. I own a 1080ti (12GB RAM) - and I consider this "high-end" for many people who aren't actively employed for machine learning (College kids and younger especially). I know you can "use the cloud" but I would really prefer not to... |
|
|
You can always just use smaller models and/or lower resolutions though; of course the results won't be on par but it may reach a qualitative result (for research and experimentation purposes) or good enough result (for personal application purposes). E.g. hobbyists don't need AlphaGo-level go playing AI (which I'm sure had aggregate costs in 5 figures or more to train), reduced versions play all far above our levels -- although in this case there's the interesting effort of pooling hobbyist resources to indeed reach SOTA, see LeelaZero[1] and LCZero.
Some kinds of research will be effective only at large orgs, that's always been true. There was indeed a brief period when people realized GPUs could unleash deep learning/CNNs that you could do anything with a good GPU, but that was very much an exception. To borrow from another field, you cannot do a level of car engine research without all infrastructure to fabricate and test engine prototypes (though you can do some/other kinds of theoretical analysis).
[1] http://zero.sjeng.org/home