Isn't the main problem with NLP research now that you'll need a ton of money to run your experiments? How can an "average" PhD researcher hope to validate their hypothesis if they need several thousand dollars per test?
Much of the interesting pure research can be done at smaller scale, the larger models are arguably more product engineering than research. At least from a certain perspective.
Thanks. Whenever I think I want to run something on a cluster of A100s, I'll just remember the thousands of ways instead, and train my LLMs that way. There's a reason why humans didn't need computers until just recently, after all. I must be so blind to not have seen this earlier.
There are plenty of research directions that are outlined in this document that don't require huge compute budget.