[cainxinth]

>I like that this shows how hard even conceptually simple ideas are to achieve in fine-tuning LLMs. Even given a pretty good starting dataset, a decent starting model, etc. this appears to have been a challenge.

Surely, but we can't gloss over the fact that this was accomplished by a single person.

[zoogeny]

Yes and no I think. I've seen individuals achieve things in their bedrooms that would make most corporations blush. Demoscene type stuff comes to mind as an example. Often a single person can become hyper obsessed with achieving some goal and in the absence of any interference can achieve something impressive beyond what can be achieved within a company.

Consider a PM involved in this project, feeding in requirements from a business. Instead of the "just get it done at any cost" mentality of a single person you would have KPIs and business objectives that would muddy the water.

I just mean to say that there is a gulf between what can be done by a single hacker in his basement when they have no constraints other than their imagination compared to what can be accomplished by a business. Sometimes the single-hacker achievement doesn't scale.

So, it is impressive that this is possible for a single person at all. But, from a business/operation perspective, I don't actually think that is as relevant as it may seem.