[digilypse]

I suspect it could soon be feasible to fine-tune on code with limited open data through a bootstrapping approach.

Give it the source code, a test library and access to a dev environment and with some prompting it could start to experiment with increasingly complex use cases, learning from successful attempts. This would depend on the model’s ability to understand what a successful outcome is so it can define test cases, which might be harder if the output isn’t text but not impossible.

Being able to give a model expert knowledge on an undocumented library or language seems like it could help accelerate adoption of new technologies that might otherwise suffer from network effects as users get used to AI-assisted development. Not to mention automated testing, finding edge cases and making pull requests to fix them, security, etc.

A human taking time to experiment with their assumptions and gain experience with an unfamiliar subject is in a sense creating their own training data as well.