[kingstnap]

For now we train LLMs on next token prediction and Fill-in-the-middle for code. This exactly reflects in the experience of using them in that over time they produce more and more garbage.

It's optimistic but maybe once we start training them on "remove the middle" instead it could help make code better.