[epolanski]

This so much.

To me it seems so strange that few good language designers and ml folks didn't group together to work on this.

It's clear that there is a space for some LLM meta language that could be designed to compile to bytecode, binary, JS, etc.

It also doesn't need to be textual like we code, but some form of AST llama can manipulate with ease.

[senbrow]

At that point why not just have LLMs generate bytecode in one shot?

Plenty of training data to go on, I'd imagine.

[dyauspitr]

The code would be un reviewable.

[TeMPOraL]

It would also be harder for the LLM to work with. Much like with humans, the model's ability to understand and create code is deeply intertwined and inseparable from its general NLP ability.

[senbrow]

Why couldn't you use an LLM to generate source code from a prompt, compile it, then train a new LLM on the same prompt using the compiled output?

It seems no different in kind to me than image or audio generation.

[senbrow]

...by a human :)

[dyauspitr]

Hence very important in the transitional phase we are currently in where LLMs can’t do everything yet.

[seb1204]

Would this be addressed by better documentation of code and APIs as well as examples? All this would go into the training materials and then be the body of knowledge.

[LZ_Khan]

readability would probably be the sticking point