[TrueDuality]

I would have to guess it has something to do with that task not actually being suitable for language models at their current stage. Even if they could be trusted to perform the task, its actually not that much work to just... write code to handle keeping this kind of thing in sync. It's really really not that much more work. You really don't even need to do it, both training and inference can be done within PyTorch or in C++.

If it was necessary for some reason... Running a language model to keep something like this is sync over long term training and iteration would likely be more expensive than a developer's time AND block the researcher in a verification loop on the output which still probably needs to be checked by the developer (they could be the same person which will just deepen the frustration they experience).

The use of a lot of garbage accounts in this thread and lack of model details also looks pretty shady...

[solverist]

I'm confused. If these tools aren't good enough for AI research then why would they be good enough for consumer applications? If language models can not help with the AI development loop then the technology is not going to be useful for consumer use cases. Code can be very easily verified by linters and type systems so the problem of verification is much simpler than in consumer use cases without linters and type systems.

[quadrature]

>Code can be very easily verified by linters and type systems so the problem of verification is much simpler than in consumer use cases without linters and type systems.

you are confusing (syntactic) validation from verification. verifying code is an incredibly hard problem.

You can get a lot of value out of a models even if they are not capable of AI development because most people aren't doing things that are as complicated as AI development.

[developist]

I don't think AI development is complicated. It's just a bunch of functions with parameters which are optimized with gradient descent. The AI development loop is extremely simple and most AI "research" is basically stacking standard tensor operations and seeing what works. It's surprising there is no company that is applying AI to AI development since it is essentially a game with symbols and very well defined measurable outcomes.

[stale2002]

> If language models can not help with the AI development loop then the technology is not going to be useful for consumer use cases.

it quite literally is useful for consumer usecases though.

For example, one consumer usecase that is being used by a lot of students right now is cheating on their homework.

It is right now being used for all sorts of consumer things like that.

Also, if you have an opinion you can just say what your opinion is. You don't have to hide it behind questions and alt accounts.

You can just have an opinion and say it.

[developist]

I don't have an opinion. I am legitimately surprised that very easy problems in AI research have not already been solved with some foundation model. Translating and optimization of code from one formal language to another seems like a very obvious application of AI and yet most of the work is still done manually.

[TrueDuality]

I don't think you read what I said, or you don't know what you're talking about. Language models are actively being researched to be useful, but as of right now _they are not ready, capable, or trustworthy enough_ to perform tasks without supervision, they're especially bad at complex code and code that doesn't have many examples in the internet corpuses such as optimized CUDA kernels...