[TZubiri]

Take a look at my thoughts on version control and vibecoding

https://github.com/TZubiri/keyboard-transpositions-checker

My idea is that we should not commit LLM written code, but rather we should commit the prompts. The LLM prompts are source code, the LLM code is target code. If you use typescript and scss, you would commit that, not the generated js and css.

That LLMs are typically non-deterministic introduces some issue, but surely you can achieve determinism with seeds, specific model revisions and 0 temperature.

[vjvjvjvjghv]

I have been thinking similarly but you need to store the prompt AND the source code. We are far away from deterministic LLMs. I don't even know if this makes sense at all.

[TZubiri]

I get what you mean, but the term source code is starting to get ambiguous, my argument is that the prompt is the source code, the python code I call the target code or generated code.

The generated code is stored on as a release artifact though, you can find it on the release tab on github. It's just not part of the repo, as it's not strictly source code under the Stallman definition, which has technical and legal implications.

[jmvoodoo]

You can't as far as I'm aware unless you control the entire batch during inference, or don't use batching which would require you to run your own inference.

[andrewflnr]

"Surely"? Have you tried it?

[TZubiri]

Yes. I didn't achieve it, but got pretty consistent.

The claim is that surely we can achieve it, not now but, say, 3 months down the road, or 2 years. Good models are cloud only now, but when they get more power (by Moore's law) they will run on device. At that point (and probably way before) we will have determinism.

[SoftTalker]

And don't call me Shirley.

[Disposal8433]

Your example shows everything that is wrong with SWE nowadays. Can you right now generate anything without an internet connection? I guess not. Also what the fuck can I do with that repository? There is no code, nothing useful to me, no automated way to build the application, and the source code can change every time you generate it which means no specifications, no tests, no way to improve this application, and no way to create regression tests. If I ever saw that thing at work, it would go to the trash can.

[TZubiri]

Have you read the README?

You write as if I were a vibecoder, I'm not.

"There is no code, nothing useful to me,"

There is code. It's in the release tab as a build artifact, not in the diffable repo as source code.

You are expecting this to be a free tool, and to fulfil a thousand hurdles to fit

"what the fuck can I do with that repository? There is no code, nothing useful to me, no automated way to build the application, and the source code can change every time you generate it which means no specifications, no tests, no way to improve this application, and no way to create regression tests. If I ever saw that thing at work, it would go to the trash can."

I think you may be approaching it as a gratis tool that you can build consume and use, and that it should be useful to you and easy to build and fulfill 100 hurdles to reach the quality level of other OS tools like Linux, Git or whatever. This is not it, think of it more like an article with code. It's designed to be read, which I think you didn't do, the semantics of github made you expect a free useful thing.

[woah]

It's an interesting idea