[kloud]

Specs might become one solution for coping with the need to review increased volume of code. A spec is a higher level of abstraction than code, which is a higher level of abstraction than machine code. The industry made the transition to higher-level once, paradigm is changing so it might happen again.

The workflow I imagine is either deriving specs from the conversation or reverse engineering the code to spec, review and edit the spec which should be tighter and much more compressed, then deterministically compile to code. Of course we don't want to be spec-first only, that would be going back waterfall, but doing iterations back and forth.

Now, Claude is not a compiler because it is closed and non-deterministic (they do opaque processing on server, hiding reasoning tokens), but LLMs might be. We refer to a piece of code from npm/pip by name to get some code by downloading it. We then have lockfiles with hashes to ensure integrity. Currently we are vibing it, but in the future we could refer to a piece of code by prompt/spec and getting the code by inferring it. To ensure integrity, the lockfile would be hashes of open weights and inference code (and ironing out implementation details like non-determinism due to GPU scheduling, etc.).

[toonvanvr]

It's funny how you describe something very close to what I was attempting to design. It was meant to trickle down from tickets to deterministic code. Data wise, it should be like a pyramid of tickets diluting layer by layer into leaf nodes which were "implementable" as statements in code. I'm not sure if that description makes sense read by someone else.

I think what made sense was envisioning a nanoswarm of LLMs (anticipating ASIC performance) diluting specs into semantic logic nodes, but a conversion from these to deterministic code made the vibe coded experiment come to halt. Your lock approach could be a shortcut to that.

Totally off-topic: it's quite intriguing that you can feel once friction starts building during a design phase. Suddenly everything slows down. I wonder if it's quantifiable and therefore can identify "wrong" design choices made by either humans or LLMs.

(Hoping some claw bot pics this up to finish my idea on my github tix repo in the initial-design branch, as main is empty ~ MPL2)

[kloud]

This resonates, I can imagine the development process as a cascade: we get business goals and user feedback, product is defined which can be pretty fuzzy, designers define UI, engineering works data models and tech constraints. Inputs at the top can be fuzzy, but each node in the graph applies some intelligence and clarifies it.

Related to the friction you mention, since the flow graph of the context is not preserved when it gets transformed into code, I find LLMs getting confused and making worse changes, because the explicit decisions with good reasons are getting mixed with implicit decisions from LLM surfaced during designing/planning. Sometimes when things change the code should change radically to match the new reality, but LLMs often "play it safe" by accumulating code and adding to tech debt. With a flow graph when change is made in the upper levels, it could propagate and allow decisions in lower levels to be revisited.

[skydhash]

> A spec is a higher level of abstraction than code, which is a higher level of abstraction than machine code

No it’s not.

Just like a quick doodle is not an higher level representation of the Mona Lisa. Sure for someones that knows the latter, it can suggest it. Or for someone that doesn’t know it, it may provide some basis of conversation. But it’s not the real thing. You can’t provide a doodle of something and expect an artist that hasn’t seen it to paint it.

Doodling’s value is that it lets you iterate on ideas without the accidental constraints that comes with the implementation (choosing paints, finding references materials, deciding colors,…). Some necessary choices are kept for later, while you decide on the most important ones.

So a spec is useful in designing software, but it’s severely lacking in implementing it. And the decisions made in implementation are as important as the ones made in the design. Even more after a while in production.

[drdexebtjl]

This might work if we have completely new harnesses for this, because the current one’s intentionally introduce non-reproducible context and tools.

The current way we’re doing SDD is extremely pointless. You reach a decision point that is under specified, make a decision, update the spec and continue the implementation. There’s no guarantee that the implementation followed from the updated spec, and not from the decision.

[kloud]

I agree, another issue I have with existing SDD frameworks I've seen like spec-kit or openspec, is they seem to miss the mark by coupling specification (what the software should be) and execution (how to get there). If the description of execution is needed, it means the spec is weak. As a rule of thumb if a framework defines execution concepts like tasks or tickets, that is a red flag for me.

[kimjune01]

prompts are not specs because most often, prompts are underspecified. You can prompt fizzbuzz well enough for it to be functional, but not ffmpeg. So the distinction is less about Claude's capability, more about the quality of the input. Garbage in garbage out

[hombre_fatal]

I think something can be considered a spec to some degree if outcome was derived from it.

It might underspecify, but that is just a quality of the spec. And as long as there is any discretion at implementation time, then the spec necessarily underspecifies to some degree. The goal of a spec is often to be clear on key directional concerns. Consider RFCs.

There's a trade-off between specification and implementation discretion. But if you think about starting at one end ("an internet forum") and then incrementally expanding the specifications ("it has topics with many posts in a linear paginated list"), there's some point where everything you care about is in the spec, and the rest is left to implementor discretion.

That's what I'd consider a good spec and a good LLM prompt.

[kimjune01]

and LLMs are really good at filling in the blanks. It's like a really big import on generic application knowledge, and if you draw the circles well enough, it will draw the rest of the owl.

[kloud]

You can reference other specs in a prompt. Video codecs are well specified with hundreds of pages of specs, they are probably in some form in the weights already, but you could also reference them and hash them as inputs directly to create a functional alternative to ffmpeg.

[kimjune01]

i meant to make the distinction between 'prompt' and 'spec' in that prompts are something that a human types in by hand. Interesting programs are a combination of canon and novel input

[kloud]

Gotcha, yeah if a prompt does not produce the result you want, you need to add more details to it, depending on the scope it can be a lot. When it includes all sufficient details it becomes a spec.