[m_ke]

I tried really hard to do this, but it turns out the models don't care about your ideas and want to do what's popular in their training data, so they will happily ignore anything you try to force down their throats, especially as context length grows or if you hit compaction.

So to make best use of the models steer them down familiar paths, mention common pattern and frameworks, use popular packages and languages that have the high median quality online.

I started my project with a few simple interface definitions and a short design / architecture doc that I include in the AGENTS.md file, but no matter how hard I try all of the models just end up ignoring it and sprinkled new seeds of variants of the same stuff all over my code base, that with each new session grow new branches.

[logicprog]

I do highly unusual, off the beaten path projects with AI.

For instance, a game engine written entirely in LuaJIT, but allocating almost all data structures using CFFI, a very unusual, custom object-orientation DSL I designed myself, and that uses SDL3's SDL_gpu library to do all rendering, a ton of GLSL 4.6 compute shaders cross compiled to SPIR-V and Metal (which constraints what GLSL can be written), and is a hybrid tile-oriented and 3D engine.

Another example is my Emacs-like live-modifiable image-based editor, again in LuaJIT+CFFI, with a kqueue+atomic lock-free ring buffer+pthread based actor model, each running a separate LuaJIT interpreter, with a custom advicing system, event bus, async/await implementation riding on the event bus, a custom process management and IO library, and testing via Tmux.

Even DeepSeek V4 Flash mostly does fine with this, once it's built a few memories in the memory system. Let alone something like DSv4-Pro or GLM 5.2. All I can say is skill issue.

[m_ke]

It's not an issue of usual vs unusual, I'm saying the models are way better at writing and maintaining django or react code bases than your own hand rolled architecture that you define in some docs that it has to learn and keep in context. All of the models do an amazing job making local edits and working in small greenfield projects but once you get to full production systems with close to a million lines things start to rot. The code still works and the models are able to make progress but all of a sudden you have 3-4 different versions of your concepts sprinkled in random corners of your code base.

My second point is that the models are way better at things like Rust or Lua than Python or JS, because the average person producing code in those languages has way more programming experience, so the code quality of training data online in those languages is higher than the programming 101 medium blogspam type content you see in more popular intro languages.

[justincormack]

At 600k lines of Rust, not finding those issues yet. You can search for different versions of same concepts, I mean you need to refactor a lot anyway.

[Chu4eeno]

Did you actually manually review the 600k lines of code?

And the reason LLMs tend to produce such insanely large codebases is partially what the comment you're replying to explained, they duplicate everything all over the place.

[justincormack]

The codebase is large because its a complex problem and its around 75% tests. Have used various tools to find duplicates and they arent there to any significant extent.

[SR2Z]

"You are violating the DRY principle with code like <blahblah>. Look through the codebase in depth, and identify possible places to consolidate common logic."

Yeah, there's no substitute for taste, but this is not that big of a deal. I infinitely prefer repeated code to crappy, leaky abstractions. Let the model generate some slop, then tighten it up either by hand or with more prompting.

[rapind]

I just finished an LLM assisted Rust rewrite of a production project after a couple months. It's around 150k LOC, mostly Rust, excluding generated code and tests. Around 200k if I include tests. I'm also now starting to extract / refactor some of it's code into their own reusable crates, so my LOC isn't going to grow and might even shrink a bit. I can't speak to 1M LOC, but I'd imagine that takes a lot of human context to reason about too, and you probably want to break it up into manageable chunks if possible.

LLMs have been a boon. I have a lot of guardrails in place though to keep the LLMs focused and producing correct and ergonomic code. A lot of effort and review has gone into creating these guardrails (choice of language, linting, forced conventions, tests, etc.). A lot of these practices will be useful in future projects. I highly recommend this approach. Don't just copy skills or processes that others publish. Figure it out yourself so the lessons stick. Human in the loop, especially one that has learned through practice and failure, is still immensely valuable IMO. It's a fine line between getting stuff done and yakshaving over your tooling... but that's always been the case, as proven by my vim dotfiles. :)

It's worth noting that the AI industry is filled with tons of snakeoil and BS. You don't need to take someone's course in "21 steps to be productive with AI" or w/e. Just like learning a new programming language, dig right in and try to build something that scratches an itch. Pick a project you have complete autonomy over instead of that "spend all the tokens" crap managers were pushing until recently (this has corrected now right?).

Parent saying "Skill issue" though... What a jackarse.

[logicprog]

I haven't hit nearly a million lines, that's true, these projects are "only" 20k and 45k lines of code or so, respectively; however, I do have directed acyclic graphs of worker agents, managed asynchronously by the agent I chat directly with, do huge architectural refactors (the most recent, was to update everything in the codebase that used the sychronous Lua standard library `io` functions to use the asynchronous, multithreaded IO system I created for the editor — at least, everything that could use the new library, which I kept deliberately simple in a few ways, and everything that wasn't already temporary, but you know) and that's turned out quite well, and at least for me, 45k sloc is nothing to sneeze at, especially since with a green field project there's less examples to keep them on track. I do regularly check in to make sure it's using my design to the fullest, as well.

[saghm]

> All I can say is skill issue.

I don't necessarily disagree with the overall point you're making, but I think calling it a "skill issue" is a bit reductive. These are relatively new tools that are changing quickly, and the amount of flexibility in how you can use them is a lot higher than pretty much anything else we've built up patterns for in the industry in recent years. Rather than dismissing people who express having trouble producing the results you can, I feel like we should be giving lifting others up by providing insights into how we're able to do those things when we know that they're possible. Otherwise, how do we know it's truly a "skill" issue and not a "knowledge" issue, and the only thing stopping them is that no one has helped them understand?

[logicprog]

In my experience, because as you say, the flexibility of these tools and how you engage with them is so broad, and how much the nuances of thay engagement can often matter, the only way to teach someone how to do what you do with them is to sit down and pair a program with them. Barring that, any instructions you could give would massively under-determined what they actually try as a result of your instructions, and so it ends up being an infinite loop of them just coming back and saying "it doesn't work." So in the end, people really just have to discover how these things work for themselves.

[saghm]

That's a completely reasonable take. I wouldn't have any issue with a nuanced comment like that. My issue with your above comment is that it's a lot more arrogant than nuanced.

[logicprog]

That's fair, I think my point is there's a nuanced theory behind my choice to be rude, which is essentially "you need to experiment more, be more open minded, really work at it, and then you'll see what others are seeing, it isn't impossible." It's like Dark Souls, nobody can beat it for you, or teach you how to use dodge rolls, you've gotta build that skill yourself, and blaming the tools won't help.

[_doctor_love]

I think your attitude is pretty lazy. It's a low-effort answer to say "skill issue" and then walk away feeling superior. Much harder to lean in to the other person, understand them, then help guide them onto the path. When you do this, you cause them to accelerate tremendously.

IMHO as you get more senior in software, after a while the only interesting metric becomes: are you raising the level of the people around you?

EDIT: looking at your Github seems like you are still in university. So I'll say "age and experience issue" on your end ;)

[saghm]

> That's fair, I think my point is there's a nuanced theory behind my choice to be rude, which is essentially "you need to experiment more, be more open minded, really work at it, and then you'll see what others are seeing, it isn't impossible."

Why not just say that then, instead of being rude? I pretty firmly disagree with the idea that being rude is ever a useful way to convey information rather than intent. Content and tone are separate axes that you can calibrate independently, so what you're saying is never an excuse for how you say it. Being rude conveys emotion, not information, and you didn't really convey information in your comment other than "look at how amazing the stuff I do is", and that's not a particularly effective mechanism for getting people to see your point of view.

[tech-ninja]

I agree with you and I didn't think your comment was rude. Some people are against using LLMs and they will always find an excuse instead of experimenting with them so it doesn't matter how helpful you try to be they already made their mind.

[simonw]

I think "skill issue" is often deployed (including here) as a shortcut for what you said - "the only thing stopping them is that no one has helped them understand".

(I try not to say "skill issue" myself because it comes across as rude.)

[saghm]

Yes, but with the context that the person saying it is almost always the one who does understand, and is choosing to be snarky instead of helpful.

[atoav]

It is easier to get an LLM to tell an unusual story using the most popular phrases than have it tell a common story using off-kilter uncommon phrases.

The big arc is easy to get using an LLM, getting it to do the details correctly, that is the actual hard part. So in my cases I (by other people's account a legendarily patient person) lost my cool trying to get a LLM to write CSS in a way that makes any sense if you have a design background. I ended up rewriting the whole CSS by hand, because it is inconsistent, piecemeal and just doesn't make sense. Yeah let's make everything bold and important. Yeah let's write internal implementation details into a label in the UI.

I decided to use LLMs as an aid to my own thinking, not as a replacement.

[rstuart4133]

> All I can say is skill issue.

I regularly see claims by people that they have produced large, reliable bodies of code using vibe coding. Unfortunately, no vibe coded project I've used comes anywhere close to code reviewed by a human. Admittedly, my sample size is very small: things like claude-code, OpenClaw, and the occasional project I see mentioned here. But compared to the things I used regularly like GNU coreutils, Firefox, Linux, vim, git and the hundreds of other software projects I use daily, vibe coded apps are in a different ballpark. Vibe coded apps of a few 100k lines often glitch several times a day, the traditional ones that have several orders of magnitude more lines often don't fail in several years of use.

AI is now being used to assist in creating and maintaining these traditional bodies of software, of course. It looks like it has enhanced productivity as open source contributions have grown by 20% or so, and we've only just started to learn how to use it. I haven't noticed any degradation in quality. It may be too early to tell, but I suspect the engineers involved wouldn't let that happen.

As you say, it's a skill issue - it takes considerable skill to produce reliable maintainable code. But I suspect that wasn't the skill you are referring to.

[mpweiher]

Maybe the key difference is that you are so far off the beaten path that there simply are no examples of what you are doing that the models "want" to emulate?

I've also had reasonable success with the models generating fairly idiomatic Objective-Smalltalk, my own language of which there are likely few to no examples in the training data.

I do steer them towards my own sample programs.

[dominotw]

things you described are not "highly unusual" in the way gp comment is describing. you are just remixing bunch of stuff. Thats the type of thing ai is really good at.

[logicprog]

The GP was just talking about not using common languages like TypeScript and frameworks like React, which is a bar of unusual that seems arguably far lower than mine, especially since I literally defined special custom DSLs with unusual semantics, and used libraries like SDL_gpu with little to no pre-existing training data. So this seems like shifting the goalposts.

Also, what would be highly unusual to you?

[dominotw]

> The GP was just talking about not using common languages like TypeScript and frameworks like React

i dont see anything like that in that comment.

[logicprog]

"It's not an issue of usual vs unusual, I'm saying the models are way better at writing and maintaining django or react code bases than your own hand rolled architecture that you define in some docs that it has to learn and keep in context. All of the models do an amazing job making local edits and working in small greenfield projects but once you get to full production systems with close to a million lines things start to rot. The code still works and the models are able to make progress but all of a sudden you have 3-4 different versions of your concepts sprinkled in random corners of your code base.

My second point is that the models are way better at things like Rust or Lua than Python or JS, because the average person producing code in those languages has way more programming experience"

[Chu4eeno]

The most esoteric I've seen from an LLM was this light rendering fix in a TempleOS game engine: https://gitgud.io/CrunkLord420/cyberchud/-/commit/c49879751c...

Though it's still cheating with a compatibility layer, I don't think claude touched any HolyC.

[ssivark]

> All I can say is skill issue

I know nothing about you or your code, but till you make your code + workflows public and have all of it reviewed / critiqued (so that other can replicate the performance), it's hard to accept it as a serious case study.

For all we know, this could just as well be a severe case of Dunning-Kruger.

[williamcotton]

https://williamcotton.github.io/datafarm-studio/

https://williamcotton.github.io/algraf

https://williamcotton.github.io/pdl

https://github.com/williamcotton/algraf

https://github.com/williamcotton/pdl

Check out the /docs dir in the repos for each plan file and the language specs.

They are my 4th and 5th DSLs and I learned quite a bit along the way, most importantly how to separate a WASM runtime wired up to a Monaco editor “lsp” with the same crate for editor-services that works with the actual system binary LSP.

[logicprog]

Feel free to peruse the repos! I follow compression-oriented programming, so there are places where the code is "insufficiently" DRY intentionally, as I feel out the precise use cases, and I explicitly made some other decisions (like a God editor object) for meta design reasons (I want it to be easy to modify the editor state from anywhere, just like in Emacs), so all I ask is that you review these without being excessively punative to prove your point.

https://github.com/alexispurslane/cursed/

https://github.com/alexispurslane/industrialworld/

Feel free to look at my past projects, prior to me starting to use AI (which started with gemini-ocr) to assess my baseline skill level.

[fzeroracer]

I've found this to be more consistently the case than not. People almost never post their project and in the rare cases that they do, it's usually some shambling mess that doesn't actually function. They put together a keyword soup post that sounds good on an AI generated resume but falls apart upon investigation.

[jryle70]

> People almost never post their project and in the rare cases that they do, it's usually some shambling mess that doesn't actually function

It's funny because you chose to target the one who is pro-AI, who has since posted their code, and not the GP who is against AI, who also didn't post any code to support his opinion that "it turns out the models don't care about your ideas and want to do what's popular in their training data".

Do you see the irony?

[logicprog]

No, they don't. They're more interested in pathologizing and talking down to and about me.

[logicprog]

Its hard not to feel insulted here. "Projects using AI like this don't exist, and if they do, they're broken and the people who made them have psychosis." However, although I haven't tested my projects cross machine, or cross platform yet (my editor only works on MacOS because it uses kqueue), I would be happy to take an unedited video demonstration of both, I assure you they both run.

[InkCanon]

Not specifically about the comment you replied to, but there has been an absolute Cambrian explosion of plausible sounding projects that use really complicated words that are just AI psychosis. Recently there was someone who posted a huge Lean proof on GitHub supposedly of proving P=NP. The problem is AI is really good at deluding people if that person doesn't have domain specific knowledge. AI severely makes dunning Kruger worse

[_doctor_love]

The key is to remove degrees of freedom from the agent. IMHO this is the really hard work of agentic engineering. You need to strongly constrain what the agent can even do in the first place to force it into a certain area.

That runs the danger of putting the model out of distribution, of course, so it takes some experimenting.

I'm working on a Ruby project where - on purpose - I'm disallowing any frameworks other than what's standard Ruby. So no Rails or Sinatra, just the inbuilt Webrick web server. Initially it was hard to keep the model on the rails but as I've learned more it gets easier. That said, it's clear to me that the model pulls very strongly in familiar directions in terms of how it writes code, i.e. a lot of what gets written still ends up looking "Railsy" even though my learning project doesn't use it.

[iwontberude]

It makes everyone more average and same-y. It’s a good average, better than bad, but I cringe everytime I see someone claiming it’s a force multiplier for their personal expression. Bullshit.

[cyanydeez]

right, the model will listen to you for ~32k context; half of that is just getting them into the "downhill" path for being a coding assistant. You can throw in whatever your heart's desire and if it's a single fix, it might listen to you; but treating it like a smart-IDE basically means it'll follow whatever dominant pattern is in the training data.

If we had smart capitalism (the SAME PROBLEM) we'd realize what we want are per-framework or per-OS models that simply are 80% how to english and 20% the tools we need for a given project.

But that's no where the money is.

[m_ke]

yeah I tried rules, hooks and forbidding things like pip / python3 but it just led to the model failing to do what it wants and a bunch of token churn trying to get around my more rigid constraints.

main problem is that the harness files get loaded into context early in the session and slowly wash away as new information comes in.

[cyanydeez]

i've been playing around with https://github.com/bytedance/deer-flow and they seem to figure out how to get around context sizing issues and has a more automated prompt system.

Essentially the problem with current hanrsses is they just can't find/evict context and want to ensure KV cache isn't evicted.

If they evict the old prompt and tried to keep all the proper rules, then you get token churn anyway, as what you want is an entirely new chain of events where your rules/guidance sit at the front of the model chain.

What we're getting to is there really isn't an economical way for cloud models to do it; you can do it with a local model, but at comfortable speeds, you need smaller sized.

I think a good harness via Qwen3.6-35B-A3B is possible, as the token gen is only 3B since that's the active parameters.

So it's mostly that to keep rules pinned to the top of the context will either bloat the model or it'll evict the KV cache.