[maccard]

> A lot of people here seem to be living in a different universe than me

I feel this way on this topic too.

> I think detractors believe you should just let AI do the job blindly instead of leveraging it as a tool to accelerate you.

The problem is; how _should_ I use AI? On a previous thread, I had two replies to the same comment, one saying "provide the LLM all the context it needs and let it go ham", and the other saying "Carefully guide and craft it and review everything". Both subthreads had people agreeing.

My experience with LLMs is leaving them unattended gives very poor results within a handful of iterations, and that carefully guiding them can be a value add, but the effort ot do that is very often as much as just writing the damn code myself, particularly if there's a few iterations that go on.

[hokapo]

I think the main cause of different views is just the fact that people have very different products and codebases they're working on. Therefore, the requirements and applicability of AI varies a lot, case-by-case. It would be useful if people always explained their exact use cases when making claims one way or another. I understand it's not feasible for many use cases to reveal exact details, but at least some transparency would be great.

I've found AI to be immensely useful in gamedev, mainly UX development, research, and curation. In those areas I let it go quite wild, with mechanical and visual tests for UX, and summaries for research and curation. I've also made skills, custom subagents, hooks, and scripts (all made with AI assistance) to streamline development.

Gamedev is a good example of where AI shines, since the risks of fucking things up are not so big. If I worked on, let's say banking, I'd be much more careful. On my day job, as a data scientist (moving towards product owner) in a large industrial company, I'm much more careful. UX can run on vibes, but anything that touches data processing or business logic needs to be looked at by a human, row by row. Depending on use cases, it may be fine to look at only the tests. The main problem is that there's so much stuff to review, and maybe 98% of it is fine. It's then very boring to try to spot the 2%. I don't have good solutions for that yet, and idk if anyone does.

[maccard]

> It would be useful if people always explained their exact use cases when making claims one way or another. I understand it's not feasible for many use cases to reveal exact details,

It would be useful if people _ever_ explained the use cases. Even Anthropic when they're publishing their materials are hand wavey about this.

> Gamedev is a good example of where AI shines, since the risks of fucking things up are not so big

I couldn't disagree more. I run an engineering team in GameDev, and AI is great at making a gigantic mess. It's great for generating a lot of code, but the minute that doesn't do what you want it to, you now need to go back and learn _everything_ that it's generated. And IME, doing that is as bad as having just written it yourself in the first place.

> he main problem is that there's so much stuff to review, and maybe 98% of it is fine. It's then very boring to try to spot the 2%. I don't have good solutions for that yet, and idk if anyone does.

I tentatively agree here, except I've found about 50% of it is "fine", 40% of it is "this works but isn't really what I want to put into my product" and 10% is "absolutely not".

[jimberlage]

AI is great at reading, so give it things to read. Claude will try to throw big markdown docs in your repo, but you don't have to do that. Make it keep docs by the code.

I include a line in AGENTS.md that says "We *always* add docstrings to methods, classes, structs, and namespaces - there should be 100% coverage of docstrings."

It will make better choices about what functions to make or remove if you force it to justify why the function exists.

And when you have to go back and understand, it becomes easier.

[maccard]

It’s clear at this point that agents don’t actually follow agents.md

They try to but they don’t. As an example, we use perforce in games. I regularly have to stop an agent and remind it to use p4 edit rather than blasting through the read only flag, despite the first paragraph of Claude.md being “this is a project using perforce. Batch call p4 edit on all files before modifying, do not manually remove the read only flag”

[hokapo]

> It would be useful if people _ever_ explained the use cases. Even Anthropic when they're publishing their materials are hand wavey about this.

Yes, good addition.

> I couldn't disagree more. I run an engineering team in GameDev, and AI is great at making a gigantic mess. It's great for generating a lot of code, but the minute that doesn't do what you want it to, you now need to go back and learn _everything_ that it's generated. And IME, doing that is as bad as having just written it yourself in the first place.

Yeah, I should have been more specific since our use cases are very different. I'm working solo on small games, so it's easier to keep the AI on a leash. I can imagine it can be a nightmare on large code bases with many collaborators.

> I tentatively agree here, except I've found about 50% of it is "fine", 40% of it is "this works but isn't really what I want to put into my product" and 10% is "absolutely not".

Yeah, probably our use cases are very different here too. We mostly develop/maintain dozens of microservices for internal use, mostly greenfield, relatively small in scope, and with 1-3 developers in each. And maybe 98% was a bit too optimistic. The larger and older the codebase, the more problems AI tends to create. But it's very good at helping to understand old codebases (even those written by myself haha).

Oh, and esoteric data science topics can be a minefield too. If there's a topic that's new to me, It's difficult to know what is hallucination and what is not. It's best to let AI suggest alternative methods and then study those yourself.

[fzeroracer]

> Gamedev is a good example of where AI shines, since the risks of fucking things up are not so big

Hi, professional game dev here. No it's not. Thank you for reading.

[hokapo]

Yeah, as I said in a reply to another comment, I oversimplified there. What I should have said is that solo or almost-solo gamedev with relatively small games is a good example of where AI shines. I don't have experience of developing larger games in larger teams.

[ndriscoll]

I've found that these models are extremely good at reasoning since late last year, but they're basically 200 IQ Amelia Bedelias. So my working model earlier this year was to give it a brief instruction and watch how it failed, which was generally a lack of context that it couldn't have possibly known. Then I'd add that context to the team AI docs repo, try again from scratch, and iterate until it did what I wanted without me having to give a bunch of detail in the prompt. Over the course of 2-3 months, I needed to give new context and steering less and less.

I also encouraged my team to paste any inquiries or alerts we got in our slack channel verbatim into codex and follow that procedure until it could figure it out as a one-shot.

I still treat it as more of a design and coding partner day-to-day, so I'm typically not trying to one-shot, but it can e.g. be triggered by a pagerduty alert, reference our code, check grafana panels, query application state across different servers/clusters, and come up with bug fix PRs all autonomously, and then post its analysis, mitigation suggestions, and PR link to slack for the on-caller to review. This was all just some docs, scripts, and a little listener service to trigger a one-shot model prompt that we built.

As a design partner, it's made it easier to add more telemetry or create rapid prototypes to check my assumptions. I've encouraged my team to regularly ponder on what facts they wish they knew, no matter how difficult it would be to find out. Pretend you could just ask God anything you want to know. Then ask the AI to go add whatever's needed to find out. We should all move toward a perfect intuition for what our system is doing because all questions can now be easily answered if we can only think to ask them.

[vlmutolo]

It’s a very soft skill that changes as the capabilities of AI change.

But one thing that doesn’t change is the need to specify an end goal correctly and precisely. I think the emphasis of knowledge/information-processing work is going to increasingly be placed on verification mechanisms. This is practically equivalent to precisely defining an end goal.

Spend time deeply thinking about what it means for a solution to be correct. What properties will a correct solution have? What of those properties are testable? Write those things down and tell the agent.

As models get better, agents will be able to target more and more difficult end goals. The strategy just becomes more useful. (It’s useful for people as well.)

For example, if you want an agent to write a photo editor, think about what end properties the editor should have. There are reference images for color space and rendering transformations. That’s a good start.

Sometimes the goal will be fuzzier. “I want a feature like CaptureOne where I provide a reference image and it makes my image look like that.” Well, time to think really hard about what that means. Iterate with AI on how to test for that precisely. Come up with some good metrics/heuristics. Maybe it means local contrast should match. Maybe the overall distribution of colors. Maybe something more complicated.

Then you have a target and you can let an implementation agent work against that. If it fails, it’s either because the agent is bad or your target was incorrect or incomplete. As models get better, the limiting factor becomes your ability to correctly define a problem.

[icepush]

The reality of the current situation is that we have dozens of families of different model; each of which is differentiated further into various revision numbers and power levels, and depending on which one you use and what the nature of the tasks is, you very much need a different usage style that can only be discerned after multiple (Sometimes very many) sessions of using that individual model for that category of task. It is not one size fits all, and trying to pretend that it is will lead to failure.

[maccard]

> It is not one size fits all, and trying to pretend that it is will lead to failure.

I'm not asking for a one size fits all solution. I'm asking for "vibe coding with Codex on GPT 5.6 with high is the way, but if you want to be more ivnolved, Sonnet 4.6 on Claude is the path. Here's the proof."

Saying that it depends and it's impossible to quantify allows the camp who are claiming it's more productive to say "you're holding it wrong".

[tzone]

Latest models like Opus 5.0 work well in both scenarios. You can just give it a goal and let it make a plan, review the plan and then let it go ham.

Or you can also use it piece-meal and have it do parts of work that you want. It works in both ways and works really well.

I don't think there is any model right now that you can use with 0 oversight, but you can do pretty complex stuff without writing single line of code at this point.

[glimshe]

I don't generally use AI to write code I feel I can write efficiently. In these cases, I'll just ask AI for a review. But when I need to write code in areas I'm less confident about, I'll let AI take the reins and the do a thorough review myself. These instances are not too common, though.

I also ask direct questions to AI for brainstorming and advanced programming language usage. In these cases, AI isn't touching the code.

[addaon]

> But when I need to write code in areas I'm less confident about, I'll let AI take the reins and the do a thorough review myself.

I see this mindset a bunch, and I just don’t get it. Reviewing code is so much harder than writing code! To write code, I need to find one path to achieve my goal, and convince myself, the compiler, and the reviewer that it does so. To review code, I need to consider all reasonable paths to achieve the goal, and confirm that a sufficiently-good one was selected. This is why we have junior engineers write code and seniors review it. If you’re beyond your comfort level writing it, how can you possibly review whether the author (AI or otherwise) made good choices? You only have exposure to one possible choice lane, and by problem statement you weren’t comfortable populating the other lanes…

[zahlman]

To write code, I effectively need to do a mini-review as I go, and type vastly slower than the AI. To review code, given that all the automated checks are out of the way, I need to consider the objectives of the code (in terms of fitting into the existing codebase, having a reasonably appropriate style, etc.) and ensure there isn't anything too far out of line. Knowing that something is "sufficiently good" doesn't entail awareness of all possibilities, because you aren't ranking the thing but measuring it.

Seniors review code because they have a much better understanding of the necessary taste and discretion, and an awareness of the larger system the code will integrate into, whereas there's only so much you can learn about coding that's relevant to most of the code you'll write (and only so fast you'll ever really be able to type it, and most of the issues can be caught by automated, deterministic systems).

[glimshe]

Seeing a solved problem and verifying the solution's correctness is much easier than solving the problem yourself. it's a bit like riddles. You can find examples of that when studying any topic. I disagree that reviewing code is harder than writing code in all/most instances.

Also, there's code that is just tedious and repetitive to write. AI can write that and reviewing is pretty easy.

[hegstal]

You could ask someone a math question, and they can write an equation down, and the equation can be mathematically sound, but it still not be the right form of the equation, or even relate to the question correctly at all. If all you're being asked to do is validate the mathematical soundness of the equation then sure, it's not that hard and it often mechanical, probably easier than having to generate the equation. But that isn't review -- that's cargo-culting what review is supposed to be.

To bring back to code: They are making the point that verifying "correctness" is not the be all and end all of code review. In every situation, there are multiple "correct" solutions, and only a few of them are actually correct and fit within the past, current and future context (technical and business). If someone isn't doing this, they aren't reviewing effectively and are acting as a glorified linter, which is an astonishingly low value-add use of human capacity.

There are situations where nobody really cares about the output, and nobody wants the true form of review, so it reverts to (at best) human-linting. In those situations, engineering capability of any kind functionally has no moat and all participants are equally fungible. If the business is dependent on engineering it's also a horrendous business. I would personally avoid being involved in these environments in an engineering capacity. I mean this in a diagnostic sense - it's a terrible career move, though there will always be busy work that's exactly what we have LLMs for.

[tossandthrow]

What can be concluded is that some get immense value out of Ai and some are incredibly critical of that claim.

I do not know what the secret sauce is.

But I have a responsibility to the team I lead. I need to protect our velocity. Which means that my hiring practices now include specific checks that people are able to work with ai.

In the end, it might be a step thing. Some people are just inherently incapable of working with this technology...

[teravor]

it depends on what it is you are doing. if the complexity is low (frontend, GUI) you can just prompt it once and refine at the end.

if the complexity is high but you know how the task is to be done then what you can do is divide the task into pieces yourself or with the help of the LLM and use an LLM for each piece.

if the task is complex and you don't have a good idea how it is to be done then you need to start exploratory work with the LLM (multiple sessions, don't reuse the sessions for code).

how to use an LLM correctly is a skill in and of itself. the people who say that LLM's are bad either haven't used any good ones and have such disorganized thinking processes that they are unable to encode them into English good enough for an LLM.

[mgfist]

The truth is that using AI well is a skill much like anything else. There's no one single way to use it much like there's no one single way to write a program that serves hackernews.

[Thanemate]

>There's no one single way to use it

The fact that people jump on the "you're simply using it wrong" wagon in every HN comment section whenever someone shares their negative/neutral experience with it implies otherwise.

[AussieWog93]

There can be many correct ways to use a tool, while still having incorrect ways to use it.

Sometimes you need to set your oven to grill, other times fan force. But if you're complaining about getting burned then, yes, you're simply using it wrong.