[AgentOrange1234]

"Every optional field is a question the rest of the codebase has to answer every time it touches that data,"

This is a beautiful articulation of a major pet peeve when using these coding tools. One of my first review steps is just looking for all the extra optional arguments it's added instead of designing something good.

[shepherdjerred]

There's nothing specific to AI about this. Humans make the same mistake.

To solve this permanently, use a linter and apply a "ratchet" in CI so that the LLM cannot use ignore comments

[oblio]

Is there a Python linter that does this?

[datsci_est_2015]

Not that I’m aware of (writing Python 10+ years). Suppose you could vibecode one yourself though.

[raulparada]

Not out-of-the-box afaik, but we use https://ast-grep.github.io (on a pre-commit hook) for such cases, which bridges the linter gaps nicely.

[KronisLV]

I've been writing my own linter that's supposed to check projects regardless of the technology (e.g. something that focuses on architecture and conventions, alongside something like Oxlint/Oxfmt and Ruff and so on), with Go and goja: https://github.com/dop251/goja

Basically just a bunch of .js rules that are executed like:

  projectlint run --rules-at ./projectlint-rules ./src

Which in practice works really well and can be in the loop during AI coding. For example, I can disallow stuff like eslint-disable for entire files and demand a reason comment to be added when disabling individual lines (that can then be critiqued in review afterwards), with even the error messages giving clear guidelines on what to do:

  var WHAT_TO_DO = "If you absolutely need to disable an ESLint rule, you must follow the EXACT format:\n\n" +
  "// prebuild-ignore-disallow-eslint-disable reason for disabling the rule below: [Your detailed justification here, at least 32 characters]\n" +
  "// eslint-disable-next-line specific-rule-name\n\n" +
  "Requirements:\n" +
  "- Must be at least 32 characters long, to enforce someone doesn't leave just a ticket number\n" +
  "- Must specify which rule(s) are being disabled (no blanket disables for ALL rules)\n" +
  "- File-wide eslint-disable is not allowed\n\n" +
  "This is done for long term maintainability of the codebase and to ensure conscious decisions about rule violations.";

The downside is that such an approach does mean that your rules files will need to try to parse what's in the code based on whatever lines of text there are (hasn't been a blocker yet), but the upside is that with slightly different rules I can support Java, .NET, Python, or anything else (and it's very easy to check when a rule works).

And since the rules are there to prevent AI (or me) from doing stupid shit, they don't have to be super complex or perfect either, just usable for me. Furthermore, since it's Go, the executable ends up being a 10 MB tool I can put in CI container images, or on my local machine, and for example add pre-run checks for my app, so that when I try to launch it in a JetBrains IDE, it can also check for example whether my application configuration is actually correct for development.

Currently I have plenty in regards to disabling code checks, that reusable components should show up in a showcase page in the app, checking specific configuration for the back end for specific Git branches, how to use Pinia stores on the front end, that an API abstraction must be used instead of direct Axios or fetch, how Celery tasks must be handled, how the code has to be documented (and what code needs comments, what format) and so on.

Obviously the codebase is more or less slop so I don't have anything publish worthy atm, but anyone can make something like that in a weekend, to supplement already existing language-specific linters. Tbh ECMAScript is probably not the best choice, but hey, it's just code with some imports like:

  // Standalone eslint-disable-next-line without prebuild-ignore
  if (trimmed.indexOf("// eslint-disable-next-line") === 0) {
    projectlint.error(file, "eslint-disable-next-line must be preceded by: " + IGNORE_MARKER, {
      line: lineNum,
      whatToDo: WHAT_TO_DO
    });
    continue;
  }

Can personally recommend the general approach, maybe someone could even turn it into real software (not just slop for personal use that I have), maybe with a more sane scripting language for writing those rules.

[pipes]

Optional field can be addressed with good defaults. Well, that is how I think about it. I.e. if they aren't passed in then they are set to a default value.

[AgentOrange1234]

Here is an example of what I mean when I say optional stuff is a sign of failure to design.

I inherited some vibe-coded scripts that dealt with AWS services like Bedrock and S3. These scripts needed create various AWS SDK clients. These clients needed to know which account/region to use.

Had this been well designed, there would have some function/module responsible for deciding what account/region to use. This decision point might be complex: it might consider things like environment variables, configuration files, and command line arguments. It might need to impose some precedence among these options. Whatever these details, the decision would be authoritative once made. The rest of the code base should have expected a clear decision and just do what was decided.

Instead, the coding assistant added optional account/region arguments in many submodules. These arguments were nullable. When left unspecified, "convenient" logic did its own environment lookups and similar. The result was many "works on my machine" failures because command-line arguments affected only certain portions of the program, environment variables others, config files still others.

This is grim stuff. It's a ton of code that should not exist, spreading the decision all over the code.

[pipes]

I see what you mean. Thanks.