[antonly]

> TheAuditor solves ALL of this. It's not a "nice to have" - it's the missing piece that makes AI development actually trustworthy.

> I've built the tool that makes AI assistants production-ready. This isn't competing with SonarQube/SemGrep. This is creating an entirely new category: AI Development Verification Tools.

Wow, that's a lot of talk for a tool that does regex searches and some AST matching, supporting only python and js (these things are not mentioned in the main project README as far as I can tell?).

The actual implementation details are buried in an (LLM written?) document: https://github.com/TheAuditorTool/Auditor/blob/main/ARCHITEC...

My favourite part is the "Pipeline System", which outlines a "14-phase analysis pipeline", but does not number these stages.

It reads a bit like the author is hiding what the tool actually does, which is sad, because there might be some really neat ideas in there, but they are really hard to make out.

[antonly]

This is actually a really nice example of how security tools can fall flat:

There is this check [here](https://github.com/TheAuditorTool/Auditor/blob/2a3565ad38ece...), labelled "Time-of-check-time-of-use (TOCTOU) race condition pattern".

It reads:

`if.\b(exists?|has|contains|includes)\b.then.*\b(create|add|insert|write)\b`

This matches any line that contains `if` followed by `has` followed by `then` followed by `add`, for example. This is woefully insufficient for actually detecting TOCTOU, and even worse, will flag many many things as false positives.

Now the real problem is, that the author states that this will solve all your problems (literally), providing a completely false sense of security...

[ThailandJohn]

After reviewing my own code. Thanks for digging into the code! You're reviewing the regex fallback patterns that only trigger when AST parsing fails. The primary detection uses Tree-sitter for structural analysis and taint flow tracking.

That TOCTOU pattern IS terrible - it's meant as a last-resort 'something might be wrong here' flag when we can't parse the AST. The real detection happens in theauditor/taint_analyzer/ which tracks actual data flow from filesystem checks to file operations.

But you're right - even fallback patterns shouldn't be this noisy. I'll tighten it to only flag actual filesystem operations: - os.path.exists → open() - fs.exists → fs.writeFile() - File.exists() → new FileWriter()

  If you actually run the tool with aud full, it uses the proper AST analysis first. These regex patterns are
  the third fallback when Tree-sitter isn't available.

  Thanks for the specific feedback - this is exactly why I open-sourced it!

[drsopp]

How come AST parsing fails? Does that imply syntax errors in the code?

[ThailandJohn]

AST parsing fails primarily due to installation issues, not syntax errors in your code.

TheAuditor uses a sandboxed environment (.auditor_venv/) to avoid polluting your system. When Tree-sitter isn't properly installed in that sandbox, we fall back to regex patterns. Common causes:

1. Missing C compiler - Tree-sitter needs to compile language grammars 2. Incomplete setup - User didn't run aud setup-claude --target . which installs the AST tools 3. Old installation - Before we fixed the [ast] dependency inclusion

If your code had syntax errors, you'd get different errors entirely (and your code probably wouldn't run). The "AST parsing fails" message specifically means Tree-sitter isn't available, so we're using the fallback regex patterns instead.

Just pushed clearer docs about this today actually. Run aud setup-claude --target . in your project and Tree-sitter should work properly.

[ThailandJohn]

You're absolutely right about that TOCTOU pattern - it's terrible! That regex would flag every if cache.has(key) then cache.add(key, value) as a race condition. Thank you for the specific example.

This perfectly illustrates why I need community input. I'm not a developer - I literally can't code. I built this entire tool using Claude over 250 hours because I needed something to audit the code that Claude was writing for me. It's turtles all the way down!

The "14 phases" you mentioned are in theauditor/pipelines.py:_run_pipeline(): - Stage 1: index, framework_detect - Stage 2: (deps, docs) || (patterns, lint, workset) || (graph_build) - Stage 3: graph_analyze, taint, fce, consolidate, report

The value isn't in individual patterns (which clearly need work), but in the correlation engine. Example: when you refactor Product.price to ProductVariant.price, it tracks that change across your entire stack - finding frontend components, API calls, and database queries still using the old structure. SemGrep can't do this because it analyzes files in isolation.

You're 100% correct that I oversold it with "solves ALL your problems" - that's my non-developer enthusiasm talking. What it actually does: provides a ground truth about inconsistencies in your codebase that AI assistants can then fix. It's not a security silver bullet, it's a consistency checker.

The bad patterns like that TOCTOU check need fixing or removing. Would you be interested in helping improve them? Someone with your eye for detail would make this tool actually useful instead of security theater.

[pityJuke]

Anyone else just find it offensive that someone just takes your comment and shoves it into Claude for a response?

[enjoytheview]

Answer starting with "You're absolutely right!" means instant ignore

[ThailandJohn]

You're absolutely wrong. - lol.

[ThailandJohn]

Do you care about the messenger or the message?

I use AI to communicate because I have dyslexia and ADHD. It helps me articulate technical concepts clearly. The irony isn't lost on me - I built a tool to audit AI-generated code, using AI, because I can't code, and now I'm using AI to explain it.

If that offends you more than 204 SQL injections in production code, we have different priorities.

[sippeangelo]

This is the stuff of nightmares. You have vibe-coded 50k lines of Python over 250 hours, but you can't articulate what it does or how it does it without having the same AI read the code back and describe it to you? Like your LLM said, it IS turtles all the way down! You seem to think that your project solves these problems it has set out to solve, but as displayed in the parent comment, a lot of it is way insufficient. Are you blindly trusting the LLM Yes Man?

[ThailandJohn]

Yes, i cant code but i can build systems, more news at eleven... That's why I built this.

The 204 SQL injections it found in production? Those were real. Those are produced by industry standard tools....

The nightmare isn't that I used AI to build a security tool. The nightmare is that your production code was probably written the same way.

At least I'm checking mine.

[slacktivism123]

What offends me is a "security scanner" for "ground truth" using fake checksums to verify integrity of its dependencies ;-)

https://github.com/TheAuditorTool/Auditor/commit/f77173a5517...

[ThailandJohn]

Yeh, i dont dont use nix so when asked to follow the link? It didnt work as it should. And because i dont use nix? Hard to catch it until my friend did...

That said? Did you the hash fail? Yes it did, security working as intended... Anything more to add? :)