[h1fra]

what I saw using 5-6 tools like this:

- PR description is never useful they barely summarize the file changes

- 90% of comments are wrong or irrelevant wether it's because it's missing context, missing tribal knowledge, missing code quality rules or wrongly interpret the code change

- 5-10% of the time it actually spots something

Not entirely sure it's worth the noise

[bwfan123]

code-reviews are not a good use-case for LLMs. here's why: LLMs shine in usecases when their output is not evaluated on accuracy - for example, recommendations, semantic-search, sample snippets, images of people riding horses etc. code-reviews require accuracy.

What is a useful agent in the context of code-reviews in a large codebase is a semantic search agent which adds a comment containing related issues or PRs from the past for more context to human reviewers. This is a recommendation and is not rated on accuracy.

[asdev]

the code reviews can't be effective because the LLM does not have the tribal knowledge and product context of the change. it's just reading the code at face value

[theonething]

Isn't it possible to feed that knowledge and context to it? Have it scan your product website and docs, code documentation, git history, etc?