[NewsaHackO]

> The text sections take implementation details that don't matter and present them to the user like they need to know them. It's also outdated.

The point of the wiki is to help people learn the codebase so they can possibly contribute to the project, not for end users. It absolutely should explain implementation details. I do agree that it goes overboard with the diagrams. I’m curious, I’ve seen other moderately sized repo owners rave about how DeepWiki did very well in explaining implementation details. What specifically was it getting wrong about your code in your case? Is it just that it’s outdated?

[blopker]

I dunno, it seems to be real excited about a VS Code extension that doesn't exist and isn't mentioned in the actual documentation. There's just too many factual errors to list.

[NewsaHackO]

>I dunno, it seems to be real excited about a VS Code extension that doesn't exist and isn't mentioned in the actual documentation. There's just too many factual errors to list.

There is a folder for a VS Code extension here[0]. It seems to have a README with installation instructions. There is also an extension.ts file, which seems to me to be at least the initial prototype for the extension. Did you forget that you started implementing this?

[0] https://github.com/blopker/codebook/blob/c141f349a10ba170424...

[curl-up]

This thread should end up in the hall of fame, right next to the Dropbox one.

From a fellow LLM-powered app builder, I wish you best of luck!

[raincole]

Yeah, this is a thread worth saving. Even just as an example of multiple people who can't read as well as an LLM.

[oblio]

Plot twist, OP has a doc mentioning it as unreleased.

[sceptic123]

In that folder is CHANGELOG.md[0] that indicates that this is unreleased. I'd say that including installation instructions for an unreleased version of the extension is exactly the issue that is being flagged.

[0] https://github.com/blopker/codebook/blob/main/vscode-extensi...

[NewsaHackO]

You are going to want to reread the file you are quoting buddy. That changelog is indicative that the extension has been released. The Unreleased section seems to list features that are not yet included in the released version of the VS Code extension, and the future plans are features that have not been developed yet.

[throwaway290]

here the maintainer says it doesn't exist. there's basically no way another interpretation is "more correct". presence or files can be not intended for use, deprecated, internal, WIP, etc. this is why we need maintainers.

[NewsaHackO]

Maintainers are not gods, and don't get to rewrite plainly true facts. In the Changelog, it actually says it is a "Initial release of Codebook VS Code extension".

[throwaway290]

compared to an llm they are an authoritative source...

[Phelinofist]

What a plot twist

[NewsaHackO]

It’s funny, I accidentally put a link to the commit instead of the current repo file because I was investigating whether or not he committed it versus he recently took over the project and didn’t realize the previous owner had started one. But he is the one who actually committed the code. I guess LLMs are so good now that they’re stopping developers from hallucinating about code they themselves wrote.

[blopker]

I brought up this issue because I thought it illustrated my previous points nicely.

Yes, there is a VS Code folder in that repo. However, it doesn't exist as an actual extension. It's an experiment that does not remotely work.

The LLM generated docs has confidently decided that not only does it exist, but it is the primary installation method.

This is wrong.

Edit: I've now had to go into the Readme of this extension to add a note to LLMs explicitly to not recommend it to users. I hate this.

[ninininino]

Is it possible that a random person who discovered your repo from Google search would make the same mistake the LLM did and assume it works and not realize it was an unfinished experiment?

[rng-concern]

Yes, and so the value of the persons opinions on the repo is low. Far lower than real documentation written by someone who knows more, that would not have made that mistake.

The value proposition here is that these llm docs would be useful, however in this case they were not.

[NewsaHackO]

>Far lower than real documentation written by someone who knows more, that would not have made that mistake.

But his own documentation did said that there was a VSCode extension, with installation instructions, a README, changelog, etc. From what he said, it doesn't even compile or remotely work. It would be extremely aggravating to attempt to build the project with the maintainer's own documentation, spend an hour trying to figure out what's wrong, and then contact the maintainer for him to say, "oh yeah, that documentation not correct, that doesn't even compile even though I said it did 2 months ago lol." It is extremely ironic that he is so gungho about DeepWiki getting this wrong.

[raincole]

Wow. Better advertisement for LLM in three comments than anything OpenAI could come up with.

[lionkor]

It might be internal, unfinished, a prototype, in testing and not yet for public use. It might exist but do something else.

This is not an ad for LLMs. If you think this is good, you should probably not ever touch code that humans interact with.