Hacker News new | ask | show | jobs
by darby_eight 817 days ago
I still find it hard to believe that LLMs provide any real edge over the kind of googling/man-paging that's necessary for understanding scripts, especially given the false positives chatbots are known for. Typically language + feature + library is enough to look something up in under ten seconds for me.

Granted, this probably takes a fair amount of experience to even know what you're looking at well enough to search for it.
1 comments
simonw 817 days ago
The good ones (Claude 3 Opus, GPT-4) are really incredibly good at understanding scripts. It's very rare that they hallucinate anything, especially important details for the more widely used tools that I tend to stick to.

I trust LLMs with tools like Bash and jq and ffmpeg which have been around for years. I wouldn't trust them with anything released within the past 12-24 months.

An example from just the other day: https://til.simonwillison.net/go/installing-tools

I wanted to understand this:

    go install github.com/icholy/semgrepx@latest
How does that @latest reference mention? There's no branch or tag on that repo called "latest".

I tried and failed to find documentation. I gave up and asked GPT-4, which said:

> @latest: This specifies the version of the package you want to install. In this case, latest means that the Go tool will install the latest version of the package available. The Go tool uses the versioning information from the repository's tags to determine the latest version. If the repository follows semantic versioning, the latest version is the one with the highest version number. If there are no version tags, latest will refer to the most recent commit on the default branch of the repository.

Is that correct? I have no idea! But it still gave me more to go on than my failed attempts with the real documentation.

link
darby_eight 817 days ago
I appreciate the illustration! I can certainly see how this would be useful especially for tools like ffmpeg, which is both difficult to google and understand and doesn't have a large overlap with patterns other tools use.
link