[AvAn12]

I think the gap is because 1. For coding, Claude is amazing - mainly because of its curated skills and because massive amounts of working code has already been carefully labeled over the last decade or so via GitHub. And because with any Turing complete language, there is only so much one can do.

But 2. For most other things, LLMs are fairly underwhelming. Research is usually mediocre. Try being rigorous and repeat your research prompt many times - then make a confusion matrix to tally up how many false positives and false negatives occur. And for the rest, be honest and ask yourself if the LLM is doing much more than a basic search engine query or trip to Wikipedia would have told you. For “normie” use cases, it’s handy-ish but far from revolutionary

[gamblor956]

Basically this. For tax, and law in general, LLMs are prone to giving the wrong answer for anything specific (though they're great at the super general stuff because they're just restating one of the millions of summaries written by humans).

Gemini still isn't sure what details are in the version of OBBA that actually passed, because there was more discussion about various proposals (that didn't make it into the bill) than there was about the final version of the bill itself.

Unfortunately, it's an intractable problem based on the ways that LLMs work. In order to overcome those limitations, you have to provide so much detail to the prompt that you would find the answer faster searching manually.

[Gigachad]

Also because programming is self contained in a computer where the results can be tested and iterated easily. For programming the agent can just run the compiler and tests and keep retrying until it works. If I wanted to for example sew a T shirt, AI is useless.

[calf]

I didn't understand your comment about Turing complete languages, could you explain that part please?

[AvAn12]

At the end of the day, the processor can only do Turing operations: assign values to variables (registers, memory locations, storage), loops, bitwise operations, and conditionals. Whether the source code is python, java, or lisp, it has to compile or interpret down to machine code ultimately. Likewise if the running software is a word processor, DOOM, or an LLM, at the end of the day it will be executed by the processor using the three operations. Lots of other fancy hardware and software may accelerate things but ultimately it is those ops that are the running code. The rest is many wonderful conveniences and abstractions.

[otabdeveloper4]

LLMs are always amazing doing stuff you're bad at. (And always underwhelming doing stuff you know.)