[lucubratory]

>But they can't, they still fail at arithmetic and still fail at counting syllables.

You are incorrect. These services are free, you can go and try it out for yourself. LLMs are perfectly capable of simple arithmetic, better than many humans and worse than some. They can also play chess and write poetry, and I made zero claims at "counting syllables", but it seems perfectly capable of doing that too. See for yourself, this was my first attempt, no cherry picking: https://chatgpt.com/share/ea1ee11e-9926-4139-89f9-6496e3bdee...

I asked it a multiplication question so it used a calculator to correctly complete the task, I asked it to play chess and it did well, I asked it to write me a poem about it and it did that well too. It did everything I said it could, which is significantly more than a narrow AI system like a calculator, a chess engine, or an image recognition algorithm could do. The point is it can do reasonably at a broad range of tasks, even if it isn't superhuman (or even average human) at any given one of them.

>I think that LLMs are really impressive but they are the perfect example of a narrow intelligence.

This doesn't make any sense at all. You think an AI artifact that can write poetry, code, play chess, control a robot, recommend a clutch to go with your dress, compute sums etc is "the perfect example of a narrow intelligence." while a chess engine like Stockfish or an average calculator exists? There are AI models that specifically and only recognise faces, but the LLM multitool is "the perfect example of a narrow intelligence."? Come on.

>I think they don't blur the lines between narrow and general, they just show a different dimension of narrowness.

You haven't provided an example of what "dimension of narrowness" LLMs show. I don't think you can reasonably describe an LLM as narrow without redefining the word - just because something is not fully general doesn't mean that it's narrow.

[riffraff]

For arithmetic, this is ChatGPT:

    how much is (0.2 + 0.1) * 10?
    The result of (0.2+0.1)×10 is approximately 3, with a slight rounding difference leading to 3.0000000000000004.   

My 10yo does not make this error, ChatGPT does because it does not understand math, but knows how to use Python.

For poetry: counting syllables is a significant part of most poetry forms, so if you can't count syllables, you can't do poetry.

Let's say you want a 5-7-5 haiku, this is ChatGPT

    write a 5-7-5 haiku about windstorms

    Fierce winds howl and spin,
    Branches bend, clouds race the sky,
    Storm leaves quiet calm. 

this is not a 5-7-5 haiku.

LLMs are not general, but they show that a specific specialization ("guess next token") can solve a lot more problem that we thought it could.

[lucubratory]

This argument generalises to all possible AI systems and thus proves way too much.

>[AI system]s are not general, but they show that a specific specialization ("[process sequential computational operations]") can solve a lot more problem that we thought it could.

Or if you really want:

>Humans are not general, but they show that a specific specialization ("neuron fires when enough connected neurons fire into it") can solve a lot more problem that we thought it could.

This is just sophistry - the method by which some entity is achieving things doesn't matter, what matters is whether or not it achieves them. If it can achieve multiple tasks across multiple domains it's more general than a single-domain model.