[Rooster61]

I took it to mean that the way LLM's use natural language causes the typical observer to feel as if they can perform far more than what they actually can. Akin to the analogy of humanoid robots.

It plays off of the "if it looks like a duck, quacks like a duck, and walks like a duck" idiom, which of course isn't foolproof and gives avenue to the kind of spectacular advertising that is fueling this hype.

[the_sleaze_]

I agree. A personal anecdote.

My mom was lamenting car insurance quotes, so I told her to ask AI. She did, then had it do a Monte Carlo simulation for all the insurances she the AI felt she was qualified for.

It happily replied that it did 1 million monte carlo simulations and here was the result.

To this day I don't think she fundamentally groks that LLMs cannot calculate.

[radlad]

For me, it was a friend that was wildly impressed by ChatGPT (before it could search the web) had "analyzed recent market news and stock price graphs" to give him stock recommendations.

[coldtea]

>To this day I don't think she fundamentally groks that LLMs cannot calculate

Can't most LLMs trivially use Python and other languages and libs and calculate?

[aitchnyu]

I used Gemini to take "0.3 grams of KNO3 will raise the nitrate level of 10 gallons of water 4.84 ppm" and generate tables of how many grams of dry fertilizer for 1ppm, 5ppm, 10ppm for my planted aquariums of 144 and 3000 liters. It calculated them perfectly.

https://rotalabutterfly.com/rex-grigg/dosing.htm

[SamPatt]

LLMs cannot themselves calculate, but they are given tools which can.

They're getting quite good at that now.

[BillinghamJ]

ChatGPT can easily do Monte Carlo simulation in its "thinking" step, and has done many times for me. e.g. I asked it to compare savings interest between regular banks and median returns from premium bonds. It's not difficult at all for it to do, you can see the code it's generated to do it + the output, easy to inspect