[HerbManic]

It was partially a joke but someone posted a image of Co-pilot in Excel to demonstrate the limits of these things. Three cells with three numbers (1, 2, 3) and co-pilot asked to sum these three up.

Instead of answering with 6, it came up with 15. The comment was "If AI is doing this, a global financial crash is inevitable."

Might not be real but it is something to keep an eye on. Hopefully, they are a bit more cautious on how this is implemented.

[kgeist]

I wonder why it's so bad. Do they just paste a CSV into the raw model? Because in my experience, even small local models can handle it reasonably well if the harness forces them to write & run a Python script that parses the table and performs the calculations, instead of relying solely on next-token prediction.