[Ostrogoth]

A few months ago I asked CGPT to create a max operating depth table for scuba diving based on various PPO2 limits and EAN gas profiles, just to test it on something I know (its a trivially easy calculation; and the formula is readily available online). It got it wrong…multiple times…even after correction and supplying the correct formula, the table was still repeatedly wrong (it did finally output a correct table). I just tried it again, with the same result. Obviously not something I would stake my life on anyway, but if it’s getting something so trivial wrong, I’m not inclined to trust it on more complex topics.

[tom_m]

Well it doesn't really do math.

[Ostrogoth]

Interesting; just went down a rabbit hole on LLM training and math. For this example, it could have simply copied a table from online, but I wasn’t aware how poorly some LLMs perform on even basic math functions. I’ve not run into that issue before.