[cglan]

Not exactly seeing great performance with this model. It gets tripped up by ridiculously silly questions.

For example "if you have a spoon, a knife, a fork, a carrot and a calculator, which one would you use for math?"

> None. The question is grammatically incorrect as it implies that the person has all of those items in their possession at once

or

"The bus was going so fast it passed the racecar". What vehicle was going the fastest?"

> It's impossible to tell which one went faster, as they were both traveling at high speeds

[pumanoir]

ChatGPT and Claude also get tripped by both questions

[refulgentis]

I think Claude is correct on the merits (Claude’s awesome in general)

Without more context about the speeds of the bus and racecar, I cannot determine which vehicle was going fastest based on the given statement. Simply saying that the bus passed the racecar does not provide enough information to compare their speeds.

[CuriouslyC]

This is true, the question isn't inconsistent with the bus and the racecar travelling in opposite directions, it's just implied that they're going in the same direction.

[cubefox]

GPT-4 (Bing) solves them correctly.

[andriym]

yeah, gonna need a few more M's of data work for that unfortunately.