[mpoteat]

Reading the question, I thought the answer was six as well. Such a question is so misleading that I would hope that an AI answers as it has in this case, because not all problems are fully logically specified, and you need to remain robust against the question being posed incorrectly.

In other words, I would posit the probability of you having meant for the answer to be 5 low, because the question itself becomes trivial. For a system that needs to deal with many people asking questions, this robustness is helpful in my view.

In other words, it's not totally clear that time has not passed between sentences in your prompt. So the second instance of "now" could be a different "now".

[heavyset_go]

Except the AI is not responding this way because it thinks the probability of the OP having meant something else was high. It answered the way it did because it is incapable of understanding the question it was asked, along with any question it is asked.

The "robustness" you're perceiving in this case is just a mere coincidence, and doesn't reflect an aptitude for answering poorly phrased questions, but it does reflect a fundamental problem with these systems.

[Jensson]

Reading and understanding exactly what they are asking for is a part of the test, adding superfluous information to problems is perfectly valid to test that students actually understand instead of trying to pattern match to solutions they memorized.

[antihipocrat]

Good point, I added a clearer version of the question to my comment above. It still has trouble with it.