[maeil]

So far ever since the initial release of gpt 3.5 turbo every ""upgrade"" has mostly been an actual downgrade. I have a battery of tasks that the initial 3.5 turbo (Nov 2022) was able to perform but the newer ones very consistently fail at, regardless of prompting.

I've been moving tasks from 3.5-turbo to Llama3-70b for this reason.

Very curious to see whether this time it'll be an actual upgrade instead of a downgrade.

[tempaccount420]

The original GPT-4 was an upgrade IMO. GPT-4 Turbo and GPT-4o were downgrades. GPT-4o seems especially bad (on text-to-text).

[maeil]

Yup! OpenAI's best public English-language text model to date is GPT-4, which came out more than a year ago, March '23.

But this hasn't just held for GPT-4, it's also the case for GPT-3.5 turbo, where I'd say the difference is even bigger! 0301 was the strongest (March 2023). Then we got 0613 (June 2023) and 1106 (November 2023), both significantly worse than 0301.

It's always fun to see on e.g. Reddit, ChatGPT users discussing whether GPT is getting worse or not, with clear "for" and "against" camps. To any production user that has done 1:1 comparisons, it's clear as day. Par for the course for Altman to go for this approach though, it's clear he'll do anything it takes. Taking a page out of the Tesla "FSM in 20XX " playbook of blatant lying to sell a product.

Note: For vision input, things have in fact been getting better. 4-o clearly beats the initial gpt-4-vision.