[macrolime]

There are no signs of diminishing returns just yet though, and no one knows if that will be at GPT-5 or GPT-5000. I suspect the performance will keep increasing drastically at least until we have a model that's been trained with essentially all available text, video and audio data. Who knows what will happen after we have something that's been trained on all of YouTube. After this maybe we (or an AI) will have figured out how to keep improving without any more data.

[coldtea]

>There are no signs of diminishing returns just yet though

Sure about that? GTP-4 doesn't seem 5 times better than 3, much less 10x. Despite having 5/10x the parameters.

[gremlinsinc]

yeah it is, gpt3 scored in the 80th percentile for the bar, gpt4 scored top 20 percentile and is much better at math, plus having 4x the context alone gives it much more power.

it's just it's different in capabilities. chatgpt delivers different results and both have unique characteristics.

gpt4 being able to not only create images but also decipher what's in then is another huge advancement.

Gen2 another ai can create amazing videos from a text prompt. Any director or film maker wannabe with more prowess on creating the story than filming it, can now just use ai to create the film from their vision.

even more exciting is the speed that things are progressing. it was supposed to take 8 years to get chatGPT quality training down to 400k price instead of millions. Stanford did it in 6 weeks with llama and alpaca. it can run for under 600 or slower on home PCs.