[happypumpkin]

Another data point, I definitely find Opus better for coding, but not by much. The problems I give them are generally short (<= 100 lines) and well-defined so any advantage Opus has in larger contexts won't be apparent to me. They're also generally novel problems but NOT particularly challenging (anyone with a BS CS should be able to solve them in < 1hr).

I have them working with mostly C++ and Clojure, a bit of Python, and Vimscript every once in a while. Both models are much better at Python and fairly bad at Vimscript. Clojure failure cases are mostly from invented functions and being bad at modifying existing code. I can't pick out a strong pattern in how they fail with C++, but there have been a few times where GPT4 ends up looping between the same couple unworkable solutions (maybe this indicates a poor understanding of prior context?).

[xetsilon]

Spot on. People need to say what they are actually using the models for and not just "coding".

I mostly use it to make react/javascript front ends to a python/fastapi backend and chatGPT4 is great at that.

I tried to write a piece of music though in the old Csound programming language and it barely even works.

It will be interesting to see how the context plays out because I have noticed that I can often give it extra context that I think will be helpful but end up causing it to go down a wrong path. I might even say my best results have been from the most precise instructions inside the smallest possible context.