[mentos]

Shouldn’t the next model focus not be on code but system design?

Seems like the work from a good system design to code is practically solved.

Now it’s a matter of the design of the system. Or is that represented in these evals?

[dist-epoch]

Have you tried system design with LLMs? I find them pretty good at suggesting 5 architectures for a problem and then iterating on the solutions.

Even if I had no idea, going with the default suggestion would not be a terrible mistake, assuming you did describe your requirements relatively well.