[lisasays]

So how would GPT fare at writing a simulation for a problem ... that has no source code or even literature for it in the (crawlable) public domain?

Also, as to what the GPT group was able to produce -- sure it was a lot of code, and apparently a quite a bucket of features -- but did it actually produce a usable simulation? Or even a coherent statement of what a "baseball simulation" should do, actually, and how its accuracy is to be measured?

I'm not casting aspersions here - I'd really like to know.

[Balgair]

It does not produce a usable simulation of baseball right out of the box. It'll give you skeleton code that you kinda have to then fill in yourself. But it's really good skeleton code. Like, the functions are used wrong, but it's the correct function. The explanation of the code that it give you is really spot on though. Like, yes, those are the correct steps a coder should implement.

It's easy enough to try it out for yourself too! Give yourself a challenge and see where it takes you.

[lisasays]

I'll definitely give it a whirl sometime. And I appreciate the detailed field report.

It's just that, if someone gave me 3 hours, and asked me to come back with constructive, actionable progress toward creating a simulator for X (where X is sufficiently rich and complex, like baseball) -- I wouldn't mess around with skeleton code at all.

Instead I'd try my best to come up with a statement of what the simulator should do, and why.

[Balgair]

Yeah, in our case it was baseball and most of us are fans. So, we all knew what to do and what 'good' looked like. It was still pretty open ended though, which was fun. It was good to see what my coworkers came up with and the different approaches taken.