[atrevbot]

As someone who has almost no familiarity with game engines, it seems the success of this port was largely possible due to a comprehensive test suite written in a runtime agnostic way. What might be the equivalent test suite implementation required to successfully port a game engine to another language?

[jamesfinlayson]

Gosh, I don't think any game engines have particularly good test suites at all. GoldSource and Source are the only ones that I have any real experience with and neither seems to have anything (Source may have a handful of things but nothing approaching baseline let alone comprehensive).

I have no idea how game devs handle big refactors other than lots of manual testing.

[sisve]

> Gosh, I don't think any game engines have particularly good test suites at all.

What? I do understand that a CRUD app with little to no logic do not focus on test.

But any kind of engine.. i can not understand that it's not a priority? In general I'm not a big fan of unit test on simple websites, they do not give any real value compared to effort

But I remember working on a complex codename. It was extremely important to have close to 100% unit test in the core part. Saved me a lot.

[ACCount37]

Mostly, because a lot of game engines are ancient relics, tracing their lineage all the way back to Quake 1.

The development practices are not exactly up to date, and game development is in no hurry to change. It doesn't help that software development wages there are not at all competitive - game development selects for passion, not skill. People who want to build robust modern codebases and people who want to build AAA games are different people. So there aren't many game devs who want to push for better test coverage.

But it's also because game engines are dealing with many, many things that are hard to test for.

You know how messy it is to test a website for "does this layout look right" or "can you navigate from A to B"? Now multiply that by complex 3D geometry. A lot of what game engines do is dealing with complex 3D geometry, where the primary verification is "does it look right" and "does this interaction feel right". Which is why game development traditionally has wide human QA, and slim unit testing.

Only now do we have software that can sort of, semi-reliably, automate testing for "does it look right".

[mike_hearn]

A game engine has a vast space of possible outputs, all of which will be considered good enough.

Its primary output is pixels and sound. Those are hard to test in a reasonable way. Screenshot testing is useless in a codebase where most of the changes are about making the pixels prettier.

[hvb2]

> In general I'm not a big fan of unit test on simple websites, they do not give any real value compared to effort

And then your site gets more traction, features are added, things are refactored.

And now you're looking at what used to be a simple site, still with no tests.

It's a habit, that pays off long term. You don't do it for yourself, you do it for who changes the code next. Might be you, or not. Even when you do it yourself, you can't hold all the requirements in you brain, you write those up in tests

[jamesfinlayson]

I agree - maybe some modern game engines have unit tests for stuff like their physics engine and their "standard library" code (optimised data structures, maths utilities, string utilities etc) but as others have said, testing sound and vision is hard, and something that hasn't been mentioned is all the different flags in a game engine - testing even a sensible subset of all the different sound and rendering variables would be a huge undertaking.

[zamalek]

One option would be to have an input replay alongside captured outputs (audio visual), at some fixed framerate. Capturing intermediates (scene graph etc.) would probably also be valuable, as that could help nail down why something is failing.

Or you could do it [as I recall the project being called] the scientist way. You still have the old code, so you could replay inputs against each and compare. Probably more realistic because uncompressed video would be a ridiculously huge dataset. This would be more resilient in the face of testing hardware and driver drift.

Historically game engines are the worst offenders when it comes to unit testing. I'm not sure if that's still the case - but that's why I erred on the side of integration tests.

[hdjrudni]

Box3D just showcased some stuff including deterministic replays. If you wanted to port that, you could probably import the replay and make sure it plays back the same way in your new language. I think it captures the inputs and forces applied, not the pixels.

I suppose rendering is a component of a game engine too though, not just physics. I don't know how to do that reliably. Even if you captured pixels, it'd be annoying. If you've ever tried doing screenshot based diffing on web you will know that slight changes in aliasing in Chrome bugger everything up. Things that should be equivalent randomly aren't but not in a way that any human would care.