[SCdF]

Can you give some examples of what you're actually testing? Perhaps an example of a specific test statement, the kind of level the tests sit at etc, what gets covered and what doesn't etc?

Are you testing the rendering engine? Lighting / physics / shaders / collision detect?

Are you testing scripting? Win conditions? AI (probably not for SoT I guess)?

Are you testing the core engine? File mgmt? memory mgmt? input reading etc?

As someone who is a developer and a gamer but has never combined the two, the idea of TDD in that scenario is fascinating.

[speps]

We are unit testing all new code that can be unit tested, trying to do dependency injection when possible so mocking is easier.

We are testing the rendering engine, lighting, shaders, etc. in a visual test framework we created ourselves that compares each run of the tests with the previous and uses SSIM to output a comparison metric. At the moment, that's still checked by someone every week but could be automated. However, when changing rendering code or shaders, we do require that a test result is shown during code review.

For physics and collision detection, we do have tests covering that too, a lot of them involving our FFT water simulation as well.

We are testing the whole flow of the game from start menu to entering a session, matchmaking, loading a map etc. I think AI also has some good coverage, notably for the navmesh which is essential.

The core engine is Unreal Engine 4. It's not an engine made for testing unfortunately, even though the vanilla version has some (but not nearly enough). We've stuck with a specific version for a while now as upgrading usually meant that we'd also get new bugs from the new version. All the core is covered by all the existing tests, we've added tests where we thought it was critical and where we fixed or improved internal code of the engine.

Before joining Rare, I really wasn't aware of any testing whatsoever in the video games industry. It's been an eye opener and I wouldn't do it any other way.

Reading other comments, we might not follow TDD all the time or to the letter but the point is that testing itself is a very part of everything we do as engineers. That allows our QA team to focus on stuff that is actually broken and can't be tested (such as if a feature is actually fun).

[SCdF]

Thanks for your reply.

> We are testing the rendering engine, lighting, shaders, etc. in a visual test framework we created ourselves that compares each run of the tests with the previous and uses SSIM to output a comparison metric

So would this be something like rendering a frame of a test scene (utah teapot? real in-game scene?) in various states of rendering (with / without lighting, with / without specular / normals etc), and then making sure they look the same each time?

> Before joining Rare, I really wasn't aware of any testing whatsoever in the video games industry. It's been an eye opener and I wouldn't do it any other way.

You should definitely consider a GDC/etc talk, I imagine it would be an eye opener for a lot of people.

[speps]

> So would this be something like rendering a frame of a test scene (utah teapot? real in-game scene?) in various states of rendering (with / without lighting, with / without specular / normals etc), and then making sure they look the same each time?

Yes, exactly, it's small test scenes we setup ourselves whenever we add a new rendering feature or fix a rendering bug. Some of them are still not deterministic as we're using Compute Shaders for some stuff which is hard to make deterministic.

> You should definitely consider a GDC/etc talk, I imagine it would be an eye opener for a lot of people.

We are currently looking for topics. We'll see what comes out of that discussion internally :)