[tokyodude]

I'm not sure what most teams do now-a-days but I went to GDC in Koln and saw the Croteam talk. Over 10 or so years by programmers just adding a little here and there as it occurred to them they had build a pretty cool testing system.

First they had made it so if someone was playing the game and saw a bug they could press the "file a bug" key, type in a description and the game would save out enough info to bring someone back to that point in the game, same camera, and possibly other state. From the bug database they could click a link that would launch the game back into that state, let someone verify the fix and mark it as fixes.

The also had a waypoint system for bots to play through the puzzles (this was the Talos Principle they were talking about). If the bots ever got stuck, as in didn't make to the next waypoint within some time limit they bots would file a bug using the system above.

https://www.gdcvault.com/play/1022784/Fast-Iteration-Tools-i...

As another interesting idea apparently the creator of Thumber built a URL system so people press a button which would generate a URL into the clipboard, they could then paste that URL into Slack (or email/chat/etc) that would launch the game in a particular state to pass that other users on the team.

[corysama]

Similarly, on a game I worked on we had a “controller monkey” that would spam all possible player actions (move, jump, attack, special powers) rapidly and randomly. We then had the testers record multiple paths exploring through each level. Every few seconds the monkey-controlled player character would be teleported a bit further down the path to ensure progress and coverage. Dozens of people would set the monkey to run all night when they went home for the day. In the morning we would have a dozen fresh crash dumps.

[pfranz]

I feel like when people just say "tests" there's a lot of conflating. Usually, on a site like this or on a blog post, "tests" are referring to automated unit-testing or talking about TDD...which is probably most rare and most fought against in game dev (from what I've seen from an adjacent industry).

Testing, in general, is pretty essential to writing code that does what you want. Test code is just automating what you'd be doing manually and it's a lot faster to have the code do it than for me to do it 10 times. Even if that's printing out a value or showing it in a debugger. Testing frameworks or libraries to fuzz out problems and harden code are quite common. Game dev often has a lot of manual play testers. Most engines and dev consoles have a lot of tools to either just record the screen or save state when problems are seen.

Heck, most "cheat codes" were added to jump around the came to test for bugs. That's technically "test code."

[tokyodude]

> Testing, in general, is pretty essential to writing code that does what you want.

Shipped 17 games, several AAA games with no automated testing. Not saying that was good only that it happened and so is at least some evidence that automated testing is not essential.

The problem with automated testing in games is most of it will be content specific and the content changes multiple times a day. A product like say GTA5 has 20-30 programmers and 300 artists and game designers. Those 300 artists and game designers are adding new content and or changing old constantly. Often they also use a scripting language or a blueprint like visual language to setup in-game logic like "open blue door if player has blue key". There's just too much to test. Every day 1000s of tests would have to be re-written because some boxes, doors, etc were moved .5 units to the left.

There isn't zero value in automated testing (see above) but my experience with testing on a large project, Chrome, was that it really slowed down my velocity. Easily by much more than 50%. Of course Chrome is a platform and absolutely needs the tests IMO. I think a game engine would benefit from lots of automated tests. The game itself though it gets harder to figure out where the balance is between automated testing and manual testing.

As others pointed out as well big teams, like an OS team or a browser team often have dedicated staff to setup and maintain a testing infrastructure. Game teams rarely have this. Maybe they should but few games are given the budget.