Last year, I had the great pleasure of attending the Games Industry Conference (GIC) in Poznań, Wielkopolska, where I sat in on a number of talks about game development and craft. In one of those talks, given by Petr Nohejl, devops programmer for Warhorse Studios, I heard a fascinating anecdote that got me thinking about the silliness of video games, and the lengths testers go to in order to ensure they run smoothly for players.
Nohejl’s talk was about Kingdom Come: Deliverance II and the challenges faced by programmers trying to debug the game, resolve crashes, and deal with gameplay hitches. It was far deeper and more advanced than the bit I’m presenting here, and admittedly pretty over my head in terms of technical specifics, as I’m not a game developer myself. But at one point, he was talking about setting up simple, repeatable scenarios in-game that can be tested automatically, and gave the following example:
There are some test rotations, so you just spawn the player, you wait, you as I mentioned…and then the player rotates a few times. So it just loads up all the stuff, the streams. And we have these random rotations all around the maps. So the rotation test just like teleports and performs the rotation on thousands and thousands of places on the map. And then we have actually a heat map of the whole rotation test.
Essentially, Nohejl is describing what I am told is a quality assurance test where the player is spawned in, they spin around a few times in place, and that’s it. I am told by several individuals I spoke to with QA experience and expertise that this is a very common test and, to most QA professionals, probably pretty boring. But to me, who’d never considered the granular specifics of what went into testing games, it was hilarious.
In the wake of this discovery, I put out a call on social media and through various PR contacts for QA professionals to share their most ridiculous game testing stories: essentially, what’s the silliest thing they’ve had to do to test something in a game? What came back was a delightful flood of anecdotes ranging across AAA blockbusters to small indies, canvassing a decade or more of game development. I am now thrilled to share these stories with you in hopes that you find the goofy incidentals of QA testing as funny as I do.
Shootin’ the Breeze
“While testing Unreal II for Xbox, if we angled our gun upwards by 45 degrees and shot the assault rifle at the bridge strut as we walked across it, the game would consistently crash. We tested that stupid bridge for weeks.” -Ben Kosmina
“When I was on Overwatch I had to test to make sure that the materials on the various surfaces were set up properly for hit effects. For example, when shooting wood it would sound like wood was being impacted instead of metal. There are a LOT of materials and surfaces in the various levels, but I had access to the editor tools so I actually made a testing hero to help make things easier. It was basically Widowmaker with a 1s cooldown on her hookshot, Hanzo's wall climb ability, and an expanded magazine size so I could shoot more freely. I also gave myself like 10x movement speed and the ability to toggle the bullet sounds on and off so I could hear the impacts better. Then I proceeded to methodically run around every map shooting every unique material surface to make sure they were making the right noises. When I moved off the team I was chatting with one of my friends and he was saying he still used that hero for testing, which made my day.” -Andrew Buczacki
“[On] Anno 2070, I had to do a series of tests and videos for the USK rating board. Spent several days dropping nukes over and over, zooming in to the little people showing that it wasnt graphic. Had a whole video & Images folder of a mother with a baby buggy no-selling a direct nuke at different angles.” -Ruairí Rodinson, Rho Labyrinths studio head
Inventing a Guy
“Hello! The whole story is a bit more complicated. The NPC in Mental Refreshment [in The Outer Worlds 2] was actually my 1st encounter with a peeing NPC and I thought that, for some reason, the invisible collision is the intended behavior displayed by a urinating NPC. Then, some days later, I saw an NPC performing an unfamiliar animation.
“I curiously walked up to him and, to my great surprise, he was peeing! At that moment the confusion started. ‘Why does the game let me get so close to him?’ ‘I couldn't even dream of coming this close to the other peeing NPC!’ And so, the investigation has begun.
“After bothering a few other innocent pissing NPCs it turned out that my fundamental assumption that NPCs are supposed to be shielded while relieving themselves was wrong! I dutifully reported the excessive collision on the mentally refreshed subject and now the User can peacefully watch him do his business from up close.” -Aleksander Gozdzicki, FQA Tester, QLOC, The Outer Worlds 2
“So I was a programmer rather than QA, but I remember having to get creative when I was testing some of my code on Kinect. I had some code that was meant to detect someone making air guitar motions, and I had to shove my hoodie up my T-shirt in order to check that it would work with pregnant people.
“When testing the 'new user' Kinect login sequence we had to wear carnival masks to stop it from recognizing us.” -Tim Aidley
“On Saints Row we had a debug npc named Skinballs (lol) that was literally just four spheres wrapped in different shades of skin texture so we could test lighting for different skin tones during development…teams need to plan for that kind of testing with intentionality, and not enough do.” -Elizabeth Zelle
“My favorite example of this was ‘The Carwash’ on Mortal Kombat (X? 11?) where a T-posed character would slowly move through a series of particle emitters, each one causing a specific type of damage - cuts, stabs, burns, gashes, ice, etc. so we could test damage models on characters.” -Daanish Syed, former Netherrealm Artist
“We used placeholder characters earlier in Sunderfolk’s development whilst character art was still pending. One in particular was a free character model from Adobe’s Mixamo site called ‘Brute’ which was essentially a large oily barbarian. He was regularly used to fill in the board and for cinematics to see how animations look and function. Affectionately named Oily Man, we still use him in-engine for hero spawn indicators in the encounter designer. Oily Man lives on!” -Ali Tirmizi, Sunderfolk QA Lead at Secret Door
WoW!
Andrew Buczacki graciously supplied me with a number of anecdotes from his time on World of Warcraft, which I’ve included all as one chunk for the WoW fans in the room:
“When testing the final boss of the Icecrown raid in Wrath of the Lich King, we had a bug where Frostmourne (the Lich King's sword) didn't appear in cinematic on a specific hardware configuration. They mobilized the entire QA floor for WoW to test every permutation of graphics setting, resolution, etc. to see if it was happening anywhere else. By the end of the testing day we had all watched the cinematic (and seen most of the raid fight), to the point where many of us had memorized the monologues and the cinematic. You'd randomly hear "Bolvar!" for months after.
“Also on that fight, I was assigned a hotfix where one of the AoEs the boss did was supposed to expand 15% faster. There were no good precise measuring tools so my friend and I set up a ridiculous testing contraption involving engineering smoke flares, a hunter pet, and liberal use of in-game cheats. At the end we were able to verify that it was at least expanding faster, although I argue we got a degree of precision that probably wasn't necessary by the end.
“Old time WoW fans may remember that over the course of a few patches the shoulderpads on some race/gender combos (male Orcs most notably) shrunk. This was eventually caught and fixed, but to make sure it didn't happen again one of the tests my team ran was to look at a fixed gallery of character models wearing various equipment on the current version and the newest build. We had an automated script that would log in and take pictures from fixed angles and equip the same things so we could compare screenshots to make sure that everything stayed the same size.
“Once I had to test and verify that the ‘Win a loot roll with a 100’ achievement was working. To do this I went to Onyxia's Lair (a popular sandbox for QA), and after not winning the lottery and winning loot with a 100 on Onyxia I spawned in about 100 raid bosses at once with god mode on. Then I used super powerful AoE cheat spells to kill them all and began the laborious process of looting and rolling on all of them. The achievement totally worked, by the way.”
CAN you pet the dog?
“So here's something that I'm pretty sure most people don't ever think about: Testing "Can you Pet the dog?" in games. Like, it's a feature a lot of people look for and adore in video games, but I don't think your every day average person realizes that, as cute as it is, it took people HOURS upon HOURS to get it right haha. So in Demonschool for instance, when you pet the dog, you'll get unique dialogue per interaction. The dog can also be found on different maps and depending on the date and how many times you've pet him, he might move to different locations. If you sit there and let Faye pet the dog, the camera will (very) slowly zoom in to give the player a better view of it.
“At one point in development, we came to find out of that one of the random softlocks we were encountering was actually tied to the adorable, lovable action of petting the dog. This meant that we had to spend HOURS testing every possible variable of petting the dog. Every location it showed up in, every dialogue string attached to it, every date it was available to pet. We'd skip petting it certain days and in certain locations to try to mix up where it appeared in the game on different dates to see if THAT was causing the issue. We had to make sure that the slow camera zoom was working on each and every map that the dog can appear on, and every VERSION of those maps (day, evening, and night).
“So what ultimately should be a fun, cute little easter egg that pretty much every random player enjoys seeing in a game, ended up becoming hours and hours of work behind the scenes to make sure it worked just right and wouldn't softlock the game for everyone haha. And after tons of hours and hundreds of pets, we can now all happily pet the dog without issue!” -AJ McGucken, lead QA at Ysbryd Games
Make Some Noise
“One time when I was doing audio for god of war, we had put a sound of a frog croaking on the frog asset. Well, one of the level designers took that frog and COVERED an island in them. So I had to make a JIRA ticket to kill frog island.” -Shayna Moon, senior technical producer
“Off the top of my head, I found a bug working on New Super Mario for Wii that was at the start screen. The game shows the title and plays a little song. At the end of the song, it transitions to "demo" footage of Mario &co jumping around. If you press start, you are taken to the starting menus (new game, load game, options etc) IN THEORY, but if you happen to press start at the EXACT moment between the two states of the start screen, the game instead hard locks and must be force-restarted.
“So, every day we got a new build, I had to regress this bug by sitting at the start screen and trying to press start at exactly the right moment. If it didn't lock, I had to back out and try again. It was extremely tedious and difficult to get right, proving a negative. Another tester was a little better at the timing than me so we would do this together every morning, just pushing start, for hours, like the world's worst rhythm game.” -Anonymous
In addition to all the anecdotes I received for this piece, I also conducted an interview with Camden Stoddard, audio director at Double Fine working on Keeper. Stoddard chatted with me about the complex process he went through to get Twig, the bird, to sound exactly right, which included a lot of strange testing practices.
The emotional bond between Twig and the Lighthouse in Keeper was critical to the game, so Stoddard spent a lot of time testing different things to get the communication between the two just right. He tried actual bird sounds and recordings and libraries. He tried using a foghorn with the Lighthouse, "which no matter what you do, a foghorn doesn't get that emotional," he said. So he muted the Lighthouse, which worked out better, but there was still the matter of the bird, Twig. I'll let Stoddard take it from here:
"I wasn't reaching it with the recordings of birds. So I just started studying how a bird talks, and it turns out their larynx is unique in nature. They have two. They have an avian larynx where they have a lower one called a syrinx. And basically, that means that their pitch control and their pattern control is ridiculous. They can just do things that most animals cannot with their voice.
"So I started thinking about that. I'm like, 'Well, okay, this bird is way beyond the emotional tone of an actual bird. So we need a human performance here. So how do I make a human sound like a bird?' So I just started fooling around with my own voice, and I wound up finding this software that was meant for electronic music. It was from an electronic group in Europe.
"And I had used it before, but what was interesting about it is a lot of things you can adjust, pitch and things like that, but this, you could adjust the harmonics trail. That was what was interesting. And as soon as I put my voice through it and I started messing around with it, I became a bird. My voice just started sounding like a bird, which made a lot of sense.
"So I started experimenting with, I would do the performance as the bird. I started studying how birds communicate, and then I started thinking about, "Okay. Well, if I can have the sound of the bird, but I can have the emotional punctuation and pattern that we want emotionally, then I think we got it nailed." So over about two years, I finally got how to do Twig. And the hard part on me was, the best way to do Twig was to inhale while I was doing the sound and not exhale, because when you inhale, your larynx becomes much more raw and kind of nonhuman-sounding. So if I inhale, you're like... You get this crazy range that you're not used to. And once I manipulated that, that's where Twig came from.
"So then all of a sudden, I became like an actor, and I had to figure out these heavy, heavy scenes, which I hadn't counted on, because it's a very emotional game. There's some heights of anger and sadness and surprise and true existentialism in the end. It's wild. So I not only had to go to the limits of my sound design know-how, I had to figure out how to make this bird very emotional and really care about this lighthouse."
Below is a recording shared with us by Stoddard showing the three-step transformation of his voice from his own bird sounds to the final sounds used in the game. It uses a Granelli SM57 dynamic microphone going through an Avalon 737sp preamp, and various software to edit it, including a combo of Eventide 910 and 949 harmonizers (software) and Manipulator by Polyverse (software). It was all recorded in Pro Tools.
Climb Every Mountain, Drop Every Weapon
"When working on the Mr. X Nightmare DLC for Streets of Rage 4, the team noticed that sometimes, when a player dropped their weapon, it would fall through the floor. We had to drop all the weapons to identify the issue, which turned out to be specific to a single weapon... the swordfish! Perhaps it's a new type of flying fish?" -Laura Peitavi, lead senior QA, Streets of Rage 4 DLC.
"In NINJA GAIDEN: Ragebound, the player character (Kenji) can climb up walls. However, when attempting to grab a wall with a small surface, he would often assume an unnatural position where it appears that he is supporting his entire body weight with a single arm. The level designers at The Game Kitchen could set an "is climbable" property for each wall in the Unity game engine, but there was no way to set this property to false automatically for every short wall found in the game. Therefore, Dotemu's entire QA team had to attempt to grab every single wall in the game in order to find which ones could lead to animation issues so that The Game Kitchen could alter the properties of the problematic walls." -James Petit, QA analyst - Ninja Gaiden Ragebound
A Castle Made of WHAT
“Back in 2022, well before we announced Wildgate, ‘Banana Castle’ was the nickname we gave to this example project that we sent to external vendors so they could help us identify performance bottlenecks in our physics simulation, without revealing what we were actually working on. Using stock Unreal assets, we glued these models together to test our in-game physics.” - Grant Mark, Wildgate technical director at Moonshot
“I had a test level [in The Outer Worlds 2] that was a tower where each room was a different physical material, which meant that some of the rooms had walls/ceilings/floors made entirely of hair or skin.
“Different ‘physical materials’ will have different sound/visual effects depending on the material. For example, if you fire a bullet at a concrete floor, the decal (a bullet hole), the impact VFX (sparks flying or whatever) and the sound effects will be different than if you had fired at the ground on a sandy beach. Footsteps also make different sounds when walking around on these different materials. In addition to things like concrete and sand, we also have physical materials for hair and skin (so that they can have the appropriate response to being shot at or whatever).
“If I wanted to quickly verify that these effects were working correctly it was useful to be able to load into this test level, teleport to the room with the physical material I was looking for, and then I could test for all of these material-specific features by shooting the walls and walking around on the floors and such.” -Josh Ledford, QA analyst, Obsidian for The Outer Worlds 2
Rebekah Valentine is a senior reporter for IGN. Got a story tip? Send it to rvalentine@ign.com.
Connexion
