The world of Quality Assurance can be a potentially complex job and to be fair... that’s correct! You get to oversee test cases on 3 different trackers under the pressure of knowing that if you screw up, your client is out 50k!
(I have 5 years of experience working for an outsourcing QA company (Full disclosure: I got fired months ago for something that was totally my fault). More importantly, I’m not American - Thus there’s caveats everywhere as my experience won’t be the same as what you’ll find in the good ol’ US of A (Mainly, we have some sort of worker protection here) and it won’t be the same as someone who worked directly for a developer or a publisher. Practically every former QA employee has a different story and opinion on the business.)
“You’re joking about that 50k, right?”
I most assuredly am not. Today, I’ll be talking about Certification, which is the process that major platform manufacturers go through when judging whether a game can be released on their platform or if they refuse the game as is. That process, when I was more embedded and could learn these things, cost 50k for at least one of them. Think of it as the game’s final exam before it can be released on that platform.
As such, since this is a critical part of a game’s development, it stands to reason that there’s QA for it. Now, I do not believe that many outsourcing companies out there offer that as a vendor item, but it is still A Thing and thus must be brought up as part of this series.
“Wow, guess that must be a lot of pressure.”
My co-workers who often did it did not really experience that kind of pressure - The company I worked for had several processes that had not been implemented for regular QA to avoid mistakes as much as possible, and had the kind of vetting and review that other bugs did not ordinarily have.
One thing that I did not talk about in my nitty-gritty article and that I really should have is that every bug entered in a database usually has a classification of severity, priority or both. It depends on how the database is set-up. All bugs relating to certification are always of the highest priority, as anything that can fail a certification pass from the manufacturer is pretty much something that prevents the game from releasing on time.
So these bugs need to be as precise and clear as possible so that the issues can be fixed. Even if the issue is, quite literally, “Xbox Live is not capitalized properly”.
That is correct - Terminology is a big thing across all major platforms and messing with that is a good way of getting a failed submission. Of course, there are some other, more basic things such as “Game can be completed” that are present on the lists. As the requirements for each platform change per manufacturer, I will not go into details here, but typically anyone serious about releasing a game on console will have access to those on the developer section of the platform manufacturer’s website (or can contact the platform manufacturer for a copy of these requirements).
Since these documents encompass practically every possible scenario possible to happen on a platform, there are times that certain aspects simply will not apply, or a specific game functionality (Example: Any game that willingly induces a crash as part of a gameplay mechanic) goes against a specific points in the manufacturer’s requirements. It’s up to the developer to ask for a waiver for these requirements and then communicate to the QA team when the waiver is received (or the QA team to ask if that is possible) so that they do not check against it, or mark the test case of that requirement as a failure.
“I guess that makes sense. Anything else to know about certification?”
Honestly, the practices vary so much between platform manufacturers that without getting into each one specificially, I really can’t say much more. While I was trained in certification QA, I never actually did any certification work. Still, Certification is a key part of the process of releasing a game, and even if I didn’t have much to say but generalities, I hope you learned something from it. As always, don’t hesitate to fire up questions and I’ll answer if I can.