It’s easy to overlook the importance of quality assurance as a software developer. We all know that QA is something that we need to give appropriate attention to, but it can sometimes feel like a drag on your productivity. After all, building a new feature often gives you a tangible result; something you can immediately see and use. This is much more satisfying than performing QA tasks, which either confirm that your code is working as expected, or tell you that you have more work to do.
Even though I work on DoneDone, a product that is built specifically for QA, I still run into “tester’s block” from time to time. So it’s nice to occasionally take a step back and look at the QA process at a different angle.
Lately I’ve been reading a few books about the history of the video game industry, such as Super Mario: How Nintendo Conquered America, and Console Wars: Sega, Nintendo, and the Battle that Defined a Generation. The stories in these books appeal to me on two fronts: as a consumer, I love reading about some of the most nostalgic products from my childhood, like the NES and all its games. As a developer, it’s interesting to see how these companies planned, developed, and marketed these products (and how some their biggest successes nearly failed before they launched).
But one story that appears in these books has stuck with me over the past few weeks, and it centers on just how well Nintendo’s staff understood and respected the QA process. And more importantly, how dedicating their company to QA helped them build a multi-billion-dollar empire.
The USA gaming industry was booming in the early 1980s. Arcade cabinets could be found in restaurants, bars, laundromats, and just about any walk-in business. Gaming companies were not only producing hardware and software for businesses, but were also building home console systems targeted at consumers, such as the Atari 2600, the ColecoVision, Magnavox’s Odyssey 2, and Mattel’s Intellivision (plus many more).
As more console options became available, more and more publishers jumped into the industry and began producing new games as quickly as possible. Many consider the hastily-produced 1982 Atari game E.T. the Extra-Terrestrial as the perfect example of what went wrong with the market: a heavily-promoted game based on a blockbuster property, but developed by a tiny team (essentially a single developer) in just a few weeks to get it out the door in time for Christmas. The game was so bad, and sales were so dismal, that Atari famously buried truckloads of E.T. cartridges in a New Mexico landfill just to get rid of the excess inventory.
Many problems combined to cause the 1983 video game crash, but the biggest issue seemed to be a lack of quality control. Console manufacturers were not effective in proofing the games for their systems, allowing poorly-produced titles to quickly flood the marketplace. When parents experienced a disaster on the scale of E.T., many swore off video games as an expensive passing fad, and refused to buy any more systems or software.
Nintendo, a Japanese company who had found great success in America with their 1981 arcade game Donkey Kong, saw an opportunity to reinvent the home console market in 1985. They had developed a new home console called the Famicom (Family Computer), which was selling well in Japan despite the USA gaming crash. But they understood that American parents were extremely apprehensive about wasting money on any more shoddy games, so Nintendo knew they needed to avoid the same mistakes as their competitors.
So Nintendo established a publication process centered entirely around quality assurance. To avoid badly-produced software, games sold for the American version of the Famicom (named the Nintendo Entertainment System, or NES) must include the Nintendo Seal of Quality, which was only granted once a game was tested and verified by Nintendo itself. Nintendo’s testing process was especially vigorous and time-consuming, as they understood that just one overlooked bug could ruin their system’s reputation (and sales).
To further enforce quality software, Nintendo only allowed a publisher to submit a maximum of 5 NES games per year. This prevented companies from churning out title after title and flooding the market with subpar games. While this might reduce the number of games consumers could purchase, and result in less revenue year-to-year, it ensured that the games that were available would be memorable and high-quality. This long-term strategy would build consumers’ trust in the system and company, which more than made up for the smaller sales volume.
Finally, to prevent shoddy hardware, Nintendo required that all publishers source their game cartridges from Nintendo itself. And to prevent manufacturers from building and selling knockoff NES cartridges, the NES was designed to reject any cartridge that did not contain a special proprietary microchip.
Many publishers and developers were unhappy with these new QA practices. They felt it limited their ability to produce software at their own pace, and that it cut too far into their bottom line. They felt Nintendo’s philosophy gave Nintendo too much power, but unfortunately, the NES was selling so well that they had no choice. If a company wanted to make a video game, they had to play by Nintendo’s rules and live up to their standards.
Consumers, on the other hand, were ecstatic. The NES hardware was extremely reliable, not including the occasional game read errors that could be supposedly be solved by blowing into the cartridge (officially denied, by the way). Nintendo also stood behind their products with an excellent warranty and repair service, and the NES was officially supported by its manufacturer until 1995. NES games were high-quality and didn’t include the confusing or game-breaking bugs that were more common in the Atari generation. And while it might take games longer to be released, the NES library eventually grew to over 700 licensed titles in its 10-year lifespan.
Nintendo’s vigorous QA practices might have only been part of their overall strategy, but it established trust and reliability in their American customers and turned them into a household name. Nintendo even established a toll-free hotline that children and parents could call if they got stuck in a game, proving that customer support can also play a vital role in a product’s QA process.
Nintendo’s process also paved the way for other home console manufacturers’ QA policies. Sega, Sony, and Microsoft all followed suit with similar practices for their home consoles to ensure their published games are as bug-free as possible before release.
So the next time you’re sick of testing, take a break to think about Nintendo’s playbook. After all, one could argue that good QA is what helped the NES grow to an estimated $5 billion in annual sales, and re-established home video games as a major industry that’s still growing today.