test plan

Ever ventured unwittingly into the shadow world of NoTest? It’s the hazy utopia where the ivory tower types proclaim that if the design is good and manufacturing is good then why test.  Sometimes this even came up in a snarky way, back when I worked in an integrated contract design and manufacturing (CDM) environment, as in “You did my design and you’re going to manufacture it, so then you don’t need test . And oh by the way take the test line item out of the open book pricing I made you provide.”  Followed by a sneer, or at least I envision a sneer in my memory when I think back unfondly on those days.

Of course doing without test in the real world is, well, fantasy.

Don’t get me wrong I’m a huge fan of doing a solid design, employing six sigma techniques where cost effective, and conducting serious design validation testing long before production release.  I’m also a fan of spending the effort needed to productize a design–get it ready for production and get production ready for the design.  But despite best efforts and intentions, I am also a believer in both Murphy, he of that oft quoted Law, and also in statistics, which basically say that even with a robust design and a low DPM (defect per million) manufacturing yield loss there are still product that will fail, and its better to know about a failure sooner rather than later.

Of course the level of test absolutely should be commensurate with the product robustness, yield issues, and the total cost of field failures (which while high are not infinite) including impact on brand, returns, customer satisfaction.  And while in reality NoTest is an extreme that is rarely ventured, many companies do neglect developing and implementing a production test plan that is as robust as their design methods and production practices. Often times this is due to lack of understanding of how test works; sometimes its part of the “beat up your contract manufacturer,” and all too often it’s just part of the productization chasm that widens with time as design teams and their manufacturing counterparts drift further and further part. (for more on this see our productization blog on the chasm analogy, http://zebulonsolutions.com/productizationblog/?p=41).

The right solution is really to put the same effort into making sure your test plan is robust as you do for design, marketing or any other function.  Start with requirements, look at tradeoffs, come up with a draft plan, scour it for cost effectiveness and get buy in from the cross functional team.  Make sure the risk / reward equation is properly balanced, and, as my Swedish friends would say, make sure the amount of test is lagom–neither too much nor too little.  And avoid the sirens as they lure you toward the shadowy land of NoTest.

Chuck