DVT

You are currently browsing articles tagged DVT.

Nope, this blog isn’t about test-driving a sports car, although a guy can dream.  Rather it’s about testing products at the various corners, not just at nominal. Which we do a lot–part of our We Design, We Test, We Break Things mantra.

So what’s a corner?  It’s when two or more use or process parameters are at an extreme. For example, a sports car designed to operate from -40 °C to +60 °C and from 85 to 100 octane fuel(hey we don’t actually design sports cars so this is just an example) should be tested at the following corners:

  1. High temperature  / High octane
  2. High temperature  / Low octane
  3. Low temperature  / High octane
  4. Low temperature  / Low octane

If there are N parameters, there are N squared corners. Of course some corners may be unnecessary–say if we wanted to test High snow or Low snow, the High temperature / High snow corner could probably be neglected.

While we neither design nor test sports cars, we do work on a stupidly wide variety of products, from clean-tech widgets to baby products, electric bikes to scientific instruments. Beyond that,we are involved from everything from very early engineering verification tests (EVT)  to formal design validation testing (DVT)  and even(especially) factory floor production testing. Early testing may be quick and dirty–we recently did – 10 °C to + 40 °C EVT testing utilizing cold Colorado mornings at one extreme and an impromptu sauna thanks to two space heaters-to full on regulatory testing at a certified lab (not us, but we know who to call). Besides temperature, other corner parameters could include humidity, operating voltage, tension, as well as physical tolerances, torque settings and a zillion or three other parameters.  And of course a good dose of Design of Experiments is needed when the number of corners gets excessive.

Chuck

PS: If anyone wants to loan me a Tesla, I’d be more than happy to test its corners.  As long as slow isn’t one of the parameters. No charge.

Tags: , , , ,

Test it till it breaks. The dirty side of product development, although admittedly a bit of an adrenalin rush at times.  Like when you hear that distinctive crack or when the top half of the UUT (unit under test) flies across the lab (and doesn’t break further–a good thing). Not that we want the stuff we design to fail, but better for it to fail in the lab before steel is cut for tooling, before inventory is purchased, before the customer touches it.

So some days we get to go into work and try to break things. We’re smart about it, we plan, we order the tests to maximize the information gain.  We rig up force gauges and o-scopes, temperature chambers and drop tables.  Inclinometers too (yes that is a word). We analyze specifications, we establish controls, we pre-test, we separate variables.

Then we try really hard to break things.  But more importantly we try to understand why something breaks.  So we can design it better.  So factory yields are better. So the product fails less often and less drastically for the end user.

And yeah, we do kind of enjoy it.  Breaking things I mean.

Chuck

Tags: ,

In this modern, über cost conscious age, where every organization is trying to lean up or cut corners, depending on one’s perspective, there are a few places where it’s just plain dumb to cut corners (see also It’s Not All About Costs).  One of those is safety.  But safety issue are not limited to medical instruments or heavy machinery or toxic gases.  One of the most potentially dangerous objects in most homes and work environment is the ubiquitous lithium battery.  They’re in toys, phones, laptops and yes, electric vehicles.  And they can catch fire and blow up if not properly handled, as exhibited by recent highly publicized instances  of iPhone  and Chevy Volt fires.

But here’s the rub: they absolutely will catch fire or even worse explode if the circuitry surrounding the batteries is not implemented extremely well.  Not can, will.  And furthermore, with products containing such batteries becoming more and more common place, many design teams and especially non technical management are starting to think the the design and test of the power management circuits that surround these batteries is simple.  It is not.

The technical issues are complex and completely unsuitable for solving via a blog.   But we see far too many teams hoping to bang out a “simple” design ignore the complexities of this. Just pick up a reference design  and ka-ching, a working design ready to sell.  Or KA-BOOM. Possibly literally.

The devil is in the details, designing  the power management circuitry carefully, considering both hardware and software.  If the software freezes while charging, or if, as was reportedly the case in one of the iPhone instances, heavily utilized, the charging circuitry still needs to function properly.  Related to this is the imperative of  identifying all possible fault conditions and failure modes via an Failure Mode Effects Analysis, or FMEA (see FMEA: the Most Important Acronym that No One Knows) and in developing and executing a proper design validation test, or DVT, plan (see also Common Misconceptions about Design Validation Testing). Test these circuits in the lab–smoke in the lab beats smoke on an airplane any day (see also (promise this is the last) The Smoke Test)

One more sobering thought, and then I’ll get off this soap box:  both the iPhone and the Volt were designed and tested by some of the best engineering teams on the planet with massive budgets and superb labs.  These are world class products designed by world class teams who most likely cut far, far fewer corners than 99.9% of the product development teams out there, and they still have problems. Do not treat power management design and testing lightly. There are engineers who know how to do this, at Zebulon Solutions or at many other product development companies, as well as in many OEM labs.  Listen to these engineers. Please, before someone gets hurt.

Chuck

Tags: , , , , , ,

Ever seen the movie where the wild-haired designer looks up from her or his laptop only to be greeted by a stern-faced production manager who says, “Your new design is failing at final test”?  And the designer responds, “Why are you talking to me.  It’s a test problem.” All the test engineers out there are grousing, “Seen that movie? Heck, I was the guy in the red shirt who died in that movie.”

No, of course they don’t make movies about engineers of any stripe, but the point is still the same…

It’s an all too common problem, blaming the person who finds the problem for the problem.  Since a good production test process–or validation test for that matter–is supposed to find problems and reject parts that do not function properly, it’s small wonder that such test processes detect problems.  Test engineers are supposed to design tests that identify failures.  It makes no more sense to blame the test engineer for identifying a design fault then it does to blame a doctor for identifying an illness.

Fortunately, after everyone gets “plumb tired of all that bell-fired finger-pointing” (old Zeb is looking over my shoulder and threw that there Zebulon-ism in…) there are proactive solutions.  Investing in a rigorous design validation testing (DVT) program should detect issues before they get to the production floor.  Further upstream, a Design Failure Mode and Effects Analysis (DFMEA) can help predict what might fail, the better to focus engineering and testing efforts.

And as for all those design engineers out there, myself and our design team included–best we take a test engineer out for lunch now and again. Because even with the best engineering rigor, the most through validation efforts, mistakes will still happen and we’ll be once again be glad that the test guys caught yet another problem before our customers do.

And no more undue blaming of test. Blame Murphy instead.

Chuck

Tags: , , , ,

The best word I know in any language is a Swedish word, lagom.  Basically doesn’t translate well to English but more or less means “just right.” In the sense of not too much, not too little, kind of a Goldilocks type concept.  To me lagom means seeking the proper balance, finding the right solution that is a proper compromise between all competing factors.  We at Zebulon Solutions are currently in the progress of preparing a DVT and production quality plan–spanning DFMEAs, DVT testing, production test development and supply chain quality plan–for a customer.  This customer is a late stage start-up, so well beyond the idea in a garage but still a start-up.  Frankly our first internal pass at defining this production quality plan was detailed, well thought out, and all in all really excellent–for a much bigger customer.  But it was anything but lagom for our start-up customer.

So we’re back to the proverbial drawing board (OK, no one uses drawing boards, or even drafting tables for that matter, any more–its all on PowerPoint) to try to right size this plan for the needs of our customer.  It still needs to be detailed and well thought out, but needs to be tailored like a Nathan Road suit, not off the rack from everyone’s least favorite mass retailer.  We’re working on it.

For more on lagom, see http://en.wikipedia.org/wiki/Lagom.

For more on productization (warning, shameless plug) see http://www.zebulonsolutions.com

No foolin’

Chuck

Tags: , , , , , ,

Ever ventured unwittingly into the shadow world of NoTest? It’s the hazy utopia where the ivory tower types proclaim that if the design is good and manufacturing is good then why test.  Sometimes this even came up in a snarky way, back when I worked in an integrated contract design and manufacturing (CDM) environment, as in “You did my design and you’re going to manufacture it, so then you don’t need test . And oh by the way take the test line item out of the open book pricing I made you provide.”  Followed by a sneer, or at least I envision a sneer in my memory when I think back unfondly on those days.

Of course doing without test in the real world is, well, fantasy.

Don’t get me wrong I’m a huge fan of doing a solid design, employing six sigma techniques where cost effective, and conducting serious design validation testing long before production release.  I’m also a fan of spending the effort needed to productize a design–get it ready for production and get production ready for the design.  But despite best efforts and intentions, I am also a believer in both Murphy, he of that oft quoted Law, and also in statistics, which basically say that even with a robust design and a low DPM (defect per million) manufacturing yield loss there are still product that will fail, and its better to know about a failure sooner rather than later.

Of course the level of test absolutely should be commensurate with the product robustness, yield issues, and the total cost of field failures (which while high are not infinite) including impact on brand, returns, customer satisfaction.  And while in reality NoTest is an extreme that is rarely ventured, many companies do neglect developing and implementing a production test plan that is as robust as their design methods and production practices. Often times this is due to lack of understanding of how test works; sometimes its part of the “beat up your contract manufacturer,” and all too often it’s just part of the productization chasm that widens with time as design teams and their manufacturing counterparts drift further and further part. (for more on this see our productization blog on the chasm analogy, http://zebulonsolutions.com/productizationblog/?p=41).

The right solution is really to put the same effort into making sure your test plan is robust as you do for design, marketing or any other function.  Start with requirements, look at tradeoffs, come up with a draft plan, scour it for cost effectiveness and get buy in from the cross functional team.  Make sure the risk / reward equation is properly balanced, and, as my Swedish friends would say, make sure the amount of test is lagom–neither too much nor too little.  And avoid the sirens as they lure you toward the shadowy land of NoTest.

Chuck

Tags: , , , , , , , ,

Old Zeb has been bellyaching again over on his blog, http://www.zebulonsolutions.com/zebsblog/, about the proliferation of what he calls alphabet leek soup—acronym—that we use in the productization business.  So as a courtesy to Zeb and everyone else, I thought I would take a shot at listing a few of the acronyms we use around the shop regularly.

In return maybe Zeb can tell me what the #&$%$ acknowledge the corn means.

Chuck

Product Development

DFE or DfE    Design for Environment

Environmental conscious design and design for environmental regulations such as WEEE and RoHS

DFM               Design for Manufacturing

DFT                 Design for Test

DFx                 Design For x  

Design review of everything: manufacturing, assembly, cost, test, molding, fabrication, supply chain, environment etc

DVT                Design Validation Test

FMEA             Failure Mode Effect Analysis

DFMEA          Design Failure Mode Effect Analysis

PFMEA           Process Failure Mode Effect Analysis

FEA                Finite Element Analysis

GD&T             Geometric Dimensioning and Tolerancing

HW                 Hardware

SW                  Software

FW                  Firmware

Embedded software

R&D               Research and Development

BOM               Bill of Materials

CR                   Cost Reduction

RF                   Radio Frequency

RFID               Radio Frequency Identification

LED                Light Emitting Diode

SLA                Stereolithograpically produced plastic prototypes

SLS                 Selective Laser Sintering prototypes

PCB                Printed Circuit Board

The raw board or fab

PCBA             Printed Circuit Board Assembly

The assembled PCB

Regulatory

CE                   Conformité Européenne,

“European conformity” in French.  The European Union’s mark of product conformity.

RoHS              Regulation of Hazardous Substances

Government regulations, originally in Europe, pertaining to what hazardous materials may not be included in electrical and electronics products

WEEE             Waste Electrical and Electronic Equipment directive

Government regulations, originally in Europe, pertaining recycling and disposal of electrical and electronics products

UL                   Underwriters Laboratory

FCC                Federal Communications Commission

Program Management

PM                  Project Manager or Program Manager

A program is larger in scope than a project and typically includes multiple projects

PGM               Program Manager

PJM                 Project Manager

PMO               Program Management Office or Project Management Office

PPMO             Program and Project Management Office

P3MO             Portfolio, Program and Project Management Office

PMI                 Project Management Institute

EOL                End of Life

PLC                 Product Life Cycle

Financial

ROI                 Return on Investment

IRR                 Internal Rate of Return

NPV and IRR are inverse functions

NPV                Net Present Value

NPV and IRR are inverse functions

P&L                Profit and Loss

B/S                  Balance Sheet

OP                   Operating Profit

EBITDA         Earnings before interest, taxes, depreciation and amortization

VC                  Venture Capitalist

Tags: , , , , , , , , , , , , ,

Design Validation Testing, often abbreviated as DVT, is an important part of productization. Put simply, DVT is what is done to prove that the product meets its requirements as manifested trough its specifications. This includes validating that the product meets functional and parametric requirements, including mechanical, hardware, software, environmental and regulatory.  Many products also have industry and / or end user specific requirements.

Like many aspects of productization, there are a number of common misconceptions about DVT, which in general tend to over simplify and ignore the importance.

Misconception #1: Verification = Validation

Perhaps due in part to the unfortunate similarity of first letter and suffix, many engineers and managers tend to confuse the two concepts, which leads to the hey I already verified the design; I’m done   issue.  Verification is the act of looking at the design implementation and verifying step by step that the requirements will be met.  In practice this is done ongoing as part of the design process and most verification is done before a product design is finalized.  Verification includes testing engineering prototypes, typically as part of the design iteration process.  Validation on the other hand starts when the design is complete (which means verification is also complete), the product is tooled  and configured like it would be used by customers.  Validation needs to be done at least semi-independently from design to avoid both an appearance of conflict of interest (see item # 4 below) but also to provide fresh eyes that can catch bugs long overlooked by those used to seeing them.

Misconception #2: DVT is just regulatory and environmental testing

Many engineers and organizations equate DVT with regulatory–FCC, CE mark, UL, etc– and environmental testing–temperature testing, vibration, drop test, etc.  This is not surprising as these are indeed the two areas that tend to get subcontracted out–most companies do not have these skills in house.  But validation in fact includes testing and validating all requirements.  Often this means testing the product not only in nominal lab conditions but in simulated worst case real world conditions, which then overlaps back in with environmental.  But worst case could also mean “the customer pushes two buttons at once” or other much more complex exceptions.

Misconception #3: DVT eats up to much schedule

This follows from the naive as soon as the design is done I should be in production mindset that in general is the bane of productization.  The tactical situation here is that tooling is often a long pole in the schedule and companies have a habit of wanting to get into production as soon as parts come off the tool.  Of course with closer scrutiny this misconception falls apart as at the very least the legal requirements inherent in regulatory testing tend to trip this one up.  In practice it is in fact rare that DVT is the true long-pole if planned and executed properly.  Clever productization professionals will craft a DVT plan that gets a head start pre tool finalization on parts of the design that are not impacted by mechanicals; conduct pre screens on all regulatory requirements prior to submission; and devise with management a risk vs reward schedule that allows for release to limited production at an intermediate milestone.

Misconception #4: The design team can do their own DVT, contracting out regulatory and environmental as needed

While in some organizations the design team do in fact have the detailed knowledge and skills needed to also do DVT, this is rare.  Furthermore market forces have long since forced most third party regulatory and environmental labs to focus on bulk testing at the expense of advising on what to test.  So often tests are needlessly repeated because the designers did not know exactly how to specify a test.  Lastly, from a governance POV, since DVT is a “check” it is rarely wise to let the proverbial foxes guard the hen-house.

More to come–I’m working on a set of white papers on each aspect of productization, including DVT

Chuck

Tags: , , , , , , ,

One aspect of productization, Design Validation Testing, often called DVT, and not to be confused with Design Verification Testing with the unfortunate same acronym, has been a hot topic in the halls of Zebulon Solutions the past few weeks (okay, hall, singular–we’re not big enough to have multiple halls).  How much to test, when to test, what to test?

We have a couple of companies that we are working with right now, who will go unnamed and even disguised up a bit.  Both are in the “industrial” space, but one is a startup (let’s call them “StartupRus LLC”) designing a  robotic manufacturing system and the other is an old-line mega company (let’s call them “Megastuff  Company”) adapting what is essentially a commodity consumer product for their industrial products.

So here’s the rub: StartupRus has essentially no plans for doing DVT on their product (or at least they didn’t until we offered to help them set up a plan) while Megastuff has 20 pages of specs referencing dozens of other specs for DVT–everything from salt spray to 10 year life tests.  Yet StartupRus’s product has extreme tolerances and many moving parts, while Megastuff’s product is of a class (although they do have a few legitimate special requirements to be fair) that literally can be bought at Walgreen’s with change back from a Jackson.  Go figure.

The answer for StartupRus is relatively simple–we’re helping them define and implement a lean, staged DVT plan that keeps the testing modest in their current beta stage (where they are, as their CEO proudly says, still using a bit of duct tape) but then putting together a robust final DVT plan before launch.  The answer for Megastuff is more complicated since they have 50 years of culture to fight, and it’s likely a losing battle to even mention that maybe they should lighten up on the testing.

All for now.  Gotta get back into the lab for more testing.

Chuck

Tags: , ,