test

You are currently browsing articles tagged test.

Fail early, fail often. Fail quickly, fail smart. Repeat.

May 6, 2013 in Uncategorized | No comments

People learn from failures far more than success.  And if you are to fail, do it early. And do it quickly, for this gives you time to fail again.  The last word is the key: repeat  Valid for people, but also valid for products.  For products, we want to make them fail often and early, as if before the product gets to market.  And failing repeatedly in the design process is a great way to find the 2999 ways not to build a light bulb. So we fail, and fail and fail, and we correct our approach, and fail again.  Often times this approach yields far superior results to analyze, analyze, analyze.

That sad of course, some failures carry a higher price than others.  And the cost of failure can’t be ignored, even in the early stages.  Destroying a prototype can not only have financial considerations but also may set back the schedule.  So failing smart is also part of the equation.

A great new engineering tool for frequent failers is the 3D printer, a godsend for mechanical designers.  We’ve spent so much money on 3D printing the last few months that we’ve made a consderable downpayment on our favorite prototype shop’s new company Ferrari. Actually, this is an exageration, because 3D printing has become cheap enough that we can design, test, fail, redesign.  Often.  We try out new ideas, we develop test plans, we break things. We break necks.  And motors and PCBAs and hinges and latches and circuits. We learn from these failures and try again.  And sometimes again. And again. And again.

On the flip side of the coin, products that fail in manufacturing, in the field, in the end customer’s home, well, that’s bad.  The whole reason we want to test the ___ out of our products is to lower the odds of that happening.  Besides testing, we use predictive tools like Failure Mode Effects Analysis (FMEA) to get ahead of the potential failure modes, and to optimize our testing budget.  This comes back to the fail early part of the equation.

To failure!

Chuck

Tags: , , ,

Bridging the productization chasm

April 23, 2013 in Uncategorized | No comments

Hidden from view to most travellers on the Design Highway, the productization chasm looms into view just as the pinnacle of Mt. Volume Production seems obtainable.  Many a valiant design project has cracked on the rocks below like a gull dropping an oyster.

Bridging the productization chasm

To cross the productization chasm we need a bridge. And the best way to build a bridge is to start from both sides and work toward the middle.

For design folks, this means thinking about production early on, designing manufacturability in from the get-go, thinking about test and process and quality and yes, the dreaded S word, supply chain. It means being proactive, doing tolerance analysis before assembly problems are flagged, conducting DFMEAs (Design Failure Mode Effects Analysis) early on.  It means talking to manufacturing.

For manufacturing folks, it’s actually not just saying, hey design guys you have to follow our rules.  It’s about understanding the nature of the design process understanding that first prototypes are meant to be tested, broken, and redesigned (see We design, we test, we break things). It means taking he time to understand the product functionality and also its market positioning.  Productizing a million-piece-a-month consumer product is vastly different from productizing the transmission of a pickup truck. And it means talking to design.

For both sides understanding and often times compromise are needed.

As for us, well, all too often we get called in after the bridge fails or when no one even considers the need for a bridge.  Good for business maybe but we’d far prefer to be involved early on, helping both sides bridge the chasm.

Chuck

Tags: , , , , , , , ,

We design, we test, we break things–but why does that matter?

April 16, 2013 in Uncategorized | 1 comment

Steph, our new marketing maven, likes the pithy blog post title I did a while back, “We design, we test, we break things.” She thinks maybe that should be a new mantra for our prductization services.  I like it too (after all I did think it up), but I also worry that it’s missing one key element: why this matters to our customers?

Design is the easy part: before our customers can sell a product, it has to be designed.  Ergo value.  Ergo this is why there are thousands of design firms out there ready, and for the most part, able to perform that value add service.

Test is a bit tougher to quantify the value.  We test to verify the product meets requirements (aka Engineering Verification Testing, or EVT). We test to validate that the product meets specs (aka Design Validation Testing, or DVT.  We test each individual unit to make sure it’s a good one (aka Production Test).  This adds value indirectly, in avoidance of field returns, in avoidance of damage to brand.  It’s a necessary evil, or at least a necessary cost, to selling a product. Because failing in the lab is a whole lo cheaper than failing at the customer’s site.

Breaking things, well that sounds like fun but how does it add value? Because if we do this early enough in the product development process, we can learn how products might fail.  We can learn what tests a product might not pass.  And most importantly we can feed back that information into the design process and modify the design so that our customers’ customers don’t get the same pleasure of breaking things that we do (are we greedy?).  We learn by breaking necks, by running products so hot they stop working, by testing hinges so many times they fracture, by testing electronics until the solder joints fatigue. And this, very very indirectly, makes a big,very very tangible,  difference to our customers’ long term profitability. For the better.

Still looking for a revised pithy saying: “We design, we test, we break thing so ________________”

Chuck

Tags: , , , , ,

Done is better than perfect sometimes even applies to productization

March 18, 2013 in Uncategorized | 2 comments

Buried in all the articles about Facebook COO Sheryl Sandberg, a mention about a sign on her wall at Facebook that reads “Done is better than perfect” or some such caught my eye.  In the software universe this is the hacker creed in a nutshell, and obviously for Facebook and Facebook wannabees around the globe this mantra works.

But does it work for physical products, where changes are not as easy as just releasing V1.3.5.7 of the software? For a physical product, any changes post design release must be covered by Engineering Change Notices, or ECNs.  ECNs in turn can lead to scrapping of inventory, mandate changes to processes and tests, cause production delays due to lead time of any newly designed in parts and can trigger the need for additional validation testing or even regulatory approvals. A heck of lot more difficult than just releasing a software patch.

As such, we have often been a little loathe to adopt such hackers creeds fo productization.  But.. sometimes Done is better than Perfect, even for productization. Done but slightly imperfect products can get into beta customers’ hands, can garner marketing feedback, can start to build brands.  Can start to build revenues.  Of course too imperfect and the customer complaint hot line starts ringing, returns roll in, and the ECNs stack up on the production manager’s desk. So moderation is required, much more so than in the software industry.  But we can also pick our battles: firmware, for example, can be revved much like software in some cases.  Steel-safe tooling changes are another great example. And testing prototypes till the cows come home, while satisfying (we do love to break things), can lead to paralysis.

Like much in productization (and in life),my favorite word in the whole world, Lagom, the Swedish Goldilocks word–not too much, not too little, just right-applies. We can accept almost done and close to perfect sometimes, even for productization.  Stamp it Done and move on.

Chuck

PS–I blogged on the same topic today, how Done is better than Perfect can apply to creative writing, my side gig.

Tags: , , , , , ,

Broken necks

March 12, 2013 in Uncategorized | 1 comment

We’ve been breaking a lot of necks lately.  And bases and circuits and hinges.  No bodily damage to humans, or animals for that matter, just prototypes.  Prototypes that we breed just to be broken, stunted things missing non-essential body parts.  Plastic necks and metal rods, printed circuit boards and printed latches. Gears and latches and power transistors too.

Bang. Snap. Crash. Click.

Click is the worst, because you hear something give but no pieces are flying across the lab.  Sometimes we need a microscope to find click. Crash can be fun. “Duck,” well that’s scary for different reason.  And crash followed by “&%^$!” followed by “Todd, are you all right?” well that’s downright terrifying.

Don’t worry, Todd is all right. He’s very careful actually.

We design, we test, we break things. Almost our motto these days (maybe it should be).  The last piece is the most fun.  And sometimes even the most useful, because by breaking prototypes we learn how to optimize the design and manufacture so that the end product does not break. And of course we aren’t just breaking stuff willy-nilly and no sledge hammers are employed.  Instead we use force gauges, 50-lb kettle balls and the occasional deep sea fishing scale to pull and prod, poke and pry.  Then we build more prototypes and break them too.  Buy stock in all those companies that build plastic printers, ’cause they are great for all this.

Gotta go–just heard an odd clink from the lab.

Chuck

Tags: , ,

Testing the corners

February 20, 2013 in Uncategorized | No comments

Nope, this blog isn’t about test-driving a sports car, although a guy can dream.  Rather it’s about testing products at the various corners, not just at nominal. Which we do a lot–part of our We Design, We Test, We Break Things mantra.

So what’s a corner?  It’s when two or more use or process parameters are at an extreme. For example, a sports car designed to operate from -40 °C to +60 °C and from 85 to 100 octane fuel(hey we don’t actually design sports cars so this is just an example) should be tested at the following corners:

  1. High temperature  / High octane
  2. High temperature  / Low octane
  3. Low temperature  / High octane
  4. Low temperature  / Low octane

If there are N parameters, there are N squared corners. Of course some corners may be unnecessary–say if we wanted to test High snow or Low snow, the High temperature / High snow corner could probably be neglected.

While we neither design nor test sports cars, we do work on a stupidly wide variety of products, from clean-tech widgets to baby products, electric bikes to scientific instruments. Beyond that,we are involved from everything from very early engineering verification tests (EVT)  to formal design validation testing (DVT)  and even(especially) factory floor production testing. Early testing may be quick and dirty–we recently did – 10 °C to + 40 °C EVT testing utilizing cold Colorado mornings at one extreme and an impromptu sauna thanks to two space heaters-to full on regulatory testing at a certified lab (not us, but we know who to call). Besides temperature, other corner parameters could include humidity, operating voltage, tension, as well as physical tolerances, torque settings and a zillion or three other parameters.  And of course a good dose of Design of Experiments is needed when the number of corners gets excessive.

Chuck

PS: If anyone wants to loan me a Tesla, I’d be more than happy to test its corners.  As long as slow isn’t one of the parameters. No charge.

Tags: , , , ,

Return on Luck

December 11, 2012 in Uncategorized | 2 comments

One of my first mentors told me, “All things equal, I’d rather be lucky than good.” As much as we would all like to think that business is about skill, talent, planning, and the like, it’s also, just as with most things in life, also about luck.

But there is such as thing as making one’s own luck, and more importantly, being prepared when luck strikes. Return on Luck is just as important as Return on Investment.  Great companies have their ups and downs like everyone else, but they are ready to pounce when the die favor them.  Having a plan to deal with an uptick in orders. Building impulse capacity into the supply chain. Being prepared to accelerate market launch if the preliminary design validation testing results are promising. Having a plan for three design spins but also a plan if no spins are needed.

On the flip side, being prepared for luck does not mean neglecting contingency planning, backup plans and the like.  And sometimes luck favors the prepared.  We  at Zebulon Solutions have been looking for  a bigger space for some months. On of the first places we looked at was perfect for our needs but when we inquired about leasing it we found out someone had already put an offer in.  So we kept looking, found a couple of other spaces that would work, and made arrangements to stay in our existing space a little longer if needed.  But we heard through the grapevine that the landlord for that space we really liked and the perspective tenant were not seeing eye to eye on TIs (tenant improvements) so we kept bugging the realtor to keep us in the loop if that deal somehow fell through, even while we started to haggle on our second choice. Then out of the blue we found out that the deal had fallen through on our preferred space, and we snapped it up, getting a fair deal and the modest TIs we needed.  And since we had already done our homework on market rates, lease terms and the like we were prepared to move quick. We moved in yesterday, into a space that is perfect for our needs, with offices, a nice conference room, and a big lab where we can build, test and break things. And room to grow.  Yup, we got lucky.  But we also had made an investment in that luck, and it paid off with a handsome return.

Stop by and see out new digs: 1822 Skyway, Unit A, in Longmont, Colorado.

Chuck

Tags: , , , ,

We design, we test, we break things

September 19, 2012 in Uncategorized | 6 comments

Test it till it breaks. The dirty side of product development, although admittedly a bit of an adrenalin rush at times.  Like when you hear that distinctive crack or when the top half of the UUT (unit under test) flies across the lab (and doesn’t break further–a good thing). Not that we want the stuff we design to fail, but better for it to fail in the lab before steel is cut for tooling, before inventory is purchased, before the customer touches it.

So some days we get to go into work and try to break things. We’re smart about it, we plan, we order the tests to maximize the information gain.  We rig up force gauges and o-scopes, temperature chambers and drop tables.  Inclinometers too (yes that is a word). We analyze specifications, we establish controls, we pre-test, we separate variables.

Then we try really hard to break things.  But more importantly we try to understand why something breaks.  So we can design it better.  So factory yields are better. So the product fails less often and less drastically for the end user.

And yeah, we do kind of enjoy it.  Breaking things I mean.

Chuck

Tags: ,

Gauging R&R

August 16, 2012 in Uncategorized | No comments

Nope, R&R doesn’t stand for rest and relaxation, or at least not here at Zebulon Solutions, at least not this quarter–we’re busy as spit.  It’s short for repeatability and reproducibility, a mouthful to be sure (hence the acronym), specifically Gauge Repeatability and Reproducibility, or Gauge R&R (or sometimes Gage R&R).

In layman’s terms, Gauge R&R is used to quantify the variability in measurement attributable to the instrument (nee gauge) and the operator.  If pick up a pair of calipers and measure a block of unobtanium 5 times, I’d like to think I’ll get the same result each time, or close to it. That’s repeatability.  And if Todd takes those same calipers and measures that same block, hopefully he gets close to the same result I did (OK, he’ll more than likely get a better result).  That’s reproducibility.   Oddly enough, Gauge R&R is about variance not accuracy, so in theory both Todd and I could be consistently measuring a 4 cm wide block as being 10 cm.  That’s bias, which can be addressed by calibration, a whole ‘nother can of worms, as Zeb would say.

In simple terms the total variance, sigma, is related to the variance of the part and the variance of the measure via a quadratic equation.

Again, without going into some really head-numbing math, basically we want to make sure that the errors in our measurements should be << the inherent tolerance of the part.  And we’d also like that the six sigma measurement error be a small fraction of the specification.

We’re currently working on setting up a Gauge R&R plan for a really complex piece of custom test equipment. It’s not made of unobtanium but our customer has some really brilliant engineers doing some really amazing things in terms of designing it. There isn’t really an operator component, but there are at least 7 axises of variation we will need to consider.  And an equally tough challenge in terms of creating and maintaining gold units–the standards by which the Gauge R&R tests will be run relative to.

As for unobtanium, we really don’t recommend designing with it.  (see Designing with Unobtanium to find out why)

Chuck


Tags: , ,

Keepin’ it local, part 1: Requirements

July 20, 2012 in Keepin' it local | No comments

This is the first installment in a series chronicling Zebulon Solutions’s efforts to keep the supply chain local for a large product development project that we’re working on.  We don’t know where this will end up yet, but we’ll report on the path we take along the way.

Zebulon Solutions is a productization services company–we help our customers turn their R&D projects into manufacturing ready products.  We don’t own the product, and often times we don’t even do the design; we just help make it manufacturable. And help is a key word here–we’re rarely the only voice, and we don’t always get the last say in decisions, even when it comes to manufacturing and supply chain.

So we have a major customer, who shall remain nameless (and moreover, somewhat deliberately disguised–we take confidentiality seriously) who is designing a large scale, system level product that we shall call System X.  Think of it as a piece of capital equipment with a BOM cost running into the hundreds of thousands and a number of significant subsystems. We’re not doing the design but we are tasked with setting up the supply chain, from components to subsystems to final assembly and test, and also in helping to optimize the design for manufacturability, testability and the supply chain.

Honestly, keeping it local is not a hot button for our customer.  Other than a firm ban of using China sources, for IP reasons,  we have no direct customer imposed constraints on where we build this nor on the supply chain. What we have are requirements on performance, lead time, price, order flexibility, and end delivery point.  We’ll let these drive the supply chain, and see if they lead to a local solution or no.

  • The product is highly complex, a product in some ways pushing the leading edge beyond anything out there.
    • This means that there will likely be some pretty challenging assemblies and testing challenges
    • It also means that churn is likely
    • Both of the above favor a local supply chain
  • Lead time is also an issue
    • Again this favors a local supply chain
  • As always price is an issue. But our customer is wise enough to look at cost of ownership
    • Development pricing seems to favor keeping it local–less cost for supplier quals, getting on airplanes etc
    • But unit pricing, as always, makes us think seriously about low cost regions, for all the normal reasons
      • For non critical components, of which there are of course many, why wouldn’t we choose the lowest cost supplier?  Which are often Asia based.
  • The customer does have one somewhat unique requirement, in that they need to be able to order these systems in batch size = 1 increments, at sporadic intervals, with little to no forecasting
    • This is what drove us to start thinking local, because we need a lot of flexibility and will have to scramble to hit lead times
  • The end delivery point is tbd, but most likely Asia
    • This does not favor a local solution, but the tbd aspects keep us from looking globally, at least for now

Nothing is locked in yet, but the requirements are at least causing us to think that it would be good to try to keep as much of the supply chain local.  Not quite sure what local means: in Colorado? In the US? In North America? And also, as a complex system, it’s not as easy as asking Where is it built? as we need to look at components, subassemblies, subsystems and system integration.

To date we know that one of the major buy subsystems will come from Massachusetts as a sole source for technical reasons.  There is a custom interconnect component that we have identified a number of US sources for that should work out, and we are starting to look at Colorado based PCBA houses for doing board level assembly.  But many of the electronics components, while coming out of disty, are in fact Asia built, and that is already causing us a few concerns on lead-time but also providing some favorable pricing. Many custom mechanical components as well, but we haven’t scratched the surface there yet.

Still much, much to do.

To be continued.

Chuck

Tags: , , , , , , ,

« Older entries