When pursuing aggressive benchmarks, engineers must employ portfolio thinking, running multiple design projects simultaneously. But choosing winners requires a decisive way to eliminate projects that are not feasible to continue innovating, often referred to as a "project killer".
In this episode, we analyze Tesla's battery development as a case study. We delve into their use of five clear-cut constraint categories that define failure conditions upfront: the Economic filter, Performance filter, Scalability filter, Resource filter, and System filter.
We discuss the challenges engineers face in letting go of projects due to the sunk cost fallacy, where prior investments irrationally influence future choices, leading to the creation of "zombie projects".
Learn why defining explicit kill criteria before development begins is a vital, often overlooked exercise that saves resources and ensures rational decision-making.
Blog for this episode: https://deeneyenterprises.com/qdd/podcast/define-kill-criteria-to-avoid-zombie-projects/
Episode with Dianna's review of Annie Duke's "Quit": Exploring Product Development and AI Through Literature: Insights from 'Loonshots', 'AI 2041', 'Quit', and "How Big Things Get Done' (QDD Book Cast) - Deeney Enterprises
Ready to apply this to your project?
→ Schedule a free discovery call: Dianna's calendar
Want insights like this?
→ Subscribe to my newsletter: qualityduringdesign.substack.com
Learn the full framework:
→ Get the Book: Pierce the Design Fog
ABOUT DIANNA
Dianna Deeney is a quality advocate for product development with over 25 years of experience in manufacturing. She is president of Deeney Enterprises, LLC, which helps organizations and people improve engineering design.
Episode Transcript
Welcome back to Quality During Design.
Today we're looking at Tesla'saggressive battery targets.
To manage the risk of pursuingmultiple battery chemistries
simultaneously, a necessaryelement of portfolio thinking,
Tesla developed a framework toidentify winners and eliminate
losers.
(01:43):
As engineers, the hardest thingwe do is kill a project, often
falling victim to the sunk costpolicy, where prior investments
of our resources influence ourfuture choices, risking the
creation of a zombie project.
Tesla proactively definedfailure conditions up front
(02:03):
using five constraintcategories, including economic,
performance, and system filters,to measure project viability.
Join me as we explore whydefining these explicit kill
criteria before developmentbegins is the most valuable and
often overlooked exercise inproduct engineering.
(02:25):
Welcome to Quality DuringDesign, the place to use quality
thinking to create productsothers love, for less time, less
money, and a lot less headache.
I'm your host, Diana Deaney.
I'm a senior quality engineerwith over 20 years in
manufacturing and productdevelopment and author of Pierce
the Design Fog.
I help design engineers applyquality and reliability thinking
(02:47):
throughout product development,from early concepts through
technical execution.
Each episode gives youframeworks and tools you can
use.
Want a little more?
Join the Substack for monthlyguides, templates, and QA where
I help you apply these to yourspecific projects.
Visit qualityderingdesign.com.
(03:07):
Let's dive in.
Welcome back.
I've been reading up on Tesla'sbattery development program and
looking at it from a case studypoint of view, starting with
their 2020 battery day.
All sorts of versions of thisare on YouTube that you can go
(03:27):
and view it yourself.
In 2020, they announced somepretty aggressive Go criteria,
like 56% reduction in batterycost per kilowatt hour, and a
54% increase in vehicle range.
To meet these kind of targets,they started to self-develop
their own battery, the 4680cell, a custom design, custom
(03:50):
manufacturing.
They not only had these types ofbenchmarks that they wanted to
meet, but they also had to keepproducing and meeting other
targets for their existingproduct line.
So they started running multiplebattery chemistries
simultaneously.
And we can all sort of guesswhy.
When you're in development,you're not sure about the sure
(04:12):
thing.
You don't want to put all youreggs in one basket, so to speak.
You don't want to focus all ofyour resources onto one idea.
In case it fails, you have abackup plan.
That's the portfolio thinking ofthis case study, right?
You run parallel options, andthen throughout you
systematically choose thewinners.
(04:33):
So the company has these targetsthey want to achieve, and
they're doing design andinnovation to be able to meet
that.
And those are a good criteria tohave because they are
benchmarks, your goal, whatyou're trying to achieve.
And the closer you can get tothat, the better.
But there's a whole lot of otherthings going on too.
We also need a project killcriteria.
(04:53):
If we're developing multiplethings at once to be able to
reach the same end goal, youneed to be able to decide when
to stop pursuing some of theother options.
Not only where to focus withwhat's performing the best, but
what's not going to be feasibleto continue to innovate.
Some of the constraintcategories that I'm seeing that
(05:15):
Tesla used were cost, this isthe economic filter.
For example, this is the 56%target they announced on their
battery day.
So if the innovation raises thepack price or fails to hit the
cost curve, it's rejectedregardless of density.
And speaking of density, that'stheir performance filter.
(05:37):
They wanted to significantlyimprove the volumetric and
gravimetric density overexisting cells.
So if the innovation path thatthey're developing only offers
marginal improvements orcompromises safety for density,
it's rejected.
So those are two constraintcategories or kill criteria for
(05:57):
this portfolio management,economics and performance.
But I'm seeing that they hadmaybe three other constraint
categories.
One was scalability, which isrelated to manufacturability.
Whatever they're developing musthave been compatible with rapid,
high volume production.
And if the process requiredexotic materials, complex wet
(06:19):
processing, or it can't beimplemented on the dry battery
electrode line, they abandonedit.
The next to last category was aresource filter related to the
supply chain.
They wanted to utilize theglobally abundant and
environmentally stablematerials.
For example, they wanted to moveaway from the high cobalt
(06:40):
materials.
So if their development relieson the scarce, geopolitically
sensitive, or environmentallydamaging materials, they
rejected it.
Finally, the last constraintcategory that I saw was the
system filter.
They wanted to make sure thatwhatever they were developing
integrated seamlessly into thevehicle structure, the cell to
(07:04):
pack or the cell to vehicle.
And if the design couldn'tfunction as a structural
element, the technology wasrejected as it compromises the
overall vehicle efficiency andcost.
So if you think about this, thisis Tesla's battery innovation
development kill criteria,economics, manufacturability,
(07:28):
performance, resources, and thesystem filters.
And I think they're reallywell-defined and clear-cut
criteria against which tomeasure whatever it is that
they're developing, whatever itis they're managing.
So think about your currentprojects that you're working on.
You definitely have Go criteria.
(07:49):
Those are the user needs,requirements, and other
performance requirements thatyou're trying to meet.
What is your kill criteria?
Do you have any?
You might have some, but youdon't know what it is.
How can you find out?
And how can you use that to helpyou make decisions?
(08:12):
When we're developing killcriteria, the project kill
criteria, we're actuallydefining the failure conditions
up front.
Doesn't that make thingsclearer?
You see this in non-engineeringand product development
communities also, talking a lotabout New Year's resolutions
right now.
(08:33):
Sometimes people will givethemselves a cutoff time.
You know, I'm going to startthis new business.
I'm going to give myself I'mgoing to give myself two years
to have this many customers.
And if I don't, I'm going topivot and do something
different.
The hardest thing to do whenyou've invested so much time and
energy, research and resourcesinto a project is to kill it.
(08:59):
You're invested in it and youwant to see it succeed.
You want to see it through tothe end.
I think many of us engineersthink that way.
It may have something to do withhow we were trained as engineers
to find the problem and thenkeep digging or evolving or
innovating until you find thesolution.
We might find a solution, but itmay not be feasible against all
(09:22):
the other criteria because it'snot just performance, is it?
It's also cost andmanufacturability and what the
customer wants, what thebusiness needs.
I sometimes refer to some ofthese other aspects of product
design and engineering as ourinternal customers.
(09:42):
Yes, we need to meet someoutward external goals for our
customers or whoever's going tobe buying our product, but we
also have a responsibility toengineer for many other people
and divisions and things.
So kill criteria is somethingthat we want to be able to
define for our projects.
(10:05):
Project manager or some of thepeople that are deciding what
projects are pursued in thebusiness probably have this kill
criteria.
Maybe you don't know what it is.
It might be worth finding out.
If there is no kill criteria andyou're just going to keep
developing until you get it,your project might turn into a
(10:26):
zombie project where it neverreally gets done, it never sees
the light of day, and it eitherjust fizzles out or eventually
somebody pulls the plug.
There can be sunk cost fallaciesin our engineering projects,
also, where we have an emotionalattachment to our design or our
project.
(10:47):
The sunk cost fallacy is acognitive bias where we continue
investing in a decision based onour prior investments.
So we've already put so muchtime, energy, and effort into
whatever we're developing thatit doesn't make sense for us to
not continue developing it.
Whereas really we're eliminatingor not factoring in the future
(11:10):
benefits or future non-benefits.
If we continue investing in thisinnovation or this development,
is it going to pay out in theend, really?
Or is it better worth our timeand resources to be working on
something different?
When we're being rational aboutour decision making, these costs
(11:32):
that we've already incurred inour project should not influence
our future choices.
They're irretrievable.
During product development, wemay be working under our sunk
cost fallacy.
It's something we need to beaware of and recognize and pivot
our own decision making if weneed to when we recognize it.
(11:53):
So in addition to the successthresholds of our projects,
which we need, we should alsodefine and get alignment on the
kill criteria of our projects.
And this is an often overlookedexercise.
In her book, Quit (12:09):
The Power to Know When to Walk Away, author
Annie Duke shows us the value ofknowing when to quit.
She offers a lot of examplesfrom different points of view,
different industries.
She even looks at the sport ofboxing.
She also gets into some of thepsychological and cognitive
(12:29):
biases that we have when itcomes to quitting.
And she promotes defining killcriteria in the beginning, quit
criteria.
I recommend reading it.
It will help you identify whenpeople are ignoring the quit
signals or avoid settingcriteria in the first place.
I reviewed this book in aprevious podcast episode.
(12:50):
I'll link to it in the shownotes.
As engineers, we're also used torunning tests with acceptance
criteria, especially if they'relate-stage designed tests.
Our product has to performwithin these windows or it's a
failure.
Defining kill criteria forprojects is sort of like that.
You're defining criteria foryour project up front before you
(13:15):
get into it.
And that'll help you define whento quit so that your project
doesn't become a zombie project.
Define your kill criteria beforeyou start testing, and for your
projects, even before you startdeveloping.
In the next episode, I will talkabout the expected value
(13:37):
calculation that helps end thedebate of whether to keep going
or not.
Join me on Substack for the AskMe Anything post.
This month it's called ChooseIt: How to Decide Between Viable
Options when you'll never have100% confidence.
And then later in the month,I'll be publishing another post,
(13:58):
including some of theinformation that I've discovered
about Tesla's batterydevelopment and what we can
derive from that and how we canuse it for ourselves.
Find me on Substack at qualityduring design.
And if you're just finding me onthis podcast, please subscribe
to your favorite podcastprovider.
This has been a production ofDeanny Enterprises.
(14:20):
Thanks for listening.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Burden
The Burden is a documentary series that takes listeners into the hidden places where justice is done (and undone). It dives deep into the lives of heroes and villains. And it focuses a spotlight on those who triumph even when the odds are against them. Season 5 - The Burden: Death & Deceit in Alliance On April Fools Day 1999, 26-year-old Yvonne Layne was found murdered in her Alliance, Ohio home. David Thorne, her ex-boyfriend and father of one of her children, was instantly a suspect. Another young man admitted to the murder, and David breathed a sigh of relief, until the confessed murderer fingered David; “He paid me to do it.” David was sentenced to life without parole. Two decades later, Pulitzer winner and podcast host, Maggie Freleng (Bone Valley Season 3: Graves County, Wrongful Conviction, Suave) launched a “live” investigation into David's conviction alongside Jason Baldwin (himself wrongfully convicted as a member of the West Memphis Three). Maggie had come to believe that the entire investigation of David was botched by the tiny local police department, or worse, covered up the real killer. Was Maggie correct? Was David’s claim of innocence credible? In Death and Deceit in Alliance, Maggie recounts the case that launched her career, and ultimately, “broke” her.” The results will shock the listener and reduce Maggie to tears and self-doubt. This is not your typical wrongful conviction story. In fact, it turns the genre on its head. It asks the question: What if our champions are foolish? Season 4 - The Burden: Get the Money and Run “Trying to murder my father, this was the thing that put me on the path.” That’s Joe Loya and that path was bank robbery. Bank, bank, bank, bank, bank. In season 4 of The Burden: Get the Money and Run, we hear from Joe who was once the most prolific bank robber in Southern California, and beyond. He used disguises, body doubles, proxies. He leaped over counters, grabbed the money and ran. Even as the FBI was closing in. It was a showdown between a daring bank robber, and a patient FBI agent. Joe was no ordinary bank robber. He was bright, articulate, charismatic, and driven by a dark rage that he summoned up at will. In seven episodes, Joe tells all: the what, the how… and the why. Including why he tried to murder his father. Season 3 - The Burden: Avenger Miriam Lewin is one of Argentina’s leading journalists today. At 19 years old, she was kidnapped off the streets of Buenos Aires for her political activism and thrown into a concentration camp. Thousands of her fellow inmates were executed, tossed alive from a cargo plane into the ocean. Miriam, along with a handful of others, will survive the camp. Then as a journalist, she will wage a decades long campaign to bring her tormentors to justice. Avenger is about one woman’s triumphant battle against unbelievable odds to survive torture, claim justice for the crimes done against her and others like her, and change the future of her country. Season 2 - The Burden: Empire on Blood Empire on Blood is set in the Bronx, NY, in the early 90s, when two young drug dealers ruled an intersection known as “The Corner on Blood.” The boss, Calvin Buari, lived large. He and a protege swore they would build an empire on blood. Then the relationship frayed and the protege accused Calvin of a double homicide which he claimed he didn’t do. But did he? Award-winning journalist Steve Fishman spent seven years to answer that question. This is the story of one man’s last chance to overturn his life sentence. He may prevail, but someone’s gotta pay. The Burden: Empire on Blood is the director’s cut of the true crime classic which reached #1 on the charts when it was first released half a dozen years ago. Season 1 - The Burden In the 1990s, Detective Louis N. Scarcella was legendary. In a city overrun by violent crime, he cracked the toughest cases and put away the worst criminals. “The Hulk” was his nickname. Then the story changed. Scarcella ran into a group of convicted murderers who all say they are innocent. They turned themselves into jailhouse-lawyers and in prison founded a lway firm. When they realized Scarcella helped put many of them away, they set their sights on taking him down. And with the help of a NY Times reporter they have a chance. For years, Scarcella insisted he did nothing wrong. But that’s all he’d say. Until we tracked Scarcella to a sauna in a Russian bathhouse, where he started to talk..and talk and talk. “The guilty have gone free,” he whispered. And then agreed to take us into the belly of the beast. Welcome to The Burden.