We learn the secrets of a safe deployment practice while continuing to study The DevOps Handbook as Joe is a cartwheeling acrobat, Michael is not, and Allen is hurting, so much.
For those of you that are reading these show notes via their podcast player, you can find this episode's full show notes at https://www.codingblocks.net/episode140.
Sponsors
- Datadog – Sign up today for a free 14 day trial and get a free Datadog t-shirt after your first dashboard.
- Secure Code Warrior – Start gamifying your organization's security posture today, score 5,000 points, and get a free Secure Code Warrior t-shirt.
Survey Says
News
- Rob Pike's Rules of Programming (utexas.edu)
No, you click the button …
Enable Feedback to Safely Deploy Code
- Without a quick feedback loop:
- Operations doesn't like deploying developer code.
- Developers complain about operations not wanting to deploy their code.
- Given a button for anyone to push to deploy, nobody wants to push it.
- The solution is to deploy code with quick feedback loops.
- If there's a problem, fix it quickly and add new telemetry to track the fix.
- Puts the information in front of everyone so there are no secrets.
- This encourages developers to write more tests and better code and they take more pride in releasing successful deployments.
- An interesting side effect is developers are willing to check in smaller chunks of code because they know they're safer to deploy and easier to reason about.
- This also allows for frequent production releases with constant, tight feedback loops.
- Automating the deployment process isn't enough. You must have monitoring of your telemetry integrated into that process for visibility.
Use Telemetry to Make Deployments Safer
- Always make sure you're monitoring telemetry when doing a production release,
- If anything goes wrong, you should see it pretty immediately.
- Nothing is "done" until it is operating as expected in the production environment.
- Just because you improve the development process, i.e. more unit tests, telemetry, etc., that doesn't mean there won't be issues. Having these monitors in place will enable you to find and fix these issues quicker and add more telemetry to help eliminate that particular issue from happening again going forward.
- Production deployments are one of the top causes of production issues.
- This is why it's so important to overlay those deployments on the metric graphs.
Pager Duty – Devs and Ops together
- Problems sometimes can go on for extremely long periods of time.
- Those problems might be sent off to a team to be worked on, but they get deprioritized in lieu of some features to be added.
- The problems can be a major problem for operations, but not even a blip on the radar of dev.
- Upstream work centers that are optimizing for themselves reduces performance for the overall value stream.
- This means everyone in the value stream should share responsibility for handing operational incidents.
- When developers wer
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Burden
The Burden is a documentary series that takes listeners into the hidden places where justice is done (and undone). It dives deep into the lives of heroes and villains. And it focuses a spotlight on those who triumph even when the odds are against them. Season 5 - The Burden: Death & Deceit in Alliance On April Fools Day 1999, 26-year-old Yvonne Layne was found murdered in her Alliance, Ohio home. David Thorne, her ex-boyfriend and father of one of her children, was instantly a suspect. Another young man admitted to the murder, and David breathed a sigh of relief, until the confessed murderer fingered David; “He paid me to do it.” David was sentenced to life without parole. Two decades later, Pulitzer winner and podcast host, Maggie Freleng (Bone Valley Season 3: Graves County, Wrongful Conviction, Suave) launched a “live” investigation into David's conviction alongside Jason Baldwin (himself wrongfully convicted as a member of the West Memphis Three). Maggie had come to believe that the entire investigation of David was botched by the tiny local police department, or worse, covered up the real killer. Was Maggie correct? Was David’s claim of innocence credible? In Death and Deceit in Alliance, Maggie recounts the case that launched her career, and ultimately, “broke” her.” The results will shock the listener and reduce Maggie to tears and self-doubt. This is not your typical wrongful conviction story. In fact, it turns the genre on its head. It asks the question: What if our champions are foolish? Season 4 - The Burden: Get the Money and Run “Trying to murder my father, this was the thing that put me on the path.” That’s Joe Loya and that path was bank robbery. Bank, bank, bank, bank, bank. In season 4 of The Burden: Get the Money and Run, we hear from Joe who was once the most prolific bank robber in Southern California, and beyond. He used disguises, body doubles, proxies. He leaped over counters, grabbed the money and ran. Even as the FBI was closing in. It was a showdown between a daring bank robber, and a patient FBI agent. Joe was no ordinary bank robber. He was bright, articulate, charismatic, and driven by a dark rage that he summoned up at will. In seven episodes, Joe tells all: the what, the how… and the why. Including why he tried to murder his father. Season 3 - The Burden: Avenger Miriam Lewin is one of Argentina’s leading journalists today. At 19 years old, she was kidnapped off the streets of Buenos Aires for her political activism and thrown into a concentration camp. Thousands of her fellow inmates were executed, tossed alive from a cargo plane into the ocean. Miriam, along with a handful of others, will survive the camp. Then as a journalist, she will wage a decades long campaign to bring her tormentors to justice. Avenger is about one woman’s triumphant battle against unbelievable odds to survive torture, claim justice for the crimes done against her and others like her, and change the future of her country. Season 2 - The Burden: Empire on Blood Empire on Blood is set in the Bronx, NY, in the early 90s, when two young drug dealers ruled an intersection known as “The Corner on Blood.” The boss, Calvin Buari, lived large. He and a protege swore they would build an empire on blood. Then the relationship frayed and the protege accused Calvin of a double homicide which he claimed he didn’t do. But did he? Award-winning journalist Steve Fishman spent seven years to answer that question. This is the story of one man’s last chance to overturn his life sentence. He may prevail, but someone’s gotta pay. The Burden: Empire on Blood is the director’s cut of the true crime classic which reached #1 on the charts when it was first released half a dozen years ago. Season 1 - The Burden In the 1990s, Detective Louis N. Scarcella was legendary. In a city overrun by violent crime, he cracked the toughest cases and put away the worst criminals. “The Hulk” was his nickname. Then the story changed. Scarcella ran into a group of convicted murderers who all say they are innocent. They turned themselves into jailhouse-lawyers and in prison founded a lway firm. When they realized Scarcella helped put many of them away, they set their sights on taking him down. And with the help of a NY Times reporter they have a chance. For years, Scarcella insisted he did nothing wrong. But that’s all he’d say. Until we tracked Scarcella to a sauna in a Russian bathhouse, where he started to talk..and talk and talk. “The guilty have gone free,” he whispered. And then agreed to take us into the belly of the beast. Welcome to The Burden.