The Databricks Data Engineer
Helping 18k+ Databricks data engineers become seniors: interview like seniors, execute like seniors, think like seniors.
Episodes
Your team is three weeks into a Databricks performance push. Broadcast hints in PRs. AQE flags toggled like christmas lights. Partition counts re-tuned for the third time. The manager is asking, gently, when the gains are showing up in the bill.
The staff DE on the next team finished theirs in two afternoons. Same workloads, bigger drop. They were running a triage you have never been taught.
In this episode:
- Why most of what...
You scroll past the cancelled junior req, the "serverless first" line on your director's planning slide, and the third Lakebase mention from your Databricks rep this quarter. Each one looks like a news item. None of them feel like they're about you.
They are. Four structural shifts have already happened in the field, and the words "Databricks data engineer" don't mean what they meant in 2024. Most...
Mid-level is a down escalator. It looks like flat ground. You feel productive, your tickets close on Friday, your burndown chart is healthy, and your review says "reliable executor of well-defined work" for the third cycle in a row.
That sentence is the official label for "not getting promoted this year" - and most Databricks data engineers never decode it. It isn't a skill gap. It's nine habits that ...
You check the usage logs on a dashboard you spent two weeks building. Zero views. Not low views. Zero. The stakeholder who requested it hasn't logged in once. Three months later they ask the exact question the dashboard answers, in a meeting, out loud, as if the dashboard doesn't exist. Because for them, it doesn't.
In this episode:
- Why the most technically impressive Databricks dashboards are often the least used
- ...
She could explain shuffle hash join versus sort merge join. She knew when Adaptive Query Execution kicks in. She had six weeks of notes on Delta Lake, Spark memory, and cluster configs.
She walked into the Databricks senior interview feeling genuinely confident. Then the interviewer asked her to walk through a diagnosis, not recite a definition, and everything she studied was aimed at the wrong target.
In this episode:
- Why ...
She kept a terabyte-scale pipeline running for six months without a single incident. Not one page, not one late dashboard. Then review season came and the engineer who spent two weekends fixing an outage he partly caused got the promotion instead. Her name wasn't in a single incident report, because when you prevent problems, there's no report to put your name on.
In this episode:
- Why the skills Databricks data engi...
Popular Podcasts
Ding dong! Join your culture consultants, Matt Rogers and Bowen Yang, on an unforgettable journey into the beating heart of CULTURE. Alongside sizzling special guests, they GET INTO the hottest pop-culture moments of the day and the formative cultural experiences that turned them into Culturistas. Produced by the Big Money Players Network and iHeartRadio.
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy, Jess Hilarious, And Charlamagne Tha God!
Nancy Grace dives deep into the day’s most shocking crimes and asks the tough questions in her new daily podcast – Crime Stories with Nancy Grace. Nancy Grace had a perfect conviction record during her decade as a prosecutor and used her TV show to find missing people, fugitives on the run and unseen clues. Now, she will use the power of her huge social media following and the immediacy of the internet to deliver daily bombshells! Theme Music: Audio Network