Julian LaNeve (@JulianLaneve, CTO @astronomerio) discusses data pipelines, Apache Airflow, Astronomer’s managed offering, and the benefits of data pipelines for both developers and operations.
SHOW: 939
SHOW TRANSCRIPT: The Cloudcast #939 Transcript
SHOW VIDEO: https://youtube.com/@TheCloudcastNET
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST: "CLOUDCAST BASICS"
SPONSORS:
- [VASION] Vasion Print eliminates the need for print servers by enabling secure, cloud-based printing from any device, anywhere. Get a custom demo to see the difference for yourself.
- [FCTR] Try FCTR.io (that's F-C-T-R dot io) free for 60 days. Modern security demands modern solutions. Check out Fctr's Tako AI, the first AI agent for Okta, on their website
SHOW NOTES:
Topic 1 - Welcome to the show, Julian. Give everyone a quick introduction.
Topic 2 - Our topic today is Data Pipelines with Apache Airflow. For those unfamiliar, provide an introduction to Apache Airflow and how Airflow manages data pipelines.
Topic 3 - What are the advantages of Apache Airflow vs. others in the space? What are the downsides? How does Airflow fit in with other Apache projects?
Topic 4 - I would imagine this is where Astronomer potentially comes into play. What makes Astonomer different from Airflow? What problems are you trying to solve for both developers and operations folks?
Topic 5 - What does a typical implementation look like? What growing pains do developers typically face when they need to introduce pipelining tools and begin standardization? Is it a scale issue? A complexity of tools issue? Integrations with infrastructure?
Topic 6 - One aspect I typically see with automation is security, especially at scale. What recommendations do you have for developers regarding security, particularly in the context of multi-tenancy, for data pipelines?
FEEDBACK?
- Email: show at the cloudcast dot net
- Bluesky: @cloudcastpod.bsky.social
- Twitter/X: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Popular Podcasts
NFL Daily with Gregg Rosenthal
Gregg Rosenthal and a rotating crew of elite NFL Media co-hosts, including Patrick Claybon, Colleen Wolfe, Steve Wyche, Nick Shook and Jourdan Rodrigue of The Athletic get you caught up daily on all the NFL news and analysis you need to be smarter and funnier than your friends.
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com