February 9, 2026 • 27 mins
Viktor Gamov talks to Jeremy Custenborder (Confluent) about his career in large-scale systems. Jeremy’s first job: paper boy. His challenge: keeping MySpace running at a massive pre-cloud scale while building the tools that didn’t exist yet and learning to fail fast.
SEASON 2
Hosted by Tim Berglund, Adi Polak and Viktor Gamov
Produced and Edited by Noelle Gallagher, Peter Furia and Nurie Mohamed
Music by Coastal Kites
Artwork by Phil Vo
- 🎧 Subscribe to Confluent Developer wherever you listen to podcasts.
- ▶️ Subscribe on YouTube, and hit the 🔔 to catch new episodes.
- 👍 If you enjoyed this, please leave us a rating.
- 🎧 Confluent also has a podcast for tech leaders: "Life Is But A Stream" hosted by our friend, Joseph Morais.
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
SPEAKER_01 (00:00):
This episode takes us back in time into MySpace
era.
Times where if you need a tool,you had to build this yourself.
This is Confluent Developer.
SPEAKER_02 (00:10):
That was one of those spots where it didn't
exist yet, so we built our own.
The biggest thing that thattaught me was you don't have to
get everything right the firsttime.
Fail fast, figure out what's uhwhat your problem is, and then
keep moving on from there.
SPEAKER_01 (00:23):
Welcome back to this episode of Confluent Developer
Podcast.
My name is Viktor Gamoff, and Iwill be your host today.
And today I have a very specialguest.
Every guest in this show isspecial, but this one is truly
something.
And uh you will understand bythe end of this episode why I'm
saying this.
(00:44):
Jeremy Custom Border, the legendof Kafka Connect.
And uh we're gonna talk aboutthis.
Jeremy, welcome to Kampf andDevelopment.
Thanks for having me.
Thanks for having me.
It's great to uh finally uhreconnect after you know many
years that I haven't seen you inperson.
I've seen you um, you know, allover like internet and LinkedIn
and other places, but it'sfinally good to be with you on
(01:07):
the same room and talk aboutsome cool tech stuff.
SPEAKER_02 (01:09):
Of course, my I'm Mr.
Man.
SPEAKER_01 (01:11):
Yeah.
Uh for those of you, uh those ofyou who uh don't know uh Jeremy,
can you tell us like in a fewwords what you do um and like
how people you know can uh canknow you?
SPEAKER_02 (01:24):
Yeah, so um I'm an I'm an engineer like a lot of a
lot of y'all.
Um I worked at Confluent.
I I joined uh originally joinedConfluent in in 2016 and worked
till uh 2022.
Um I um I'm the type type ofperson that I'd say um I I
worked in the field and thatworked out pretty well for me
because uh I have a shortattention span.
(01:45):
Yeah.
So it's like I go from problemto problem to problem and and I
enjoyed that.
I had a I had a great timeworking here, and it's it's fun
to be back chatting with y'all.
SPEAKER_01 (01:54):
Yeah, and um Conflict Developer Podcast, this
is the show where we explore theorigins of this uh technical
excellence and how um theluminaries or some like a
visionaries and some some verycool people and some awesome
nerds are came came to be, andlike uh what um you know how
they started and how they end upand the things.
(02:16):
And one of the first questionsthat I like to ask people is uh
what was your first job?
Not necessarily in related totechnology, not necessarily
relating to software.
What was your first time whenyou get this kind of like a
crispy uh like a bill or maybelike a few coins and stuff like
that?
Depends when you started andwhat was the rate of inflation
that time.
SPEAKER_02 (02:36):
Well, I mean, my my absolute first job was um I was
a paper boy from 10 to 16.
Nice.
And like I had that I had thatdown pr pretty well.
And that was a great job.
And I made um a hundred bucks amonth.
And like that money went in andthen went immediately back out.
SPEAKER_01 (02:55):
Yeah, you probably could afford you lots of uh
candies and some some otherinteresting entertainment.
SPEAKER_02 (03:02):
Exactly.
But I mean like that was uh thatwould be the 90s.
SPEAKER_01 (03:06):
So you know, you can go, yeah, you can you can do a
lot with a hundred bucks a monthin nineties, I guess.
SPEAKER_02 (03:13):
And then and then my first uh my first real IT job, I
I used to be the networkadministrator for the um the
Green County Library System inuh in Ohio.
SPEAKER_01 (03:25):
Okay, so tell us about this.
What year it was, what was thenetwork landscape at the time,
like what kind of like hardware,software things do you're
working with?
Tell us everything.
SPEAKER_02 (03:35):
Oh, it was it was a blast.
It was so what I loved aboutthis organization is um I was
one of the first people I thinkthat was that was nice.
Okay.
And it's like, you know, you'reworking for with for a bunch of
grandmothers.
Yes.
And you know, they would befreaked out if they broke a
computer.
And you know, I'm in this phasein my life, and you know, this
is like 98, so I'm I'm 18 yearsold.
(03:55):
Yeah.
And um, I didn't go to college,I'm self-taught.
So, you know, I'm I'm I'm inthis like like pre playground
that had a decent budget.
I had 400 machines that I that Iuh administrated with you know
T1 connections betweenbuildings, and I had uh, you
know, I had I had a decentbudget and it was like a great
(04:16):
place to learn because was itlike a Windows machines, Linux
machines?
All Windows machines.
Windows machines.
Yeah, and so it it was it wasgreat from the perspective of of
that things weren't perfect, sothat you could make a lot of
improvements.
So and then the other part withthat is you're working for all
these these grandmas.
Yes.
And you know, like so it wasgreat for me because you know, I
(04:38):
wasn't living on my own at 18,and you know, it's not like that
job paid that great.
Yeah.
And you know, you so I would Iwould I would say, hey, I'm
gonna be coming in to work on ony'all's computers on you know,
Tuesday.
I show up and there's all thesebaked goods.
It was amazing.
It was amazing.
SPEAKER_01 (04:57):
So you're working for all these grandmothers.
So you you have you have likeincentives, maybe not uh with
the monitor, but at least youwill be well fed, and you had
the your you know, the dose ofsugar that was, you know, the
the homemade cookies the best.
SPEAKER_02 (05:12):
Yes.
And the the cool part about thatis like I could uh I had enough
time where I could I could workand learn learn stuff, and
that's where I I wrote my firstprograms.
SPEAKER_01 (05:21):
What was the language you you used first
time?
SPEAKER_02 (05:23):
Uh first time was was ASP.net.
Okay.
So the the first portion of mycareer was was pretty heavily uh
Windows-based.
Yeah, yeah, me too.
SPEAKER_01 (05:31):
I I also um after I get my first job in you know the
Russian like bank calledZburbank, um I started learning.
Yes.
I I learned uh.NET and uh uhsurprisingly not to not to um I
didn't go uh PHP route becauselots of uh folks from my school
(05:53):
went to PHP route.
I went actually SP.net as well.
Yeah.
It was version uh.NET uh.NET2.0.
So it's just it just came out.
SPEAKER_02 (06:02):
I spent a lot of time with that later on.
Yeah.
So I I I did that for a whileand uh I wrote um we basically
had these CD ROMs with uh um uhlike genealogy things on it.
So you could come in and doresearch on genealogy in one
group.
Uh-huh.
So I I put something together tohelp searching and help kind of
find uh which CDs had what forfolks.
SPEAKER_01 (06:26):
Nice.
SPEAKER_02 (06:26):
And that was my that was the first thing I ever I
ever wrote.
SPEAKER_01 (06:29):
Yeah.
So basically you're uh theclassical three-tier
application, right?
You have a you have uh like afront end, you have a like
application server, which isprobably what internet
information server, right?
And a database.
SPEAKER_02 (06:40):
Yeah, well, I mean, man, you know, it's one of those
deals in those days that didn'tcompile, where I would, I would
say just it compiles, ship it.
SPEAKER_01 (06:47):
That's that's what's pretty cool.
My uh my first um uh.NETapplication was uh first complex
application.
It was attempt to implement kindof like a trading system.
Kind of I thought like, okay, soI was working in the bank as a
as my uh internship uh programuh during the summertime.
I was like, okay, so what aboutkind of like I write the
(07:08):
training application alsocompiled, uh and uh but my demo
uh for uh when I was went toshow this to to this like a
committee of the professors andstuff like that, um it only ran
on my laptop, obviously.
So runs on my laptop, so that'sthat's how and uh also I and now
we know where we got Dockerfrom.
(07:29):
Yeah, exactly, exactly.
Um so so you started the uh youalready mentioned that you
self-taught, you learned uhthese technologies um uh
basically on your own uh time onyour own pace, and looks like
you also very was veryinterested.
One of the one of the thingswhat we like to discover uh
discuss is uh the the biggestprofessional challenge.
(07:51):
Like I um I know you fromprobably it was you know you can
think about this.
It's it's really was a complexchallenge to find the ways to
bring the virtually any datasource in the world into Kafka.
Uh and we will talk we will talkabout this.
I really want to talk aboutthis, but on your opinion,
something that you would besuper proud or super embarrassed
(08:14):
if it's like some of the complextasks that you solved, like the
the biggest challenge of yourlife like today, or something
that you kind of likeunderestimated your your your
your your power and it's like uhyeah it didn't work out, but
still I learned a lot.
So that's that's what I want totalk about.
SPEAKER_02 (08:31):
Yeah, yeah.
So if I had to go there, I willtell you probably probably the
place I learned the most, and uhthis is gonna age me.
This is definitely gonna age me.
Um so you know, I I I was aroundOhio for a while, and then you
know, just at some point Idecided like I gotta get out of
Ohio.
And so like I I went to uh I Iwent out west and then I ended
(08:52):
up in LA and I ended up workinguh for for MySpace, the social
network.
And uh I I did that for fiveyears.
So um working.
SPEAKER_01 (09:02):
For those of you kids who don't know what the
MySpace is, is something like uhokay, Facebook also ages.
Those kids are not usingFacebook these days, it's
probably using TikTok orsomething like that.
But before Facebook was a thing,there was a MySpace.
Exactly, exactly.
SPEAKER_02 (09:21):
And I I uh worked on that.
So and the the fun part aboutthat is like you know, when you
talk to folks, they would say,like, okay, we needed to have a
caching tier.
Yeah.
They're like, well, why didn'tyou use Redis?
Yeah.
It's like go look at the firstcommits on Redis.
That was like five years after.
I think it was I I think Rediswas was built in 2010.
(09:42):
I I'd have to go look.
SPEAKER_01 (09:43):
You know, but before that, there was a memcache.
SPEAKER_02 (09:45):
We had one in yeah.
Uh Memcache came out in like2007 or 2000.
SPEAKER_01 (09:50):
I think it's even came up for you know this type
of like applications, they startpopping up the like initial wave
of web 2.0, kind of like what weknow today as uh web scale
things.
Um and I think the um yeah,memcache was also around that
time, around the 2000s.
SPEAKER_02 (10:10):
Yeah, yeah, but that was one of those spots where it
didn't exist yet.
So we we built our own and wewere we were uh running
monstrous scale on Windowsmachines, which uh and and
actually making great numbers.
So uh you were talking aboutyour ASP.net 2, you know, 2.0.
SPEAKER_01 (10:30):
Yeah.
SPEAKER_02 (10:30):
That was mine.
That's what we were working on.
It was the the whole web frontend was ASP.net 2.0.
SPEAKER_01 (10:36):
What was the uh I guess right now probably it
would be like a very like smallnumbers, but the dead time, what
was the profile of load?
Like how many um daily users ofthe of the platform?
SPEAKER_00 (10:48):
Now a quick word from our sponsor.
Confluent Developer the Podcastis brought to you by Confluent
Developer the website, which haseverything you need as a
developer of data streamingsystems.
And it's completely free.
We've got curriculum, hands-onexercises, executable tutorials,
the online data streamingengineer certification, also
free, a way to find a meetupnear you, those are free.
(11:12):
Everything is there.
I really want you to besuccessful in your journey as a
data streaming engineer, andthis is the site that has what
you need.
Check it out atdeveloper.confluent.io.
That's developer.confluent.io.
Now back to the show.
SPEAKER_02 (11:28):
Well, when I when I joined, uh when I joined, they
had about uh uh 10 million usersand were was pushing about uh I
want to say at that time, abouttwo or three uh gigabits a
second uh from their datacenters.
And then when I left, um we had320 million, if I remember
(11:50):
right, users.
And uh they were pushing aterabit from their data centers.
SPEAKER_01 (11:56):
So what was the again about uh we leave in 2025,
we have a great deal of um opensource technologies available
for building stuff because ofthis uh era of uh web scale.
But uh if we you know back intime, what was the like biggest
challenges apart from what wealready discussed with the uh
(12:16):
with caching, probably like aload balancing would be so much
charting of databases and stuff?
SPEAKER_02 (12:22):
So much has changed, and just so like you know, like
for example, like everybody usesKubernetes now.
There was nothing like that.
And then you know, Kubernetescan do like a slow rollout and
handle all the health checks,and we had to build everything
around that.
So, like all of these were wasstitching things like net
scalers and other uh componentstogether to get that type of
(12:46):
functionality.
And you know, today I look atwhat you have and it's like it's
it's amazing.
Yeah.
And then, you know, like we weremoving files around, and that's
how you would do deployments.
And you know, to you would youwould image machines by setting
up monstrous multicast groupsand sending tons of image
traffic down to to to machines,and that's how we like a way
(13:08):
that's how we are uh vs orsomething like that, right?
SPEAKER_01 (13:12):
Or not even that.
SPEAKER_02 (13:13):
No, it's all all but all bare metal.
Oh wow, yeah.
And then like at the end, it wasuh like right around when AWS
was was starting to come up.
We were also running Zen as welland and uh virtualizing it's
instead so instead of patching,we would just swap out machines.
SPEAKER_01 (13:27):
Yeah.
Wow, that's uh that was prettycool time.
Um so what this uh this time uhtaught you about data?
SPEAKER_02 (13:38):
That's that's the question that I'm trying to
slowly slide in to the I meanI'll tell you the the the the
the biggest thing that thattaught me was you don't have to
get everything right the firsttime.
It's like so you you need toobviously you need to to have
good principles around aroundstoring your data, but you can
you can increment and you canyou can add on and you can you
(14:00):
can do more at a later time.
It doesn't, you know, but foryou know, I'd say that's right
around the time agiledevelopment start has started
kind of kind of adopting.
Yeah.
It's like fail fast, figure outwhat's uh what your problem is,
and then and then keep keepmoving on from there.
So uh you said how how longyou've been in the MySpace?
SPEAKER_01 (14:18):
I was there for five years.
So I think this is the one ofthe things that I heard from
also former colleagues of us,maybe probably we need to get
him on this podcast as welleventually, uh Sriram
Suburbanian.
He mentioned one of the thingsthat these days you can put on
resume whatever you want, andit's kind of like uh should be a
(14:38):
little bit alarming for some ofthe hiring managers when you
have like a distributed systemsengineer in your resume and you
only spend like a one year ineach company.
So there's not enough time toyou to be like a fully solidify
as a distributed systemsengineer if you're not went
through the multiple iterationsof the same product.
Oh yeah.
Like there is a joke or maybesemi-joke when the people saying
(15:00):
that if you're not embarrassedfor the first version of your
product, you're probably doingsomething wrong.
A hundred percent.
So that's that's that's uhaligned with what you just said,
kind of like you don't have todo everything you know right at
the first time, but you have auh you need to know that you
need to iterate and learn fromthis and how you can do this
better next time.
So that's why the idea of umrelieving multiple major
(15:22):
versions when you justconstantly swapping not the
constantly, but you know, you'reswapping stack every like two
years.
You saw that something works,the technology of the time works
um after two years, there wouldbe totally different landscape
that there might be some of theproblems that you try to solve
that someone else is will besolving.
Like that's how we end up uhhaving Nginx instead of Apache
(15:48):
for for for web front end.
This is how we ended up havingsome other like a lighter like
HTTP load balancers in in frontof instead of like having some
vendor uh based you know loadbalancers and things like that.
SPEAKER_02 (16:01):
So it's funny, and you know, I go back and I look
at like uh like how many uses ofKafka I would have had there.
Yeah, it would have been uh itwould have been insane.
Yeah.
And then you know, if you lookat, hey, how would you do things
now, you know, like like metricswe built our own.
Yeah, you know, well Prometheusis pretty amazing.
Why don't you use that?
Yeah, but it was not thereanymore.
None of those things existed.
(16:21):
And you know the other the otherthing I got from that was um a
good understanding of how to douh production debugging.
And you you kind of you kind ofget a lot of uh of principles.
So one of my jobs was kind ofbeing an SRE before that.
SRE was award, yeah.
SPEAKER_01 (16:38):
Yeah, yeah.
System administrator.
That's how uh there was calledthat time.
Like called Jeremy, you know.
Yeah, Jeremy's just like showingup the for for like for cookies
and like doing something withfixing the thing.
SPEAKER_02 (16:50):
It was fun.
I mean, you'd you get in thesespots where like in all of these
charts go red, and then you'vegot to go figure out, hey,
what's the problem?
And we learned a lot, and it wasa great team.
And you know, like I I had thatsimilar experience and like uh
working at Confluent.
You just had a had a great teamwhere you all have the same
goal, yeah, and you're there tohelp each other and win.
SPEAKER_01 (17:09):
Yeah.
That's yeah.
Yeah, and I think um interestingthing when you you have this
like this mentality where thosethings were not existing, so you
had to build yourself.
And when I started um learningKafka, it was around 2000 uh
maybe 17, 16.
(17:30):
Uh, we had the product that wassimilar to uh to do stream
processing at Hazelcast thattime.
And I would start looking how toget data, and there was already
availability of uh differentways how we can get data in.
But you came a little bitearlier and you didn't have this
luxury, and again, bring backthis mentality that okay, we I
(17:50):
didn't have these tools, I needto build those tools.
So you went up and built some ofthe some of the very popular uh
connectors.
SPEAKER_02 (18:00):
Yeah, yeah.
So you know when I when I joineduh Confluent, the only products
that Confluent had at that pointum that was an Apache Kafka was
schema registry.
And so and and I I believe wehad the JDBC connector and the
HDFS connector.
And I think that was that wasthe entire products the product
stack.
(18:21):
Um and so like I I ended up, Iwas one of the first folks in
the field.
And so I would I'd end uptalking to a customer and there,
and a lot of it would be like,man, if I can get data from this
and then do this to it and putit in that, you know, we'll
we'll actually be able to getthis into production.
And so for me, you know, I waslike, okay, you know, the short
(18:44):
attention span can send, allright, I like this quick
tactical deployment.
SPEAKER_01 (18:48):
Waiting in the in the in the boarding in the
airplane, and you have a time uhwhen you can have a snack and
open laptop and start.
SPEAKER_02 (18:56):
Yeah, I I actually used to do that a lot because
I've I've lived in Austin forexcept for like like three
months that I worked atConfluent.
I've lived in Austin, and I usedto have to come out here to the
bay all the time.
So what I would do is I'd meetwith the customer, find out what
they what you know they neededto do.
Okay, pull down the Dockercontainers, yeah, and then work
on it on the flight home.
SPEAKER_01 (19:15):
Yeah.
SPEAKER_02 (19:15):
And usually I could get, you know, I like to say it
compiles, ship it.
Yeah, I could get that qualityusually by the time I got home.
SPEAKER_01 (19:23):
Yeah.
SPEAKER_02 (19:23):
And then, you know, hey, Mr.
Customer, play with this.
Does this kind of like whatyou're you're you're trying to
do?
Yeah.
And then that's really where alot of it came from is, you
know, I wanted to, I wanted tohelp people, I wanted them to be
successful.
Yeah.
And then, you know, at the sametime, I wanted, I wanted
Confluent to grow and besuccessful.
SPEAKER_01 (19:41):
Yeah.
SPEAKER_02 (19:42):
And the the easiest thing for me is I mean, if
you've if you've ever talked tome before, I'm always pushing
everything has to be Avro,everything, blah, blah, blah,
blah.
But in a lot of cases, peopledon't want to do that.
And the um the connectorecosystem let me focus on the
system.
Yeah.
So like I just need to get dataout of the system and into
(20:04):
connector format.
Yep, and into a struct format.
Yes, right.
And then I I hand it off, andcustomer can say, Hey, I don't
want your Avro.
I want to use use JSON.
And it's all transparent.
SPEAKER_01 (20:17):
Yeah, because you can configure uh converter, and
after that it will be handled uhby uh Kafka Connect uh
ecosystem.
SPEAKER_02 (20:25):
Yeah, and so that that worked out perfect for me.
And so that so I just startedbuilding them and then putting
them out on uh on my GitHub.
SPEAKER_01 (20:35):
Yeah.
SPEAKER_02 (20:35):
And then for a while I I used to directly publish
them to uh Confluence uh aregistry.
SPEAKER_01 (20:42):
Yeah.
But the registry was like yearsafter, kind of like uh we we we
realized that uh like havinglike App Store for connectors,
it's it's not only uh beneficialfor company uh to kind of like
uh have a sort of taxonomy, butalso beneficial for people to
search them.
And have a little bit ofassurance that if it was in the
part of some uh marketplace orsome sort of like a connectors
(21:05):
hub, uh there would be someassurance that it was at least
tested.
It's not only kind of like youknow, compile shipped, but also
it was tested and validated byby um uh kind of reputable
vendor.
So I would say that was huge umhuge help for people to
understand having this registry.
And I think that time was likeyour connectors and there was
(21:28):
like uh what the datamountaineer's uh connectors.
They were they were building.
SPEAKER_02 (21:32):
Yeah, they did a they did a great job.
SPEAKER_01 (21:34):
Yeah, so there was a um huge ecosystem of this of
different uh data sources umthat would definitely benefit
for bringing data from thesources into Kafka in order to
enable some other um other usecases.
Um so during this uh like whatwas your favorite connector to
(21:54):
build?
What was the most exciting or umsomething that maybe you know
the closest?
How close the biggest deal inthat time, maybe.
SPEAKER_02 (22:03):
Um oh man, you cut caught me off guard with that.
So you didn't you didn't likesend me a question?
SPEAKER_01 (22:09):
You didn't ask uh which of your kids you love
more, Victor.
Um I uh like the My favorite wasTwitter Connector.
Before Twitter was kind of freesocial network.
Uh Twitter Connector was greatbecause it was able to enable
such a great uh demos.
SPEAKER_02 (22:30):
You but you had to be brave to do it uh uh like a
live Twitter demo.
SPEAKER_01 (22:35):
I I love this doing the trust y'all in that.
I was doing this in uh in uh inRussia where the audience is uh
wired to be kind of okay, let'ssee how we can break this stuff,
how we can create some somemachine malicious thing.
So the Twitter connector wasgreat.
But it was for me, it was greatbecause I was using specific
hashtag, so I would capture someactivities and they said, hey,
(22:58):
you see, the people were havinga blast during my talk.
And um it's it's actually whatgreat demonstration of um what's
the streaming data, how thiswould look like.
100%.
100%.
SPEAKER_02 (23:09):
Yeah, I mean that that one that one was a fun one.
Um I uh the syslog one, thatone, that one was a lot of fun.
I I tend to like a lot ofnetwork, yeah, uh, um network
admin type things.
And so I always wanted to dolike a I never had the
opportunity to do it, but Ialways wanted to do like a
large-scale network monitoringuh project using using Kafka.
(23:31):
That was one thing I never nevergot the the opportunity to do.
But I I did build some uh someof that and some like some
Netflow collection stuff.
Um I like I liked uh uh uhbuilding and implementing uh
protocols, doing doing some ofthose.
Yeah.
Um the file system connector,that one, that one opened a lot
of.
SPEAKER_01 (23:52):
Correct.
SPEAKER_02 (23:53):
The the spoolder connector.
Yeah, yeah.
Yeah.
So that one was that one was alot of fun.
And and um I because that onewas fun for me because I always
um I if if you ever talk to meas a customer, I was always
everything's gotta be Avro.
And I didn't like that, youknow, the other connectors out
(24:14):
there wouldn't let me get thingsin with a strong type.
And so, you know, that one wouldif you would use my um kind of
messed up idea of a of a schema,you could apply schema to data
as you put as you put in.
And that ended up being usedpretty pretty heavily.
SPEAKER_01 (24:32):
Yeah.
Awesome.
So, Jeremy, um, if you wouldsummarize like everything that
you learn over the time andalready see kind of like there's
uh uh one of the things that wealready talk about is kind of
like it's not has to be like aperfect and the first step.
Um maybe some advice uh forpeople who listen to us.
Like we I don't want to soundlike it's like two old dudes
(24:53):
sitting here just oh in mytimes, blah blah blah.
Uh we still we still uh theteaching the technology of the
future, there's still you knowhuge uh uh the cluster of the of
the people who would love tolearn things about Kafka
connection processing and allthose kind of things.
So maybe some some advice forthose type of listeners.
SPEAKER_02 (25:13):
I would I would say design principles that I like to
live by is if you can't explainyour idea to another engineer in
less than five minutes, it's toocomplex and needs to be broken
up.
And that's that's something Ilike to to to live by.
The thing I will tell youthroughout my career I've seen
and in and some of the bigenvironments I've worked in,
(25:36):
simplicity scales.
And the other thing to keep inmind, you don't have access to
production.
So make sure make sure you haveproper logging.
SPEAKER_01 (25:44):
Yes.
SPEAKER_02 (25:44):
And then also logging that you can turn up.
SPEAKER_01 (25:47):
Yeah.
SPEAKER_02 (25:47):
So like sometimes info is not enough.
Exactly.
So like, you know, it's it'sokay.
Uh if if you look throughout myconnectors, there'd be there I
would use uh trace and trace youshouldn't turn on in in
production because it could be alot of it's gonna put a ton out
there and it could put and itcould uh dump some data.
SPEAKER_01 (26:05):
Yeah.
SPEAKER_02 (26:05):
Debug is gonna say, hey, I'm thinking about doing
this.
Yeah.
And I did that.
You know, but you you need toyou need to uh um love yourself
and make sure that you know if Ihave to uh uh get this data,
there's a way for me to get it.
So like pushing a config changeto production is a lot easier
than pushing an entire newrelease.
SPEAKER_01 (26:27):
And that was Jeremy Custom Border, ladies and
gentlemen.
Uh amazing person to talk to.
Uh and that was another greatepisode of Confluent Developer
Podcast.
I'm your host, Victor Gamoff,and as always, have a nice day.
This episode takes us back in time into MySpace
era.
Times where if you need a tool,you had to build this yourself.
This is Confluent Developer.
SPEAKER_02 (00:10):
That was one of those spots where it didn't
exist yet, so we built our own.
The biggest thing that thattaught me was you don't have to
get everything right the firsttime.
Fail fast, figure out what's uhwhat your problem is, and then
keep moving on from there.
SPEAKER_01 (00:23):
Welcome back to this episode of Confluent Developer
Podcast.
My name is Viktor Gamoff, and Iwill be your host today.
And today I have a very specialguest.
Every guest in this show isspecial, but this one is truly
something.
And uh you will understand bythe end of this episode why I'm
saying this.
(00:44):
Jeremy Custom Border, the legendof Kafka Connect.
And uh we're gonna talk aboutthis.
Jeremy, welcome to Kampf andDevelopment.
Thanks for having me.
Thanks for having me.
It's great to uh finally uhreconnect after you know many
years that I haven't seen you inperson.
I've seen you um, you know, allover like internet and LinkedIn
and other places, but it'sfinally good to be with you on
(01:07):
the same room and talk aboutsome cool tech stuff.
SPEAKER_02 (01:09):
Of course, my I'm Mr.
Man.
SPEAKER_01 (01:11):
Yeah.
Uh for those of you, uh those ofyou who uh don't know uh Jeremy,
can you tell us like in a fewwords what you do um and like
how people you know can uh canknow you?
SPEAKER_02 (01:24):
Yeah, so um I'm an I'm an engineer like a lot of a
lot of y'all.
Um I worked at Confluent.
I I joined uh originally joinedConfluent in in 2016 and worked
till uh 2022.
Um I um I'm the type type ofperson that I'd say um I I
worked in the field and thatworked out pretty well for me
because uh I have a shortattention span.
(01:45):
Yeah.
So it's like I go from problemto problem to problem and and I
enjoyed that.
I had a I had a great timeworking here, and it's it's fun
to be back chatting with y'all.
SPEAKER_01 (01:54):
Yeah, and um Conflict Developer Podcast, this
is the show where we explore theorigins of this uh technical
excellence and how um theluminaries or some like a
visionaries and some some verycool people and some awesome
nerds are came came to be, andlike uh what um you know how
they started and how they end upand the things.
(02:16):
And one of the first questionsthat I like to ask people is uh
what was your first job?
Not necessarily in related totechnology, not necessarily
relating to software.
What was your first time whenyou get this kind of like a
crispy uh like a bill or maybelike a few coins and stuff like
that?
Depends when you started andwhat was the rate of inflation
that time.
SPEAKER_02 (02:36):
Well, I mean, my my absolute first job was um I was
a paper boy from 10 to 16.
Nice.
And like I had that I had thatdown pr pretty well.
And that was a great job.
And I made um a hundred bucks amonth.
And like that money went in andthen went immediately back out.
SPEAKER_01 (02:55):
Yeah, you probably could afford you lots of uh
candies and some some otherinteresting entertainment.
SPEAKER_02 (03:02):
Exactly.
But I mean like that was uh thatwould be the 90s.
SPEAKER_01 (03:06):
So you know, you can go, yeah, you can you can do a
lot with a hundred bucks a monthin nineties, I guess.
SPEAKER_02 (03:13):
And then and then my first uh my first real IT job, I
I used to be the networkadministrator for the um the
Green County Library System inuh in Ohio.
SPEAKER_01 (03:25):
Okay, so tell us about this.
What year it was, what was thenetwork landscape at the time,
like what kind of like hardware,software things do you're
working with?
Tell us everything.
SPEAKER_02 (03:35):
Oh, it was it was a blast.
It was so what I loved aboutthis organization is um I was
one of the first people I thinkthat was that was nice.
Okay.
And it's like, you know, you'reworking for with for a bunch of
grandmothers.
Yes.
And you know, they would befreaked out if they broke a
computer.
And you know, I'm in this phasein my life, and you know, this
is like 98, so I'm I'm 18 yearsold.
(03:55):
Yeah.
And um, I didn't go to college,I'm self-taught.
So, you know, I'm I'm I'm inthis like like pre playground
that had a decent budget.
I had 400 machines that I that Iuh administrated with you know
T1 connections betweenbuildings, and I had uh, you
know, I had I had a decentbudget and it was like a great
(04:16):
place to learn because was itlike a Windows machines, Linux
machines?
All Windows machines.
Windows machines.
Yeah, and so it it was it wasgreat from the perspective of of
that things weren't perfect, sothat you could make a lot of
improvements.
So and then the other part withthat is you're working for all
these these grandmas.
Yes.
And you know, like so it wasgreat for me because you know, I
(04:38):
wasn't living on my own at 18,and you know, it's not like that
job paid that great.
Yeah.
And you know, you so I would Iwould I would say, hey, I'm
gonna be coming in to work on ony'all's computers on you know,
Tuesday.
I show up and there's all thesebaked goods.
It was amazing.
It was amazing.
SPEAKER_01 (04:57):
So you're working for all these grandmothers.
So you you have you have likeincentives, maybe not uh with
the monitor, but at least youwill be well fed, and you had
the your you know, the dose ofsugar that was, you know, the
the homemade cookies the best.
SPEAKER_02 (05:12):
Yes.
And the the cool part about thatis like I could uh I had enough
time where I could I could workand learn learn stuff, and
that's where I I wrote my firstprograms.
SPEAKER_01 (05:21):
What was the language you you used first
time?
SPEAKER_02 (05:23):
Uh first time was was ASP.net.
Okay.
So the the first portion of mycareer was was pretty heavily uh
Windows-based.
Yeah, yeah, me too.
SPEAKER_01 (05:31):
I I also um after I get my first job in you know the
Russian like bank calledZburbank, um I started learning.
Yes.
I I learned uh.NET and uh uhsurprisingly not to not to um I
didn't go uh PHP route becauselots of uh folks from my school
(05:53):
went to PHP route.
I went actually SP.net as well.
Yeah.
It was version uh.NET uh.NET2.0.
So it's just it just came out.
SPEAKER_02 (06:02):
I spent a lot of time with that later on.
Yeah.
So I I I did that for a whileand uh I wrote um we basically
had these CD ROMs with uh um uhlike genealogy things on it.
So you could come in and doresearch on genealogy in one
group.
Uh-huh.
So I I put something together tohelp searching and help kind of
find uh which CDs had what forfolks.
SPEAKER_01 (06:26):
Nice.
SPEAKER_02 (06:26):
And that was my that was the first thing I ever I
ever wrote.
SPEAKER_01 (06:29):
Yeah.
So basically you're uh theclassical three-tier
application, right?
You have a you have uh like afront end, you have a like
application server, which isprobably what internet
information server, right?
And a database.
SPEAKER_02 (06:40):
Yeah, well, I mean, man, you know, it's one of those
deals in those days that didn'tcompile, where I would, I would
say just it compiles, ship it.
SPEAKER_01 (06:47):
That's that's what's pretty cool.
My uh my first um uh.NETapplication was uh first complex
application.
It was attempt to implement kindof like a trading system.
Kind of I thought like, okay, soI was working in the bank as a
as my uh internship uh programuh during the summertime.
I was like, okay, so what aboutkind of like I write the
(07:08):
training application alsocompiled, uh and uh but my demo
uh for uh when I was went toshow this to to this like a
committee of the professors andstuff like that, um it only ran
on my laptop, obviously.
So runs on my laptop, so that'sthat's how and uh also I and now
we know where we got Dockerfrom.
(07:29):
Yeah, exactly, exactly.
Um so so you started the uh youalready mentioned that you
self-taught, you learned uhthese technologies um uh
basically on your own uh time onyour own pace, and looks like
you also very was veryinterested.
One of the one of the thingswhat we like to discover uh
discuss is uh the the biggestprofessional challenge.
(07:51):
Like I um I know you fromprobably it was you know you can
think about this.
It's it's really was a complexchallenge to find the ways to
bring the virtually any datasource in the world into Kafka.
Uh and we will talk we will talkabout this.
I really want to talk aboutthis, but on your opinion,
something that you would besuper proud or super embarrassed
(08:14):
if it's like some of the complextasks that you solved, like the
the biggest challenge of yourlife like today, or something
that you kind of likeunderestimated your your your
your your power and it's like uhyeah it didn't work out, but
still I learned a lot.
So that's that's what I want totalk about.
SPEAKER_02 (08:31):
Yeah, yeah.
So if I had to go there, I willtell you probably probably the
place I learned the most, and uhthis is gonna age me.
This is definitely gonna age me.
Um so you know, I I I was aroundOhio for a while, and then you
know, just at some point Idecided like I gotta get out of
Ohio.
And so like I I went to uh I Iwent out west and then I ended
(08:52):
up in LA and I ended up workinguh for for MySpace, the social
network.
And uh I I did that for fiveyears.
So um working.
SPEAKER_01 (09:02):
For those of you kids who don't know what the
MySpace is, is something like uhokay, Facebook also ages.
Those kids are not usingFacebook these days, it's
probably using TikTok orsomething like that.
But before Facebook was a thing,there was a MySpace.
Exactly, exactly.
SPEAKER_02 (09:21):
And I I uh worked on that.
So and the the fun part aboutthat is like you know, when you
talk to folks, they would say,like, okay, we needed to have a
caching tier.
Yeah.
They're like, well, why didn'tyou use Redis?
Yeah.
It's like go look at the firstcommits on Redis.
That was like five years after.
I think it was I I think Rediswas was built in 2010.
(09:42):
I I'd have to go look.
SPEAKER_01 (09:43):
You know, but before that, there was a memcache.
SPEAKER_02 (09:45):
We had one in yeah.
Uh Memcache came out in like2007 or 2000.
SPEAKER_01 (09:50):
I think it's even came up for you know this type
of like applications, they startpopping up the like initial wave
of web 2.0, kind of like what weknow today as uh web scale
things.
Um and I think the um yeah,memcache was also around that
time, around the 2000s.
SPEAKER_02 (10:10):
Yeah, yeah, but that was one of those spots where it
didn't exist yet.
So we we built our own and wewere we were uh running
monstrous scale on Windowsmachines, which uh and and
actually making great numbers.
So uh you were talking aboutyour ASP.net 2, you know, 2.0.
SPEAKER_01 (10:30):
Yeah.
SPEAKER_02 (10:30):
That was mine.
That's what we were working on.
It was the the whole web frontend was ASP.net 2.0.
SPEAKER_01 (10:36):
What was the uh I guess right now probably it
would be like a very like smallnumbers, but the dead time, what
was the profile of load?
Like how many um daily users ofthe of the platform?
SPEAKER_00 (10:48):
Now a quick word from our sponsor.
Confluent Developer the Podcastis brought to you by Confluent
Developer the website, which haseverything you need as a
developer of data streamingsystems.
And it's completely free.
We've got curriculum, hands-onexercises, executable tutorials,
the online data streamingengineer certification, also
free, a way to find a meetupnear you, those are free.
(11:12):
Everything is there.
I really want you to besuccessful in your journey as a
data streaming engineer, andthis is the site that has what
you need.
Check it out atdeveloper.confluent.io.
That's developer.confluent.io.
Now back to the show.
SPEAKER_02 (11:28):
Well, when I when I joined, uh when I joined, they
had about uh uh 10 million usersand were was pushing about uh I
want to say at that time, abouttwo or three uh gigabits a
second uh from their datacenters.
And then when I left, um we had320 million, if I remember
(11:50):
right, users.
And uh they were pushing aterabit from their data centers.
SPEAKER_01 (11:56):
So what was the again about uh we leave in 2025,
we have a great deal of um opensource technologies available
for building stuff because ofthis uh era of uh web scale.
But uh if we you know back intime, what was the like biggest
challenges apart from what wealready discussed with the uh
(12:16):
with caching, probably like aload balancing would be so much
charting of databases and stuff?
SPEAKER_02 (12:22):
So much has changed, and just so like you know, like
for example, like everybody usesKubernetes now.
There was nothing like that.
And then you know, Kubernetescan do like a slow rollout and
handle all the health checks,and we had to build everything
around that.
So, like all of these were wasstitching things like net
scalers and other uh componentstogether to get that type of
(12:46):
functionality.
And you know, today I look atwhat you have and it's like it's
it's amazing.
Yeah.
And then, you know, like we weremoving files around, and that's
how you would do deployments.
And you know, to you would youwould image machines by setting
up monstrous multicast groupsand sending tons of image
traffic down to to to machines,and that's how we like a way
(13:08):
that's how we are uh vs orsomething like that, right?
SPEAKER_01 (13:12):
Or not even that.
SPEAKER_02 (13:13):
No, it's all all but all bare metal.
Oh wow, yeah.
And then like at the end, it wasuh like right around when AWS
was was starting to come up.
We were also running Zen as welland and uh virtualizing it's
instead so instead of patching,we would just swap out machines.
SPEAKER_01 (13:27):
Yeah.
Wow, that's uh that was prettycool time.
Um so what this uh this time uhtaught you about data?
SPEAKER_02 (13:38):
That's that's the question that I'm trying to
slowly slide in to the I meanI'll tell you the the the the
the biggest thing that thattaught me was you don't have to
get everything right the firsttime.
It's like so you you need toobviously you need to to have
good principles around aroundstoring your data, but you can
you can increment and you canyou can add on and you can you
(14:00):
can do more at a later time.
It doesn't, you know, but foryou know, I'd say that's right
around the time agiledevelopment start has started
kind of kind of adopting.
Yeah.
It's like fail fast, figure outwhat's uh what your problem is,
and then and then keep keepmoving on from there.
So uh you said how how longyou've been in the MySpace?
SPEAKER_01 (14:18):
I was there for five years.
So I think this is the one ofthe things that I heard from
also former colleagues of us,maybe probably we need to get
him on this podcast as welleventually, uh Sriram
Suburbanian.
He mentioned one of the thingsthat these days you can put on
resume whatever you want, andit's kind of like uh should be a
(14:38):
little bit alarming for some ofthe hiring managers when you
have like a distributed systemsengineer in your resume and you
only spend like a one year ineach company.
So there's not enough time toyou to be like a fully solidify
as a distributed systemsengineer if you're not went
through the multiple iterationsof the same product.
Oh yeah.
Like there is a joke or maybesemi-joke when the people saying
(15:00):
that if you're not embarrassedfor the first version of your
product, you're probably doingsomething wrong.
A hundred percent.
So that's that's that's uhaligned with what you just said,
kind of like you don't have todo everything you know right at
the first time, but you have auh you need to know that you
need to iterate and learn fromthis and how you can do this
better next time.
So that's why the idea of umrelieving multiple major
(15:22):
versions when you justconstantly swapping not the
constantly, but you know, you'reswapping stack every like two
years.
You saw that something works,the technology of the time works
um after two years, there wouldbe totally different landscape
that there might be some of theproblems that you try to solve
that someone else is will besolving.
Like that's how we end up uhhaving Nginx instead of Apache
(15:48):
for for for web front end.
This is how we ended up havingsome other like a lighter like
HTTP load balancers in in frontof instead of like having some
vendor uh based you know loadbalancers and things like that.
SPEAKER_02 (16:01):
So it's funny, and you know, I go back and I look
at like uh like how many uses ofKafka I would have had there.
Yeah, it would have been uh itwould have been insane.
Yeah.
And then you know, if you lookat, hey, how would you do things
now, you know, like like metricswe built our own.
Yeah, you know, well Prometheusis pretty amazing.
Why don't you use that?
Yeah, but it was not thereanymore.
None of those things existed.
(16:21):
And you know the other the otherthing I got from that was um a
good understanding of how to douh production debugging.
And you you kind of you kind ofget a lot of uh of principles.
So one of my jobs was kind ofbeing an SRE before that.
SRE was award, yeah.
SPEAKER_01 (16:38):
Yeah, yeah.
System administrator.
That's how uh there was calledthat time.
Like called Jeremy, you know.
Yeah, Jeremy's just like showingup the for for like for cookies
and like doing something withfixing the thing.
SPEAKER_02 (16:50):
It was fun.
I mean, you'd you get in thesespots where like in all of these
charts go red, and then you'vegot to go figure out, hey,
what's the problem?
And we learned a lot, and it wasa great team.
And you know, like I I had thatsimilar experience and like uh
working at Confluent.
You just had a had a great teamwhere you all have the same
goal, yeah, and you're there tohelp each other and win.
SPEAKER_01 (17:09):
Yeah.
That's yeah.
Yeah, and I think um interestingthing when you you have this
like this mentality where thosethings were not existing, so you
had to build yourself.
And when I started um learningKafka, it was around 2000 uh
maybe 17, 16.
(17:30):
Uh, we had the product that wassimilar to uh to do stream
processing at Hazelcast thattime.
And I would start looking how toget data, and there was already
availability of uh differentways how we can get data in.
But you came a little bitearlier and you didn't have this
luxury, and again, bring backthis mentality that okay, we I
(17:50):
didn't have these tools, I needto build those tools.
So you went up and built some ofthe some of the very popular uh
connectors.
SPEAKER_02 (18:00):
Yeah, yeah.
So you know when I when I joineduh Confluent, the only products
that Confluent had at that pointum that was an Apache Kafka was
schema registry.
And so and and I I believe wehad the JDBC connector and the
HDFS connector.
And I think that was that wasthe entire products the product
stack.
(18:21):
Um and so like I I ended up, Iwas one of the first folks in
the field.
And so I would I'd end uptalking to a customer and there,
and a lot of it would be like,man, if I can get data from this
and then do this to it and putit in that, you know, we'll
we'll actually be able to getthis into production.
And so for me, you know, I waslike, okay, you know, the short
(18:44):
attention span can send, allright, I like this quick
tactical deployment.
SPEAKER_01 (18:48):
Waiting in the in the in the boarding in the
airplane, and you have a time uhwhen you can have a snack and
open laptop and start.
SPEAKER_02 (18:56):
Yeah, I I actually used to do that a lot because
I've I've lived in Austin forexcept for like like three
months that I worked atConfluent.
I've lived in Austin, and I usedto have to come out here to the
bay all the time.
So what I would do is I'd meetwith the customer, find out what
they what you know they neededto do.
Okay, pull down the Dockercontainers, yeah, and then work
on it on the flight home.
SPEAKER_01 (19:15):
Yeah.
SPEAKER_02 (19:15):
And usually I could get, you know, I like to say it
compiles, ship it.
Yeah, I could get that qualityusually by the time I got home.
SPEAKER_01 (19:23):
Yeah.
SPEAKER_02 (19:23):
And then, you know, hey, Mr.
Customer, play with this.
Does this kind of like whatyou're you're you're trying to
do?
Yeah.
And then that's really where alot of it came from is, you
know, I wanted to, I wanted tohelp people, I wanted them to be
successful.
Yeah.
And then, you know, at the sametime, I wanted, I wanted
Confluent to grow and besuccessful.
SPEAKER_01 (19:41):
Yeah.
SPEAKER_02 (19:42):
And the the easiest thing for me is I mean, if
you've if you've ever talked tome before, I'm always pushing
everything has to be Avro,everything, blah, blah, blah,
blah.
But in a lot of cases, peopledon't want to do that.
And the um the connectorecosystem let me focus on the
system.
Yeah.
So like I just need to get dataout of the system and into
(20:04):
connector format.
Yep, and into a struct format.
Yes, right.
And then I I hand it off, andcustomer can say, Hey, I don't
want your Avro.
I want to use use JSON.
And it's all transparent.
SPEAKER_01 (20:17):
Yeah, because you can configure uh converter, and
after that it will be handled uhby uh Kafka Connect uh
ecosystem.
SPEAKER_02 (20:25):
Yeah, and so that that worked out perfect for me.
And so that so I just startedbuilding them and then putting
them out on uh on my GitHub.
SPEAKER_01 (20:35):
Yeah.
SPEAKER_02 (20:35):
And then for a while I I used to directly publish
them to uh Confluence uh aregistry.
SPEAKER_01 (20:42):
Yeah.
But the registry was like yearsafter, kind of like uh we we we
realized that uh like havinglike App Store for connectors,
it's it's not only uh beneficialfor company uh to kind of like
uh have a sort of taxonomy, butalso beneficial for people to
search them.
And have a little bit ofassurance that if it was in the
part of some uh marketplace orsome sort of like a connectors
(21:05):
hub, uh there would be someassurance that it was at least
tested.
It's not only kind of like youknow, compile shipped, but also
it was tested and validated byby um uh kind of reputable
vendor.
So I would say that was huge umhuge help for people to
understand having this registry.
And I think that time was likeyour connectors and there was
(21:28):
like uh what the datamountaineer's uh connectors.
They were they were building.
SPEAKER_02 (21:32):
Yeah, they did a they did a great job.
SPEAKER_01 (21:34):
Yeah, so there was a um huge ecosystem of this of
different uh data sources umthat would definitely benefit
for bringing data from thesources into Kafka in order to
enable some other um other usecases.
Um so during this uh like whatwas your favorite connector to
(21:54):
build?
What was the most exciting or umsomething that maybe you know
the closest?
How close the biggest deal inthat time, maybe.
SPEAKER_02 (22:03):
Um oh man, you cut caught me off guard with that.
So you didn't you didn't likesend me a question?
SPEAKER_01 (22:09):
You didn't ask uh which of your kids you love
more, Victor.
Um I uh like the My favorite wasTwitter Connector.
Before Twitter was kind of freesocial network.
Uh Twitter Connector was greatbecause it was able to enable
such a great uh demos.
SPEAKER_02 (22:30):
You but you had to be brave to do it uh uh like a
live Twitter demo.
SPEAKER_01 (22:35):
I I love this doing the trust y'all in that.
I was doing this in uh in uh inRussia where the audience is uh
wired to be kind of okay, let'ssee how we can break this stuff,
how we can create some somemachine malicious thing.
So the Twitter connector wasgreat.
But it was for me, it was greatbecause I was using specific
hashtag, so I would capture someactivities and they said, hey,
(22:58):
you see, the people were havinga blast during my talk.
And um it's it's actually whatgreat demonstration of um what's
the streaming data, how thiswould look like.
100%.
100%.
SPEAKER_02 (23:09):
Yeah, I mean that that one that one was a fun one.
Um I uh the syslog one, thatone, that one was a lot of fun.
I I tend to like a lot ofnetwork, yeah, uh, um network
admin type things.
And so I always wanted to dolike a I never had the
opportunity to do it, but Ialways wanted to do like a
large-scale network monitoringuh project using using Kafka.
(23:31):
That was one thing I never nevergot the the opportunity to do.
But I I did build some uh someof that and some like some
Netflow collection stuff.
Um I like I liked uh uh uhbuilding and implementing uh
protocols, doing doing some ofthose.
Yeah.
Um the file system connector,that one, that one opened a lot
of.
SPEAKER_01 (23:52):
Correct.
SPEAKER_02 (23:53):
The the spoolder connector.
Yeah, yeah.
Yeah.
So that one was that one was alot of fun.
And and um I because that onewas fun for me because I always
um I if if you ever talk to meas a customer, I was always
everything's gotta be Avro.
And I didn't like that, youknow, the other connectors out
(24:14):
there wouldn't let me get thingsin with a strong type.
And so, you know, that one wouldif you would use my um kind of
messed up idea of a of a schema,you could apply schema to data
as you put as you put in.
And that ended up being usedpretty pretty heavily.
SPEAKER_01 (24:32):
Yeah.
Awesome.
So, Jeremy, um, if you wouldsummarize like everything that
you learn over the time andalready see kind of like there's
uh uh one of the things that wealready talk about is kind of
like it's not has to be like aperfect and the first step.
Um maybe some advice uh forpeople who listen to us.
Like we I don't want to soundlike it's like two old dudes
(24:53):
sitting here just oh in mytimes, blah blah blah.
Uh we still we still uh theteaching the technology of the
future, there's still you knowhuge uh uh the cluster of the of
the people who would love tolearn things about Kafka
connection processing and allthose kind of things.
So maybe some some advice forthose type of listeners.
SPEAKER_02 (25:13):
I would I would say design principles that I like to
live by is if you can't explainyour idea to another engineer in
less than five minutes, it's toocomplex and needs to be broken
up.
And that's that's something Ilike to to to live by.
The thing I will tell youthroughout my career I've seen
and in and some of the bigenvironments I've worked in,
(25:36):
simplicity scales.
And the other thing to keep inmind, you don't have access to
production.
So make sure make sure you haveproper logging.
SPEAKER_01 (25:44):
Yes.
SPEAKER_02 (25:44):
And then also logging that you can turn up.
SPEAKER_01 (25:47):
Yeah.
SPEAKER_02 (25:47):
So like sometimes info is not enough.
Exactly.
So like, you know, it's it'sokay.
Uh if if you look throughout myconnectors, there'd be there I
would use uh trace and trace youshouldn't turn on in in
production because it could be alot of it's gonna put a ton out
there and it could put and itcould uh dump some data.
SPEAKER_01 (26:05):
Yeah.
SPEAKER_02 (26:05):
Debug is gonna say, hey, I'm thinking about doing
this.
Yeah.
And I did that.
You know, but you you need toyou need to uh um love yourself
and make sure that you know if Ihave to uh uh get this data,
there's a way for me to get it.
So like pushing a config changeto production is a lot easier
than pushing an entire newrelease.
SPEAKER_01 (26:27):
And that was Jeremy Custom Border, ladies and
gentlemen.
Uh amazing person to talk to.
Uh and that was another greatepisode of Confluent Developer
Podcast.
I'm your host, Victor Gamoff,and as always, have a nice day.
Popular Podcasts
Betrayal Weekly
Betrayal Weekly is back for a new season. Every Thursday, Betrayal Weekly shares first-hand accounts of broken trust, shocking deceptions, and the trail of destruction they leave behind. Hosted by Andrea Gunning, this weekly ongoing series digs into real-life stories of betrayal and the aftermath. From stories of double lives to dark discoveries, these are cautionary tales and accounts of resilience against all odds. From the producers of the critically acclaimed Betrayal series, Betrayal Weekly drops new episodes every Thursday. If you would like to share your story, you can reach out to the Betrayal Team by emailing them at betrayalpod@gmail.com and follow us on Instagram at @betrayalpod and @glasspodcasts. Please join our Substack for additional exclusive content, curated book recommendations, and community discussions. Sign up FREE by clicking this link Beyond Betrayal Substack. Join our community dedicated to truth, resilience, and healing. Your voice matters! Be a part of our Betrayal journey on Substack.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com