January 15, 2025 • 62 mins
This episode discusses the complexities of VXLAN and EVPN in modern networking, highlighting the transition from traditional layer 2 designs to more efficient layer 3 systems. With insights from expert Aninda Chaterjee, listeners learn the importance of understanding underlying technologies for effective network management.
• Personal experiences with home lab setups and troubleshooting
• Explanation of VXLAN as a data plane encapsulation method
• Challenges faced in traditional data center architectures
• Transitioning from layer 2 to layer 3 using leaf and spine architectures
• How EVPN enhances VXLAN functionality by managing MAC addresses
• Importance of understanding technology for effective troubleshooting
• Aninda’s insights on becoming a technical author in the networking field
Find everything AONE right here: https://linktr.ee/artofneteng
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:00):
This is the Art of Network Engineering podcast.
In this podcast, we'll exploretools, technologies and talented
people.
We aim to bring you informationthat will expand your skill
sets and toolbox and share thestories of fellow network
(00:21):
engineers.
Welcome to the Art of NetworkEngineering.
I am AJ Murray and for thisepisode I am joined by Andy
Laptev.
Andy, how you doing man?
Speaker 2 (00:33):
AJ, I'm good, I was just telling you guys right
before the show started.
I've been banging away in myhome lab for two days trying to
tear out ESXi and replace itwith Proxmox, for reasons and I
have advice for anyone listeningIf you didn't know this do one
thing at a time.
The reason it's taken me twodays to get Proxmox working is
(00:55):
because I also decided tosegment my home network at the
same time.
So I'm creating all these zonesand VLANs and I have like a
Ubiquiti UDM Pro and I thought Iknew what I was doing.
But I could not.
The connectivity wouldn't work.
I'm on my PC in one VLAN, I'mtrying to get over to the VM in
another.
I thought I had inter-VLANcommunication.
I'm like just give me a CLIlike this ClickOps stuff.
(01:17):
So I finally just toreeverything out, put it all in
one blast radius.
Everything's in a default VLAN.
Yay, it works.
So do one thing at a time is myadvice.
Speaker 1 (01:29):
What's the expression Like?
If you do multiple things, youcan't half-ass a bunch of things
, you can only half-ass one orwhatever.
Speaker 2 (01:35):
Yeah and I couldn't figure out what was wrong.
I tried to install Proxmoxprobably 15 times in the past
two days.
I just kept reinstalling itbecause I'm like, well, maybe
the sd card's bad or maybe maybeit had to be in the array, or
maybe raid six, I mean, whateverright, I'm, I'm making stuff up
and I had no idea how totroubleshoot it.
And then it occurred to me I'mlike, huh, I wonder if it's
(01:56):
routing, and yeah, so anyway,well, that's a good like
troubleshooting step right.
Speaker 1 (02:01):
Like you know, make make only one change at a time
and then test it and see what'shappening.
Speaker 2 (02:05):
Yeah, and the public, the community, had plenty of
feedback.
People were trying to give meLinux commands and TCB dump and
check this and look at that.
I mean, ultimately I brokesomething and then got mad at
the wrong thing.
I'm like why?
Speaker 1 (02:18):
won't.
Speaker 2 (02:18):
Proxmox work.
I'm like, oh, I apparentlystill don't understand routing.
That's how my two days, butit's, it's good right now.
Tomorrow I'll be able to dowhat I need to do and start
labbing.
So I'm always learning right.
Like I didn't realize, I Iconfigured a raid five array
years ago on my server, and thensome, and then I had a disk
fail.
So I'm like, hey, buddy, youget another disk fail, you're
(02:39):
screwed.
I'm like, oh, like raid six isbetter, two disks can fail.
I'm like, okay.
Oh, like RAID 6 is better, twodisks can fail.
I'm like, okay.
So like I did that too.
I'm like learning stuff as I go.
How are you doing, aj?
Speaker 1 (02:48):
Doing good man Doing good.
I've been getting back to myroots.
I've been playing on the CLI atwork a little bit, which has
been fun.
I haven't done that for a while.
So yeah, it's been good,something different.
I am excited to welcome ourguest, aninda, this evening
because we are going to talkabout EVPN VXLAN.
I love VXLAN.
(03:09):
I'm not an expert in it.
I've cut my fingers, bloodiedmy knuckles and had a lot of fun
playing with it in varioussituations.
But, aninda, thank you so muchfor joining us.
Thanks for having me.
Guys Happy to be here.
So we know who you are, butwhat do you do and what's your
(03:30):
relationship status?
Speaker 3 (03:32):
with VXLAN.
So I am always I'm a TME slashsolutions engineer at Nokia as
of now, so previously Cisco andJuniper and I work in the DC
space, and I have been workingin the DC space for several
years now.
I do all sorts of things underthis umbrella, which includes a
(03:57):
lot of hands-on, so it's a lotof lab work, obviously in the
EVP and VXN space, and thenpreviously I was doing a lot of
SD access for Cisco when I was aTME at Cisco.
Last few years have mostly beenDC and has mostly been EVP and
VXN across enterprise telcoclouds, web scalers,
(04:19):
hyperscalers, you name it.
So it's been an interesting fewyears in this space and, of
course, the fun AI stuff that'sbeen going on.
Speaker 1 (04:30):
Right, yeah, absolutely.
So you've been working withVXLAN for a while, so it's kind
of a mouthful there, evpn.
There's actually a couple ofdifferent things going on, so
let's start first with VXLAN.
What is it?
Speaker 3 (04:53):
So I mean VXLAN is just, it's a data plane
encapsulation method.
So you're essentially adding abunch of layers to elevate
services from a different layerto a layer.
They call the overlay, andthink of it like MPLS or GRE as
an example.
So MPLS would add a bunch oflabels and then the labels would
essentially take you from onePE to the other.
(05:14):
So VXLAN does something similar.
It's the actual data planeheaders that you add when you
want to take the packet from onePE to another, and in VXN
terminology they call them a VXNtunnel endpoint or a VTAP.
So the VTAP essentially is anode that supports this
(05:37):
functionality or it has theability to NCAP and DCAP VXN
packets.
So you're just adding a bunchof headers and the headers are
it's an Ethernet header, an IPheader and a UDP and a VXN
header.
So that's 50 bytes overheadthat you add on top of the
original payload or packet.
Speaker 2 (05:59):
Did you say four different headers?
Yeah, I thought it was one.
Speaker 3 (06:05):
Yeah, should have been 10.
Might have been easier withmaybe 10.
Speaker 2 (06:11):
Why does this exist?
Like what problem does VXLANsolve?
Why are we adding four headersto every frame in our data
center?
Speaker 3 (06:21):
So you kind of sort of have to go back to how, in
general, dcs have evolved overtime and the problems that we're
trying to solve for today.
That existed with justgenerally how we design things
in the past.
Past.
What you would have is we wouldhave like a three tiered
(06:44):
architecture as an example,which would be an access layer,
distribution layer or a core anda core, and then occasionally
you would collapse the core andthe distribution and you would
just have like a collapsed coredesign.
Right and predominantly youwould have layer two links
between the access and thedistribution and good old
(07:05):
spanning tree to make surethere's no broadcast storms you
don't have layer two loopsbecause that's bad and then the
distribution and above you wouldtypically have layer three
routing.
So it would be just get me outof the network as quickly as I
can towards the core and thecore takes me out towards the
(07:25):
WAN.
Obviously, span actually had alot of inefficiencies.
You couldn't predict failuresand when something fails it is
very hard to predict how youwould converge.
You had convergence issues andof course you were sort of
always susceptible to a layer toloop, even if you've designed
(07:48):
the network really well.
You have all of your guards inplace.
You know there were some prettyhorrible things that could go
wrong.
That would melt yourinfrastructure and across your
domain, right.
Speaker 2 (08:04):
Yeah, go ahead.
Quick question, sorry so, didvMotion exist yet at this time
and at this point in time thatwe're talking about?
Speaker 3 (08:11):
Yeah, so you obviously had these Layer 2
requirements at that point intime also, which is why Layer 2
was such a big driving factorbehind some of these designs,
right, but the other factor alsowas the network was always
looked at from the perspectiveof these designs, right, but the
other factor also was thenetwork was always looked at
from the perspective of scalingup, which was the other problem.
So you would have like amonolithic architecture from a
(08:35):
server point of view, and whatthat means is, you know, let's
say, I have like a webapplication and it has to talk
to a database, and maybe it haslike a browser engine which
caters to all of your inboundHTTP requests.
Right, now, all of thesedifferent constructs were
(08:57):
typically coupled into a singleserver and you would have that
single server doing everything,right, right?
So any conversation that had tohappen between these
applications to serve theumbrella application was all
within one single physicalhardware that would connect to
your access or your or your leaf, right?
Speaker 2 (09:19):
I call these the good old days good old days Good old
, simpler days.
Yes.
Speaker 3 (09:27):
But the problem that that presented was now, let's
say, you want to scale up andyou want to handle more requests
, right?
And again this is scaling up.
So you start scaling up thatparticular server.
This could be in terms ofwhatever CPU memory, the NIC
(09:48):
speeds themselves.
But that starts to have directimpact to the network
infrastructure it's connected to, because now you have to scale
up your immediately connectednetwork infra as well, so you
have to scale up the speeds onyour network infrastructure side
, and so on.
So again, scaling up was not avery good design.
(10:10):
So eventually the industrystarted moving away from layer
two to layer three.
So it's a fully routedarchitecture between your leaf
so your access and yourdistribution and your core, and
you get a lot of benefits fromthis.
Now you can start to do ECMPacross.
(10:33):
I could have a routing protocolthat sort of distributes all of
my server infrastructureinformation and then I could
start to do ECMP from access upand down to the other access
layer where my other servers areconnected.
Ecmp is obviously morepredictable, routing is more
(10:56):
predictable, convergence is morepredictable, latency is more
predictable.
So you get a host of benefitsnow through this design, right.
And then of course, there wasthis shift from this three-tier
core distribution access towardsa Clow sort of architecture.
(11:17):
So again sorry, like goingreally back, Charles Clow was
kind of this guy who formalizeda way of doing fabrics for
telephony systems where he wouldhave this so for like telephony
systems you would use crossbars, right, and crossbars would
(11:38):
have X number of inputs and Ynumber of outputs and as you
start connecting them you couldget like an input line going to
an output line and somebodytalking to somebody else, right.
So you take the same sort oflogic.
And then what he did was hesaid the way to sort of reduce
complexity but continue to scaleup these telephony systems was
(12:00):
to start building my fabric instages.
So I would have an ingressstage where the call comes in,
and then I would have a middlestage which acts like a fan out
for the egress stage where thecall exits and connects to
somebody on the other side,right.
So they took the same logic andwe eventually applied it for
(12:22):
data center fabrics where youknow you have an ingress stage
which we call as the leaf, youhave the middle stage which we
started calling the spines, andthen we had the egress stage
where traffic exits out, whichwe do call the leaves again.
So it's essentially collapsedinto this leaf spine
(12:44):
architecture, which is whatyou'll hear that terminology
thrown around quite a bit today.
Speaker 2 (12:51):
Thank you for pronouncing his name correctly.
Speaker 3 (12:54):
Yeah, he's French, so Charles Clow.
Speaker 2 (12:58):
Almost everyone says Closs, and I don't know why it
drives me crazy.
Speaker 1 (13:04):
It's one of those things where it's as a native
English speaker, or an.
American probably is more right, like most Americans are going
to look at that and say clothsbecause they see CLOS.
But unless you have any sort oflinguistic knowledge, you know,
being where I am up near Canada, we, like a lot of our our
local languages is driven fromfrom French Right.
(13:26):
So when, I look at it, I waslike, oh okay, all right, I got
it.
Speaker 3 (13:29):
I mean I, I didn't.
I didn't know the currentpronunciation either.
So credit to Russ White, who isprobably the guru in terms of
everything DC.
Speaker 2 (13:40):
So I've just I've learned a ton from him,
including pronunciation of names.
I never knew when I I got tolove Russ.
Thank you for all yourknowledge, Russ.
Speaker 3 (13:52):
When.
Speaker 2 (13:52):
I was working on a Verizon central office.
I worked on a telco frame and20 years later I learned that
this was a Charlie Clow designthat I was running wires on.
Speaker 1 (14:06):
So let's kind of like try to summarize what we've
just talked about here.
So we've got VXLAN, we have ourspines, we have our leaves and
we have our border leaves wheretraffic will ingress, egress out
of the fabric.
We have a lot of flexibility inthat we can scale and we can
scale in a couple of differentways.
If we need additional bandwidthbetween our leaves, we can
(14:27):
scale and we can scale in acouple of different ways.
If we need additional bandwidthbetween our leaves, we can
scale and add spines.
Speaker 3 (14:31):
Correct, we're just scaling out, right, scaling out,
and if we need to addadditional ports, then we can
just add additional leaves,correct exactly so you move away
from scaling up where I gotta,you know, swap out my nicks to a
higher speed nick, or I got toswap out the leaf to something
that supports more line cards ormore speeds, to just scaling
(14:53):
horizontally instead.
Speaker 1 (14:55):
Right.
And then all of theseinterconnects are layer three,
so you have the benefits of nospanning tree.
We're using all of our linksand we're using ECMP rather than
, you know, lacp or somethingsimilar.
Right, you know?
I think a lot of people justassume that you know you're
going to get equal loadbalancing out of LACP and you
(15:15):
don't.
There's a lot of hashing andguessing going on, whereas with
ECMP it is what it is it's equalcost multipathing.
So if you have two spines,you're sending the traffic
equally to both of those spines,if you have three, so on and so
forth.
Speaker 3 (15:31):
Again, I would say that it depends.
It's not as equal as you think,even though it is ECMP, because
most of the hashing that you dotoday is per flow and not per
packet, because you don't want.
If you start doing per flow,then you kind of have to deal
with reordering at the serverside and with TCP.
(15:54):
Obviously that's going to causea host of problems and again,
with UDP it's problematic.
So you do per flow loadbalancing and it's typically
like a five double hash.
So you're taking your sourcedestination MAC, your source
destination IP and, let's say,the port number, the UDP part or
(16:15):
TCP part, as an example, right?
So as long as you have enoughvariability in your flow, you
should get a good sort of loadbalancing mechanism where it's
more or less well done and equalout of all your ECMP bots.
Yeah.
Speaker 1 (16:34):
Gotcha Okay, all right.
So if everything is layer three, what are we doing layer
two-wise?
Are we dedicating entire leavesto particular layer two domains
Like how are we using ournetwork now?
Speaker 3 (16:49):
Yeah, so that's sort of now the fundamental problem
that you run into is hey, Imoved my infrastructure to be a
smart layer three infrastructure, but I still want to live in
the olden days and I want toextend layer two.
And then of course there aresome fundamental requirements of
vMotion and things like that,which does require a layer 2
(17:12):
adjacency.
So how do you solve for that?
Because I obviously stop mybroadcast domain as soon as I
hit the leaf, because everythingabove is now layer 3.
So that's where VXLAN as anencapsulation sort of also helps
.
What you're essentially doingis you're transporting layer two
(17:37):
within this layer threeinfrastructure.
So, through various constructsand various means of
disseminating sort of layer twoinformation across the fabric,
which is where BGP VPN comes inI can start learning layer two
(17:59):
MAC addresses against aparticular VXLAN tunnel endpoint
which is a VTAP, and I can sayhey, if you want to go to this
MAC address, it sits behind thisVTAP, right against this bridge
domain or broadcast domain,which translates to a VXLAN
(18:21):
identifier, a VNI, on the VXLANside.
So then I can start toencapsulate the packet and
provide this Layer 2 extensioneven though you have a fully
routed Layer 3 infrastructureunderneath it.
Speaker 1 (18:37):
And, as you said, that's where the EVP and BGP
comes in, because now you'resharing the MAC addresses over
Layer 3 and you're essentiallymaking routing decisions based
off MAC location, correct?
Speaker 3 (18:50):
So I know, andy, you have a question, so I'll let you
ask and then maybe I'll expandon this a little bit more.
Speaker 2 (18:55):
I have all the questions.
I'm sorry.
Yeah, so when I learned thatEVPN is BGP exchanging MAC
addresses instead of routes, mybrain exploded.
I'm like wait, I didn't knowBGP could advertise MAC
addresses.
I thought it was for routes.
So try to remember where youare, because I have a couple of
(19:16):
questions.
Speaker 3 (19:18):
Can I break your brain?
Speaker 2 (19:19):
a little bit more.
Well, yeah, yeah.
Speaker 3 (19:24):
You know this isn't new right.
So in fact, did you know thatIS-IS does the same thing, like
what Cisco implemented, liketheir version of Trill, which
was FabricPath back in the day,they had IS-IS exchanging Macs
too.
Speaker 2 (19:41):
I did not know that, and thanks for pointing that out
to the audience.
Let me show you something else.
Andy doesn't know.
No, that's fascinating.
Speaker 1 (19:49):
I did not know that myself.
That's good yeah.
Speaker 2 (19:51):
So I'm following the historical evolution and the two
reasons I thought we went fromthree-tier to CLO or Leaf Spine.
At least what I learned in thebooks was you know, 80% used to
be north-south traffic with theold and then, once we went to
microservices and everythingeverywhere in the DC, it
switched to 80%, went east-westand we needed more speed and
(20:14):
less latency and everything'sone or two hops away.
I forget what it is and what isit?
Two hops, everything's two hops, yeah, no matter where it has
to go.
So that traffic shifted right.
And was what behind that?
Was it the microservices?
Yeah, it was virtualized.
Speaker 3 (20:30):
Yeah, so like when the network evolved, the network
evolution was pushed through, Ithink, like server evolution,
Like there was so much ofvirtualization that was
happening.
And now of course, it's allmicroservices, it's all
containerized and all of yourlike constructs of an
application that would sit onone physical server.
They would not exist either asVMs or containers and they would
(20:54):
sit behind different attachmentpoints, like different leafs.
And now when I, when like asingle HTTP request comes in to
service that I actually have totalk more east-west between all
of these application constructsthat sit across the fabric.
So now I'm just within myfabric, going east-west before I
(21:15):
actually go north-south.
So you're right, A lot of itwas driven by.
But that's what drove it A lotof it was driven by the advent
of virtualization andmicroservices and things like
that.
Speaker 2 (21:25):
Yep the other thing I wanted to ask.
I thought that um, evpn, vxland solved the problem with v
motion because you need all thev lands available everywhere.
So I guess the question Iwanted to ask earlier was when
it was three tier, layer two atthe access, how did v motion?
Like how could you move aworkload if a VLAN only existed
(21:45):
in one place?
Like you couldn't move itsomewhere else.
Speaker 3 (21:48):
Does that make sense?
Even in three tier the VLANswould exist in multiple places.
But what would also bedifferent is you would have like
a centralized layers threegateway for all of them, like on
the distribution.
You would typically have all ofthem.
Like on the distribution, youwould typically have all of your
(22:09):
SVIs in Cisco terminology orIRBs and Juniper, nokia
terminology that would existthere and everything below the
distribution was layer two.
So for all your services, thedistribution was your first L3
hop, which was the gateway, andyou would typically have some
form of FHRP, some redundancyprotocol running which is HSRP,
(22:30):
brrp, whatever, right?
So even if you're remulsioningacross, you're doing it across
your layer two adjacent accesslayer, but your layer three
never changes, right?
The problem now in a layer threeinfrastructure is well, I don't
have layer two adjacencyanymore unless I provide it in
(22:53):
some form through this tunnelingmechanism.
And then the other problem is,once I start positioning my
layer three gateways, where do Iactually position them?
Because every leaf is now L3 uptowards the spines, right?
So that's where we also broughtin this concept of like an any
(23:14):
cost distributed gateway, right.
So I would have the IRBs withthe same IP, same MAC that now
exist across all your leafs.
So I don't really care if Imove from leaf one to leaf 10,
because when I move I retain mydefault gateway, like I'm not
(23:34):
changing that, which is a coreconstruct that is necessary.
And as long as that defaultgateway exists on the leaf that
I have now moved under, from mypoint of view there's really no
change, right?
The resolution of the IP, ofthe gateway to the Mac, is going
to be exactly the same.
Speaker 2 (23:53):
Okay, last question on my torrent of questions, and
I hand it back to AJ.
So you said something about, sonow we've gone.
The end of the conversationearlier was so we've completely
gone to layer three.
Infrastructure, right,everything's layer three, but we
have to maintain layer two andI guess that's what the XLAN is
supporting.
Is that maintenance of layertwo down at, I guess, the
servers?
So that we can transport itacross.
(24:15):
Why I guess there's no othersolution, like we can't go layer
three from the leaf down theserver and get rid of the layer
two.
No, you absolutely can.
Speaker 3 (24:26):
Yeah, you can.
So a lot of people actuallybuild, especially when you look
at, let's say, a lot of cloudservices such as, like an
Anycast DNS as an example A lotof people build that
infrastructure with L3 all theway down to the servers.
Right, Because what theseservers do and you have multiple
(24:50):
nodes that are advertising thesame anycast address, as an
example right, and by doing L3all the way down, you're able to
achieve ECMP to all of theseguys.
Right, so it doesn't all theseleafs will advertise the same
Anycast address, so I don't carewhich leaf I eventually come in
on, that leaf will eventuallythen ECMP down to any of the
(25:15):
nodes that support this AnycastDNS, these Anycast DNS services.
Right, and that's a network,like a non-hardware way of doing
load balancing for Anycast DNSservices, Because otherwise what
you would typically do is youwould have to purchase like a
very expensive hardware loadbalancer and put it in the
(25:37):
middle and then put all of yourDNS servers behind it and then
the load balancer does all ofthat, right, so this is a
different way, a protocol-onlyway of actually doing any cast
DNS services as well.
So you do, and especially for,like cloud services, Kubernetes,
clusters, CNIs like Calico asan example.
(26:00):
A lot of these are nowdelivered through L3 to the
leaves.
Speaker 2 (26:07):
I guess it's dependent on use case and the
problem you're solving.
If you were building aGreenfield, data center.
You wouldn't just go, youwouldn't completely get rid of
L2, because that's the bestpractice.
Now it just it depends on whatyou're doing, I guess.
Speaker 3 (26:18):
It depends on what your application needs Like.
Sometimes these applicationsthey need like layer two
keep-alives between them as anorganization.
If you bought into thatapplication, what do you do?
You have to facilitate a way oftransporting that, even if you
(26:38):
want to go towards more modernways of building your DCs.
So, once again, it's theapplication's fault.
More modern ways of buildingyour DCs, right?
So yeah?
Speaker 2 (26:45):
So once again, it's the application's fault.
Speaker 3 (26:48):
It is.
I mean, that's the funny thing.
Right, we are building all ofthis network infrastructure to
solve and service your businessuse cases and what outcome you
want, so it is driven by theapplication.
100%.
Yeah, it's a lot.
I can see the brain spinning alittle bit, andy.
Speaker 2 (27:16):
Why is it so hard?
Like am I not you and I, so youknow?
Full transparency to theaudience.
I mean, you spent three days ofyour life with me teaching me
VPN, vxlan, and we labbed it andI videoed it.
And I keep telling myself I'mgoing to go back to those videos
, build it at home and really,until I build it, I don't really
think this stuff's going to bein my bones, but it just.
(27:36):
It seems so easy to you and forthe rest of us.
Why is it so complicated?
Speaker 3 (27:41):
is it not?
Really?
I don't think it's easy.
I think it's just it'sconsistency, it's, um, having a
day job that requires me to dothis every day, um, like, it's
the same thing with automation,right?
Um, you've tried, you'vestarted, stopped, I think.
Several times before I havedone the same thing and I suck
(28:03):
at it, like, really, really like, I'm terrible at it, right, but
I think, if that would becomepart of my day job and you have
to do it every day, I think thatthat changes the dynamics a
little bit, right?
You're forced to learn.
You're forced to encountercustomers who are doing this.
You're forced to, like, look atdesigns.
(28:24):
You're forced to encountercustomers who are doing this.
You're forced to look atdesigns and make sense of it,
and it just starts to sit inyour brain a little bit.
Speaker 2 (28:31):
So when you build an EVPN VXLAN fabric, let's say in
your lab, are you doing itartisanally by hand?
You're not automating it.
Speaker 3 (28:38):
No, so it depends, right.
So I made sure.
So I did a lot of DC when I wasat.
So I made sure.
So I've done a lot of.
I did a lot of DC when I was atCisco because I was supporting I
was like an escalation engineerfrom the engineering side of
things, but I was heavilyinvolved on the platform side,
right, so we wouldn't getinvolved in like initial
(28:58):
troubleshooting.
A lot of stuff that would cometo us would be you know, the
ASIC is dropping the packet hereand why is that happening?
On the Nexus side of things,since it was, nexus was
obviously heavily deployed onthe DC side.
So I would cater to a lot of DCdesigns and DC infrastructure
sided VXLAN there, but not asmuch as I've done it in the last
(29:19):
three years when I was atJuniper, which is where I really
got involved with customersdeployments and I was in the
thick of things.
So I made sure that I waslabbing, like I was building a
simple topology, let's say, twospines, four to five leafs and
some services, l2 and L3, right,I was doing it by hand, at
(29:41):
least like three or four times aweek like break it, build it
all over again, and I did it byhand for the entirety of my time
at Juniper and then again atNokia, because the CLI is
obviously different.
Unless I wanted to spin upsomething very, very quickly,
then I could either use JuniperApp Store or I'm using, like
(30:02):
Nokia, eda as an example.
Now, right, but I made sure todo it by hand, because that
forces you to make a lot ofmistakes, forces you to
troubleshoot when you haven'tconfigured things properly, and
it forces you to learn,obviously.
Speaker 2 (30:18):
And it's a lot of config.
That's why I asked right, Likeit's dense, it's yeah.
Speaker 1 (30:22):
It is, it is.
Speaker 2 (30:24):
This isn't a 15 line stanza, like you don't?
You don't enable the vxlan andwalk away no, and that's exactly
.
Speaker 3 (30:30):
There's so many moving parts and it's so
difficult to understand.
Um, you know why you needcertain things, and only when
you start to configure it, andthen you don't configure
something you need you, it hitsyou in the head really hard and
you're like, oh shit, I reallyneeded this for.
Uh, so and so reason right, um,and then you do it again 200
times and then maybe it sits inyour head for for some time.
Speaker 2 (30:51):
So yeah, hey, jan, like you've built fabrics right,
or or like, I mean, are peoplebuilding this by hand?
Is this something that mostfolks rely on, like like an ACI
or a fabric automation tool?
Speaker 1 (31:06):
You know, I don't know how many people are
building it by hand manually fora production environment.
I think a lot of the VXLANimplementation that I've
personally seen has usually beentied with some sort of product
package, right?
So Cisco's ACI, and thenthere's a bunch of other out
there that use, you know, vxlanunder their hood.
(31:27):
When I initially startedlearning it, even though I was
going to do it for ACI, I tookthe time to learn it manually so
I could understand how itworked.
I could troubleshoot it to adegree and have the appreciation
of, you know, what is what isACI doing for me as a network
operator, of what is ACI doingfor me as a network operator.
But I haven't seen many manualimplementations of it throughout
(31:51):
the course of my career.
Now, that being said, I haven'tspent a ton of time with it.
So just before I left my lastjob as a pro services engineer,
I was actually working on amanual implementation of VXLAN,
because it was just between twodata centers and they were two
very small data centers too, sobut they needed to stretch layer
two between the two, and thisis really the best way to do it.
Speaker 2 (32:11):
And is it a safe assumption that writing it by
hand is definitely the way tohammer what's happening in your
head, Like so you understand howthis is all playing together?
Yeah, absolutely.
Speaker 1 (32:21):
You know cause, as Aninda said, you said you're
going to make mistakes andyou're going to have to
troubleshoot it, and then, onceyou fix and understand how this
stuff goes together, you're justgoing form of orchestrator or
some form of automationframework Ansible, whatever
(32:48):
right, and more than that.
Speaker 3 (32:49):
You want, like the ROI, when you're doing it
through these orchestrators isyou want some guarantee of how
the network's actuallyfunctioning.
So you want to look at like daytwo stuff, like all of the op
stuff right, is all my peeringsup?
Is the fabric deployed asintended, or has somebody gone
(33:12):
in and changed some stuff and myfabric's no longer adhering to
what it should be and thingslike that.
So that's where a lot of thebenefits come in.
The automation problem has beensolved for a long time now.
But the day two stuff is that'sa place where things are really
happening and for DC, of course, for DC that's very, it's very
(33:34):
important and critical that youknow what's actually happening
in your infrastructure.
Speaker 2 (33:39):
And this is creeping into enterprise right.
It's not just data center.
Speaker 3 (33:42):
Yeah, I mean, and this is creeping into enterprise
, right?
Speaker 2 (33:44):
It's not just data center.
Yeah, I mean yeah of courseyeah.
Speaker 1 (33:47):
Of course yeah, because I think one of the
things that it can do for you isnow you're not tying an IP
address to a specific location,right?
So now you don't have to worryabout writing the same ACLs over
and over and over again for allyour different networks.
You can now stretch thatnetwork and put it wherever you
want, and now you have like aunified set of rules that you
don't have to necessarily worryabout.
Speaker 3 (34:08):
Yeah, especially for enterprise use cases where
there's so much of wireless andthere's so much of roaming, like
Cisco obviously pushed as theaccess, which was originally
Lisp plus VXLAN, so Lisp wasdoing all of the control plane
stuff and then VXLAN, lisp wasdoing all of the control plane
stuff and then VXLAN was yourdata plane.
But I think from an adoptionperspective Lisp was much harder
(34:32):
because nobody really used itas much.
They also moved to BGP VPN andI think you could do both.
I haven't done anything on theSDS space in like three years,
four years or more, but I'mpretty sure you could do BGP
EVPN with VXN today on theircampus side, also through DNAC
(34:53):
or Catalyst Center, whateverthey call it today.
Speaker 2 (34:56):
Is there anything to say about EVPN?
Besides, it's the control planethat carries MAC addresses.
Is there anything there?
I mean yeah.
Speaker 3 (35:05):
So again, like, a lot of this is through evolution.
Like BGP, evpn was not howVXLAN was deployed originally,
right?
So you actually had VXLAN floodand learn, which is very
similar to how, like, if you'vedone VPLS in the past, you're
(35:28):
sort of flooding a data planepacket, like an ARP as an
example, through your fabric andthen that enables all of your
other you know, terminatingendpoints to learn the source
packages, right, but there are alot of consequences of doing
that.
Obviously, flooding is not thatgreat and then the support
flooding.
(35:49):
You sort of need some form ofinfrastructure that facilitates
that flooding, right, andtypically this would be like a
multicast underlay that youwould build through PIM sparse
mode as an example, right, andthis multicast underlay that you
would build through PIMS parsemode as an example, right, and
this multicast underlay wouldflood all of your data plane
(36:10):
packets and then you would havedata plane driven learning,
right.
A lot of drawbacks, especiallyaround layer 2, jsmc when things
move, there's a lot of delayand convergence issues and
things like that.
So, again, industry evolved tofigure out that you could
actually use BGP through a newaddress family and a sub-address
(36:35):
family, which was L2, vpn, evpn, in short, bgp EVPN.
And now you could use BGP EVPNas a control plane to sort of
disseminate all of your learningacross these VXLAN tunnel
endpoints or the leafs or theVTEPs, right?
So as soon as I have a dataplane, learn on my leaf, let's
(36:57):
say a GARP comes in or an ARPcomes in, or an ARP comes in,
I'll process that in whateverway I have to process that ARP.
But I'll also go and tell BGPEVPN that hey, I had a local
learn and this is the MACaddress on this port, on this
bridge domain, which maps tothis VXLAN identifier.
(37:19):
So why don't you go and tellthe rest of the fabric where
this MAC address exists, right?
So BGP VPN takes that, it buildslike a BGP update and sends it
to everybody else.
And now everybody elseprocesses that update and it
knows, okay, mac A residesbehind leaf one right.
And if I want to get to MAC A,I can end, cap the packet with
(37:44):
all of these headers and send itto leaf one right.
And then the only thing I'lladd there is it's more than just
Mac addresses, andy.
So eventually they alsorealized that there are layer
three, very clear layer threeuse cases also, like I want to
transport just IP addresses aswell.
So you have a host of EVPNroutes that do different things.
(38:09):
Some of them do Mac only, thesame thing does Mac plus IP, and
some of them do like just an ipprefix as an example.
So I'm just decoupling macaddresses from ip addresses are
there route types or am Iconfusing protocols?
yeah, so they're.
They're bgp, evp and routetypes um one through.
I want to say 11 or 12.
(38:31):
No, no, it's yeah.
Speaker 2 (38:33):
So yeah, type five is stuck in my head for some
reason.
Speaker 3 (38:38):
Yeah, that's the IP prefix now.
So that's the one that justdoes away from MAC addresses and
you're only advertising, youknow, like a V4, v6.
That's correct.
You passed this quiz.
Good job, yay, do I get the jobor not?
You?
Speaker 2 (38:52):
got the job man.
Speaker 3 (38:56):
Yeah, that's a mouthful.
So yeah, if you have anyquestions, let's.
Let's talk about it.
Speaker 2 (39:03):
Stunned silence well, I I have a completely unrelated
question what the hell is afive tuple hash?
I'm gonna get us off of thisevp and vx for a second.
You said five tuple hash andI'm like question mark
exclamation.
I don't even know what a tupleis.
I've heard people talk abouttuples and well, there's been a
lot of information here.
I didn't want to stop the trainfor that dumb question, but
(39:23):
here we are.
There was a break.
Speaker 3 (39:26):
I mean you have to have like an algorithmic way of
determining if you have equalcost bots to send traffic out.
And this is either like layerthree, ecmp, or let's say you
have like a like a port channelright and and you have multiple
links in the port channel.
Now you can't, you can't sendtraffic out every link because
(39:47):
you can't guarantee the order inwhich the packets go out.
And I could, I could send,let's say, packet two before I
can receive packet two beforepacket one, which causes a host
of problems.
So typically you would do perflow, right.
And to do per flow you need tohave some algorithmic way of
determining which flow is put onwhich link.
(40:10):
If I have, let's say, two linksto send traffic out, right,
links to send traffic out right.
So that algorithm essentiallyuses different variables which
you, let's say, you could callthem tuples here, and these
variables include the source.
Typically by default they wouldinclude your source MAC, your
(40:31):
destination MAC from theinternet header, your source IP,
destination IP from the IPheader if this is an IP packet,
and, let's say, your destinationport, as an example from a UDP
header or a TCP header.
So these five things form yourfive tuples that go into the
algorithm and then the algorithmjust spits out like a hash that
(40:54):
says, hey, you take this linkand you go out of this link,
right?
Speaker 2 (40:59):
So do I understand that each of the flows are
pinned to a link?
Does that sound accurate?
Speaker 3 (41:05):
Yes, unless of course the number of available links
change and you sort of have torecompute how many links are
still available and then assign,like a bucket, to each of these
links.
Yeah, so it's typically.
Speaker 2 (41:21):
And is it pinning flows to a link to avoid the
out-of-order packets that youmentioned earlier?
Is that why?
Speaker 3 (41:29):
It's just a consequence of the algorithm,
right?
If there's no variability inthe algorithm, the outcome will
always be the same, which is avery interesting question when
it comes to VXLAN, because?
So here's a problem statement,right?
So once I do the VXLAN end cap,I add, like an outer Ethernet
(41:50):
header, I add an outer IP header, and then I add a UDP header
and then the VXLAN header.
Like these are the four headersthat I'm adding right Now.
The goal of my outer Ethernetand IP headers is purely for
transport, right?
I want to take the originalpayload from one VXLAN tunnel
(42:13):
endpoint to the other, right,and these endpoints are
identified by, you know, theirloopback addresses, as an
example.
So how do I do that?
I add an IP header that sayshey, the source IP will be
VXLand tunnel, endpoint one,destination IP is going to be
VXLand tunnel, endpoint two, andthen my Ethernet header.
(42:35):
Basically, hop, hop by hop, Itake it from my leaf spine down
to the other leaf, right?
But the problem that that nowpresents is, let's say, I have
10 different flows, um, but theyhave to go between the two
vtabs, right?
So the outer ip header and theouter ethernet headers that I
now add will always be the same,even though my original flows
(43:01):
or the original payload isactually changing and there is
enough variability, right.
So now, from a load balancingperspective, that's a big
problem, like, how would I solveit?
I would always start pinning itagainst a particular link,
going out towards my spines,even though I have multiple ECMP
links available, right.
So do you want to like take aguess at how do we solve this,
(43:26):
or do you want me to answer no,okay, Well, my brain is probably
full, so I'm going to let youlet's have you answer this.
Speaker 2 (43:40):
I have one other question and then.
Speaker 1 (43:42):
I'd like to maybe pivot to your authoring?
Speaker 2 (43:44):
uh, okay, sure, honestly, my brain this is, but
this is what it's, no reflectionon you or the technology.
This is what happens to me whenI try to talk about as an
example I get about 20 to 30minutes.
It happened when you wereworking with me, remember,
trying to teach me.
We get like about this time inand you would ask me a question
and I would just stare.
I think my brain is full.
(44:04):
I think you've reached thepoint that my you've reached the
perimeter of my intelligence.
Yeah.
Speaker 3 (44:13):
So just I put this problem statement because you
asked a really importantquestion, which is how are we
deciding where to position thisflow?
And that presents a problem whenyou're doing NCAP like this,
because you abstract away thevariability of the original
flows and you're addingsomething on top which is almost
(44:36):
always the same if the servicessit behind the same two leaves.
You need to have some mechanismthat sort of adds that
variability right.
So that's why the UDP headerexists before the VXLAN header.
So obviously it identifies thatwhat comes next is a VXLAN
header.
But what we do now is we usethe original payload and we do
(45:02):
like a five-tuple hash again onthe original payload to generate
the UDP destination port.
So as long as the originalpayload varies, the UDP port
starts to vary right.
And then to load balance thisVXLine encapsulated packet, I'll
use my outer Ethernet headerstuff, my outer IP header stuff
(45:25):
and then the UDP destinationport which is now changing.
So I'm able to introduce thisconcept of variability which
allows me to now load balanceeven encapsulated packets better
across these equal cost parts.
Speaker 2 (45:40):
AJ, I wanted to be a TME until I met Aninda and then
I realized I might not be assmart as I thought I was.
Speaker 3 (45:48):
Hey, don't say that no it's a compliment, man.
Speaker 2 (45:53):
I'm just always blown away by your depth of knowledge
.
I'll say this like I don't.
Speaker 3 (45:57):
I honestly don't think I'm smart.
Like there are like people whoget this like I would.
I would take like six months toget this, and there are people
who would get this in like acouple of weeks.
What I feel I do a littledifferently is, um, like I lab
up so much of this day in andday out, like that's that's the
only thing that has really madea difference for me.
Um, it's, it's just in mymemory because I've done it so
(46:20):
much, right?
Uh, so it's not that I'm smart,it was just like a crazy amount
of hard work and a crazy amountof like just in the lab, um,
looking at broken shit.
Uh, day in and day out yeah,you've put in the reps.
Speaker 2 (46:33):
Um.
The last question before wepivot away into your life as an
author I wanted to ask was sothese are standards correct?
Evpn, vxlan.
Speaker 3 (46:42):
Yeah, some of them are like RFCs, some of them are
IETF jobs.
But yes, they're sort ofstandardized Correct.
Speaker 2 (46:50):
But so I've heard people say that different
vendors implement themdifferently.
So I guess is that accurate?
And then, if it is, how dovendors implement them
differently?
So I guess, is that accurate?
And then, if it is, how dovendors implement standards
differently?
Maybe they don't know what thestandard means because no, there
are.
Speaker 3 (47:06):
So, yeah, there is.
Sometimes the verbiage in thedraft or the rfc is a little
vague and it comes.
It's up to the vendor and itdepends on how the vendor
chooses to implement this insoftware.
So that's where some of thedifferences come in.
And then just there, like, forexample I don't want to get into
(47:28):
this, but like data center,interconnects are obviously a
big deal, because when you'rebuilding DCs you're probably
building multiple DCs.
Typically simplest form wouldbe like a DRDC as well, like a
primary DC and a DRDC, and youwant to connect the two and have
them talk to each other.
So you need some form ofinterconnect.
(47:51):
Now, as an example, there is anRFC that exists that defines how
to do a DCI, which is called anintegrated interconnect.
But then Cisco decided thatthey could do, they wanted to do
things differently, perhaps, intheir eyes, more efficiently.
So they put in another draftfor doing DCI, which now Cisco
(48:14):
uses, but a lot of the othervendors in the industry use the
RFC that already exists.
So now you're in a positionwhere if I have mixed vendors
and I have Cisco on one side and, let's say, juniper and Aris on
the other side and I want to doDCI.
Well, they're doing twodifferent forms of DCI.
(48:34):
Now, to Cisco's credit, theyalso figured out that
interoperability is a big deal.
So they do have some knobswhere you could sort of make it
compatible with other vendors,but you see where the complexity
starts to creep in, right.
So yeah, it is a problem.
Speaker 2 (48:56):
So what's it like writing a book?
Why did you write a book?
What, what, what is your book?
I mean, you're a family guy.
I've worked with you.
You're a very hard worker.
You put in a lot of time.
You have a wife, you have ayoung daughter.
I can't imagine how much timeit took you so like, why would
you, why did you write your book?
Speaker 3 (49:13):
I guess the book stuff has been on my mind for,
uh, for a long time, like when Iwas like I've been a tech
blogger for um for quite a whilenow, um, and it was off and on.
And then I really heavily gotinto it when I was a tme at
cisco doing all of their sdastuff, and I realized like
there's such a big gap, um, uh,in in just pointing and clicking
(49:39):
on the UI and thenunderstanding what's actually
happening behind the hood.
Because it's cool to do all theUI stuff and you can probably
figure that out and have thefabric maybe in 30 minutes to an
hour, but when things break, Idon't think the UI is really
going to solve it for you, maybeto a small extent, but you
(50:01):
really need to understand howall of the stuff under the hood
works, right?
So I started taking all of thatand writing very tech-heavy
blogs.
The marketing part of a TME isnon-existent, it's all tech.
Like we weren't, we weren'treally doing all the fluffy
(50:21):
marketing stuff, right?
And I haven't really done thatever.
I don't particularly enjoy iteither, right?
So I was just doing all of thetech stuff and I realized that,
okay, there's, there's anaudience for it and I really
like doing it anyway.
So I actually tried to do theSDA book for some time, but that
really didn't gather steam andI couldn't really see like a way
(50:46):
in, because Cisco was just sohumongous and so big.
I was just a very tiny part ofit.
Speaker 2 (50:51):
But to approach your manager and say I'd like to
write a book.
I'm sorry to interrupt you, buthow do you even?
Who do you talk?
To in Cisco to write a book.
I'm sorry to interrupt you.
Like how do?
Speaker 3 (50:58):
you?
How do you?
Even who do you talk to?
It's just gonna write a book.
Yeah, we've.
I've tried that a few times.
I just like I just couldn't seea way in, right, yeah, um, and
then I had sort of given up, orI was actually thinking about
like just self-publishing somesda stuff at that point, um, but
I didn't.
I didn't know if, like, I saw alot of value in it at that
point in time.
Then I moved into Juniper Againa lot of heavy hitting DC stuff
(51:20):
, and again I realized, man,this stuff is hard.
So I went back to a lot of techblogging again.
And then Mike Puchong, who wasmy big boss at that point and
who is my big boss again, andthen Kathy Gadecki, who was
under him.
And then I had another managerat Juniper, rida Hameedi.
(51:48):
Exceptional people, verysupportive of things you want to
do outside of just your day job.
So I then pitched the idea ofthis Juniper book, focused just
on Dana centers, because nothingexisted from a Juniper
perspective at that time.
So I knew there was a gap thatwe could fill.
So Kathy got me in touch withRuss and Russ was working at
(52:08):
Juniper at that point.
So I was very lucky.
I had some overlap with him forabout a year and then he
introduced me to this gentlemanat Pearson called Brett, and
Russ works with Brett quiteheavily.
So Brett was really interested.
Originally this was actuallysupposed to be just video
(52:28):
content and I wasn't too keen ondoing like a tutorial or like
on demand stuff.
At that point I said I think myforte is writing.
I think that's what I'm morepassionate about.
So I pitched an entire outlinefor the book and they were like
really happy to take it up.
Um, and yeah, there was nolooking back after that.
(52:48):
Um took me about a year andthree months, uh, in total, from
starting to write to publishing.
I was hard man, I was workingnights, I was working every
weekend and again, I don't thinkthere's a way to balance it.
There's sacrifice.
There's a lot of family timethat is sacrificed, for sure,
(53:17):
and then of course, you're doingyour day job, but then you sort
of have to write the book aswell.
Um, I was okay with thatbecause I felt a lot of my day
job was contributing back towriting the book because it was
so heavily just DC specific.
So I actually told all of mybosses that don't take me out of
my day job because I feel thatmakes the book more realistic,
(53:39):
like I'm not writing in a bubble, I know what's actually
happening outside and I want tomake it relatable.
So I continued a hundredpercent on the day job and then
150% on the on the book and Ilost a lot of like family stuff.
Speaker 2 (53:54):
But um, yeah, you make up, I guess you had full
support of your wife right, likea lot of like family stuff.
Speaker 3 (53:56):
But um, yeah, you make up.
Speaker 2 (53:57):
I guess you had full support of your wife right like
of course there's a lot on theshow how important it is right
to have the support of your,your support system, to be able
to do something like that 100like it's.
Speaker 3 (54:07):
It can be very frustrating because, like
there's times where you'retrying to build labs for the
book and like it's just itdoesn't work and stuff's
breaking, which sometimes youthink it's a bug, sometimes it's
a mistake, and you spend daystroubleshooting it's that, like
a lot of that comes out, um, asfrustration when you're like
with with your family away fromlike the book stuff also.
Speaker 2 (54:29):
So, yeah, my life is like a pillar of support, um,
and I would either like thiswouldn't exist without, without,
for sure and I think I heardyou correctly the reason that
you wrote the book is you wantedpeople to understand what was
happening under their hood, forwhen things broke right like
here's how it's all working, sowhen it breaks, you'll have the
tools you need to know where tolook.
(54:49):
Was that why you wanted towrite this book?
Speaker 3 (54:51):
that is my general um .
Uh yeah, the general ideabehind why I do a lot of the
tech blogging is is that, yeah,I like even a lot of the vendor
stuff, like even juniper, evennokia, right, um, it's not easy
to figure out what's happeningbehind the hood, and sometimes
the vendors don't make it easy,right, um, and I feel like it's
(55:14):
up to us as the community tosort of help each other out,
especially for people who aremanaging this day in, day out,
who are in the upside of things,because I have been on that
side right when hospitals godown, when people call in and
say they're losing millions ofdollars per second and shit hits
(55:37):
the fan.
Like you want the knowledge tobe able to troubleshoot some of
these things.
So yeah, it's just, it's ahelping hand, it's a way for the
community to understand this ishow stuff works behind the
scene.
If, if you were able to put inthe time to lab it up and lead
it up.
So, yeah, I just I loved givingback.
(55:59):
And I have another book that I'mwriting for Juniper, because I
had, like I had committed to it.
And then I have a third bookthat we just signed a contract
for, which is interesting.
It's a it's a multi vendor DCbook that's going to cover Cisco
, adista, juniper and and Nokia,so maybe like 2000,000 pages of
a big.
But yeah, I have a lot of funwith it.
Speaker 2 (56:20):
And it's so now you're on fire.
You've gotten started andyou're on fire.
I think that it's.
I just want to put a.
You know, I think it'simportant to have people on
staff that have the deepknowledge for when things break
and the reason I think it'simportant.
I don't know if you guys haveever had this perception from,
like, vendor marketing and I'mnot pointing a finger in any
(56:40):
particular direction butthere'll be vendors who say that
their automation platform is sogood that it you know things
won't break, right.
Oh well, if you, you know, ifyou use this, you know you won't
break it.
I feel like that's a bigmarketing angle.
That and listen, it's, it's.
It's usually better than doingit artisanally by hand in the
CLI.
So I get it.
But things break.
(57:01):
You hit bugs, things get wedged, weird stuff happens, right,
and and if you were like you hadsaid something like that
earlier if you're relying on theUI and you know this tool to
manage it, fine, you know yougot to change connectivity or
add or delete.
But man, when stuff breaks,like you better have Aninda
around, because I don't know ifthat automation platform is
(57:21):
going to do that for you, right,like, even if there's a
rollback.
It just seems to me that youreally need somebody around who
knows what the hell's happeningunder the hood.
Speaker 1 (57:30):
Yeah, that makes a big difference, for sure.
Speaker 2 (57:32):
If you want to, you know, get your network back up.
Aninda, this has been amazingand and and I'm sorry that I had
a dumb look half of theconversation, because it's just
so much, but it's just so muchto take in.
Speaker 1 (57:46):
There's so many layers of complexity, which is
why I thought it'd be helpful tohave this conversation and and
trying to suss out like what'sgoing to be the most important
stuff to cram into an hour isdifficult like vxlan, yeah, yeah
where would you direct peoplelike?
Speaker 2 (58:01):
so I mean, I'm, you know, I'm halfway through
studying for the ccmp andthere's a vxlan chapter in that
book, right?
So I guess vendors havematerial people can consume to
learn this stuff.
But some of them are coming toyou saying, hey, man, you know
this overlay, it's all the rage.
You can be excellent.
Like where would you directpeople who want to get started
learning about everything we'retalking about here?
Speaker 3 (58:22):
so I learned a majority of what I wrote in the
book um from dinesh dutt, who isagain pioneer um in this space,
from, uh, russ white um, jeffdoyle, jeff Russ White, jeff
Doyle, jeff Tansura.
So a lot of them have eitherbooks content or they just have,
like Jeff and Jeff do BetweenTwo Nerds.
(58:44):
They have a lot of good contenton EVP and VXLAN.
They have a lot of interestingguests that come in and talk
about this.
Russ has a huge amount of freestuff out there Rule11.tech,
rule11.academy, which is his newacademy for teaching.
(59:05):
And then Dinesh Dutt has twoexcellent books.
I think they're called BGP inthe data center and then EVPN in
the data center and then cloudnative networking, I think
center, and then evpn in thedata center and then cloud
native networking, I think um.
So I was, yeah, I was just Iwas watching their stuff, uh,
reading their stuff uh, day in,day out.
When I joined juniper, when Iwas writing the book, um, yeah,
(59:27):
so go, go check them out um,they are the original sources of
truth for any of this um.
And then, yeah, maybe, maybecheck out the book I wrote,
because I hope I sort of took alot of what they were teaching
me and then hopefully translatedthat into easily consumable
information, with someimplementation information
(59:49):
around Juniper.
Speaker 2 (59:51):
What's the name of your book and how can people
find it?
Speaker 3 (59:54):
So the book is called Deploying Juniper Dinner
Centers with EVP and VXLan.
You could either get it onAmazon or you could buy it
straight from Pearson, either asa hard copy or PDF.
Speaker 1 (01:00:07):
Awesome, and where can people find more from you?
Speaker 3 (01:00:12):
I am on LinkedIn, I am on Twitter for now, and then
Blue Sky recently, and at all ofthese places it's Anandchat.
So A-N-I-N-C-H-A-T.
Speaker 1 (01:00:26):
And you said you're a tech blogger.
Were you blogging for yourself?
Do you have a website that youstill blog at?
Speaker 3 (01:00:34):
Yeah, I blog for myself.
It's not very consistentbecause there's so much of work
stuff going on and then a lot ofthe stuff I want to blog about
goes into the book, but I dohave a lot of good content there
.
It is calledtheaskyconstructcom.
Speaker 1 (01:00:49):
Awesome.
We will put links to all ofthat great stuff in the show
notes and you can check out hisblog, check out the books and
other resources that hementioned, and then to thank you
so much for joining us tonight.
This has been a thank you forhaving me.
It was awesome.
Thanks guys.
Yeah, I feel like there's a lothere we still could have
unpacked, so we might have tohave you back for, uh, another.
Speaker 3 (01:01:08):
Another follow-up episode yeah, I would love that.
It's always good to talk to youguys.
Speaker 1 (01:01:13):
Awesome, all right we will see you next time on
another episode of the Art ofNetwork Engineering podcast.
Thanks for joining us.
Speaker 2 (01:01:28):
Hey everyone, this is Andy.
If you like what you heardtoday, then please subscribe to
our podcast and your favoritepodcatcher.
Click that bell icon to getnotified of all of our future
episodes.
Also follow us on Twitter andInstagram.
We are at Art of Net Eng,that's Art of N-E-T-E-N-G.
You can also find us on the webat artofnetworkengineeringcom,
(01:01:48):
where we post all of our shownotes, blog articles and general
networking nerdery.
You can also see our prettyfaces on our YouTube channel
named the Art of NetworkEngineering.
Thanks for listening.
This is the Art of Network Engineering podcast.
In this podcast, we'll exploretools, technologies and talented
people.
We aim to bring you informationthat will expand your skill
sets and toolbox and share thestories of fellow network
(00:21):
engineers.
Welcome to the Art of NetworkEngineering.
I am AJ Murray and for thisepisode I am joined by Andy
Laptev.
Andy, how you doing man?
Speaker 2 (00:33):
AJ, I'm good, I was just telling you guys right
before the show started.
I've been banging away in myhome lab for two days trying to
tear out ESXi and replace itwith Proxmox, for reasons and I
have advice for anyone listeningIf you didn't know this do one
thing at a time.
The reason it's taken me twodays to get Proxmox working is
(00:55):
because I also decided tosegment my home network at the
same time.
So I'm creating all these zonesand VLANs and I have like a
Ubiquiti UDM Pro and I thought Iknew what I was doing.
But I could not.
The connectivity wouldn't work.
I'm on my PC in one VLAN, I'mtrying to get over to the VM in
another.
I thought I had inter-VLANcommunication.
I'm like just give me a CLIlike this ClickOps stuff.
(01:17):
So I finally just toreeverything out, put it all in
one blast radius.
Everything's in a default VLAN.
Yay, it works.
So do one thing at a time is myadvice.
Speaker 1 (01:29):
What's the expression Like?
If you do multiple things, youcan't half-ass a bunch of things
, you can only half-ass one orwhatever.
Speaker 2 (01:35):
Yeah and I couldn't figure out what was wrong.
I tried to install Proxmoxprobably 15 times in the past
two days.
I just kept reinstalling itbecause I'm like, well, maybe
the sd card's bad or maybe maybeit had to be in the array, or
maybe raid six, I mean, whateverright, I'm, I'm making stuff up
and I had no idea how totroubleshoot it.
And then it occurred to me I'mlike, huh, I wonder if it's
(01:56):
routing, and yeah, so anyway,well, that's a good like
troubleshooting step right.
Speaker 1 (02:01):
Like you know, make make only one change at a time
and then test it and see what'shappening.
Speaker 2 (02:05):
Yeah, and the public, the community, had plenty of
feedback.
People were trying to give meLinux commands and TCB dump and
check this and look at that.
I mean, ultimately I brokesomething and then got mad at
the wrong thing.
I'm like why?
Speaker 1 (02:18):
won't.
Speaker 2 (02:18):
Proxmox work.
I'm like, oh, I apparentlystill don't understand routing.
That's how my two days, butit's, it's good right now.
Tomorrow I'll be able to dowhat I need to do and start
labbing.
So I'm always learning right.
Like I didn't realize, I Iconfigured a raid five array
years ago on my server, and thensome, and then I had a disk
fail.
So I'm like, hey, buddy, youget another disk fail, you're
(02:39):
screwed.
I'm like, oh, like raid six isbetter, two disks can fail.
I'm like, okay.
Oh, like RAID 6 is better, twodisks can fail.
I'm like, okay.
So like I did that too.
I'm like learning stuff as I go.
How are you doing, aj?
Speaker 1 (02:48):
Doing good man Doing good.
I've been getting back to myroots.
I've been playing on the CLI atwork a little bit, which has
been fun.
I haven't done that for a while.
So yeah, it's been good,something different.
I am excited to welcome ourguest, aninda, this evening
because we are going to talkabout EVPN VXLAN.
I love VXLAN.
(03:09):
I'm not an expert in it.
I've cut my fingers, bloodiedmy knuckles and had a lot of fun
playing with it in varioussituations.
But, aninda, thank you so muchfor joining us.
Thanks for having me.
Guys Happy to be here.
So we know who you are, butwhat do you do and what's your
(03:30):
relationship status?
Speaker 3 (03:32):
with VXLAN.
So I am always I'm a TME slashsolutions engineer at Nokia as
of now, so previously Cisco andJuniper and I work in the DC
space, and I have been workingin the DC space for several
years now.
I do all sorts of things underthis umbrella, which includes a
(03:57):
lot of hands-on, so it's a lotof lab work, obviously in the
EVP and VXN space, and thenpreviously I was doing a lot of
SD access for Cisco when I was aTME at Cisco.
Last few years have mostly beenDC and has mostly been EVP and
VXN across enterprise telcoclouds, web scalers,
(04:19):
hyperscalers, you name it.
So it's been an interesting fewyears in this space and, of
course, the fun AI stuff that'sbeen going on.
Speaker 1 (04:30):
Right, yeah, absolutely.
So you've been working withVXLAN for a while, so it's kind
of a mouthful there, evpn.
There's actually a couple ofdifferent things going on, so
let's start first with VXLAN.
What is it?
Speaker 3 (04:53):
So I mean VXLAN is just, it's a data plane
encapsulation method.
So you're essentially adding abunch of layers to elevate
services from a different layerto a layer.
They call the overlay, andthink of it like MPLS or GRE as
an example.
So MPLS would add a bunch oflabels and then the labels would
essentially take you from onePE to the other.
(05:14):
So VXLAN does something similar.
It's the actual data planeheaders that you add when you
want to take the packet from onePE to another, and in VXN
terminology they call them a VXNtunnel endpoint or a VTAP.
So the VTAP essentially is anode that supports this
(05:37):
functionality or it has theability to NCAP and DCAP VXN
packets.
So you're just adding a bunchof headers and the headers are
it's an Ethernet header, an IPheader and a UDP and a VXN
header.
So that's 50 bytes overheadthat you add on top of the
original payload or packet.
Speaker 2 (05:59):
Did you say four different headers?
Yeah, I thought it was one.
Speaker 3 (06:05):
Yeah, should have been 10.
Might have been easier withmaybe 10.
Speaker 2 (06:11):
Why does this exist?
Like what problem does VXLANsolve?
Why are we adding four headersto every frame in our data
center?
Speaker 3 (06:21):
So you kind of sort of have to go back to how, in
general, dcs have evolved overtime and the problems that we're
trying to solve for today.
That existed with justgenerally how we design things
in the past.
Past.
What you would have is we wouldhave like a three tiered
(06:44):
architecture as an example,which would be an access layer,
distribution layer or a core anda core, and then occasionally
you would collapse the core andthe distribution and you would
just have like a collapsed coredesign.
Right and predominantly youwould have layer two links
between the access and thedistribution and good old
(07:05):
spanning tree to make surethere's no broadcast storms you
don't have layer two loopsbecause that's bad and then the
distribution and above you wouldtypically have layer three
routing.
So it would be just get me outof the network as quickly as I
can towards the core and thecore takes me out towards the
(07:25):
WAN.
Obviously, span actually had alot of inefficiencies.
You couldn't predict failuresand when something fails it is
very hard to predict how youwould converge.
You had convergence issues andof course you were sort of
always susceptible to a layer toloop, even if you've designed
(07:48):
the network really well.
You have all of your guards inplace.
You know there were some prettyhorrible things that could go
wrong.
That would melt yourinfrastructure and across your
domain, right.
Speaker 2 (08:04):
Yeah, go ahead.
Quick question, sorry so, didvMotion exist yet at this time
and at this point in time thatwe're talking about?
Speaker 3 (08:11):
Yeah, so you obviously had these Layer 2
requirements at that point intime also, which is why Layer 2
was such a big driving factorbehind some of these designs,
right, but the other factor alsowas the network was always
looked at from the perspectiveof these designs, right, but the
other factor also was thenetwork was always looked at
from the perspective of scalingup, which was the other problem.
So you would have like amonolithic architecture from a
(08:35):
server point of view, and whatthat means is, you know, let's
say, I have like a webapplication and it has to talk
to a database, and maybe it haslike a browser engine which
caters to all of your inboundHTTP requests.
Right, now, all of thesedifferent constructs were
(08:57):
typically coupled into a singleserver and you would have that
single server doing everything,right, right?
So any conversation that had tohappen between these
applications to serve theumbrella application was all
within one single physicalhardware that would connect to
your access or your or your leaf, right?
Speaker 2 (09:19):
I call these the good old days good old days Good old
, simpler days.
Yes.
Speaker 3 (09:27):
But the problem that that presented was now, let's
say, you want to scale up andyou want to handle more requests
, right?
And again this is scaling up.
So you start scaling up thatparticular server.
This could be in terms ofwhatever CPU memory, the NIC
(09:48):
speeds themselves.
But that starts to have directimpact to the network
infrastructure it's connected to, because now you have to scale
up your immediately connectednetwork infra as well, so you
have to scale up the speeds onyour network infrastructure side
, and so on.
So again, scaling up was not avery good design.
(10:10):
So eventually the industrystarted moving away from layer
two to layer three.
So it's a fully routedarchitecture between your leaf
so your access and yourdistribution and your core, and
you get a lot of benefits fromthis.
Now you can start to do ECMPacross.
(10:33):
I could have a routing protocolthat sort of distributes all of
my server infrastructureinformation and then I could
start to do ECMP from access upand down to the other access
layer where my other servers areconnected.
Ecmp is obviously morepredictable, routing is more
(10:56):
predictable, convergence is morepredictable, latency is more
predictable.
So you get a host of benefitsnow through this design, right.
And then of course, there wasthis shift from this three-tier
core distribution access towardsa Clow sort of architecture.
(11:17):
So again sorry, like goingreally back, Charles Clow was
kind of this guy who formalizeda way of doing fabrics for
telephony systems where he wouldhave this so for like telephony
systems you would use crossbars, right, and crossbars would
(11:38):
have X number of inputs and Ynumber of outputs and as you
start connecting them you couldget like an input line going to
an output line and somebodytalking to somebody else, right.
So you take the same sort oflogic.
And then what he did was hesaid the way to sort of reduce
complexity but continue to scaleup these telephony systems was
(12:00):
to start building my fabric instages.
So I would have an ingressstage where the call comes in,
and then I would have a middlestage which acts like a fan out
for the egress stage where thecall exits and connects to
somebody on the other side,right.
So they took the same logic andwe eventually applied it for
(12:22):
data center fabrics where youknow you have an ingress stage
which we call as the leaf, youhave the middle stage which we
started calling the spines, andthen we had the egress stage
where traffic exits out, whichwe do call the leaves again.
So it's essentially collapsedinto this leaf spine
(12:44):
architecture, which is whatyou'll hear that terminology
thrown around quite a bit today.
Speaker 2 (12:51):
Thank you for pronouncing his name correctly.
Speaker 3 (12:54):
Yeah, he's French, so Charles Clow.
Speaker 2 (12:58):
Almost everyone says Closs, and I don't know why it
drives me crazy.
Speaker 1 (13:04):
It's one of those things where it's as a native
English speaker, or an.
American probably is more right, like most Americans are going
to look at that and say clothsbecause they see CLOS.
But unless you have any sort oflinguistic knowledge, you know,
being where I am up near Canada, we, like a lot of our our
local languages is driven fromfrom French Right.
(13:26):
So when, I look at it, I waslike, oh okay, all right, I got
it.
Speaker 3 (13:29):
I mean I, I didn't.
I didn't know the currentpronunciation either.
So credit to Russ White, who isprobably the guru in terms of
everything DC.
Speaker 2 (13:40):
So I've just I've learned a ton from him,
including pronunciation of names.
I never knew when I I got tolove Russ.
Thank you for all yourknowledge, Russ.
Speaker 3 (13:52):
When.
Speaker 2 (13:52):
I was working on a Verizon central office.
I worked on a telco frame and20 years later I learned that
this was a Charlie Clow designthat I was running wires on.
Speaker 1 (14:06):
So let's kind of like try to summarize what we've
just talked about here.
So we've got VXLAN, we have ourspines, we have our leaves and
we have our border leaves wheretraffic will ingress, egress out
of the fabric.
We have a lot of flexibility inthat we can scale and we can
scale in a couple of differentways.
If we need additional bandwidthbetween our leaves, we can
(14:27):
scale and we can scale in acouple of different ways.
If we need additional bandwidthbetween our leaves, we can
scale and add spines.
Speaker 3 (14:31):
Correct, we're just scaling out, right, scaling out,
and if we need to addadditional ports, then we can
just add additional leaves,correct exactly so you move away
from scaling up where I gotta,you know, swap out my nicks to a
higher speed nick, or I got toswap out the leaf to something
that supports more line cards ormore speeds, to just scaling
(14:53):
horizontally instead.
Speaker 1 (14:55):
Right.
And then all of theseinterconnects are layer three,
so you have the benefits of nospanning tree.
We're using all of our linksand we're using ECMP rather than
, you know, lacp or somethingsimilar.
Right, you know?
I think a lot of people justassume that you know you're
going to get equal loadbalancing out of LACP and you
(15:15):
don't.
There's a lot of hashing andguessing going on, whereas with
ECMP it is what it is it's equalcost multipathing.
So if you have two spines,you're sending the traffic
equally to both of those spines,if you have three, so on and so
forth.
Speaker 3 (15:31):
Again, I would say that it depends.
It's not as equal as you think,even though it is ECMP, because
most of the hashing that you dotoday is per flow and not per
packet, because you don't want.
If you start doing per flow,then you kind of have to deal
with reordering at the serverside and with TCP.
(15:54):
Obviously that's going to causea host of problems and again,
with UDP it's problematic.
So you do per flow loadbalancing and it's typically
like a five double hash.
So you're taking your sourcedestination MAC, your source
destination IP and, let's say,the port number, the UDP part or
(16:15):
TCP part, as an example, right?
So as long as you have enoughvariability in your flow, you
should get a good sort of loadbalancing mechanism where it's
more or less well done and equalout of all your ECMP bots.
Yeah.
Speaker 1 (16:34):
Gotcha Okay, all right.
So if everything is layer three, what are we doing layer
two-wise?
Are we dedicating entire leavesto particular layer two domains
Like how are we using ournetwork now?
Speaker 3 (16:49):
Yeah, so that's sort of now the fundamental problem
that you run into is hey, Imoved my infrastructure to be a
smart layer three infrastructure, but I still want to live in
the olden days and I want toextend layer two.
And then of course there aresome fundamental requirements of
vMotion and things like that,which does require a layer 2
(17:12):
adjacency.
So how do you solve for that?
Because I obviously stop mybroadcast domain as soon as I
hit the leaf, because everythingabove is now layer 3.
So that's where VXLAN as anencapsulation sort of also helps
.
What you're essentially doingis you're transporting layer two
(17:37):
within this layer threeinfrastructure.
So, through various constructsand various means of
disseminating sort of layer twoinformation across the fabric,
which is where BGP VPN comes inI can start learning layer two
(17:59):
MAC addresses against aparticular VXLAN tunnel endpoint
which is a VTAP, and I can sayhey, if you want to go to this
MAC address, it sits behind thisVTAP, right against this bridge
domain or broadcast domain,which translates to a VXLAN
(18:21):
identifier, a VNI, on the VXLANside.
So then I can start toencapsulate the packet and
provide this Layer 2 extensioneven though you have a fully
routed Layer 3 infrastructureunderneath it.
Speaker 1 (18:37):
And, as you said, that's where the EVP and BGP
comes in, because now you'resharing the MAC addresses over
Layer 3 and you're essentiallymaking routing decisions based
off MAC location, correct?
Speaker 3 (18:50):
So I know, andy, you have a question, so I'll let you
ask and then maybe I'll expandon this a little bit more.
Speaker 2 (18:55):
I have all the questions.
I'm sorry.
Yeah, so when I learned thatEVPN is BGP exchanging MAC
addresses instead of routes, mybrain exploded.
I'm like wait, I didn't knowBGP could advertise MAC
addresses.
I thought it was for routes.
So try to remember where youare, because I have a couple of
(19:16):
questions.
Speaker 3 (19:18):
Can I break your brain?
Speaker 2 (19:19):
a little bit more.
Well, yeah, yeah.
Speaker 3 (19:24):
You know this isn't new right.
So in fact, did you know thatIS-IS does the same thing, like
what Cisco implemented, liketheir version of Trill, which
was FabricPath back in the day,they had IS-IS exchanging Macs
too.
Speaker 2 (19:41):
I did not know that, and thanks for pointing that out
to the audience.
Let me show you something else.
Andy doesn't know.
No, that's fascinating.
Speaker 1 (19:49):
I did not know that myself.
That's good yeah.
Speaker 2 (19:51):
So I'm following the historical evolution and the two
reasons I thought we went fromthree-tier to CLO or Leaf Spine.
At least what I learned in thebooks was you know, 80% used to
be north-south traffic with theold and then, once we went to
microservices and everythingeverywhere in the DC, it
switched to 80%, went east-westand we needed more speed and
(20:14):
less latency and everything'sone or two hops away.
I forget what it is and what isit?
Two hops, everything's two hops, yeah, no matter where it has
to go.
So that traffic shifted right.
And was what behind that?
Was it the microservices?
Yeah, it was virtualized.
Speaker 3 (20:30):
Yeah, so like when the network evolved, the network
evolution was pushed through, Ithink, like server evolution,
Like there was so much ofvirtualization that was
happening.
And now of course, it's allmicroservices, it's all
containerized and all of yourlike constructs of an
application that would sit onone physical server.
They would not exist either asVMs or containers and they would
(20:54):
sit behind different attachmentpoints, like different leafs.
And now when I, when like asingle HTTP request comes in to
service that I actually have totalk more east-west between all
of these application constructsthat sit across the fabric.
So now I'm just within myfabric, going east-west before I
(21:15):
actually go north-south.
So you're right, A lot of itwas driven by.
But that's what drove it A lotof it was driven by the advent
of virtualization andmicroservices and things like
that.
Speaker 2 (21:25):
Yep the other thing I wanted to ask.
I thought that um, evpn, vxland solved the problem with v
motion because you need all thev lands available everywhere.
So I guess the question Iwanted to ask earlier was when
it was three tier, layer two atthe access, how did v motion?
Like how could you move aworkload if a VLAN only existed
(21:45):
in one place?
Like you couldn't move itsomewhere else.
Speaker 3 (21:48):
Does that make sense?
Even in three tier the VLANswould exist in multiple places.
But what would also bedifferent is you would have like
a centralized layers threegateway for all of them, like on
the distribution.
You would typically have all ofthem.
Like on the distribution, youwould typically have all of your
(22:09):
SVIs in Cisco terminology orIRBs and Juniper, nokia
terminology that would existthere and everything below the
distribution was layer two.
So for all your services, thedistribution was your first L3
hop, which was the gateway, andyou would typically have some
form of FHRP, some redundancyprotocol running which is HSRP,
(22:30):
brrp, whatever, right?
So even if you're remulsioningacross, you're doing it across
your layer two adjacent accesslayer, but your layer three
never changes, right?
The problem now in a layer threeinfrastructure is well, I don't
have layer two adjacencyanymore unless I provide it in
(22:53):
some form through this tunnelingmechanism.
And then the other problem is,once I start positioning my
layer three gateways, where do Iactually position them?
Because every leaf is now L3 uptowards the spines, right?
So that's where we also broughtin this concept of like an any
(23:14):
cost distributed gateway, right.
So I would have the IRBs withthe same IP, same MAC that now
exist across all your leafs.
So I don't really care if Imove from leaf one to leaf 10,
because when I move I retain mydefault gateway, like I'm not
(23:34):
changing that, which is a coreconstruct that is necessary.
And as long as that defaultgateway exists on the leaf that
I have now moved under, from mypoint of view there's really no
change, right?
The resolution of the IP, ofthe gateway to the Mac, is going
to be exactly the same.
Speaker 2 (23:53):
Okay, last question on my torrent of questions, and
I hand it back to AJ.
So you said something about, sonow we've gone.
The end of the conversationearlier was so we've completely
gone to layer three.
Infrastructure, right,everything's layer three, but we
have to maintain layer two andI guess that's what the XLAN is
supporting.
Is that maintenance of layertwo down at, I guess, the
servers?
So that we can transport itacross.
(24:15):
Why I guess there's no othersolution, like we can't go layer
three from the leaf down theserver and get rid of the layer
two.
No, you absolutely can.
Speaker 3 (24:26):
Yeah, you can.
So a lot of people actuallybuild, especially when you look
at, let's say, a lot of cloudservices such as, like an
Anycast DNS as an example A lotof people build that
infrastructure with L3 all theway down to the servers.
Right, Because what theseservers do and you have multiple
(24:50):
nodes that are advertising thesame anycast address, as an
example right, and by doing L3all the way down, you're able to
achieve ECMP to all of theseguys.
Right, so it doesn't all theseleafs will advertise the same
Anycast address, so I don't carewhich leaf I eventually come in
on, that leaf will eventuallythen ECMP down to any of the
(25:15):
nodes that support this AnycastDNS, these Anycast DNS services.
Right, and that's a network,like a non-hardware way of doing
load balancing for Anycast DNSservices, Because otherwise what
you would typically do is youwould have to purchase like a
very expensive hardware loadbalancer and put it in the
(25:37):
middle and then put all of yourDNS servers behind it and then
the load balancer does all ofthat, right, so this is a
different way, a protocol-onlyway of actually doing any cast
DNS services as well.
So you do, and especially for,like cloud services, Kubernetes,
clusters, CNIs like Calico asan example.
(26:00):
A lot of these are nowdelivered through L3 to the
leaves.
Speaker 2 (26:07):
I guess it's dependent on use case and the
problem you're solving.
If you were building aGreenfield, data center.
You wouldn't just go, youwouldn't completely get rid of
L2, because that's the bestpractice.
Now it just it depends on whatyou're doing, I guess.
Speaker 3 (26:18):
It depends on what your application needs Like.
Sometimes these applicationsthey need like layer two
keep-alives between them as anorganization.
If you bought into thatapplication, what do you do?
You have to facilitate a way oftransporting that, even if you
(26:38):
want to go towards more modernways of building your DCs.
So, once again, it's theapplication's fault.
More modern ways of buildingyour DCs, right?
So yeah?
Speaker 2 (26:45):
So once again, it's the application's fault.
Speaker 3 (26:48):
It is.
I mean, that's the funny thing.
Right, we are building all ofthis network infrastructure to
solve and service your businessuse cases and what outcome you
want, so it is driven by theapplication.
100%.
Yeah, it's a lot.
I can see the brain spinning alittle bit, andy.
Speaker 2 (27:16):
Why is it so hard?
Like am I not you and I, so youknow?
Full transparency to theaudience.
I mean, you spent three days ofyour life with me teaching me
VPN, vxlan, and we labbed it andI videoed it.
And I keep telling myself I'mgoing to go back to those videos
, build it at home and really,until I build it, I don't really
think this stuff's going to bein my bones, but it just.
(27:36):
It seems so easy to you and forthe rest of us.
Why is it so complicated?
Speaker 3 (27:41):
is it not?
Really?
I don't think it's easy.
I think it's just it'sconsistency, it's, um, having a
day job that requires me to dothis every day, um, like, it's
the same thing with automation,right?
Um, you've tried, you'vestarted, stopped, I think.
Several times before I havedone the same thing and I suck
(28:03):
at it, like, really, really like, I'm terrible at it, right, but
I think, if that would becomepart of my day job and you have
to do it every day, I think thatthat changes the dynamics a
little bit, right?
You're forced to learn.
You're forced to encountercustomers who are doing this.
You're forced to, like, look atdesigns.
(28:24):
You're forced to encountercustomers who are doing this.
You're forced to look atdesigns and make sense of it,
and it just starts to sit inyour brain a little bit.
Speaker 2 (28:31):
So when you build an EVPN VXLAN fabric, let's say in
your lab, are you doing itartisanally by hand?
You're not automating it.
Speaker 3 (28:38):
No, so it depends, right.
So I made sure.
So I did a lot of DC when I wasat.
So I made sure.
So I've done a lot of.
I did a lot of DC when I was atCisco because I was supporting I
was like an escalation engineerfrom the engineering side of
things, but I was heavilyinvolved on the platform side,
right, so we wouldn't getinvolved in like initial
(28:58):
troubleshooting.
A lot of stuff that would cometo us would be you know, the
ASIC is dropping the packet hereand why is that happening?
On the Nexus side of things,since it was, nexus was
obviously heavily deployed onthe DC side.
So I would cater to a lot of DCdesigns and DC infrastructure
sided VXLAN there, but not asmuch as I've done it in the last
(29:19):
three years when I was atJuniper, which is where I really
got involved with customersdeployments and I was in the
thick of things.
So I made sure that I waslabbing, like I was building a
simple topology, let's say, twospines, four to five leafs and
some services, l2 and L3, right,I was doing it by hand, at
(29:41):
least like three or four times aweek like break it, build it
all over again, and I did it byhand for the entirety of my time
at Juniper and then again atNokia, because the CLI is
obviously different.
Unless I wanted to spin upsomething very, very quickly,
then I could either use JuniperApp Store or I'm using, like
(30:02):
Nokia, eda as an example.
Now, right, but I made sure todo it by hand, because that
forces you to make a lot ofmistakes, forces you to
troubleshoot when you haven'tconfigured things properly, and
it forces you to learn,obviously.
Speaker 2 (30:18):
And it's a lot of config.
That's why I asked right, Likeit's dense, it's yeah.
Speaker 1 (30:22):
It is, it is.
Speaker 2 (30:24):
This isn't a 15 line stanza, like you don't?
You don't enable the vxlan andwalk away no, and that's exactly
.
Speaker 3 (30:30):
There's so many moving parts and it's so
difficult to understand.
Um, you know why you needcertain things, and only when
you start to configure it, andthen you don't configure
something you need you, it hitsyou in the head really hard and
you're like, oh shit, I reallyneeded this for.
Uh, so and so reason right, um,and then you do it again 200
times and then maybe it sits inyour head for for some time.
Speaker 2 (30:51):
So yeah, hey, jan, like you've built fabrics right,
or or like, I mean, are peoplebuilding this by hand?
Is this something that mostfolks rely on, like like an ACI
or a fabric automation tool?
Speaker 1 (31:06):
You know, I don't know how many people are
building it by hand manually fora production environment.
I think a lot of the VXLANimplementation that I've
personally seen has usually beentied with some sort of product
package, right?
So Cisco's ACI, and thenthere's a bunch of other out
there that use, you know, vxlanunder their hood.
(31:27):
When I initially startedlearning it, even though I was
going to do it for ACI, I tookthe time to learn it manually so
I could understand how itworked.
I could troubleshoot it to adegree and have the appreciation
of, you know, what is what isACI doing for me as a network
operator, of what is ACI doingfor me as a network operator.
But I haven't seen many manualimplementations of it throughout
(31:51):
the course of my career.
Now, that being said, I haven'tspent a ton of time with it.
So just before I left my lastjob as a pro services engineer,
I was actually working on amanual implementation of VXLAN,
because it was just between twodata centers and they were two
very small data centers too, sobut they needed to stretch layer
two between the two, and thisis really the best way to do it.
Speaker 2 (32:11):
And is it a safe assumption that writing it by
hand is definitely the way tohammer what's happening in your
head, Like so you understand howthis is all playing together?
Yeah, absolutely.
Speaker 1 (32:21):
You know cause, as Aninda said, you said you're
going to make mistakes andyou're going to have to
troubleshoot it, and then, onceyou fix and understand how this
stuff goes together, you're justgoing form of orchestrator or
some form of automationframework Ansible, whatever
(32:48):
right, and more than that.
Speaker 3 (32:49):
You want, like the ROI, when you're doing it
through these orchestrators isyou want some guarantee of how
the network's actuallyfunctioning.
So you want to look at like daytwo stuff, like all of the op
stuff right, is all my peeringsup?
Is the fabric deployed asintended, or has somebody gone
(33:12):
in and changed some stuff and myfabric's no longer adhering to
what it should be and thingslike that.
So that's where a lot of thebenefits come in.
The automation problem has beensolved for a long time now.
But the day two stuff is that'sa place where things are really
happening and for DC, of course, for DC that's very, it's very
(33:34):
important and critical that youknow what's actually happening
in your infrastructure.
Speaker 2 (33:39):
And this is creeping into enterprise right.
It's not just data center.
Speaker 3 (33:42):
Yeah, I mean, and this is creeping into enterprise
, right?
Speaker 2 (33:44):
It's not just data center.
Yeah, I mean yeah of courseyeah.
Speaker 1 (33:47):
Of course yeah, because I think one of the
things that it can do for you isnow you're not tying an IP
address to a specific location,right?
So now you don't have to worryabout writing the same ACLs over
and over and over again for allyour different networks.
You can now stretch thatnetwork and put it wherever you
want, and now you have like aunified set of rules that you
don't have to necessarily worryabout.
Speaker 3 (34:08):
Yeah, especially for enterprise use cases where
there's so much of wireless andthere's so much of roaming, like
Cisco obviously pushed as theaccess, which was originally
Lisp plus VXLAN, so Lisp wasdoing all of the control plane
stuff and then VXLAN, lisp wasdoing all of the control plane
stuff and then VXLAN was yourdata plane.
But I think from an adoptionperspective Lisp was much harder
(34:32):
because nobody really used itas much.
They also moved to BGP VPN andI think you could do both.
I haven't done anything on theSDS space in like three years,
four years or more, but I'mpretty sure you could do BGP
EVPN with VXN today on theircampus side, also through DNAC
(34:53):
or Catalyst Center, whateverthey call it today.
Speaker 2 (34:56):
Is there anything to say about EVPN?
Besides, it's the control planethat carries MAC addresses.
Is there anything there?
I mean yeah.
Speaker 3 (35:05):
So again, like, a lot of this is through evolution.
Like BGP, evpn was not howVXLAN was deployed originally,
right?
So you actually had VXLAN floodand learn, which is very
similar to how, like, if you'vedone VPLS in the past, you're
(35:28):
sort of flooding a data planepacket, like an ARP as an
example, through your fabric andthen that enables all of your
other you know, terminatingendpoints to learn the source
packages, right, but there are alot of consequences of doing
that.
Obviously, flooding is not thatgreat and then the support
flooding.
(35:49):
You sort of need some form ofinfrastructure that facilitates
that flooding, right, andtypically this would be like a
multicast underlay that youwould build through PIM sparse
mode as an example, right, andthis multicast underlay that you
would build through PIMS parsemode as an example, right, and
this multicast underlay wouldflood all of your data plane
(36:10):
packets and then you would havedata plane driven learning,
right.
A lot of drawbacks, especiallyaround layer 2, jsmc when things
move, there's a lot of delayand convergence issues and
things like that.
So, again, industry evolved tofigure out that you could
actually use BGP through a newaddress family and a sub-address
(36:35):
family, which was L2, vpn, evpn, in short, bgp EVPN.
And now you could use BGP EVPNas a control plane to sort of
disseminate all of your learningacross these VXLAN tunnel
endpoints or the leafs or theVTEPs, right?
So as soon as I have a dataplane, learn on my leaf, let's
(36:57):
say a GARP comes in or an ARPcomes in, or an ARP comes in,
I'll process that in whateverway I have to process that ARP.
But I'll also go and tell BGPEVPN that hey, I had a local
learn and this is the MACaddress on this port, on this
bridge domain, which maps tothis VXLAN identifier.
(37:19):
So why don't you go and tellthe rest of the fabric where
this MAC address exists, right?
So BGP VPN takes that, it buildslike a BGP update and sends it
to everybody else.
And now everybody elseprocesses that update and it
knows, okay, mac A residesbehind leaf one right.
And if I want to get to MAC A,I can end, cap the packet with
(37:44):
all of these headers and send itto leaf one right.
And then the only thing I'lladd there is it's more than just
Mac addresses, andy.
So eventually they alsorealized that there are layer
three, very clear layer threeuse cases also, like I want to
transport just IP addresses aswell.
So you have a host of EVPNroutes that do different things.
(38:09):
Some of them do Mac only, thesame thing does Mac plus IP, and
some of them do like just an ipprefix as an example.
So I'm just decoupling macaddresses from ip addresses are
there route types or am Iconfusing protocols?
yeah, so they're.
They're bgp, evp and routetypes um one through.
I want to say 11 or 12.
(38:31):
No, no, it's yeah.
Speaker 2 (38:33):
So yeah, type five is stuck in my head for some
reason.
Speaker 3 (38:38):
Yeah, that's the IP prefix now.
So that's the one that justdoes away from MAC addresses and
you're only advertising, youknow, like a V4, v6.
That's correct.
You passed this quiz.
Good job, yay, do I get the jobor not?
You?
Speaker 2 (38:52):
got the job man.
Speaker 3 (38:56):
Yeah, that's a mouthful.
So yeah, if you have anyquestions, let's.
Let's talk about it.
Speaker 2 (39:03):
Stunned silence well, I I have a completely unrelated
question what the hell is afive tuple hash?
I'm gonna get us off of thisevp and vx for a second.
You said five tuple hash andI'm like question mark
exclamation.
I don't even know what a tupleis.
I've heard people talk abouttuples and well, there's been a
lot of information here.
I didn't want to stop the trainfor that dumb question, but
(39:23):
here we are.
There was a break.
Speaker 3 (39:26):
I mean you have to have like an algorithmic way of
determining if you have equalcost bots to send traffic out.
And this is either like layerthree, ecmp, or let's say you
have like a like a port channelright and and you have multiple
links in the port channel.
Now you can't, you can't sendtraffic out every link because
(39:47):
you can't guarantee the order inwhich the packets go out.
And I could, I could send,let's say, packet two before I
can receive packet two beforepacket one, which causes a host
of problems.
So typically you would do perflow, right.
And to do per flow you need tohave some algorithmic way of
determining which flow is put onwhich link.
(40:10):
If I have, let's say, two linksto send traffic out, right,
links to send traffic out right.
So that algorithm essentiallyuses different variables which
you, let's say, you could callthem tuples here, and these
variables include the source.
Typically by default they wouldinclude your source MAC, your
(40:31):
destination MAC from theinternet header, your source IP,
destination IP from the IPheader if this is an IP packet,
and, let's say, your destinationport, as an example from a UDP
header or a TCP header.
So these five things form yourfive tuples that go into the
algorithm and then the algorithmjust spits out like a hash that
(40:54):
says, hey, you take this linkand you go out of this link,
right?
Speaker 2 (40:59):
So do I understand that each of the flows are
pinned to a link?
Does that sound accurate?
Speaker 3 (41:05):
Yes, unless of course the number of available links
change and you sort of have torecompute how many links are
still available and then assign,like a bucket, to each of these
links.
Yeah, so it's typically.
Speaker 2 (41:21):
And is it pinning flows to a link to avoid the
out-of-order packets that youmentioned earlier?
Is that why?
Speaker 3 (41:29):
It's just a consequence of the algorithm,
right?
If there's no variability inthe algorithm, the outcome will
always be the same, which is avery interesting question when
it comes to VXLAN, because?
So here's a problem statement,right?
So once I do the VXLAN end cap,I add, like an outer Ethernet
(41:50):
header, I add an outer IP header, and then I add a UDP header
and then the VXLAN header.
Like these are the four headersthat I'm adding right Now.
The goal of my outer Ethernetand IP headers is purely for
transport, right?
I want to take the originalpayload from one VXLAN tunnel
(42:13):
endpoint to the other, right,and these endpoints are
identified by, you know, theirloopback addresses, as an
example.
So how do I do that?
I add an IP header that sayshey, the source IP will be
VXLand tunnel, endpoint one,destination IP is going to be
VXLand tunnel, endpoint two, andthen my Ethernet header.
(42:35):
Basically, hop, hop by hop, Itake it from my leaf spine down
to the other leaf, right?
But the problem that that nowpresents is, let's say, I have
10 different flows, um, but theyhave to go between the two
vtabs, right?
So the outer ip header and theouter ethernet headers that I
now add will always be the same,even though my original flows
(43:01):
or the original payload isactually changing and there is
enough variability, right.
So now, from a load balancingperspective, that's a big
problem, like, how would I solveit?
I would always start pinning itagainst a particular link,
going out towards my spines,even though I have multiple ECMP
links available, right.
So do you want to like take aguess at how do we solve this,
(43:26):
or do you want me to answer no,okay, Well, my brain is probably
full, so I'm going to let youlet's have you answer this.
Speaker 2 (43:40):
I have one other question and then.
Speaker 1 (43:42):
I'd like to maybe pivot to your authoring?
Speaker 2 (43:44):
uh, okay, sure, honestly, my brain this is, but
this is what it's, no reflectionon you or the technology.
This is what happens to me whenI try to talk about as an
example I get about 20 to 30minutes.
It happened when you wereworking with me, remember,
trying to teach me.
We get like about this time inand you would ask me a question
and I would just stare.
I think my brain is full.
(44:04):
I think you've reached thepoint that my you've reached the
perimeter of my intelligence.
Yeah.
Speaker 3 (44:13):
So just I put this problem statement because you
asked a really importantquestion, which is how are we
deciding where to position thisflow?
And that presents a problem whenyou're doing NCAP like this,
because you abstract away thevariability of the original
flows and you're addingsomething on top which is almost
(44:36):
always the same if the servicessit behind the same two leaves.
You need to have some mechanismthat sort of adds that
variability right.
So that's why the UDP headerexists before the VXLAN header.
So obviously it identifies thatwhat comes next is a VXLAN
header.
But what we do now is we usethe original payload and we do
(45:02):
like a five-tuple hash again onthe original payload to generate
the UDP destination port.
So as long as the originalpayload varies, the UDP port
starts to vary right.
And then to load balance thisVXLine encapsulated packet, I'll
use my outer Ethernet headerstuff, my outer IP header stuff
(45:25):
and then the UDP destinationport which is now changing.
So I'm able to introduce thisconcept of variability which
allows me to now load balanceeven encapsulated packets better
across these equal cost parts.
Speaker 2 (45:40):
AJ, I wanted to be a TME until I met Aninda and then
I realized I might not be assmart as I thought I was.
Speaker 3 (45:48):
Hey, don't say that no it's a compliment, man.
Speaker 2 (45:53):
I'm just always blown away by your depth of knowledge
.
I'll say this like I don't.
Speaker 3 (45:57):
I honestly don't think I'm smart.
Like there are like people whoget this like I would.
I would take like six months toget this, and there are people
who would get this in like acouple of weeks.
What I feel I do a littledifferently is, um, like I lab
up so much of this day in andday out, like that's that's the
only thing that has really madea difference for me.
Um, it's, it's just in mymemory because I've done it so
(46:20):
much, right?
Uh, so it's not that I'm smart,it was just like a crazy amount
of hard work and a crazy amountof like just in the lab, um,
looking at broken shit.
Uh, day in and day out yeah,you've put in the reps.
Speaker 2 (46:33):
Um.
The last question before wepivot away into your life as an
author I wanted to ask was sothese are standards correct?
Evpn, vxlan.
Speaker 3 (46:42):
Yeah, some of them are like RFCs, some of them are
IETF jobs.
But yes, they're sort ofstandardized Correct.
Speaker 2 (46:50):
But so I've heard people say that different
vendors implement themdifferently.
So I guess is that accurate?
And then, if it is, how dovendors implement them
differently?
So I guess, is that accurate?
And then, if it is, how dovendors implement standards
differently?
Maybe they don't know what thestandard means because no, there
are.
Speaker 3 (47:06):
So, yeah, there is.
Sometimes the verbiage in thedraft or the rfc is a little
vague and it comes.
It's up to the vendor and itdepends on how the vendor
chooses to implement this insoftware.
So that's where some of thedifferences come in.
And then just there, like, forexample I don't want to get into
(47:28):
this, but like data center,interconnects are obviously a
big deal, because when you'rebuilding DCs you're probably
building multiple DCs.
Typically simplest form wouldbe like a DRDC as well, like a
primary DC and a DRDC, and youwant to connect the two and have
them talk to each other.
So you need some form ofinterconnect.
(47:51):
Now, as an example, there is anRFC that exists that defines how
to do a DCI, which is called anintegrated interconnect.
But then Cisco decided thatthey could do, they wanted to do
things differently, perhaps, intheir eyes, more efficiently.
So they put in another draftfor doing DCI, which now Cisco
(48:14):
uses, but a lot of the othervendors in the industry use the
RFC that already exists.
So now you're in a positionwhere if I have mixed vendors
and I have Cisco on one side and, let's say, juniper and Aris on
the other side and I want to doDCI.
Well, they're doing twodifferent forms of DCI.
(48:34):
Now, to Cisco's credit, theyalso figured out that
interoperability is a big deal.
So they do have some knobswhere you could sort of make it
compatible with other vendors,but you see where the complexity
starts to creep in, right.
So yeah, it is a problem.
Speaker 2 (48:56):
So what's it like writing a book?
Why did you write a book?
What, what, what is your book?
I mean, you're a family guy.
I've worked with you.
You're a very hard worker.
You put in a lot of time.
You have a wife, you have ayoung daughter.
I can't imagine how much timeit took you so like, why would
you, why did you write your book?
Speaker 3 (49:13):
I guess the book stuff has been on my mind for,
uh, for a long time, like when Iwas like I've been a tech
blogger for um for quite a whilenow, um, and it was off and on.
And then I really heavily gotinto it when I was a tme at
cisco doing all of their sdastuff, and I realized like
there's such a big gap, um, uh,in in just pointing and clicking
(49:39):
on the UI and thenunderstanding what's actually
happening behind the hood.
Because it's cool to do all theUI stuff and you can probably
figure that out and have thefabric maybe in 30 minutes to an
hour, but when things break, Idon't think the UI is really
going to solve it for you, maybeto a small extent, but you
(50:01):
really need to understand howall of the stuff under the hood
works, right?
So I started taking all of thatand writing very tech-heavy
blogs.
The marketing part of a TME isnon-existent, it's all tech.
Like we weren't, we weren'treally doing all the fluffy
(50:21):
marketing stuff, right?
And I haven't really done thatever.
I don't particularly enjoy iteither, right?
So I was just doing all of thetech stuff and I realized that,
okay, there's, there's anaudience for it and I really
like doing it anyway.
So I actually tried to do theSDA book for some time, but that
really didn't gather steam andI couldn't really see like a way
(50:46):
in, because Cisco was just sohumongous and so big.
I was just a very tiny part ofit.
Speaker 2 (50:51):
But to approach your manager and say I'd like to
write a book.
I'm sorry to interrupt you, buthow do you even?
Who do you talk?
To in Cisco to write a book.
I'm sorry to interrupt you.
Like how do?
Speaker 3 (50:58):
you?
How do you?
Even who do you talk to?
It's just gonna write a book.
Yeah, we've.
I've tried that a few times.
I just like I just couldn't seea way in, right, yeah, um, and
then I had sort of given up, orI was actually thinking about
like just self-publishing somesda stuff at that point, um, but
I didn't.
I didn't know if, like, I saw alot of value in it at that
point in time.
Then I moved into Juniper Againa lot of heavy hitting DC stuff
(51:20):
, and again I realized, man,this stuff is hard.
So I went back to a lot of techblogging again.
And then Mike Puchong, who wasmy big boss at that point and
who is my big boss again, andthen Kathy Gadecki, who was
under him.
And then I had another managerat Juniper, rida Hameedi.
(51:48):
Exceptional people, verysupportive of things you want to
do outside of just your day job.
So I then pitched the idea ofthis Juniper book, focused just
on Dana centers, because nothingexisted from a Juniper
perspective at that time.
So I knew there was a gap thatwe could fill.
So Kathy got me in touch withRuss and Russ was working at
(52:08):
Juniper at that point.
So I was very lucky.
I had some overlap with him forabout a year and then he
introduced me to this gentlemanat Pearson called Brett, and
Russ works with Brett quiteheavily.
So Brett was really interested.
Originally this was actuallysupposed to be just video
(52:28):
content and I wasn't too keen ondoing like a tutorial or like
on demand stuff.
At that point I said I think myforte is writing.
I think that's what I'm morepassionate about.
So I pitched an entire outlinefor the book and they were like
really happy to take it up.
Um, and yeah, there was nolooking back after that.
(52:48):
Um took me about a year andthree months, uh, in total, from
starting to write to publishing.
I was hard man, I was workingnights, I was working every
weekend and again, I don't thinkthere's a way to balance it.
There's sacrifice.
There's a lot of family timethat is sacrificed, for sure,
(53:17):
and then of course, you're doingyour day job, but then you sort
of have to write the book aswell.
Um, I was okay with thatbecause I felt a lot of my day
job was contributing back towriting the book because it was
so heavily just DC specific.
So I actually told all of mybosses that don't take me out of
my day job because I feel thatmakes the book more realistic,
(53:39):
like I'm not writing in a bubble, I know what's actually
happening outside and I want tomake it relatable.
So I continued a hundredpercent on the day job and then
150% on the on the book and Ilost a lot of like family stuff.
Speaker 2 (53:54):
But um, yeah, you make up, I guess you had full
support of your wife right, likea lot of like family stuff.
Speaker 3 (53:56):
But um, yeah, you make up.
Speaker 2 (53:57):
I guess you had full support of your wife right like
of course there's a lot on theshow how important it is right
to have the support of your,your support system, to be able
to do something like that 100like it's.
Speaker 3 (54:07):
It can be very frustrating because, like
there's times where you'retrying to build labs for the
book and like it's just itdoesn't work and stuff's
breaking, which sometimes youthink it's a bug, sometimes it's
a mistake, and you spend daystroubleshooting it's that, like
a lot of that comes out, um, asfrustration when you're like
with with your family away fromlike the book stuff also.
Speaker 2 (54:29):
So, yeah, my life is like a pillar of support, um,
and I would either like thiswouldn't exist without, without,
for sure and I think I heardyou correctly the reason that
you wrote the book is you wantedpeople to understand what was
happening under their hood, forwhen things broke right like
here's how it's all working, sowhen it breaks, you'll have the
tools you need to know where tolook.
(54:49):
Was that why you wanted towrite this book?
Speaker 3 (54:51):
that is my general um .
Uh yeah, the general ideabehind why I do a lot of the
tech blogging is is that, yeah,I like even a lot of the vendor
stuff, like even juniper, evennokia, right, um, it's not easy
to figure out what's happeningbehind the hood, and sometimes
the vendors don't make it easy,right, um, and I feel like it's
(55:14):
up to us as the community tosort of help each other out,
especially for people who aremanaging this day in, day out,
who are in the upside of things,because I have been on that
side right when hospitals godown, when people call in and
say they're losing millions ofdollars per second and shit hits
(55:37):
the fan.
Like you want the knowledge tobe able to troubleshoot some of
these things.
So yeah, it's just, it's ahelping hand, it's a way for the
community to understand this ishow stuff works behind the
scene.
If, if you were able to put inthe time to lab it up and lead
it up.
So, yeah, I just I loved givingback.
(55:59):
And I have another book that I'mwriting for Juniper, because I
had, like I had committed to it.
And then I have a third bookthat we just signed a contract
for, which is interesting.
It's a it's a multi vendor DCbook that's going to cover Cisco
, adista, juniper and and Nokia,so maybe like 2000,000 pages of
a big.
But yeah, I have a lot of funwith it.
Speaker 2 (56:20):
And it's so now you're on fire.
You've gotten started andyou're on fire.
I think that it's.
I just want to put a.
You know, I think it'simportant to have people on
staff that have the deepknowledge for when things break
and the reason I think it'simportant.
I don't know if you guys haveever had this perception from,
like, vendor marketing and I'mnot pointing a finger in any
(56:40):
particular direction butthere'll be vendors who say that
their automation platform is sogood that it you know things
won't break, right.
Oh well, if you, you know, ifyou use this, you know you won't
break it.
I feel like that's a bigmarketing angle.
That and listen, it's, it's.
It's usually better than doingit artisanally by hand in the
CLI.
So I get it.
But things break.
(57:01):
You hit bugs, things get wedged, weird stuff happens, right,
and and if you were like you hadsaid something like that
earlier if you're relying on theUI and you know this tool to
manage it, fine, you know yougot to change connectivity or
add or delete.
But man, when stuff breaks,like you better have Aninda
around, because I don't know ifthat automation platform is
(57:21):
going to do that for you, right,like, even if there's a
rollback.
It just seems to me that youreally need somebody around who
knows what the hell's happeningunder the hood.
Speaker 1 (57:30):
Yeah, that makes a big difference, for sure.
Speaker 2 (57:32):
If you want to, you know, get your network back up.
Aninda, this has been amazingand and and I'm sorry that I had
a dumb look half of theconversation, because it's just
so much, but it's just so muchto take in.
Speaker 1 (57:46):
There's so many layers of complexity, which is
why I thought it'd be helpful tohave this conversation and and
trying to suss out like what'sgoing to be the most important
stuff to cram into an hour isdifficult like vxlan, yeah, yeah
where would you direct peoplelike?
Speaker 2 (58:01):
so I mean, I'm, you know, I'm halfway through
studying for the ccmp andthere's a vxlan chapter in that
book, right?
So I guess vendors havematerial people can consume to
learn this stuff.
But some of them are coming toyou saying, hey, man, you know
this overlay, it's all the rage.
You can be excellent.
Like where would you directpeople who want to get started
learning about everything we'retalking about here?
Speaker 3 (58:22):
so I learned a majority of what I wrote in the
book um from dinesh dutt, who isagain pioneer um in this space,
from, uh, russ white um, jeffdoyle, jeff Russ White, jeff
Doyle, jeff Tansura.
So a lot of them have eitherbooks content or they just have,
like Jeff and Jeff do BetweenTwo Nerds.
(58:44):
They have a lot of good contenton EVP and VXLAN.
They have a lot of interestingguests that come in and talk
about this.
Russ has a huge amount of freestuff out there Rule11.tech,
rule11.academy, which is his newacademy for teaching.
(59:05):
And then Dinesh Dutt has twoexcellent books.
I think they're called BGP inthe data center and then EVPN in
the data center and then cloudnative networking, I think
center, and then evpn in thedata center and then cloud
native networking, I think um.
So I was, yeah, I was just Iwas watching their stuff, uh,
reading their stuff uh, day in,day out.
When I joined juniper, when Iwas writing the book, um, yeah,
(59:27):
so go, go check them out um,they are the original sources of
truth for any of this um.
And then, yeah, maybe, maybecheck out the book I wrote,
because I hope I sort of took alot of what they were teaching
me and then hopefully translatedthat into easily consumable
information, with someimplementation information
(59:49):
around Juniper.
Speaker 2 (59:51):
What's the name of your book and how can people
find it?
Speaker 3 (59:54):
So the book is called Deploying Juniper Dinner
Centers with EVP and VXLan.
You could either get it onAmazon or you could buy it
straight from Pearson, either asa hard copy or PDF.
Speaker 1 (01:00:07):
Awesome, and where can people find more from you?
Speaker 3 (01:00:12):
I am on LinkedIn, I am on Twitter for now, and then
Blue Sky recently, and at all ofthese places it's Anandchat.
So A-N-I-N-C-H-A-T.
Speaker 1 (01:00:26):
And you said you're a tech blogger.
Were you blogging for yourself?
Do you have a website that youstill blog at?
Speaker 3 (01:00:34):
Yeah, I blog for myself.
It's not very consistentbecause there's so much of work
stuff going on and then a lot ofthe stuff I want to blog about
goes into the book, but I dohave a lot of good content there
.
It is calledtheaskyconstructcom.
Speaker 1 (01:00:49):
Awesome.
We will put links to all ofthat great stuff in the show
notes and you can check out hisblog, check out the books and
other resources that hementioned, and then to thank you
so much for joining us tonight.
This has been a thank you forhaving me.
It was awesome.
Thanks guys.
Yeah, I feel like there's a lothere we still could have
unpacked, so we might have tohave you back for, uh, another.
Speaker 3 (01:01:08):
Another follow-up episode yeah, I would love that.
It's always good to talk to youguys.
Speaker 1 (01:01:13):
Awesome, all right we will see you next time on
another episode of the Art ofNetwork Engineering podcast.
Thanks for joining us.
Speaker 2 (01:01:28):
Hey everyone, this is Andy.
If you like what you heardtoday, then please subscribe to
our podcast and your favoritepodcatcher.
Click that bell icon to getnotified of all of our future
episodes.
Also follow us on Twitter andInstagram.
We are at Art of Net Eng,that's Art of N-E-T-E-N-G.
You can also find us on the webat artofnetworkengineeringcom,
(01:01:48):
where we post all of our shownotes, blog articles and general
networking nerdery.
You can also see our prettyfaces on our YouTube channel
named the Art of NetworkEngineering.
Thanks for listening.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Intentionally Disturbing
Join me on this podcast as I navigate the murky waters of human behavior, current events, and personal anecdotes through in-depth interviews with incredible people—all served with a generous helping of sarcasm and satire. After years as a forensic and clinical psychologist, I offer a unique interview style and a low tolerance for bullshit, quickly steering conversations toward depth and darkness. I honor the seriousness while also appreciating wit. I’m your guide through the twisted labyrinth of the human psyche, armed with dark humor and biting wit.