Tailscale Magic

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:00):
Hi guys, and welcome to another episode of the Uncast
show where we explore the worldof unread servers, self-hosting
and the tools that make it allpossible.
Today, I'm really excitedbecause we've got a special
guest who many of you willalready know.
Today on the show, we've gotAlex Kretschmar.
He's the co-host of theself-hosted podcast and the

(00:21):
leading voice in the open sourceworld and the head developer of
relations at Tailscale, so it'sgoing to be an interesting chat
.
First off, alex, thanks so muchfor joining us today.
In fact, I've been a longtimelistener of the self-hosted
podcast, so it's great tofinally have you on the show.
Well, that's pretty cool.
Hey, ed, how are you doing?
I'm very good.

(00:41):
Thank you, in fact, actually,we did did meet, didn't we?
Um alex in london at the umlime technology off-site meetup.
But yeah, we did.
It was in a bar with tabletennis and a lot of music, so it
was very loud.

Speaker 2 (00:56):
My overriding memory of that place is it was dark, it
was loud and robbie from nascompares was being full Essex on
us.
And it was a fun time though.

Speaker 1 (01:10):
It was Well.
We could barely hear each otherthere and at the moment it's
raining here at the moment, soI'm not sure if you can hear it
on my office roof, but hopefullywe'll have a much clearer
conversation, hopefully, yeah,anyway, it's great to have a
Brit on the show, um, someonewho understands that summer here
is just two weeks of slightlywarmer rain, but you live in

(01:30):
United States now, alex, yes,yeah, I do.

Speaker 2 (01:34):
I'm from Basingstoke, originally Amazingstoke as we
call it.
But yeah, the journey of lifehas led me to Raleigh, north
Carolina, these days.
Nice, the journey of life hasled me to Raleigh, north
Carolina, these days.

Speaker 1 (01:45):
Nice.
So you know, like I said, I'vebeen listening to your podcast
for quite a while.
I haven't been using Tailscalefor that long.
Actually, it was Brent Gerviswho convinced me to try it out.
Really, I met him in Berlinlast year when I went to visit
the NextCloud offices, and healso took me to a place called

(02:08):
Seabase.
I'm not sure if you've beenthere.

Speaker 2 (02:11):
Well, no, but Brent is possibly my best bud and he
and I talk about Seabase all thetime.
It's like a crashed spaceshipor something in there, like an
old bus.
Is that the place?

Speaker 1 (02:24):
It's absolutely.
It's insane.
I've never seen anything likeit for everyone watching.
Okay, c base.
Basically it's a ufo themedhacker space styled crash space
station, so it's basically likea sci-fi dream come to life for
me.
Um, they host like meetups,collaborations for kind of

(02:45):
techie guys.
It's one of the oldest and mosticonic hacker spaces in the
world, um, and it's entirely runby its members.
You can go and use kind of 3dprinters, um, electronic stuff,
things to fix your bike, and oneof my favorite bits is they had
a bar there, so that was um Ihear they have a weekly nixos
meetup as well.
So if they do, yep, and actuallybrent tried to get me to

(03:10):
install nixos and I did actuallyinstall it whilst I was waiting
for a delayed flight in theairport, but then I never
actually I never actuallycontinued when I got home, I
know I installed it as a vm like, logged in to the unraid server
, installing it there, but Ithink it's still on the unraid
server, but I haven't haven'tactually started it up, okay.

(03:33):
So, like I said earlier, um alexis in to do with um tail scale.
So on my youtube channelrecently I've put out a few
videos showing how to use tailscale with unraid, demonstrating
how to easily secure it andconnect back to the server,
access subnets and manage remoteaccess.

Speaker 2 (03:51):
Those videos have been great, by the way.

Speaker 1 (03:53):
Oh, thank you.

Speaker 2 (03:54):
We've been really, really excited.
There's been no you know nocahoots.
We haven't been in cahootsabout these videos, have we no?

Speaker 1 (04:01):
not at all.

Speaker 2 (04:10):
People.

Speaker 1 (04:10):
I've showed them to internally at tailscale have all
been like oh, these are great.
Thank you so much.
So thank you from a bunch oftailscalers to you.
Thank you well.
Thank you very much, alex.
I really appreciate that.
But I've got something veryexciting for all of the audience
listening that I can announceon the show today and this is
the first place that you'regoing to be hearing it Tailscale
is officially coming to Unraidand is going to be built right
into the OS.
The team's been working withDerek Kayser, who's the author

(04:33):
of the Unraid plugin, tobasically make that happen and
have it become part of the OS.
Now I know some of ourlisteners, alex.
They might not be familiar withTailscale or why it's such a
game changer.
So, alex, could you start byexplaining what tail scale

(04:53):
actually is and why it's soimportant for us kind of home
server enthusiasts and homelabbers, etc.

Speaker 2 (04:55):
yeah, well, for those of you that are listening that
don't know, I I cut my teeth onunraid, originally 10, 15 years
ago now, my goodness, back whensort of one and two terabyte
hard drives were the norm andwhat drew me to Unraid as a
product was mismatched drivesizes.
You know you could put any, youknow any disks into your array

(05:18):
and have one I think it was oneparity drive back then.
I know you can do more nowadaysand you didn't have to be
subject to the limitations thatyou were subject to with
something like zfs and I knowunranging.
So now I did zfs too, but backthen, um, I was a poor student,
so there was, there was just nochance that I was going to

(05:39):
afford an entire zfs arraysworth of disks.
What I could do was run aserver permanently in my house
with a couple of disks in it.
You know three, four, fiveterabytes worth of storage at
most.
When you've got a server runningin your house full time, you
start thinking to yourself well,what else can I do with it?
And so you start puttingsomething like Plex on there,

(06:01):
which is the gateway drug for alot of us self-hosters, and
before you know it, you've gotyou know 20 or so different
self-hosted applications,everything from invoicing to you
know trying to figure out howI'm going to host my documents
with NextCloud to who knows whatelse right.
And when you're running thatstuff on a server in your house,

(06:23):
it's all fine, you're on yourLAN, you can connect to that
server and it gets a local IPaddress and you type 192.168,
whatever and put a port numberin and it's all gravy.
And then back then I was workingat the Apple store in retail
and I was like I want to watchsome of these videos on my lunch
break that I've got on myserver at home, and so that

(06:47):
remote access bug started totrickle away and tick over in my
head.
So I started learning aboutthings like port forwarding,
which is where you open a holein your firewall pointing
directly to that server.
Essentially, you're putting thething in your house, directly
on the public internet, andyou're the behest then of the
application developers to writesafe code, which, as we all know
, application developers neverhave bugs.

(07:09):
Applications are totally secure, and so that was the way it was
for a long time.
To be honest, we had OpenVPN,which sometimes let us get into
remote networks, securelybypassing the port forwarding
thing for the most part, but youstill had to run an open vpn
server somewhere and forwardthat port to the, the node

(07:32):
that's running in your LAN, andone thing.

Speaker 1 (07:35):
One thing, alex, I always used to find about open
vpn sorry, sorry to interruptyou there, but um, and I'm sure
many other people have found itas well is you'd end up getting
a new laptop or something andyou'd have all your open VPN
configurations on the old laptopand you'd kind of be off out
somewhere and think, oh, Ireally need to connect back, and
you think, oh, I haven't got myconfigurations, but, as I'm

(07:59):
sure you're going to go on totell us, tailscale gets around
all of that as well, yeah, andit does in a really interesting
way.

Speaker 2 (08:07):
And so the reason that you do port forwarding is
to because IPV4 addresses arelimited, right.
There's only so many uhdifferent subnets and ciders and
all the rest of it that you canuh you can have on the public
internet.
So your ISP probably gives youI say probably deliberately a
public wan ip address.

(08:27):
In version four speak, someisps put you behind what's
called double nat, carrier gradenat, because they don't want to
have an entire pool of ipv4addresses that are quite
expensive to acquire these days.
So they'll give you.
Essentially they're runningtheir entire isp as a lan,
effectively, so you get a localip address, which means when you

(08:50):
want to try and connect to thatremotely, you can't because you
don't have control over theirfirewall.
Which brings me to anotherpoint.
One of the real pain points, um, besides port forwarding, was
knowing what your public wan ipaddress was.
Do you remember you had to runlike duck dns or dynamic dns,
right?
Yeah, um, this was essentiallya script that ran on somewhere

(09:14):
in your house and it updated toa remote server.
Cloudflare d still does this, Ithink, and there's a bunch of
other tools that still do it, um, and they had to update a dns
record in the cloud in somewhatreal time and there was a period
of time tools that still do it,um and they had to update a dns
record in the cloud in somewhatreal time.
And there was a period of time,I think, where it was either bt
or virgin in the uk that werejust rotating ip addresses every
like six hours or something onmy one and it was.

(09:36):
It was just getting.
Really the script just couldn'tkeep up and the dns stuff was
always wrong and, um, it wasjust a pain.
Enter Tailscale Okay, so thereason that I started working
here was because probably abouttwo years ago and I've been at
Tailscale now for about a year Inoticed in the Synology app

(09:56):
store that there was a new appcalled Tailscale.
I said, oh, what's this thing?
I should give it a look.
And it does something calledNAT traversal and this is the
absolute magic of tail scale andforgive me for sounding like a
sales pitch, but I genuinelybelieve what I'm saying, like I
wouldn't work there and Iwouldn't do this job if I didn't
personally find it solved, ahuge challenge for me.
But the reason that traversalis so amazing is because when

(10:20):
you want to make a connectionbetween two different devices,
you have to have a direct pathbetween them, and typically we
would do that, as I say, withport forwarding.
But with tail scale, the, thenat traversal piece, uses a
third party what we call a derpserver, a relay server, to
establish a known point ofcontact between those two
devices and then essentiallyabuse some advanced techniques

(10:44):
within the NAT space to create adirect connection between your
phone at the coffee shop andyour Unraid server in your
basement or under the stairs orwherever it might be.
And that really means you don'tneed to do a whole bunch of
stuff.
You don't need to do dynamicDNS anymore, you don't need to
open ports in your firewallanymore, you don't need to worry

(11:04):
about ports and all the rest ofit, because you can just use
Tailscale's native tooling toprovide TLS certificates for all
your services.
And the best part, coming backto what you said a moment ago we
handle all the key exchange foryou.
The private keys never leavethe devices in question.
So there's no risk of Tailscale reading your traffic

(11:25):
because we don't have the keysto decrypt.
It's end-to-end encrypted andthe private keys never leave
your device.
But when you install it on anew, a new phone or a new laptop
or whatever.
Like you were saying, you areauthenticating to the cloud
authentication server, the tailscale servers, the control
service, and we do all the wireguard key exchange underneath.

(11:45):
So we essentially just add yournode to the tail net and then
there's a bunch of rules.
You can configure acls policiesto say this node is allowed to
talk to that node on thisprotocol, on this port.
If you want to get reallynitty-gritty, there's a whole
bunch of stuff you can do.
But essentially it justcompletely solved the remote
connectivity piece for me to thepoint where I now have one

(12:09):
personal tail net that'sbridging a server running in
norfolk, a server running inlancaster both in the uk and all
of my servers here in raleigh,and I can use a single dns, like
I have a subdomain linked toeach of those sites.
But as long as I'm connected tomy tail net, I'm connected to
those sites as well and it's themesh network of my nerd.

(12:31):
Dreams come to life really.

Speaker 1 (12:35):
I remember as well watching one of your videos,
Alex, about how you actuallyhave your naming convention with
the subdomains to basicallylocate where they are.

Speaker 2 (12:47):
I thought that was um very, very clever this is born
out of having um two or threedifferent sites to manage really
, and trying to figure out alogical way to to manage it,
because the last thing you wantis for one of these um sites to
be dependent on a different siteand if, uh, my internet here in
raleigh goes out for whateverreason, I have a power cut or

(13:07):
someone cuts through the fiberline or whatever, um, I don't
want my mother-in-law to bewithout dns right, and so what I
try and do is I set up eachsite as a standalone unit and
then use subnet routers and abunch of tailscales features to
kind of bridge them all together.
But the naming convention Icame up with was service name,

(13:28):
so it could be jelly fin, thenhost name, so it could be you
know the name of your serverunraid, or would be tower, right
for you guys.
So it'd be jellyfintower, andthen I'd saynorfolk, for example
, and then I'd have alexktsedcom, whatever you know.
So you've got a five level deepsub name, a sub domain system,

(13:49):
going on there.
But I know just by reading thatsub domain exactly where in the
world it is, which box it's onand what the and what the
service is.
Now, if there's a if there'sonly one instance of jellyfin in
the house, which is probablylikely.
You can omit a couple ofdifferent things if you want to
to to make thing make your lifesimpler.
But the nice thing about thehaving the subdomains is you can

(14:11):
wildcard all of the dns forthat specific host to a specific
ip address in your LAN.
You don't have to do an entryper service.
You'd be like, right, well, allthe toweralexktzcom stuff
always goes to 192.168.whateverand it just, it just works like

(14:32):
you don't have to ever thinkabout it yeah, that's.

Speaker 1 (14:35):
That's really super cool.
I haven't actually tried it outmyself, but it is something I
really want to do.
I've always set up um public,um IP addresses in Cloudflare
and that kind of thing for myTailscale IP addresses.

Speaker 2 (14:50):
We've got videos about that topic and you can do
that if you want to, becauseit's only so.
Tailscale has a pool of whatare called CG NAT, basically the
100.100 CIDR range, the subnetrange.
There is a reserved range rangeand we use that internally for
the.
I think you get four millionaddresses on a, on a tail net or

(15:11):
something, and you can putthose 100.ip addresses quite
safely into cloudflare if youwant to publicly um, and they'll
only be rootable if you'reconnected to your tail net.
So yeah you're not exposinganything, you're not risking,
you're not poking holes in yourfirewalls like it's you can.
Those packets can only flowover the wire guard tunnel

(15:31):
underneath if the device has thecorrect key to unlock the door
yeah, I don't think you used tobe able to actually put kind of
like local ips into cloudflareat one time.

Speaker 1 (15:43):
I think it's only the last few years you've actually
been able to do it.
I may be a hundred percentwrong, alex, but was that right?
Yeah?
Yeah, I think they had.
They had to be kind of publicips, but anyway.
So basically, you know, itlooks like when we're going to
have the official plugin in um,unraid, users are going to have
great amount of ease of accessand security.

(16:04):
That's the idea.

Speaker 2 (16:06):
Yeah, so you'll be able to share your unraid server
that's under the stairs with afriend of yours and do remote
zfs replications, all encrypted,all over tail scale all for
free, because that's anotherthing.
We offer 100 devices and threeusers for free.
Yeah, and I've always.
I've always thought that unraidwas missing a trick by not

(16:26):
letting me carve out a fewterabytes on my friend's server
on the other side of the countryand we tell scale.
You're going to be able to doit without unraid being involved
.

Speaker 1 (16:34):
Really so you know, unraid is adding support for the
um tsnet certificates.
So, um you know, could youexplain how these certificates
will make it easier for unreadusers to actually access their
home networks?

Speaker 2 (16:50):
well, how much do you love reverse proxies?
How much do you loveconfiguring certificates?

Speaker 1 (16:56):
um, about as much as I like um mowing the lawn, and
how long is your lawn right now?
Knee height probably, yeahdon't, don't ask.

Speaker 2 (17:07):
Yeah, um, you know what's weird?
A quick tangent weird aboutmoving to america is that grass
doesn't really grow on its ownhere.
You have to seed it and carefor it and and like properly
tend.
It's weird, like I'm used toengland where grass just grows.

Speaker 1 (17:23):
Here you need to, you need to dig up a piece of turf,
alex, and bring it with you,and bring some english grass,
and then you have to do it.

Speaker 2 (17:29):
It would survive in north carolina in summer.
If I'm honest, bud uh you gofor, probably probably go for
like june, july, august is wellover 30 celsius every day, man,
nearly 40, uh, some days, andit's humid, really humid.
Anyway, we digress slightly.
What was the question?

Speaker 1 (17:51):
um, I was just asking about um, yeah, the, the certs
in in unraid and how that willmake it easier for people.

Speaker 2 (17:57):
Yeah, and maybe kind of tie that into um, if you
could explain about magic dns aswell and why that's useful for
users yeah sure, so the we'vecovered the dns a little bit
already, but essentially everytail net gets its own unique dns
name and that's on the tsnetsubdomain, right?

(18:18):
So there are, for free, severalthings you get with with
tailscale um anyway, and one ofthose is a dot ts.
Dot net domain for your tailnet, so you can refer to devices
anywhere that you can um reachthem with the with their fully
qualified domain name if you'dlike, and so the.

(18:39):
The benefit of that is that ifyou're running I'm going to use
jellyfin again if you're runningjellyfin and you think, right,
I wish this had a TLScertificate, because the
Jellyfin client's alwayscomplaining about a self-signed
TLS certificate you can runTailscale Cert and then use
Tailscale Serve to actuallygenerate automatically, via
let's Encrypt, a certificate forany of your self-hosted

(19:01):
services running on your ownRAID box.
You don't need to worry aboutAPI keys in Cloudflare to
approve ownership of a domain.
You don't even need to own yourown domain.
There's a bunch of complexitythat it removes completely from
the end user with just a coupleof commands that mean that you
can verify cryptographicallythat.

Speaker 1 (19:28):
I'm sorry, Alex.
If you can hear a funny tune inthe background.
One of my servers has a beepspeaker and I have it play a
certain little song at a certaintime to let me know it's that
time of day.
Well, it's 5pm on day.
What is?
It's 5?

Speaker 2 (19:46):
pm on a friday.

Speaker 1 (19:46):
So there we go you know, so it will stop in a
moment.

Speaker 2 (19:50):
What song is it?

Speaker 1 (19:50):
playing.
Um, I'm gonna sound very, verynerdy if I say here is actually
a jared tell commodore 64 songthat I converted into beep from
using an emulator, putting itinto an mp3 and then making it
into beep I would expect no lessof you, ed if I know, so yeah,

(20:11):
um, yeah, sorry to interrupt youwith um.

Speaker 2 (20:14):
With that, alex no, I , I don't think.
I don't think there's much moreto say.
I mean the.
The general idea is thatthere's a lot of, there's a lot
of problems that require somespecific knowledge.
In fact, just last week, I wasputting together a tutorial on
how to host a cloud VPS, installTailscale on that cloud VPS and

(20:35):
then use the Tailscale tunnelon the back end to expose
Jellyfin to the public internet.
And as I was making this video,I sort of thought to myself oh,
this is going to be a 10-15minute tutorial, easy peasy.
And then I'm like oh, shoot, weneed to generate an api key for
cloudflare, we need to doanother one for digital ocean so
we can spit up the vps withterraform, and then we need a
tailscale auth key, and then weneed to do dns.

(20:57):
And it's just like when, whenyou do this stuff for a living,
like I do it, it isn't that bad,but I, I always, I always come
across the.
I was trying to explain this tobe like would I be happy to walk
my mother through doingsomething?
And if the answer is no, thenthe.
The reality, the sad reality,is, is that many, many people in

(21:19):
the real world also, I don'tthink sometimes I live in the
real world they wouldn't bebothered either, and a lot of
what Tailscale's core mission isis to really make things simple
.
So if there's a pain point likecertificates, if there's a pain
point like connectivity, smooththose edges round those corners

(21:40):
and make it easy for people,and that really is what we're
trying to do.

Speaker 1 (21:51):
So basically, with the um dot ts certificates,
people connect to their unraidservers through a name so they
can have their server name.

Speaker 2 (21:56):
You know, like I don't know whatever it might be
called, yeah, so because unraidhas a sort of cloud connectivity
piece, right, and I mean thatis nice, but the downside of
that is it's obfuscated but it'sstill out on the public
internet and there is noidentity validation, whereas

(22:18):
with a Tailscale connection youconnect to the Tailnet client on
your phone and then only youcan reach that endpoint.
It's not like you, ed, couldguess the right string of
characters and figure it out andreach my front door.
No, literally,cryptographically you cannot,
because you don't have the rightkeys.
You cannot route packets tothat endpoint.

(22:39):
So I think it's going to bereally interesting to see how it
replaces or maybe augments theexisting unraid remote
functionality and also you don'thave to touch your router at
all.

Speaker 1 (22:52):
Yeah, that's a big one for a lot of people you know
, at the moment, to use unraidconnects you do have to forward
a port.
So oh yeah, if you don't dothat it's not going to work.
And so people who are on thingslike starlink, well, you're not
going to connect your Unraidserver.

Speaker 2 (23:07):
That way so.

Speaker 1 (23:08):
Tailscale gets around all of that.

Speaker 2 (23:11):
Starlink is a big one .
So the reason that we say thatis because Starlink does what's
called the carrier-grade NATpiece, where they don't give you
a publicly rootable IPv4address.
They give you either an IPv6 orsome kind of double matted ipv4
address, so you don't haveaccess to the uh, the starlink

(23:32):
firewall.

Speaker 1 (23:33):
So you're kind of out of luck really yeah, um, I just
wanted to ask you a question aswell, alex, on to some of the
kind of maybe more advancedparts of tail scale.
I just wonder if you canexplain to me the difference
between tail scale serve andtail scale funnel.
Oh yeah, absolutely so tailscale.

Speaker 2 (23:54):
Serve is essentially like a reverse proxy, so it's
funnel too.
Actually, the idea behind themis serve exposes things inside
your tail net using propercertificates again, because you
can integrate it with let'sencrypt just one command.
Essentially you redirect a port.
So let's say you have a Proxmoxfront end, for example, running

(24:16):
on port 8006.
You want that to be actuallyjust running at a specific
domain name on port 443.
So it's a transparent.
You know, you don't have totype a port number.
Well, you can use tailscaleserve to redirect those ports a
bit like you would with anyother reverse proxy.
Funnel does exactly the samething, except it puts it on the
public internet.

(24:36):
So obviously there's a risk.
There's a risk involved therebecause you can very easily,
with one command, put yourunraid box out on the public
internet.
Obviously that carries somerisks with it and you want to be
cognizant of those risks.
The other, the other thing toconsider with funnel is that we
proxy all of the traffic.

(24:56):
It's stilling, it's still umgoing through, uh, tail scale
once it gets to us.
But um, essentially you arebeholden to some quality of
service, uh, bandwidthlimitations.
So you're not going to bestreaming jellyfin through
funnel, not reliably?
Um, just simply because it's afree service that we offer our

(25:19):
users.
As you know, many of our peopleat tail scale are developers,
and so the reason that wecreated funnel as a as many of
our people at Tailscale aredevelopers, and so the reason
that we created Funnel as atechnology was well, I'm working
on a website prototype and Iwant a quick way to share this
database with my colleague overthere that's running a web hook
or something as part of their CIjob that they're testing real

(25:41):
quick, or it was just supposedto be a way to throw up quick
and dirty prototypes onto thepublic internet, or very simple
static websites.
You know it's not designed forthings like video streaming
right.

Speaker 1 (25:52):
Would it be okay for something like next cloud, um,
for self-hosting next cloud, ormaybe?

Speaker 2 (25:58):
again, but it depends on the meat, like if you're
putting your photos throughNextCloud, I mean it would work
but it wouldn't be performant.
And what I would say at thatpoint is that many people who
think they need Funnel actuallydon't, because with Tailscale's

(26:19):
direct connections, every devicebecomes a client on the network
, and I I kind of want to drawan analogy at this at this point
, because, uh, let me just bringup something one of my
colleagues where's my mouse gone?
That's my mouse there, it isokay.

(26:40):
Uh, essentially it's, it's thedifference between a hub and
spoke model.
So, rather than sendingeverything through a central
thing, which is what you'redoing with funnel and you're
therefore bandwidth constrainedyou're making that direct
connection from your phone backto home base, and so you get

(27:00):
full line speed, full connectionspeed between those devices,
which makes a huge difference.

Speaker 1 (27:06):
I know Unraid has got in the pipeline plans to be
able to streamline a method forsharing SMB on the tail net, so
I was wondering if you couldtell me, alex, I know it's
possible to be able to shareparts of your tail net out to
other people.

(27:26):
How does this feature work andwhat kind of controls do users
have on who actually gets access?
Great question.

Speaker 2 (27:37):
This can get a little complicated if you're not
careful.
But the short version is let'ssay, I want to give you access
to my server, and by access Imean I want to let you access
things running on port 443 orport 80 or something like that,
the web ports.
I can go into the TailScoutadmin console and share a node

(27:58):
with you.
You will then be able to routepackets from your TailNet into
mine and essentially that's it.
I can create access controllists if I want to, that limit
you to specific ports.
Like, say, I don't want you tohave any way to route packets on
port 22, for example, for SSH.

(28:19):
I can put a rule in place thatwould prevent that.
I can put a rule in place thatwould prevent that.
In fact, by default, all ofTailscale is built on a process,
a principle sorry, of zero it'snot quite zero trust, but it's
close Default deny is ourdefault state, so you do have to
explicitly allow anything thatyou want to any traffic you want

(28:43):
to pass.
But in fact we're doing this we, being Jupyteriter,
broadcasting with theself-hosted podcast that I also
do uh, we just threw up a serverinto a colo in canada and we
don't want to put that thing onthe public internet.
But I also don't want that inmy personal tail net.
I want that in a jupiterbroadcasting tail net because
it's business infrastructure andwe want it to be kosher and

(29:06):
everything to be segmented.
So I created a brand new tailnet for Jupyter Broadcasting and
I installed the Tailscaleclient on that backup server in
Canada and then I shared thatnode into my personal tail net
so that I can still do ZFS sendusing the fully qualified domain
name of that remote host in theremote subnet.

Speaker 1 (29:25):
But it still retains full independence in both places
cool and one thing people mightwonder about say, for instance,
alex, you were to share umsomething on tail scale with me,
that doesn't mean that youwould then be able to access my
town.

Speaker 2 (29:43):
That does it no, not at all.
Uh it, it's a, as I say, it's adefault deny model.
So unless you share somethingwith me explicitly um the the
blast radius I, I can't routepackets from that node you've
shared, for example, out toother nodes on your tail net,
unless you let me.

Speaker 1 (30:00):
Yeah yep, so basically, we both have to share
what we want with each other.
So, yeah, now, if you're afamily, don't need to worry it.

Speaker 2 (30:07):
It makes sense to have, uh, a single tail net with
multiple users.
So, you know, you got, you gotyourself, you got a partner, a
kid or two maybe, and you'rerunning a self-hosted um, let's
say image instance where you'rebacking up all your family
photos.
At that point, you don't wantto create four tail nets, one
each, and start sharing stuffbetween four different tail nets

(30:29):
.
You want to create one tail netand add multiple users.
And actually, coming fairlysoon I don't honestly know if
I'm supposed to say this outloud but hey, we've had one
exclusive in this show already,let's have another one.
Uh, we are looking at I'm goingto say, I'm going to say this,
uh, very carefully we arelooking at launching a personal
pro tier which is going to allowpeople to have, I think it was

(30:51):
up to half a dozen users.
Don't quote me on that.
Uh, the numbers changed alittle bit over the over, the,
the machinations and the, thebrewing period of this one, but
essentially for free, you get100 devices and three users, as
I've already mentioned.
But we're going to make itfairly cheap for families to

(31:12):
adopt a personal pro accountthat is going to support, I
think, up to six people within asingle family unit, a little
bit like what Steam have justadded and Apple family sharing.
At this point, I think we're allused to family sharing, but the
rationale there is that youdon't want to be having to worry

(31:33):
about the intricacies ofmultiple tailnets.
Again, this comes back to afounding principle of Tailscale,
of rounding off complicatedcorners and making things
smoother and more easy to use.
Complicated corners and makingthings smoother and more easy to
use.
Essentially, I want to treatany of my self-hosted services
like I'm on the land, wherever Iam, and that includes, you know
, my kid's phone taking apicture and automatically

(31:53):
backing up to image and all therest of it from I don't know
Scout Camp or wherever they'reat.
You know?
Um, I still do scout camps.
Is that still a thing?

Speaker 1 (32:06):
yeah, I don't know.
I remember I went to scout campwhen I was very young and I
just couldn't wait to get home.
But maybe they're not as goodin the uk as they are in in the
us.

Speaker 2 (32:17):
I don't know I mean, I went to a uk one.
I actually remember we built auh, there was this thing called
the mayfair.
Uh, fair, on like mayday, may4th, may 4th, where bank holiday
and we built this massivea-frame out of like tree trunks.
There was a guy called kevinwho was a volunteer firefighter,
I think, fireman.
Sorry gosh, I have americanized.

(32:39):
I've only been here six yearsand I have I have seriously
americanized.

Speaker 1 (32:42):
The thing is, alex, I still hear you actually do say
zfs, then you don't say zfs yet.

Speaker 2 (32:47):
So I switch like crazy because my, my name has a
z in it or a z in it, like I'vealways been alex ktz in my mind,
but it's zfs.
But it's z pool, it doesn'tmake any logical sense.
It really genuinely dependswhat it is.
And, uh, toronto's airport codeis yyz, even though it's a song

(33:10):
by rush, who are a canadianband, that uh would say zed like
there's.
There's zero logic in here.
I I'm just going to warn you ofthat to anybody listening.

Speaker 1 (33:20):
Don't take a word I say seriously, because it's all
just mush like you know, I I sayzfs all the time.
Now, I never I used to say zfs,but my wife is american so I
get americanisms from her um andI have a lot of people in the
uk.
They go.
You're meant to be from the uk.
You're not meant to say zfs.
Like, why are you saying that?

Speaker 2 (33:39):
oh, I said router the other day and I someone looked
at me like I was made of stone,so you know, yeah, don't say
don't say router in australia.

Speaker 1 (33:47):
No, what do they say?
Um, wow, I can't say it likebut, but um, no, sorry, you
don't say root in australia.
It has a different meaning.
So the uk router, it has a verydifferent meaning.
Yeah, people can go and look itup after they watch this
podcast.
Not safe at work, got it?
Um, I was talking to um one ofthe unraid staff recently, um

(34:12):
larry, and he mentioned actuallythis kind of travel router
thing that he picked up recentlyoff amazon.
I think it's the gli net, Ithink mt3000, yeah, and
apparently that runs tail scaleand he used it at the hotel
connecting onto their wi-fi.
Then he had the tail net goingback to his house so he can

(34:34):
route all of his traffic throughhis own internet at home and
then the travel router has itsown wi-fi network so then all of
his devices can connect to thatthrough tailscale back to his
house through his own ip.
Um, have you seen more peopleusing tailscale in similar ways
with these kind of standalonedevices?
Alex?

Speaker 2 (34:53):
well, you know that's .
That's a travel hack waiting tobe discovered.
Right there is.
This is quite popular amongstrvers.
It's popular amongst people whogo cruising, and by cruising I
mean the the safe for workversion of cruising.
Uh, go on cruise ships for theirholidays, uh, basically
anywhere where somebody whoprovides you wi-fi tries to

(35:15):
nickel and dime you based on thenumber of devices that you've
connected.
So you take one of these littletravel routers with you, you
connect it to the hotel wi-fiand then connect your phone.
This is also true on planes, bythe way.
You connect to the wi-fi of thetravel router through your
phone or laptop or whatever.
Do the captive portal thing.
These networks typicallyauthenticate based on mac

(35:37):
address, and so the mac addressyou're authenticating isn't your
phone or your laptop.
You're authenticating the macaddress of the travel router to
that remote wi-Fi network and so, so far as the hotel is
concerned, or the cruise ship orthe plane or whatever, you've
only got one device, and theyhave no idea that you've got
actually three iPhones and twoiPads and a laptop and an Apple

(35:59):
TV and all the rest of it.
The other benefit of that is itmeans you don't have to
reconnect all of these devicesto a new Wi-Fi every time you
decamp to a new place, so you'retaking a road trip and you're
going through 10 differentAirbnbs every night.
It can genuinely be a bitfrustrating to have to remember,
well, what's the Wi-Fi passwordthat this random person set

(36:20):
here.
And the really nice thing aboutthose GLI-Net devices is it
means you've just got thatsingle configuration point and
then, to put icing on the cake,you don't necessarily want all
of that traffic goingunencrypted across their network
, and so Tailscale is a bitdifferent from what we would

(36:41):
call a traditional privacy VPNlike a NordVPN or a private
internet access or a surf sharkor any of those.
Instead, we can use somethingcalled exit nodes to kind of
turn tailscale into one of thosefeatures, and so what we can do
is on the, on the GLI net andindeed any client also, because

(37:01):
it's a mesh network.
Remember, we can.
We can connect devices directlyto you know, let's say, let's
say I'm in the uk and I want tocome out of my house here in
raleigh because my onlinebanking doesn't let me access it
from another country, which istotally actually what they do.
Yeah, um, I can turn on exitnode functionality and route the

(37:23):
packets from my phone outthrough this house as if I'm
stood in or sat in this chairover what's called an exit node
functionality.
And you can do that same thingthrough the glinet too.
The nice thing about that is itmeans all of the traffic
between the glinet and the exitnode is encrypted, so the person
on the remote wi-fi has no ideawhat you're doing.

(37:44):
They won't see the ns queries.
They won't see.

Speaker 1 (37:47):
They won't see any of it I had one of the earlier um,
glinet, um devices and I'm notsure if the new one is the same.
But you also have a littleswitch on the back where you can
turn the vpn on and off.
Yeah, does um, is that?
Is that the same on the newones?

Speaker 2 (38:02):
alex, I'm not sure if you've seen them, they, they
used to only have one or twomodels, but I think they've got
like a dozen now.
So I'm going to plead ignoranceon that one.

Speaker 1 (38:10):
Take plead the fifth right, but you know I thought
that's pretty cool so you canswitch the vpn off if you kind
of need to have regular trafficyeah but then you, you toggle
the vpn and you're going backthrough um yeah the other.

Speaker 2 (38:23):
The other nice thing about it too, is that, let's say
, you find a client that can'tnatively run Tailscale, which is
quite tricky to do because werun on iOS, android, apple TVs,
firesticks, ipads, laptops,anything pretty much from BSD
right the way up to Windows werun on these days.
But there are some embeddeddevices that have Wi-Fi ESP

(38:46):
devices are a good example thatdon't natively support tailscale
yet or don't support tailscaleyet, and so if you connect those
devices to your glinet routerthat's connected through
tailscale, you can still gainsome of the same routing
benefits that you wouldotherwise get if it was a native
tailscale client.
So let's take the Jellyfinexample again.

(39:08):
Let's pretend you have a mediaplayer that can't run Tailscale
natively and you want to accessa Jellyfin server remotely.
Well, if you connect throughthe GLI net, you can actually
reach your remote Jellyfinserver, even if the client
itself isn't a Tailscale client.

Speaker 1 (39:26):
Can we talk a little bit about subnet routing as well
, please, alex?

Speaker 2 (39:29):
Oh this is a fun one, funnily enough.
So I have a backup server at mymother-in-law's house near
Norwich, so I do cross-oceangeo-replications of my ZFS data.
But that server is in a remotenetwork and I have no way to

(39:50):
access the IPMI capabilities ofthat system from here without
something called subnet routing.
So what this does is the remotesubnet is, I think it's
192.168.16.
I think it's a slash 24, sothere's 250-odd devices in that
subnet.
I've no way to access thoseunless the Linux box is turned

(40:12):
on, which can be a bit of aproblem.
Sometimes, let's say it's been apower cut or it's just not
turned on for whatever reason.
So I need a way to get to theIPMI interface of that box, and
so my firewall at that place isan OpenSense box, and so what
I've done is I've turned theOpenSense box into a subnet
router which forwards allpackets and publishes a route

(40:35):
for that subnet to all of mytailscale devices and clients so
that I can actually access anydevice in that 16 dot whatever
subnet as if I was on that sameLAN.
So essentially, the short vote,the short answer is it takes a
remote subnet and publishes aroute to all of your tailscale

(40:56):
clients so they can accessnon-tailscale native devices
printers, ipmis, robot, vacuums,whatever so basically it just
lets you pretty much be exactlythe same as if you're connected
to your wi-fi at home.

Speaker 1 (41:11):
You can connect to any anything on your subnet or,
for people who don't know what asubnet is, that's basically
your local ip range and I'vedone a totally the same thing
you have, alex, except I'm a pfsense fan as opposed to open
sense.
But, um, I've got like a um avlan I use for all of my cctv

(41:32):
cameras.
Yeah, so obviously you can'tinstall tail scale on a reolink
cctv camera, but I just use thesubnet routing.
It's a great example actuallyon pf sense to be able to
connect to my cameras and seethem, so I can run the geo what
whatever called app is umreolink app and I'm able to
actually still connect to mycameras without having to go

(41:54):
over the cameras is a greatexample well, I think we've
pretty much spoken everything Ican think of about, about tails,
tail scale personally, but isthere anything?
is there anything kind of in thepipeline for tail scale?
Um, that's that's coming, umthat you can talk about, alex,
or oh, I don't know if I'm.

Speaker 2 (42:15):
I always get confused about what I can and can't say,
so I think I'll plead ignoranceand just say watch the youtube
channel, take a, take a look atour blog and all that kind of
stuff and we'll postannouncements over there.
We are, by the way, hiring inEurope for a dev advocate.
So if you're into the DevRelspace and you are a Tailscale

(42:39):
super fan and you want to go toevents and represent the company
and write for us and do all thekind of getting people as
excited basically what I'm doing, getting people as basically
what what I'm doing, uh, gettingpeople as excited about tail
scale as as I am uh, get intouch.
You can find me at Alex dot.
Katie said dot me on theinternet and uh, I'm on, you
know, mastodon and uh, I'm goingto say Twitter, but it's not

(43:11):
more uh, all those, all thosegood places, selfhostshow as
well, the podcast.

Speaker 1 (43:12):
I'm over there, um.
So, yeah, get in touch if that,if that sounds interesting to
you, we'd love to hear from you.
Um, and whilst I've got youhere, alex, excuse me, whilst
I've got you here, alex, Iwondered if we can speak a
little bit about self-hostingyes, please, let's do that I
would love to leverage some ofyour knowledge there.
So the first question I've gotabout self-hosting is basically
when it comes to self-hosting,um, it kind of it does tie in I

(43:33):
know what you're kind ofprobably going to say with this
but, um, people are oftenconcerned about security when
they self-host stuff.
They kind of think, oh, I don'tknow about self-hosting, I know
google is going to be secure.
Um, what are some essentialsteps you'd say home users
should take to secure theirself-hosted setup?

Speaker 2 (43:51):
uh, this is a trick question.
I thought we'd moved on fromtailscale uh I think, keep it
simple.

Speaker 1 (43:58):
I'm talking more, I guess, about kind of um services
that are publicly accessible.
I'd say so you something that'snot private.

Speaker 2 (44:11):
I draw a line in the sand between services I want
public and those that I don't.
So I was Linode for a long time, it was DigitalOcean for a long
time, I think it's Hetzner.
These days I have a cloud VPSthat I run the handful of things
that I want to be public.
You know my public blog,perfect media, servercom, a
bunch of other stuff, right?
Um, most of the rest of thestuff that I want to host is not

(44:38):
for public consumption, and soI just keep it on my land, and
now with tailscale, I can justconnect to it from wherever I am
.
Um, so for me it's a very simpledelineation keep the public
things public and put it in apublic vps.
Okay, it cost me, I think,seven or eight euros a month for
a heads in a box, but I thendon't have to worry about dmzs

(45:03):
or you know weirdness with, uh,putting things on my public land
, and you know it just keepsthings simple, and that's my
philosophy is is keep it simplefor your mental model.
So when something goes wrongand it will go wrong at some
point with self-hosting, becausethat's just the way it is you
are able to quickly understandwhere something is, which again

(45:25):
speaks to my dns philosophy,where something is what it's
running on and um what the kindof impact, severity, like the
blast radius of of a problemmight be yeah, so.

Speaker 1 (45:40):
So basically, private stuff keep on a separate box,
really, and public stuff have ona vps.
I actually um set up a hetznerum box myself recently and it
was, I think, because I watchedthe self-hosted podcast and you
guys were talking about howreasonably priced they are.

Speaker 2 (45:57):
I think I pay surprisingly cheap.

Speaker 1 (45:59):
I think I pay 30 pounds a month.
I got 64 gigs, about sixterabytes um.
It's either a four or sixterabyte spinning rust drive,
two, five, twelve nvmes and Imanaged to install unraid on
there.
Um, I made like a kind ofinstall script to be able to put
it onto their onto um their usbdrive.

(46:22):
But you, what I've put on thereis the um esats tv oh ersatz so
ersatz tv, which again I gotfrom the self-hosted podcast
yeah, that was chris, that oneyeah, yeah, it's just like so
cool I haven't yet.
Um, and for people watching whodon't know what ersatz tv is, it
basically allows you to streamyour own tv channel.

(46:46):
So, yeah, me being a bit of asci-fi nerd, I've got my
favorite sci-fi shows on there,like Andromeda, star Trek, et
cetera, and it just randomlyplays episodes so you can turn
it on and you might be 10minutes before the end of an
episode.
But, like you were saying onthe podcast, like chris was

(47:08):
saying, is you don't always.
You've got plex, nb jellyfin,but you kind of go on and think
what am I going to watch, right,um, and I mean, it's the old
days of, you know, just turningon bbc2 and being like, all
right, I guess I'm watching.
Uh, I guess I'm watching deepspace nine, or whatever next
generation at six o'clock now,whatever it used to be I, when I
go to mom's house, like youknow, um my stepdad, he might

(47:31):
have something on the tv.
He's watching just some randomkind of star trek and I think,
oh, I kind of get quite into it.
Yeah, I think I forgot aboutthat.

Speaker 2 (47:39):
I think there's an aspect of home I think there's
an aspect of creative gapfilling that you do with with
with the narrative too, like youcome in halfway through an
episode.
It's almost like you'reinspector Clouseau trying to
figure out what's happened inthe first half.
How did they get here?
And you kind of miss that, ifyou.
If you, how many times have yougot to a hotel and turned it on

(48:00):
and there's just been somecrappy nineties movie playing
and it's 11 pm at night and youget so invested in the last hour
?
Yeah, it happens all the time.
And ersatz lets you essentiallyturn your media collection into
a tv station.
You can have multiple channels,different epgs, different times
of day.
You can even have the little uh, the little tester card of the,

(48:23):
the girl with the chalkboard orwhatever, if you want to at 2
am with the beat.

Speaker 1 (48:27):
Yeah, you know and people put actually the
old-fashioned adverts in don'tknow they download the adverts
from period commercials yeah, soit's really really super cool.
Um, and the reason I put it onthe over there is because I was
worried about the bandwidth.
I thought I'd rather it be thebandwidth coming from there.

(48:49):
I know it probably doesn'treally do you know that much,
but I decided to do that anywayif you're a single user, uh, it
doesn't.

Speaker 2 (48:58):
Honestly, unless you're watching 24 7, it
probably doesn't matter, but assoon as you start sharing it
with friends and family, then itcan add up pretty fast, yeah
another thing I thought of alexis um, I'm not sure you know if
someone, if no one's watching it.

Speaker 1 (49:12):
Are the hard drives still spinning up?
Is it still kind of pushing outthe content, or does?
It only okay so that was onething.
I didn't want my hard drivesalways spun up if I was running
it locally at home.

Speaker 2 (49:22):
So I thought yeah, if you take a look.
So I I'm a big fan of quicksync for hardware transcoding,
uh, but you know low energyusage transcoding and um, it
doesn't appear to be active,when I'm right okay, that's good
to know.

Speaker 1 (49:38):
Yeah, um, anyway, kind of moving on, for someone
who's new to self-hosting, whatwould you recommend as the first
few services or applications toactually try?

Speaker 2 (49:49):
solve a real problem.
Um, for me that was media.
For you it might be somethingelse completely different.
It might be recipes, it mightbe document management, it might
be invoicing, who knows what itis.
But start by looking at aservice you're paying money for,
um through a proprietaryservice that you don't own the
data for, and if you care aboutthis kind of data sovereignty

(50:11):
angle of, well, what happenswhen that company gets acquired
or goes out of business or getshacked?
If you care about that stuff,bring those services in-house
one by one.
You don't have to rush, youdon't have to do it all at once.
This is a hobby that's kept meoccupied now since I was 20, 19,

(50:34):
20.
Um, when I got my, I think Istarted off with a Drobo and
then I got a Synology and then Ibuilt my first Unraid box.
Um, you know, this is a journey, that and the, the, the
destination is the journey, likein terms of, for me at least,
it's kind of like.
Uh, if, like in terms of for meat least, it's kind of like.
If I had to compare it to 3Dprinters, self-hosting is a

(50:58):
little bit like buying a Prusa,where you have to build it
yourself and assemble ityourself.
I mean, I know you can buy apre-made one, I know I know that
, but for the most part it's atinkerer's device, right?
It's something that you'regoing to spend time fettling and
tuning and loving and bringingto life.
For some people, that's not whatthey want.

(51:19):
They want the bamboo lab, theywant the ready to go,
pre-assembled, 10 minutes outthe box, thank you.
And for most people that meansapple photos.
For most people that means likegoogle photos, or it means one
drive or google drive orwhatever it might be.
But for me, I care sufficientlyabout where that data lives

(51:41):
that I would much prefer itlived in my basement rather than
a data center powered by whoknows what in who knows where.
Um, and really it's, it's aphilosophical thing.
It no, nobody ever gets intoself-hosting to get rich or to
get you know.
Uh, I'm trying to think of thebest way to describe it, but

(52:02):
it's essentially just know whatyou're in for, right, you're in,
you're in for, you're in forowning your data.
That's a good thing, and it'salso a bad thing, because when
you screw up, there's only oneperson to blame, typically, and
it's usually you.
So you know, um, solve a realproblem.
To answer your question scratchan itch, learn some stuff, make

(52:23):
some mistakes with a low, witha low hanging service, like if.
If the question was, how do Iget started in home automation?
A great, a great answer forthat would probably be, in my
opinion, something like homeassistant with a couple of light
bulbs.
Nobody's going to get injuredif a light bulb doesn't turn on.
You can still go and flick theswitch on the wall if a light

(52:44):
bulb doesn't turn on.
So whilst you're learning, keepit low stakes and then over
time you can, as, as yourconfidence grows and as your
skill set grows, you can startto really become reliant on
these things.
Like all the lights in this youknow filming room that I'm in
my uh, my office, they're allhooked into home assistant and
it's all automated and I Ireally seriously rely on some of

(53:05):
the home automation stuff I'vegot going on in here.
But you know at the beginning,if it didn't work it it didn't
matter, you know.
So like I can be like oh look,no light, light, it's all done
from home assistant.

Speaker 1 (53:19):
So you know, uh sort of a real problem.
I think, like you say, peoplehave got to be like.
The journey is very important.
It's not the destination ofjust having the finished product
.

Speaker 2 (53:32):
I think it's a bit like kind of like my dad in his
day would tinker with his caryeah, I, I'm glad you went there
, because that's a, that's areally good comparison too we
tinker with our servers, but wecouldn't tinker with cars.

Speaker 1 (53:45):
Nowadays they're too complex, or I'm sure some people
can.
You know they you know I canbarely, you know, um, remember
to top up my car with oil.
So they do need it, though theydo.
But you know, I normally drivemy car till it breaks and I know
it's time to go to the garage.

Speaker 2 (54:02):
But yeah, fill up with some more oil.
And you get there and they lookat you weird and they say ed,
did you know?

Speaker 1 (54:07):
this is an electric car talking about going back to
our thing as well.
I want to know, alex, what wasthe first um service you ever
self-hosted yourself?

Speaker 2 (54:21):
hmm, it was probably remote desktop.
So this was.
This goes right the way back towhen I was in sixth form
college and I've been fascinatedwith remote access since
forever.
I mean, I've got a computerover here but I am over there
and you mean I can connect tothese two things.
And for me it was going tosixth form college in Winchester

(54:43):
and my house in Basingstoketrying to connect from school
back to home to get around uhcontent, uh, internet content
blocks, really.
That was what I wanted to doand that meant putting port 3389
out on the public internet backthen and my windows parks
directly out on the it horrifiesme thinking about doing that

(55:05):
now, but that was what I did asa 17 year old.
Um.
So yeah, I would say probablythe first self-hosted service
was my own computer with remotedesktop, if that counts yeah, I
think it does um.

Speaker 1 (55:19):
So you know, docker, not only in the unraid world but
everywhere, has become reallypopular in the self-hosting
community.
Now what are your thoughts onthe potential benefits and
downsides of usingcontainerization versus, say
like, um a v, um not a vpn?
A vm, that's.
That's the word I'm looking for.

Speaker 2 (55:39):
Thank you, alex how did we get an hour into this
without mentioning docker?
I think that's amazing, don'tyou?
Um, so a little bit of personalhistory here.
Uh, way, way, way back.
This is like 10 years ago.
I was very deeply involved inthe Unraid community and I used
to host something called theArch VM, which was essentially a

(56:02):
package repository, builtaround the same principles as
the AUR, for people to hostmedia acquisition apps.
Let's put it that way, mediaacquisition apps.
Let's put it that way.
Um, and essentially around thattime, john p and tom and a few
other folks were looking atadding docker to unraid.

(56:23):
I think eric schultz too.
Um, and this was pre-dockerbeing 1.0, so this is is a very
long time ago.
So they added Docker and therewas a moment when I used to
reinstall servers for fun backthen.
I don't do that now, but I didthen and so I got actually
really good at configuring allof my apps because I could just

(56:44):
click through and be like right,I remember this setting does
that?
And yada, yada, yada.
There was this one time whereI'd spun them up using Docker.
Literally just after Docker hadbeen added to the product for
the very first time, I blew myserver away, but the app data
lived somewhere else and I thenjust pointed the Docker

(57:05):
containers back at the same appdata volumes and everything was
exactly where I left it.
And I'm like, holy moly, thisis the future, this is I
understand containers now.
This is why they're so cool.
And 10 years later, I've endedup building a career on top of
it.
I ended up going into OpenShiftstuff, at Red Hat doing

(57:29):
Kubernetes work and now atTailscale.
I don't do as muchcontainerization stuff as I can,
but any chance I get all of myself-hosted infrastructure.
Everything is running out ofcontainers, like everything.
I just find the encapsulation.
It's very lightweight comparedto a virtual machine.
It just makes sense to me.

(57:49):
It did what SystemD did forLinux in terms of accessibility.
Containers allowed me tooperate at a level well above my
skill set.
Back then I couldn't compile aLinux kernel.
Back then I had no idea what Iwas doing, and yet I could type
these few Docker commands in andsuddenly, hey presto, I've got

(58:11):
Plex running Okay, sick.
Type these few docker commandsin and suddenly, hey presto,
I've got plex running okay, sick.
Um.
So around that time she musthave been 2013, 14 era uh myself
, johnny mo and uh, stian, whowent by the name lonix on the
unraid forums.
Uh, we all got together andco-founded LinuxServerio.

(58:33):
We noticed that there was, atthat point, there weren't really
many standards behind creatingthese containers for the
community.
There weren't shared baseimages.
The documentation was, if itexisted, was all over the place

(58:57):
if it existed, was all over theplace.
Um, and so we kind of took itupon ourselves to create a
unraid first containerizationproject.
Just take other people's appsand package them up.
Really, um, it kind ofsnowballed into a bit of a
juggernaut and, uh, I'm notinvolved in the project actively
anymore for various reasons,but uh, it's something I'm
deeply proud of and I know thatmany of your listeners will be

(59:19):
using it today.
I run into people all the timethat are using Linux server
containers, myself included, bythe way.
I still use them and it wasjust that standardization thing
really, that kind of tipped thescales in favor of Docker for me
and I've never really lookedback.
I mean, I still use virtualmachines for some things, but

(59:42):
for the most part I'm acontainer-first sort of guy.

Speaker 1 (59:45):
So your home assistant.
How do you run home assistant,alex?
Is that running on a real box,a VM, or in a container?

Speaker 2 (59:54):
It's running on top of Proxmox as a virtual machine
because Home Assistantspecifically provide what's
called HAOS, which is HomeAssistant OS, which is an
encapsulated way of running HomeAssistant with a bunch of
containers kind of baked intothat virtual machine image.
And the reason I do that isbecause it makes it easy to back

(01:00:15):
up that.
That virtual machine as anatomic thing, as a, as a
holistic, I can take that entireblob and just pick it up and
move it somewhere else.
And so right now it's a virtualmachine running on proxmox.
It's moved from being runningon the bare metal a couple of
times and I moved it aroundseveral times over the years and
it always just comes back.
So, uh, haos for me yeah, I dothe same.

Speaker 1 (01:00:38):
I run it as a vm.
I used to run it in a containerbut it was just not not as good
as running it in a vm and, likeyou say, if I do an update or
anything, I'll just do a cfssnapshot on the v before I do
and then if it goes wrong, I canjust literally just go back.
So I think using a VM for thatis really cool.

(01:00:58):
So any self-hosted services ortools that people might not be
aware of but have been a gamechanger for you personally.

Speaker 2 (01:01:06):
Oh, sterling PDF?
Definitely.
Have you heard of this one?
I haven't.
No, how much do you love AdobeAcrobat?
Yeah right, nobody ever sayslots.
If you ever need to do anythingwith a PDF, just type it into
Google Sterling S-T-I-R.

(01:01:27):
Sterling PDF pdf.
Um, bring this one up and havea look at it, because it's.
It's a tool that lets you splitpdfs, annotate them, rotate
them, password compress,password protect, compress them,
modify the message, like anyoperation you need to perform on
a pdf.
You can do with thisapplication in a web browser and

(01:01:50):
it is so nice and I don't thinkhardly anybody knows about it
no, I've never.

Speaker 1 (01:01:55):
Never heard of it.
I will be.
I'll be checking that out whenwe finish the podcast, alex, I
think.
Good, good, good.
Um, I also would like to knowis what's the most challenging
service or application you'veever tried to self-host, and why
, and what did you learn fromdoing that?

Speaker 2 (01:02:13):
I'm not going to include open stack, because
that's a beast, or open shift,because that's what that was.
Open shift three, because thatwas also a beast.
Uh, probably invoice ninja,because it didn't ship with a
web server built in.
So I had to figure out.
I had to figure out how to kindof tie together multiple
different containers andnamespaces and you know it was

(01:02:34):
just a whole mess.
But things have come such a longway.
I mean the standardization thatLinux Server brought to the
containerization space shouldnot be underestimated.
In my opinion, the work thatthey still do today on using the
S6 supervisor to essentiallyrecreate an init system inside

(01:02:56):
the container means thatnowadays you can run multiple
services in one container.
Even though it's a little bitof an anti-pattern from a
purist's perspective, these daysusing s6 lets you run the web
server and the app and thedatabase and who knows what else
all in the same place, whichfor most of us that are home

(01:03:17):
users, that's actually what wejust want the simplicity.
We don't need the purism thatperhaps some people would tell
you is required for containers.
Yeah, so for me I thinkprobably In invoice ninja was
the most difficult one, um, butdoesn't mean there weren't
others that I.
Maybe there's others I justcompletely gave up on.

(01:03:39):
Yeah, I managed to.
I managed to get invoice ninjaworking in the end, but uh, yeah
just going back to the linuxserver containers.

Speaker 1 (01:03:49):
We haven't actually mentioned it, but I believe all
of them now have um a docker modyou can do to actually
integrate tailscale directlyinto the container yeah, uh, I
think I think most, if not all,of them do.

Speaker 2 (01:04:00):
Um, you can.
I did this with one of thetailscale demo containers about
a year ago, so I'm a little bitrusty on the details, but
essentially you can baketailscale directly into the
linux server containers and putthe containers directly on your
tail net.
Now, another way you can dothat is to run what's called a
sidecar container, and I've gota whole video about this on the

(01:04:23):
tailscale channel.
It's about 30 or 40 minuteslong, talking about how the
linux kernel namespacing worksfor networking and stuff like
that.
It's super nerdy if you want toget into that kind of stuff.
But, uh, there's a fewdifferent ways to do it.
Um, and if you're running linuxserver containers, the docker
mod is a really good way to doit.
If you're running othercontainers, then the sidecar way
is the way to go.

Speaker 1 (01:04:43):
So, um, yeah, um, what trends do you see emerging
in the self-hosting space?
Are there any new kind oftechnologies or practices that
are kind of reshaping people'shome labs that you can see?

Speaker 2 (01:04:56):
hmm well, 10 years ago it was containers.
You know you were mad if youran containers in production 10
years ago and now you're crazyif you don't.
These days, I think a lot ofit's moving towards uh, click
ops type stuff.
You know, home assistantsmoving more towards ui-based

(01:05:16):
configuration from yaml filesand you've essentially there's a
phrase uh called crossing thechasm, that essentially we we
have mostly exhausted thetechnical early adopters for
self-hosting.
I think, think you know peoplelike you and me who are fairly
technical, who are willing toput in the time and effort to

(01:05:38):
put all the nuts and boltstogether.
And, um, you know the thejourney is the destination type
people, the trend I think I seecoming.
And there's websites like, uh,self-host.
You know s-e-l-f-h dot st.
You know the one, uh, bringingmore polish, bringing more fit

(01:05:58):
and finish to deploying apps tothe media around the space.
You know you look at peoplelike techno tim on, who I know
was on this show a few weeks ago.
Uh, tom lawrence, um, who elsewendell's like these people in
in the world, like these guysare deeply technical.
But there's there's there's anew crop coming through of folks

(01:06:20):
who are just bringing a levelof of polish to the space that
we haven't seen before.
Yeah, uh, some of it comesaround.
Some of the stuff that ourfriend robbie from nas compares
talks about like there's a wholebunch of new nas companies
starting to come through in thein the marketplace as well, like
Ugreen, have started makingNASs Synology of course have
moved away from consumerstowards business, but they still

(01:06:43):
exist.

Speaker 1 (01:06:45):
I know we've been speaking for a long time, alex.
I just got one last questionfor you, really 42.
In your latest self-hostedpodcast it was your five-year
anniversary of the podcast.
I can't actually believe it'sbeen five, it just seems like
about three weeks ago that Ilistened to the first episode.

(01:07:08):
But you were talking about, youknow, what's happened over the
last five years and you werekind of wondering what's going
to happen over the next fiveyears With the rise of
decentralized and peer-to-peertechnologies like IPFS and
Matrix.
Do you see a future where moreself-hosters are moving towards
a fully decentralized internet,and what do you think would be

(01:07:29):
the pros and cons if that shifthappened and what you think
would?

Speaker 2 (01:07:32):
be the pros and cons if that shift happened?
It's an interesting questionbecause the self-hosted podcast
itself, our primary chatplatform, is discord, which I'm
aware is ironic.
It's not a self-hosted platform.
Jupiter broadcasting hosts amatrix server, but matrix is

(01:08:02):
it's a poorly optimizedapplication.
Let's let's put it that waywhen you're running it at scale,
um, it's most of its when youcreate a new, when you bring up
a client on your phone orwhatever, and it does the sync.
That's a single-threadedoperation, and so you can end up
, if you have 20 or 30 users allconnecting at the same time,
you need 20 or 30 threadsavailable for all of those syncs

(01:08:26):
to occur.
And probably the biggestimpediment to self-hosted stuff
at scale is just the cost of theinfrastructure.
This box on linode that we'rehosting matrix on right now is
generously provided to us bythem for the duration of their
sponsorship, which ended fairlyrecently.

(01:08:49):
So we are, we are moving that tocanada fairly soon, I think, um
, but there's that saying thatif you are the, if you're, if
you're not paying for it, youare the product, right, well,
with self-hosting that's nottrue, but somebody somewhere is
paying for it and, um, I thinkthat's probably the biggest

(01:09:09):
impediment to properlydecentralized infrastructure is
just the costs of hosting.
So that's why we go withDiscord, because I actually
think that, for me at least, thebenefits of furthering the
mission ie getting more peopleinterested in self-hosting, ie

(01:09:34):
getting more people interestedin self-hosting outweighs the
the you know, the pros and consoutweigh uh, not having a place
for us to gather and talk andand meet.
And one of the things actuallyabout unraid that's really stood
the test of time are yourcommunity forums, and so we've
had these ways of asynccommunication, decentralized.
I mean, it's not decentralized,it's a forum.
It's hosted by a company butit's not controlled by, uh, you

(01:09:56):
know, a massive corporation.
At least the unread forums.

Speaker 1 (01:10:00):
Yeah.
Um, like everything else seemsto be you know, I think there's
um a balance you can havebetween self hosted and
commercial products, you knowright.
It's hosted and commercialproducts.
You know right, it's aboutpragmatism.
It's striking the balance andusing using tools.
You know um, sometimes aself-hosted tool might be better
for what you want and sometimesa closed source tool might be

(01:10:24):
better for what you want.

Speaker 2 (01:10:25):
Until fairly recently , we saw that with uh, google
photos and self-hosted photoalternatives.
I wrote for Ars Technica backin, I think, 2020.
I did a whole comparisonarticle for them on self-hosted
photo tools.
Image didn't exist back then,but now you look at what image
can do with the image search andyou know I can literally just

(01:10:47):
search for blue car in my photosthat a google photos only
solution three years ago.
An image has come from nowherein the space of a couple of
years and built something thatis as good as google photos,
which is insane.
Um so, in terms of the trendsand where things are going, I

(01:11:08):
think we're going to see moreimages appear in different
spaces.
I'm hopeful that things likejellyfin catch up to plex,
although I don't know.
I don't know historically iftheir trajectory is proof that
that can happen, although I hopeI'm wrong so what do you think
plex has got that jellyfinhasn't.

Speaker 1 (01:11:26):
Then, alex, what would you like to see?
I'm not a plex user.
I use mb myself, but I want toswitch to jellyfin.
Yeah, it's all.

Speaker 2 (01:11:34):
It's all, clients that jellyfin is the problem.
Uh, that is jellyfin's problemnow.
Probably 18 months ago, on theself hosted podcast, we did a
jellyfin january challenge.
I fully expected jellyfin to benot ready for prime time
because that was my experience ayear or two prior.
However, I'm still running it.
It's still my primary mediaserver and on my NVIDIA Shield,

(01:11:56):
which is an Android-based client, the experience is pretty great
.
But if you jump to the Apple TV, the experience is mid.
It's the best way I can put it.
You can't do things like userswitching and a bunch of other
stuff.
It's just not up to par withthe Plex experience.
And then you have things likebecause Plex has the cloud

(01:12:19):
component, which has someprivacy concerns, of course, you
can't do things like remoteaccess and sharing servers quite
as easily without some kind ofputting the batteries in
yourself with a cloud VPS orwhatever that you want to do
with Jellyfin.
It's just a case of differentbusiness models and you look at
where plex is trying to extractthe value versus jellyfin being

(01:12:41):
a free and open source projectwith people contributing their
time freely to develop it.
You know, it's just.
They're both products ofdifferent sides of the
capitalist coin, really, and I'mso glad that jellyfin exists
and I really hope it continuesto go from strength to strength,
but at the moment it's probablyprobably just a few steps
behind plex and probably alwayswill be.

(01:13:02):
I don't know.

Speaker 1 (01:13:03):
Hopefully not but at least they don't have plex's own
tv shows injected into the ui.
That's um, that was, yeah, onething that I don't like about
plex pretty egregious.
Yeah, anyway, alex, I'm notgoing to take up any more of
your time.

Speaker 2 (01:13:16):
Thank you so much for being so generous with your
time today, alex well, thank youto everybody for watching and
listening and thank you forhaving me and all the rest of it
, and maybe at some point we'llhave you on the podcast, huh
that would be absolutely awesome.

Speaker 1 (01:13:29):
it's been an incredible conversation.
I'm sure the listeners willhave a lot to take away from it.
How can people get hold of you,alex?
Obviously the self-hostingpodcast and you were saying that
Tailscale are looking for aEuropean advocate.
How would someone apply to beable to do that?

Speaker 2 (01:13:52):
Job listings will be at tailscalecom.
Slash careers, I think.
Uh, not quite live yet as werecord, although they might be,
but the time this airs I'm notsure.
Uh, you can find me atalexktzme.
I have a link tree there thatyou can go and all self-hosted,
by the way, a statically builtwebsite running out of a docker
container, of course, becausethat's how I roll.
Um, yeah, you can find more ofme at self-hostedshow

(01:14:15):
perfectmediaservercom.
Blogktzme.
Like I'm pretty easy to define.
Self-hostedshow slash discord.
Alex ktz over there.

Speaker 1 (01:14:23):
Yeah, lots of ways to find me my last question for
you is what's your dog's name,alex?

Speaker 2 (01:14:30):
well, he sat down here, actually right by my feet.
He's called archie, named afterarch linux yeah, there we are.

Speaker 1 (01:14:41):
Anyway, thank you very much, alex.
Um, you know, and for those ofyou listening, please make sure
to check out alex on theself-hosted podcast if you're
not already a listener, and keepan eye out for the official
tailscale integration coming toan unraid os near you soon.
So thanks again, alex, andthanks for all of our listeners
for tuning in.
Bye.

All Episodes

Episode Transcript

Popular Podcasts

Dateline NBC

24/7 News: The Latest

Therapy Gecko

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}Tailscale Magic

Episode Transcript

Popular Podcasts

.css-r6mb8g{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:1;overflow:hidden;}Dateline NBC

24/7 News: The Latest

Therapy Gecko

All Episodes

Tailscale Magic

Dateline NBC