September 6, 2023 • 32 mins
Join Eveline Oehrlich and Max Howell, CEO of tea.xyz and creator of Homebrew, to discuss open source including "the Nebraska problem," challenges, and more.
Max Howell is the CEO of tea.xyz and creator of Homebrew. Brew was one of the largest open source projects of all time, with tens of millions of users to this day. He has set out to make tea: a new, better package manager than brew that utilizes web3 technology to remunerate value back to open source devs.
The Humans of DevOps Podcast is incredibly grateful to be voted one of the Best 25 DevOps Podcasts by Feedspot.
Learn more about getting DevOps certified at devopsinstitute.com
Max Howell is the CEO of tea.xyz and creator of Homebrew. Brew was one of the largest open source projects of all time, with tens of millions of users to this day. He has set out to make tea: a new, better package manager than brew that utilizes web3 technology to remunerate value back to open source devs.
The Humans of DevOps Podcast is incredibly grateful to be voted one of the Best 25 DevOps Podcasts by Feedspot.
Learn more about getting DevOps certified at devopsinstitute.com
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Narrator (00:02):
You're listening to the humans of DevOps podcast, a
podcast focused on advancing thehumans of DevOps through skills,
knowledge, ideas, and learning,or the skil framework.
Max Howell (00:17):
We see this all over the place with open source. I
think a lot of the time thepeople who make successful open
source, making a hobby project,some kind, something they need,
and then it becomes an essentialpiece of internet
infrastructure.
Eveline Oehrlich (00:33):
In December of 2004, a group of IIT students
from DAP University inCopenhagen has launched a open
source beer, the one that aversion was described by those
students as a great tasting,energetic beer, and its world
first open source beer. It'sbased on a classic ale brewing
(00:57):
tradition. But with addedGuarana for a natural energy
boost. The aim was to see whathappens when an open source
structure is applied to auniversally known product like
beer. Hello there. This isevolutionarily ish. We are not
here to talk about beer. Butwe're here to talk about open
(01:21):
source. And I brought thisintroduction because I think it
fits in multiple ways you willfind out yourself. But I'm a
beer Brewer by Hobby so thatcaught my attention. Our podcast
title today is open source,Abreu and t. Now a few more
details, which is very different97% of commercial code contains
(01:45):
open source and I actuallylearned this from our guest.
When looking at some of the techtrends such as containerization,
like Docker and Kubernetes,originally developed at Google
and released as open source in2014. Big data like Apache Spark
with Hadoop or Kafka, a lot ofopen source is out there. Again,
(02:08):
as I said, Welcome to humans ofDevOps podcast, I'm Avalon early
Chief Research Officer at DevOpsInstitute. Today, we have a very
special guest, and I'm excitedto have him with us for multiple
reasons. Max Howell, CEO of Tdot x, y, z. Hello, Max. Welcome
(02:31):
to our podcast.
Max Howell (02:33):
Thank you. I'm pleased to be here.
Eveline Oehrlich (02:35):
Yes, excited for you to be with us. Tell us
the first thing really I'mcurious about is your journey
into an across open source.
Max Howell (02:48):
Yeah, well, it's a fairly good story. So I did a
chemistry degree because Ithought I wanted to be a
scientist. And in the process ofdoing that degree, and then
doing a year in the industry, Idiscovered that actually, it was
really not for me, I was workingwith this device that measured
(03:11):
surface tension for like variouschemical solutions. And I
realized that if I kept workingthere, like in 10 years, I would
still be using that singlemachine and making very minor
contributions to the world. So Ifell into this sort of
depressing funk. And Idiscovered open source.
Programming was always somethingthat I had considered a hobby.
(03:35):
My dad taught me when I was six,which, you know, very young, but
I only used it to like makevideo games essentially, like
most kids do when they learnedto program. And I never really
considered it for a career untilmy career was on the rocks. And
I didn't know what I was doingmyself. So propelled into open
source by installing Linux. AndI loved the community, I loved
(03:59):
how everyone was superpassionate about what they were
doing, and just doing it becausethey wanted to do something that
changed the world or improvedtheir workflow or just just
making cool things. And I endedup working on a few apps with
people from all over the worldand it was just like nothing
else. For me. Honestly, it's,it's difficult to find it as
(04:24):
good as it was when I first gotinto it. And we were all on IRC
together. And doing that led meto getting a job in the
industry, which you know, it'sit's kind of luck in many ways.
But there was a company inLondon that used one of the apps
I was working on and invited meto go and interview and it
wasn't really as much of aninterview as just having a chat.
(04:48):
So I actually got myself intothe industry. And a few years
later after that, you know, Ikept doing open source and so I
created homebrew, which is oneof the biggest open source
projects of all time. At thispoint, and I'm sure you'll have
some more questions about that.
Eveline Oehrlich (05:05):
Yep. Yep, keep going, because I will come,
we'll come back to that.
Max Howell (05:10):
So I created homebrew because at the time, we
were making all these differentapps. I was at last firm in
London, and we made sixdifferent apps. One was for
Linux and Mac and Windows, wehave an iPhone app, and an
Android app, and even thisBlackberry app. And we we build
(05:33):
them all on Mac, because Mac waslike the unified platform, time,
and what it really has becomethe platform for development
now. But this was 2008. And atthat time, like, developers
hadn't really decided whatplatform they're using, it was
all over the place, it was stillstill a lot of people using
(05:55):
Windows for development, whichis not as common nowadays. And
in the our office was an awfullot of people on Linux, and I
was one of the few who convertedto Mac. Because Apple had a bad
rep. With developers. It's hardto believe now. But well, it's
easier to believe over the lastfew years. But there was
(06:15):
certainly a period whereeveryone in development used to
back. But that was new. And wewere using it because it was the
platform that you could doAndroid Dev, you could do Linux
Dev, you could have a Windows VMrunning. So we could do all six
platforms quite easily. And likethe package management solutions
were the acceptable, theyweren't great. And they
(06:36):
certainly weren't designed fordevelopers, their impact
managers turned out with Linux.And in many ways, they were the
thing which defined the Linuxflavor. And it's still still the
case. But I kind of felt thatthey were designed for sis ops
and DevOps, but not development.So I wanted to build something
(06:57):
that was more for that. And so Igot on with it. And after a few
months, I realized that it waskind of neat. So I open sourced
it. And then it took a fewmonths before anyone noticed it.
But then when he got noticed, ittook off, like amazingly
quickly, huge amounts ofcontribution, but a lot of
(07:17):
excitement, because I've managedto tap into something which
people wanted, needed,especially with this burgeoning
developer platform, which waswhat the Mac was becoming. And
yeah, so here we are, like,almost 14 years later. And yeah,
it's it's an enormous project.At this point. It's very hard to
meet people that haven't heardof it or used it.
Eveline Oehrlich (07:40):
You know, I have watched your speak at Web
Summit in 2022. And you talkedabout something there called the
Nebraska problem. And Andhonestly, I've been in
infrastructure operations for along time, but I'm not a
(08:00):
developer. So maybe that's whyI've never really heard. I've
understood, I've heard of it,but I really don't understand or
didn't understand what itactually meant. So can you give
us a quick explanation of whatis meant by the Nebraska
problem? And how it relates tothe open source? And if
possible, any examples you haveseen recently? Maybe?
Max Howell (08:25):
Yeah, sure. So what we call the Nebraska problem is
the open source funding problem.And we call it that because
there's this famous XKCD comic,you'll see it trotted out
whenever there's any discussionabout open source funding
issues, and represent it showslike this tower of blocks, and
(08:48):
it represents all modern digitalinfrastructure. And then near
the bottom of the towers isprecarious, little pillar, and
it's holding the whole thing up.And there's an arrow pointing to
it, which says, this open sourcepackage maintained by some
someone from Nebraska, since2003, thanklessly. Are a
(09:12):
suddenly experienced this withhomebrew, like, it was kind of
the first time I'd reallyexperienced it, because even
though I'd done some relativelysuccessful open source before
that, well, you know, I did workfull time on a few projects
before that, but I was I didn'thave a job and didn't know what
I was going to do with myself. Iwas still living with my
(09:33):
parents. So it was to do it.With homebrew, I had to have two
full time jobs. One was paid andone which didn't. And I took
some time off here and there soI could work on homebrew full
time. But I never had enoughmoney saved up to work on it for
a long period of time, but Icouldn't abandon it because it
was well it was my easily at thetime like my greatest creation.
(09:54):
I was extremely proud of it butalso people depended on it. And
I Couldn't let them down, I justdidn't see how I could. So I
work at, you know, my officejob, and often was doing some
homebrew stuff to be honest. Andthen in the evenings and the
weekends, yeah, I didn't reallyhave a social life and just
worked on homebrew. And that waslike, wanted or needed, but it
(10:20):
wasn't sustainable. And in theend, I did burn out on it. And I
haven't really worked onhomebrew since 2016. I passed it
to the community, which, youknow, I was, I'm lucky that I
could, that suddenly otherpeople turned up. And this is
how open source is and how itshould work. Of course,
community is essential to opensource. And so it was good that
(10:40):
I could hand it over, but I canpersonally keep working on it.
And we see this all over theplace with open source, I think
a lot of the time the people whomake successful open source are
just making a hobby project forsome kind, something they need.
And then it becomes an essentialpiece of internet
infrastructure. And they'restuck in a situation where they
(11:02):
either like, abandon it. Andthen that's not what they want
to do. They can't afford to workon it as much as they need to.
While they try to find funding,and let's face it most most
funding doesn't doesn't workvery well for open source. A lot
of the time then Nebraskaprojects is what we call them,
(11:23):
like famous one couple of yearsago was locked for J which is a
logging piece of loggingsoftware for Java applications.
And there was a massive exploitfound in it where you could root
servers just by typing stuff inif the stuff you talked to him
would go through the loggingsoftware in some capacity. And
(11:43):
no one had really heard of thispackage, because it was deep in
the stack. And the soundsthemselves often works, right.
It's like these essential piecesof software have been built. But
over time, they've just beenburied on the things that are
built on top of them. And yougot really mature pieces of
software, which still need to bemaintained and everyone's
forgotten exist. A lot of forJay was a great example of that.
(12:05):
And they fixed the bug. And theyasked that maybe they could get
some funding in future so thatthey could afford to spend more
time actually maintaining thesoftware and maybe making sure
it doesn't have these securityholes. And I don't think they
ever received anything, becauseonce the bug was fixed, everyone
forgot about them again. Andthen more recently, there was
(12:26):
core J S, nine D in downloads inis since it was released, the
core of the base of every nodeapp that exists, every you know,
everything was built on Node oruses Node, and an essential
piece of infrastructure really,but the person who maintains it
(12:47):
is giving up because they'rejust fed up with the fact that
they don't get any funding.Everyone uses this thing.
Everyone is extremely entitledabout how they treat open
source. I'm afraid to say it'soften true. And he's just like,
well, I can't afford it I'mgiving up. And what's going to
happen, I don't know, it's not agood thing to use at the
(13:09):
infrastructure of and, you know,I, I've done loads of open
source. And the truth is, Ireally should just been doing
open source all these years likeI created one homebrew with the
small amount of free time I had.People I myself shouldn't have
to make a choice between workingon software that improves the
(13:33):
world and is beneficial to likeall layers of the software
stack. Or some of the some ofthe jobs I did were not
particularly beneficial to theworld, really. So that's the
Nebraska problem.
Narrator (13:48):
Skill up it learning is the perfect online
destination to learn aboutDevOps and digital
transformation, anytime,anywhere. Our digital learning
platform provides you with allthe resources you need to
upskill and learn about theseimportant topics, including
expert led course videos, accessto certification, prep courses,
(14:08):
and a community of supportivepeers. Our subscription based
model makes it easy for you togain the skills needed for
success in the modern workplace.Visit DevOps institute.com now
and explore our differentlearning plans.
Eveline Oehrlich (14:21):
Wow, wow, quite quite involved. quite
complicated. And when youstarted out just said, you know,
this noble cause of contributingto the world really with what
you have done with homebrew.We'll get to T XYZ in a minute.
But I wanted to ask a coupleother questions. Because of the
(14:43):
topic of open source, there'sstill some people seem to not
understand that perspective. Sothat's why I was very, very
interested in your thoughts andyour journey there. Let's switch
a little bit to the benefits interms of of leveraging open
source. You know, take it fromeither the benefits into from a
(15:06):
developer perspectivecollaborating and into the open
source community, or for thebenefits of the community over
in its large meaning the worldright. Talk a little bit to us
about what you see the benefitsare of open for open, not open
force, but open source.
Max Howell (15:27):
Yeah, open source is really interesting. I don't
think that human history isreally got any other equivalent
examples. Or markets usually uphere, because there's an
economic incentive there. Like Idon't know if anything else like
this. But I've been thinkingabout this quite a lot recently.
(15:48):
And I remember how in the 90s,the entire software stack was
basically owned by Microsoft.And, you know, there was
numerous issues with this. Butcertainly, as the Internet
became a thing, it became moreand more clear that Microsoft
was holding the Internet back,because Internet Explorer was
(16:08):
the only browser that anyoneused, and they didn't want to
advance it. They didn't care,too, there are no real
incentive. And it was this kindof attitude, which led to open
source taking more and more of ahold. In the end, like now, it's
difficult to imagine a worldwhere open source isn't the base
of every stack, like we say, 97%of all commercial software has
(16:31):
open source involved in it insome capacity, and like, this is
a really free percent thatdoesn't, I'd like to see what
those source bases andunderstand how they get away
with not having any open sourceat all. It Over time we, we
built the open source app, andwe swapped off the Microsoft
(16:53):
stack. And it became like thepoint where developers realized
that they not only gained likeall this functionality for free,
but also they were getting thesestep ups in productivity. And
proficiency is open source isusually designed to be like
these tiny little self containedpieces that you slop together,
and you build on top of eachother. And then you've got all
(17:15):
this slight advancedfunctionality. But in the
process of replacing Microsoftwith open source, we never
transferred all that money thatmakes over making while they
were trillions of dollars in.And instead, we just sort of
ended up with Freebase withoutreal any maintenance on it,
which, you know, we're trying tosolve AT T realized the the
(17:38):
value of open sources. Peoplethrow out ideas and the good
ones stick, there's no marketingbehind it. There's no like
someone's trying to sell yousomething that actually they
know isn't a good fit for you.You put some open source out
there. And if it works for thedevelopers of this world, they
(18:02):
adopt it? Well, this is sort ofpeople's world, they adopt it.
And then over time, if itcontinues to fit, it builds
until it becomes like a matureblock in the stack.
Eveline Oehrlich (18:17):
So with that there are challenges as well.
What are some challenges youwould say? Again, from multiple
perspectives, take it from thedeveloper, take it from the
community or take it from thesoftware world of vendors out
there?
Max Howell (18:39):
Well, I think vendors have the highest risk
right now, right? But if you'rebuilding an app, and it has 6000
dependencies, either, there's noway you can vet that. And be
sure that all those dependenciesare secure. First off, also
without any malicious insertionsof any kind. It's one thing
(19:00):
we're going to try and fix withwhat we're doing at T. But I do
wonder how, you know, if I hadto report to someone and say
that our app is secure, we wecan guarantee that I don't know
how I'd say that with a straightface 6000 depths in there. So
there's a lot of risk there forvendors. I think developers,
(19:25):
most of them just enjoy opensource existing because it
really does make computing moreenjoyable in general, especially
now. Like 14 years ago when Imade homebrew, like the amount
of people doing open sources somuch smaller. And now you can't
go a day without dozens of newthings being released
(19:46):
continuously. And some of thoseare great. But in most
developers feel bad that theyjust consume this open source
and don't give back but I don'tsee how they can effectively
give back right because you cantry and spot through a few open
source steps here and there, butwe're all using 1000s and 1000s
of open source projects. Andit's just infeasible without
(20:07):
some layer of automation tocontribute to those properly.
But open source developers arethe ones that I feel for but you
know, most of them are just doit, they start off by doing side
projects, it really is, it'slike, it's one of the beautiful
things about softwareengineering is that it's so
cheap and easy. All you need istime, that if you're gonna lay
(20:29):
there, and you got some time,you can, you can create some
open source. But what happenswith all the open source devs,
I've talked to who are, have gotsuccessful projects is that
after a while, it's a hugeamount of work. And you just
can't abandon that. You have tosomehow find the time to do it.
(20:56):
People on myself, we've, the wayI approached, it was always to
try and make it so that as muchas possible, it was robust. And
then but if you did break, itwould tell the user what they
can do to try and fix itthemselves. And they'll make
sure the documentation was asgood as I can get it. And that
would reduce the burden on me asmuch as I could, as much as
(21:17):
possible. And then, you know, ifyou if you're lucky, a community
turns up, mostly open source,I've done communities that turn
up, homebrew was
Eveline Oehrlich (21:24):
really quite an exception. Hmm, fascinating.
Absolutely fascinating. Soagain, at the Web Summit, you
said, I love this vision youpainted, you said, you want to
support those who are passionateon open source development, but
make sure that they getrewarded. And you actually call
it I love this to digitalrevolution for developers. What
(21:46):
do you mean by that? And thenwe'll get to T XYZ, because I
think there is a relationship.So tell us about the digital
revolution for the developers?
Max Howell (21:58):
Well, let's face it, in the sense of web two, as we
call it, which was, you know,mid 2000s, till, kind of
recently, I'd say, a huge amountof money was made by people
building on top of open source.And it was gradual, you know,
like the way open source becameso essential to what we're
(22:20):
building and software wasgradual. And I think a lot of
these essential pieces, if theywere built by hackers who were
building them for other hackers,and then when web two turned up
and figured out how to monetizeall that, there wasn't any
transition of wealth. And sothat's why it's a digital
revolution. These huge, megabillion dollar companies that
(22:45):
have never really given anysubstantial amount to the open
source that they depend on. It'snot right, like we were okay
with it as it happened, becauseit was gradual. But now we like
hanging on. You know, why am Iworking two full time jobs when
Microsoft has billions andbillions of dollars coming in
(23:07):
every month? Doesn't make sense.So what we're trying to build a
T is a way to successfullyremunerate how open source is
funded. There's been lots ofattempts, sponsorship, bounties,
things like this, but we seethat as being reward favorites.
(23:30):
And some people make a lot ofmoney with sponsorship. But the
truth is, there's millions ofopen source packages. And most
of them are Nebraska projectsthat nobody knows about. React
doesn't need to be funded,because Facebook, pay for it
essentially, which is anotherthing we want to fix instantly.
I don't believe that these hugecorporations should have so much
(23:51):
agenda over how these opensource projects that the
community needs to figure outwhat makes sense for open
source. Facebook could just likeclose down react tomorrow. I'd
want with what we're building atee is for these developers at
Facebook and Microsoft who dowork on open source or who are
(24:13):
just extremely talented peopleto be able to quit and work on
open source full time becausethe pay is exactly the same or
better. Even, frankly, we'rebuilding an economy on code. And
if you make a bunch ofsuccessful open source packages,
providing you're using the Tprotocol, you could be much
wealthier than you are currentlyas a software developer.
Eveline Oehrlich (24:35):
Interesting. So how can developers get
involved if I am sure we have afew of those who are going ooh,
I love this. I love hearingthis. How can they get involved?
Max Howell (24:49):
Well, four months ago I released the tea package
manager so tea is the successorto brew you know I've stopped
working on brew in mid to 2010s.And I didn't stop thinking about
what could be better aboutbroom. But I didn't really have
(25:09):
a good incentive I feel to makeanother brutal I'd already done
it. And I remember the tirelesshours that I spent working on it
robust, defying it, filling outthe package graph, answering
people's support questions, andjust like trying to make sure it
was a successful and usefulpiece of software. But I didn't
(25:29):
really want to go and do thatagain. I think I like doing
things once usually, but But 18months ago, while I was looking
into web three, and crypto,which basically was the first
time I've looked into thisstuff, because I've never been
particularly interested in it. Ithought Bitcoin was pretty
impressive. As a wonderfulmystery story to it also pretty
(25:55):
ingenious, how it works, and howit has successfully become so
large, even though it's notcontrolled. But I never really
thought that there was much morethat was interesting there for
me. But as I was diving into webthree stuff, I started to see
the different contracts wereactually pretty interesting, I
had this moment of inspirationwhile messing around an open sea
(26:18):
with NF Ts. Because NF T'sallowed you to well view the
open sea digital contract anywayforced any repurchases to put
10% back to the originalcreator. I was like, Oh, so you
can write a digital contractthat funnels money automatically
to different entities. And Irealized suddenly that the
(26:38):
package graph of all open sourcewas that kind of, if you put a
digital contract on it anyway,you could funnel money to all
the dependencies, all thepackages. So all these Nebraska
projects could get like littlebits of token from any insertion
of token near the top. Andthat's how it works, right? Like
people sponsor these bigprojects of near the top. And
(27:00):
it's all the ones underneaththat that are failing, have are
maintained by people without anythings. And they're essential
pieces of the digitalinfrastructure. Without them,
everything would collapse. Ilike everything being that
fragile is scary, like peopleremember that for a long time?
(27:20):
How it could all collapse anymoment, but somehow it doesn't.
And we're just waiting for abigger claps really. But also,
like, think about what wouldhappen if it was all funded
correctly. People myself, likemake, like occasionally big and
important projects. But we haveto do it in our spare time. What
if I was working on that fulltime? And then, you know, we
(27:44):
find security holes and open SSLevery two months? Well, if it
was funded correctly, so thatwell, either of us so could
really afford to hire people whocould, you know, like kind of
like a corporation is what we'rethinking things light, each of
these big projects will havetheir own dowels on the T
(28:05):
protocol. And they can operatethem. Similar to a kind of
corporation, they can hirepeople using the token, they can
split the rewards that they'regetting from the tee protocol
across everyone is contributingto open SSL. So if you open a
pull request, you could end upgetting paid for that pull
(28:27):
request. Because the DAO is aset of smart contracts that
figure out how to channel thetoken based on the governance
structure that those projectshave chosen. CL I came up with
the idea, like build a packagemanager and build it on top of
blockchain. So that open sourcecan be remunerated, then we
(28:48):
managed to raise 80 million, andwe've been building it since. So
I released the package manageritself four months ago, got six
sales and styles now got somereally passionate users. I think
it's special. And if anyone isinterested in getting involved
with what we're doing, that'sthe that's the place to start.
Currently, what we got isgithub.com/t. XYZ and you can
(29:12):
check it out, install it, and Ithink it's a better package
manager the brew it's, it's morethan a package manager, really,
the ideas I had over the yearsfor how the package manager
could be improved. I realizedthat it's a unique piece of
software, right? It sitsunderneath all other tooling.
There's so many things that youcan do, which nobody's really
(29:35):
tried before. I don't really getit I think perhaps package
managers just aren't very sexy.Is the truth and don't many
people that are interested inworking on them but for some
reason I've always enjoyed it. Ilove getting into that sort of
area of it.
Eveline Oehrlich (29:50):
It is a brilliant idea. You are
brilliant. I am honored to talkto you. If I come up with a name
for this that I will share itwith You but it is. It's
certainly a super cool andabsolutely honorable, really
honorable thing you're doing. Ilove the idea. I'm, I'm very
grateful. Now I have one closingquestion. Hmm, maybe it involves
(30:17):
beer drinking, but what do youdo for fun? What do you do for
fun if you don't do thesebrilliant things and manage
these fantastic things? You areYou were you were talking about?
Max Howell (30:30):
Yeah, well, so like, programming was always a
hobbies. Switch when I switch todoing it for career did become
kind of what I did one as well.But certainly, you know, I used
to live in the UK. I live in theStates now. And I used to love
going to pubs friends. So therewas some beer drinking in my
(30:50):
palace for sure. But I got a sonnow he's 10 months. And so
there's certainly less beer inmy life at the moment.
Otherwise, for fun, so Blimey, Ido like video games here and
there on Sunday not playing themas much as I used to. And I like
(31:10):
hiking or going out or campingisn't some of the things I
really enjoy.
Eveline Oehrlich (31:16):
Excellent. Well, if you make it to my part
of the world, I'm happy to takeyou out for beer. We got some
pretty good beer in Germany. Soyes, yes. Well, this has been
fantastic. Thank you so muchMax. We have been talking to max
Howell CEO of T dot XYZ andbrilliant thought leader in the
(31:37):
topic of open source Mexican,thank you so much for joining me
today on humans of DevOpspodcast. Thank you very much.
Humans of DevOps podcast isproduced by DevOps Institute.
Our audio production teamincludes Julia pape, Daniel
Newman, Schultz and Brendan Lee,shout out to my colleagues who
(31:58):
do a fantastic job. I'm humansof DevOps podcast, Executive
Producer, evolutionarily, if youwould like to join us on a
podcast, please contact us athumans of DevOps podcast at
DevOps institute.com. I'mevolutionarily talk to you soon.
Narrator (32:19):
Thanks for listening to this episode of the humans of
DevOps podcast. Don't forget tojoin our global community to get
access to even more greatresources like this. Until next
time, remember, you are part ofsomething bigger than yourself.
You belong
You're listening to the humans of DevOps podcast, a
podcast focused on advancing thehumans of DevOps through skills,
knowledge, ideas, and learning,or the skil framework.
Max Howell (00:17):
We see this all over the place with open source. I
think a lot of the time thepeople who make successful open
source, making a hobby project,some kind, something they need,
and then it becomes an essentialpiece of internet
infrastructure.
Eveline Oehrlich (00:33):
In December of 2004, a group of IIT students
from DAP University inCopenhagen has launched a open
source beer, the one that aversion was described by those
students as a great tasting,energetic beer, and its world
first open source beer. It'sbased on a classic ale brewing
(00:57):
tradition. But with addedGuarana for a natural energy
boost. The aim was to see whathappens when an open source
structure is applied to auniversally known product like
beer. Hello there. This isevolutionarily ish. We are not
here to talk about beer. Butwe're here to talk about open
(01:21):
source. And I brought thisintroduction because I think it
fits in multiple ways you willfind out yourself. But I'm a
beer Brewer by Hobby so thatcaught my attention. Our podcast
title today is open source,Abreu and t. Now a few more
details, which is very different97% of commercial code contains
(01:45):
open source and I actuallylearned this from our guest.
When looking at some of the techtrends such as containerization,
like Docker and Kubernetes,originally developed at Google
and released as open source in2014. Big data like Apache Spark
with Hadoop or Kafka, a lot ofopen source is out there. Again,
(02:08):
as I said, Welcome to humans ofDevOps podcast, I'm Avalon early
Chief Research Officer at DevOpsInstitute. Today, we have a very
special guest, and I'm excitedto have him with us for multiple
reasons. Max Howell, CEO of Tdot x, y, z. Hello, Max. Welcome
(02:31):
to our podcast.
Max Howell (02:33):
Thank you. I'm pleased to be here.
Eveline Oehrlich (02:35):
Yes, excited for you to be with us. Tell us
the first thing really I'mcurious about is your journey
into an across open source.
Max Howell (02:48):
Yeah, well, it's a fairly good story. So I did a
chemistry degree because Ithought I wanted to be a
scientist. And in the process ofdoing that degree, and then
doing a year in the industry, Idiscovered that actually, it was
really not for me, I was workingwith this device that measured
(03:11):
surface tension for like variouschemical solutions. And I
realized that if I kept workingthere, like in 10 years, I would
still be using that singlemachine and making very minor
contributions to the world. So Ifell into this sort of
depressing funk. And Idiscovered open source.
Programming was always somethingthat I had considered a hobby.
(03:35):
My dad taught me when I was six,which, you know, very young, but
I only used it to like makevideo games essentially, like
most kids do when they learnedto program. And I never really
considered it for a career untilmy career was on the rocks. And
I didn't know what I was doingmyself. So propelled into open
source by installing Linux. AndI loved the community, I loved
(03:59):
how everyone was superpassionate about what they were
doing, and just doing it becausethey wanted to do something that
changed the world or improvedtheir workflow or just just
making cool things. And I endedup working on a few apps with
people from all over the worldand it was just like nothing
else. For me. Honestly, it's,it's difficult to find it as
(04:24):
good as it was when I first gotinto it. And we were all on IRC
together. And doing that led meto getting a job in the
industry, which you know, it'sit's kind of luck in many ways.
But there was a company inLondon that used one of the apps
I was working on and invited meto go and interview and it
wasn't really as much of aninterview as just having a chat.
(04:48):
So I actually got myself intothe industry. And a few years
later after that, you know, Ikept doing open source and so I
created homebrew, which is oneof the biggest open source
projects of all time. At thispoint, and I'm sure you'll have
some more questions about that.
Eveline Oehrlich (05:05):
Yep. Yep, keep going, because I will come,
we'll come back to that.
Max Howell (05:10):
So I created homebrew because at the time, we
were making all these differentapps. I was at last firm in
London, and we made sixdifferent apps. One was for
Linux and Mac and Windows, wehave an iPhone app, and an
Android app, and even thisBlackberry app. And we we build
(05:33):
them all on Mac, because Mac waslike the unified platform, time,
and what it really has becomethe platform for development
now. But this was 2008. And atthat time, like, developers
hadn't really decided whatplatform they're using, it was
all over the place, it was stillstill a lot of people using
(05:55):
Windows for development, whichis not as common nowadays. And
in the our office was an awfullot of people on Linux, and I
was one of the few who convertedto Mac. Because Apple had a bad
rep. With developers. It's hardto believe now. But well, it's
easier to believe over the lastfew years. But there was
(06:15):
certainly a period whereeveryone in development used to
back. But that was new. And wewere using it because it was the
platform that you could doAndroid Dev, you could do Linux
Dev, you could have a Windows VMrunning. So we could do all six
platforms quite easily. And likethe package management solutions
were the acceptable, theyweren't great. And they
(06:36):
certainly weren't designed fordevelopers, their impact
managers turned out with Linux.And in many ways, they were the
thing which defined the Linuxflavor. And it's still still the
case. But I kind of felt thatthey were designed for sis ops
and DevOps, but not development.So I wanted to build something
(06:57):
that was more for that. And so Igot on with it. And after a few
months, I realized that it waskind of neat. So I open sourced
it. And then it took a fewmonths before anyone noticed it.
But then when he got noticed, ittook off, like amazingly
quickly, huge amounts ofcontribution, but a lot of
(07:17):
excitement, because I've managedto tap into something which
people wanted, needed,especially with this burgeoning
developer platform, which waswhat the Mac was becoming. And
yeah, so here we are, like,almost 14 years later. And yeah,
it's it's an enormous project.At this point. It's very hard to
meet people that haven't heardof it or used it.
Eveline Oehrlich (07:40):
You know, I have watched your speak at Web
Summit in 2022. And you talkedabout something there called the
Nebraska problem. And Andhonestly, I've been in
infrastructure operations for along time, but I'm not a
(08:00):
developer. So maybe that's whyI've never really heard. I've
understood, I've heard of it,but I really don't understand or
didn't understand what itactually meant. So can you give
us a quick explanation of whatis meant by the Nebraska
problem? And how it relates tothe open source? And if
possible, any examples you haveseen recently? Maybe?
Max Howell (08:25):
Yeah, sure. So what we call the Nebraska problem is
the open source funding problem.And we call it that because
there's this famous XKCD comic,you'll see it trotted out
whenever there's any discussionabout open source funding
issues, and represent it showslike this tower of blocks, and
(08:48):
it represents all modern digitalinfrastructure. And then near
the bottom of the towers isprecarious, little pillar, and
it's holding the whole thing up.And there's an arrow pointing to
it, which says, this open sourcepackage maintained by some
someone from Nebraska, since2003, thanklessly. Are a
(09:12):
suddenly experienced this withhomebrew, like, it was kind of
the first time I'd reallyexperienced it, because even
though I'd done some relativelysuccessful open source before
that, well, you know, I did workfull time on a few projects
before that, but I was I didn'thave a job and didn't know what
I was going to do with myself. Iwas still living with my
(09:33):
parents. So it was to do it.With homebrew, I had to have two
full time jobs. One was paid andone which didn't. And I took
some time off here and there soI could work on homebrew full
time. But I never had enoughmoney saved up to work on it for
a long period of time, but Icouldn't abandon it because it
was well it was my easily at thetime like my greatest creation.
(09:54):
I was extremely proud of it butalso people depended on it. And
I Couldn't let them down, I justdidn't see how I could. So I
work at, you know, my officejob, and often was doing some
homebrew stuff to be honest. Andthen in the evenings and the
weekends, yeah, I didn't reallyhave a social life and just
worked on homebrew. And that waslike, wanted or needed, but it
(10:20):
wasn't sustainable. And in theend, I did burn out on it. And I
haven't really worked onhomebrew since 2016. I passed it
to the community, which, youknow, I was, I'm lucky that I
could, that suddenly otherpeople turned up. And this is
how open source is and how itshould work. Of course,
community is essential to opensource. And so it was good that
(10:40):
I could hand it over, but I canpersonally keep working on it.
And we see this all over theplace with open source, I think
a lot of the time the people whomake successful open source are
just making a hobby project forsome kind, something they need.
And then it becomes an essentialpiece of internet
infrastructure. And they'restuck in a situation where they
(11:02):
either like, abandon it. Andthen that's not what they want
to do. They can't afford to workon it as much as they need to.
While they try to find funding,and let's face it most most
funding doesn't doesn't workvery well for open source. A lot
of the time then Nebraskaprojects is what we call them,
(11:23):
like famous one couple of yearsago was locked for J which is a
logging piece of loggingsoftware for Java applications.
And there was a massive exploitfound in it where you could root
servers just by typing stuff inif the stuff you talked to him
would go through the loggingsoftware in some capacity. And
(11:43):
no one had really heard of thispackage, because it was deep in
the stack. And the soundsthemselves often works, right.
It's like these essential piecesof software have been built. But
over time, they've just beenburied on the things that are
built on top of them. And yougot really mature pieces of
software, which still need to bemaintained and everyone's
forgotten exist. A lot of forJay was a great example of that.
(12:05):
And they fixed the bug. And theyasked that maybe they could get
some funding in future so thatthey could afford to spend more
time actually maintaining thesoftware and maybe making sure
it doesn't have these securityholes. And I don't think they
ever received anything, becauseonce the bug was fixed, everyone
forgot about them again. Andthen more recently, there was
(12:26):
core J S, nine D in downloads inis since it was released, the
core of the base of every nodeapp that exists, every you know,
everything was built on Node oruses Node, and an essential
piece of infrastructure really,but the person who maintains it
(12:47):
is giving up because they'rejust fed up with the fact that
they don't get any funding.Everyone uses this thing.
Everyone is extremely entitledabout how they treat open
source. I'm afraid to say it'soften true. And he's just like,
well, I can't afford it I'mgiving up. And what's going to
happen, I don't know, it's not agood thing to use at the
(13:09):
infrastructure of and, you know,I, I've done loads of open
source. And the truth is, Ireally should just been doing
open source all these years likeI created one homebrew with the
small amount of free time I had.People I myself shouldn't have
to make a choice between workingon software that improves the
(13:33):
world and is beneficial to likeall layers of the software
stack. Or some of the some ofthe jobs I did were not
particularly beneficial to theworld, really. So that's the
Nebraska problem.
Narrator (13:48):
Skill up it learning is the perfect online
destination to learn aboutDevOps and digital
transformation, anytime,anywhere. Our digital learning
platform provides you with allthe resources you need to
upskill and learn about theseimportant topics, including
expert led course videos, accessto certification, prep courses,
(14:08):
and a community of supportivepeers. Our subscription based
model makes it easy for you togain the skills needed for
success in the modern workplace.Visit DevOps institute.com now
and explore our differentlearning plans.
Eveline Oehrlich (14:21):
Wow, wow, quite quite involved. quite
complicated. And when youstarted out just said, you know,
this noble cause of contributingto the world really with what
you have done with homebrew.We'll get to T XYZ in a minute.
But I wanted to ask a coupleother questions. Because of the
(14:43):
topic of open source, there'sstill some people seem to not
understand that perspective. Sothat's why I was very, very
interested in your thoughts andyour journey there. Let's switch
a little bit to the benefits interms of of leveraging open
source. You know, take it fromeither the benefits into from a
(15:06):
developer perspectivecollaborating and into the open
source community, or for thebenefits of the community over
in its large meaning the worldright. Talk a little bit to us
about what you see the benefitsare of open for open, not open
force, but open source.
Max Howell (15:27):
Yeah, open source is really interesting. I don't
think that human history isreally got any other equivalent
examples. Or markets usually uphere, because there's an
economic incentive there. Like Idon't know if anything else like
this. But I've been thinkingabout this quite a lot recently.
(15:48):
And I remember how in the 90s,the entire software stack was
basically owned by Microsoft.And, you know, there was
numerous issues with this. Butcertainly, as the Internet
became a thing, it became moreand more clear that Microsoft
was holding the Internet back,because Internet Explorer was
(16:08):
the only browser that anyoneused, and they didn't want to
advance it. They didn't care,too, there are no real
incentive. And it was this kindof attitude, which led to open
source taking more and more of ahold. In the end, like now, it's
difficult to imagine a worldwhere open source isn't the base
of every stack, like we say, 97%of all commercial software has
(16:31):
open source involved in it insome capacity, and like, this is
a really free percent thatdoesn't, I'd like to see what
those source bases andunderstand how they get away
with not having any open sourceat all. It Over time we, we
built the open source app, andwe swapped off the Microsoft
(16:53):
stack. And it became like thepoint where developers realized
that they not only gained likeall this functionality for free,
but also they were getting thesestep ups in productivity. And
proficiency is open source isusually designed to be like
these tiny little self containedpieces that you slop together,
and you build on top of eachother. And then you've got all
(17:15):
this slight advancedfunctionality. But in the
process of replacing Microsoftwith open source, we never
transferred all that money thatmakes over making while they
were trillions of dollars in.And instead, we just sort of
ended up with Freebase withoutreal any maintenance on it,
which, you know, we're trying tosolve AT T realized the the
(17:38):
value of open sources. Peoplethrow out ideas and the good
ones stick, there's no marketingbehind it. There's no like
someone's trying to sell yousomething that actually they
know isn't a good fit for you.You put some open source out
there. And if it works for thedevelopers of this world, they
(18:02):
adopt it? Well, this is sort ofpeople's world, they adopt it.
And then over time, if itcontinues to fit, it builds
until it becomes like a matureblock in the stack.
Eveline Oehrlich (18:17):
So with that there are challenges as well.
What are some challenges youwould say? Again, from multiple
perspectives, take it from thedeveloper, take it from the
community or take it from thesoftware world of vendors out
there?
Max Howell (18:39):
Well, I think vendors have the highest risk
right now, right? But if you'rebuilding an app, and it has 6000
dependencies, either, there's noway you can vet that. And be
sure that all those dependenciesare secure. First off, also
without any malicious insertionsof any kind. It's one thing
(19:00):
we're going to try and fix withwhat we're doing at T. But I do
wonder how, you know, if I hadto report to someone and say
that our app is secure, we wecan guarantee that I don't know
how I'd say that with a straightface 6000 depths in there. So
there's a lot of risk there forvendors. I think developers,
(19:25):
most of them just enjoy opensource existing because it
really does make computing moreenjoyable in general, especially
now. Like 14 years ago when Imade homebrew, like the amount
of people doing open sources somuch smaller. And now you can't
go a day without dozens of newthings being released
(19:46):
continuously. And some of thoseare great. But in most
developers feel bad that theyjust consume this open source
and don't give back but I don'tsee how they can effectively
give back right because you cantry and spot through a few open
source steps here and there, butwe're all using 1000s and 1000s
of open source projects. Andit's just infeasible without
(20:07):
some layer of automation tocontribute to those properly.
But open source developers arethe ones that I feel for but you
know, most of them are just doit, they start off by doing side
projects, it really is, it'slike, it's one of the beautiful
things about softwareengineering is that it's so
cheap and easy. All you need istime, that if you're gonna lay
(20:29):
there, and you got some time,you can, you can create some
open source. But what happenswith all the open source devs,
I've talked to who are, have gotsuccessful projects is that
after a while, it's a hugeamount of work. And you just
can't abandon that. You have tosomehow find the time to do it.
(20:56):
People on myself, we've, the wayI approached, it was always to
try and make it so that as muchas possible, it was robust. And
then but if you did break, itwould tell the user what they
can do to try and fix itthemselves. And they'll make
sure the documentation was asgood as I can get it. And that
would reduce the burden on me asmuch as I could, as much as
(21:17):
possible. And then, you know, ifyou if you're lucky, a community
turns up, mostly open source,I've done communities that turn
up, homebrew was
Eveline Oehrlich (21:24):
really quite an exception. Hmm, fascinating.
Absolutely fascinating. Soagain, at the Web Summit, you
said, I love this vision youpainted, you said, you want to
support those who are passionateon open source development, but
make sure that they getrewarded. And you actually call
it I love this to digitalrevolution for developers. What
(21:46):
do you mean by that? And thenwe'll get to T XYZ, because I
think there is a relationship.So tell us about the digital
revolution for the developers?
Max Howell (21:58):
Well, let's face it, in the sense of web two, as we
call it, which was, you know,mid 2000s, till, kind of
recently, I'd say, a huge amountof money was made by people
building on top of open source.And it was gradual, you know,
like the way open source becameso essential to what we're
(22:20):
building and software wasgradual. And I think a lot of
these essential pieces, if theywere built by hackers who were
building them for other hackers,and then when web two turned up
and figured out how to monetizeall that, there wasn't any
transition of wealth. And sothat's why it's a digital
revolution. These huge, megabillion dollar companies that
(22:45):
have never really given anysubstantial amount to the open
source that they depend on. It'snot right, like we were okay
with it as it happened, becauseit was gradual. But now we like
hanging on. You know, why am Iworking two full time jobs when
Microsoft has billions andbillions of dollars coming in
(23:07):
every month? Doesn't make sense.So what we're trying to build a
T is a way to successfullyremunerate how open source is
funded. There's been lots ofattempts, sponsorship, bounties,
things like this, but we seethat as being reward favorites.
(23:30):
And some people make a lot ofmoney with sponsorship. But the
truth is, there's millions ofopen source packages. And most
of them are Nebraska projectsthat nobody knows about. React
doesn't need to be funded,because Facebook, pay for it
essentially, which is anotherthing we want to fix instantly.
I don't believe that these hugecorporations should have so much
(23:51):
agenda over how these opensource projects that the
community needs to figure outwhat makes sense for open
source. Facebook could just likeclose down react tomorrow. I'd
want with what we're building atee is for these developers at
Facebook and Microsoft who dowork on open source or who are
(24:13):
just extremely talented peopleto be able to quit and work on
open source full time becausethe pay is exactly the same or
better. Even, frankly, we'rebuilding an economy on code. And
if you make a bunch ofsuccessful open source packages,
providing you're using the Tprotocol, you could be much
wealthier than you are currentlyas a software developer.
Eveline Oehrlich (24:35):
Interesting. So how can developers get
involved if I am sure we have afew of those who are going ooh,
I love this. I love hearingthis. How can they get involved?
Max Howell (24:49):
Well, four months ago I released the tea package
manager so tea is the successorto brew you know I've stopped
working on brew in mid to 2010s.And I didn't stop thinking about
what could be better aboutbroom. But I didn't really have
(25:09):
a good incentive I feel to makeanother brutal I'd already done
it. And I remember the tirelesshours that I spent working on it
robust, defying it, filling outthe package graph, answering
people's support questions, andjust like trying to make sure it
was a successful and usefulpiece of software. But I didn't
(25:29):
really want to go and do thatagain. I think I like doing
things once usually, but But 18months ago, while I was looking
into web three, and crypto,which basically was the first
time I've looked into thisstuff, because I've never been
particularly interested in it. Ithought Bitcoin was pretty
impressive. As a wonderfulmystery story to it also pretty
(25:55):
ingenious, how it works, and howit has successfully become so
large, even though it's notcontrolled. But I never really
thought that there was much morethat was interesting there for
me. But as I was diving into webthree stuff, I started to see
the different contracts wereactually pretty interesting, I
had this moment of inspirationwhile messing around an open sea
(26:18):
with NF Ts. Because NF T'sallowed you to well view the
open sea digital contract anywayforced any repurchases to put
10% back to the originalcreator. I was like, Oh, so you
can write a digital contractthat funnels money automatically
to different entities. And Irealized suddenly that the
(26:38):
package graph of all open sourcewas that kind of, if you put a
digital contract on it anyway,you could funnel money to all
the dependencies, all thepackages. So all these Nebraska
projects could get like littlebits of token from any insertion
of token near the top. Andthat's how it works, right? Like
people sponsor these bigprojects of near the top. And
(27:00):
it's all the ones underneaththat that are failing, have are
maintained by people without anythings. And they're essential
pieces of the digitalinfrastructure. Without them,
everything would collapse. Ilike everything being that
fragile is scary, like peopleremember that for a long time?
(27:20):
How it could all collapse anymoment, but somehow it doesn't.
And we're just waiting for abigger claps really. But also,
like, think about what wouldhappen if it was all funded
correctly. People myself, likemake, like occasionally big and
important projects. But we haveto do it in our spare time. What
if I was working on that fulltime? And then, you know, we
(27:44):
find security holes and open SSLevery two months? Well, if it
was funded correctly, so thatwell, either of us so could
really afford to hire people whocould, you know, like kind of
like a corporation is what we'rethinking things light, each of
these big projects will havetheir own dowels on the T
(28:05):
protocol. And they can operatethem. Similar to a kind of
corporation, they can hirepeople using the token, they can
split the rewards that they'regetting from the tee protocol
across everyone is contributingto open SSL. So if you open a
pull request, you could end upgetting paid for that pull
(28:27):
request. Because the DAO is aset of smart contracts that
figure out how to channel thetoken based on the governance
structure that those projectshave chosen. CL I came up with
the idea, like build a packagemanager and build it on top of
blockchain. So that open sourcecan be remunerated, then we
(28:48):
managed to raise 80 million, andwe've been building it since. So
I released the package manageritself four months ago, got six
sales and styles now got somereally passionate users. I think
it's special. And if anyone isinterested in getting involved
with what we're doing, that'sthe that's the place to start.
Currently, what we got isgithub.com/t. XYZ and you can
(29:12):
check it out, install it, and Ithink it's a better package
manager the brew it's, it's morethan a package manager, really,
the ideas I had over the yearsfor how the package manager
could be improved. I realizedthat it's a unique piece of
software, right? It sitsunderneath all other tooling.
There's so many things that youcan do, which nobody's really
(29:35):
tried before. I don't really getit I think perhaps package
managers just aren't very sexy.Is the truth and don't many
people that are interested inworking on them but for some
reason I've always enjoyed it. Ilove getting into that sort of
area of it.
Eveline Oehrlich (29:50):
It is a brilliant idea. You are
brilliant. I am honored to talkto you. If I come up with a name
for this that I will share itwith You but it is. It's
certainly a super cool andabsolutely honorable, really
honorable thing you're doing. Ilove the idea. I'm, I'm very
grateful. Now I have one closingquestion. Hmm, maybe it involves
(30:17):
beer drinking, but what do youdo for fun? What do you do for
fun if you don't do thesebrilliant things and manage
these fantastic things? You areYou were you were talking about?
Max Howell (30:30):
Yeah, well, so like, programming was always a
hobbies. Switch when I switch todoing it for career did become
kind of what I did one as well.But certainly, you know, I used
to live in the UK. I live in theStates now. And I used to love
going to pubs friends. So therewas some beer drinking in my
(30:50):
palace for sure. But I got a sonnow he's 10 months. And so
there's certainly less beer inmy life at the moment.
Otherwise, for fun, so Blimey, Ido like video games here and
there on Sunday not playing themas much as I used to. And I like
(31:10):
hiking or going out or campingisn't some of the things I
really enjoy.
Eveline Oehrlich (31:16):
Excellent. Well, if you make it to my part
of the world, I'm happy to takeyou out for beer. We got some
pretty good beer in Germany. Soyes, yes. Well, this has been
fantastic. Thank you so muchMax. We have been talking to max
Howell CEO of T dot XYZ andbrilliant thought leader in the
(31:37):
topic of open source Mexican,thank you so much for joining me
today on humans of DevOpspodcast. Thank you very much.
Humans of DevOps podcast isproduced by DevOps Institute.
Our audio production teamincludes Julia pape, Daniel
Newman, Schultz and Brendan Lee,shout out to my colleagues who
(31:58):
do a fantastic job. I'm humansof DevOps podcast, Executive
Producer, evolutionarily, if youwould like to join us on a
podcast, please contact us athumans of DevOps podcast at
DevOps institute.com. I'mevolutionarily talk to you soon.
Narrator (32:19):
Thanks for listening to this episode of the humans of
DevOps podcast. Don't forget tojoin our global community to get
access to even more greatresources like this. Until next
time, remember, you are part ofsomething bigger than yourself.
You belong
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Therapy Gecko
An unlicensed lizard psychologist travels the universe talking to strangers about absolutely nothing. TO CALL THE GECKO: follow me on https://www.twitch.tv/lyleforever to get a notification for when I am taking calls. I am usually live Mondays, Wednesdays, and Fridays but lately a lot of other times too. I am a gecko.