July 12, 2025 • 62 mins
Wolf talks about making your programs better. There are lots of ways to make them better. It all starts with figuring out what matters and measuring it. Measuring it all the time. Measuring it more. This episode is about following that path.
Show notes:
Take-aways from the episode:
- Understand what you are optimizing for: (speed,memory,storage,developer, etc…)
- Measurement is job one, because that’s the only way to know where the money is actually going. You should be measuring. A lot. More than that. It should be part of CI/CD. You should run it before pushing. Everyone should be doing it. Measurement might be even more important than testing (and don’t get me wrong, testing is very important). When I worked on Mozilla, our build servers did timing. If your commit slowed something down, that was considered “bustage”, and required immediate fixing.
- Use the profiler for two things:
- To see if the whole thing is faster or slower so you know when it’s time to look deeper
- To dive into the actual execution and locate the bad parts you need to improve.
- It’s all about the money.
- Write clear, simple, and correct (you’ll know by testing) code. Only then should you optimize. Do I need to repeat the old adage about premature optimization? “Premature optimization is the root of all evil.” It’s easier to speed up working code, than it is to fix fast but broken code.
- Understand the (real) data you will be operating on.
- You don’t know just by looking at the source what actually costs you the most money. Yes, you can see where stupid things happen, but even for those, knowing which actually matter requires measurement.
Hosts:
Jim McQuillan can be reached at jam@RuntimeArguments.fm
Wolf can be reached at wolf@RuntimeArguments.fm
Follow us on Mastodon: @RuntimeArguments@hachyderm.io
If you have feedback for us, please send it to feedback@RuntimeArguments.fm
Checkout our webpage at http://RuntimeArguments.fm
Theme music:
Dawn by nuer self, from the album Digital Sky
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Jim (00:03):
Welcome back to another episode of Runtime Arguments.
We are really excited abouttoday's episode.
Wolf is going to be talkingabout uh uh code performance and
where does the money go?
We'll be talking aboutprofiling, measurement, and
optimization.
Uh we are we are so grateful forthe listeners out there.
(00:26):
Uh they're providing greatfeedback.
We're enjoying doing this.
We're just gonna keep on doingit as long as you guys enjoy it.
Um this particular episodestarted uh the idea for it came
when I uh was doing the researchfor the WebAssembly episode.
That was several episodes ago.
Uh we were trying to useWebAssembly and trying to make
(00:47):
it fast and trying to compare itwith other languages, and the
idea came up that boy, wouldn'tuh wouldn't an episode on
optimization and measurement uhreally be a nice episode?
And Wolf was all over that.
He jumped right on it andstarted doing some research.
And he's I don't think he had todo a lot of research.
He knows this stuff really,really well.
(01:08):
I I think this one is probablypretty easy for him.
So that's what we're gonna betalking about today.
But before we do, uh I have justa very little bit of feedback
from our previous episode.
Uh that episode was on uh uhWolf, what was that episode?
The last one we did.
Wolf (01:26):
File systems.
Jim (01:27):
File systems, yeah.
How could I forget?
Because I did that one.
Anyway, we have a little bit offeedback on that.
Uh first of all, most of theresearch uh on that episode I
can't take credit for.
Uh, most of it was done by myfriend Ron.
Uh he and I took a long roadtrip up to Winnipeg uh back in
May, and maybe it was earlyJune.
(01:49):
Um to pass the time, Ron wasresearching file systems because
he was trying to figure outwhat's the best file system to
use for a server he was settingup.
So he was reading man pages outloud to me, and we were
discussing, and we learned allkinds of stuff about file
systems, and that's really whatwent into uh the previous
episode.
So if you haven't listened to ityet, go back and find it.
(02:11):
Uh it's it was just a couple ofweeks ago.
Uh we talked all about filesystems and stuff.
So that was pretty cool.
Uh, one of the things I failedto mention during that episode,
we did talk about uh theimportance of checksums in file
systems, newer file systems likeBDRFS, or that's BTRFS.
Um, they have something calledchecksums, and that's kind of
(02:35):
important because file systemsand hard disks and and any kind
of uh uh persistent storage cansuffer from something called bit
rot, and that is the the datacan degrade, uh especially for
magnetic uh material.
Uh the data can actuallydegrade, uh uh and so checksums
(02:55):
will help you detect that and insome cases correct for that.
So I wanted to make sure that uhI got that out there.
Wolf (03:02):
So that's kind of that actually that actually makes me
want to say something.
Uh I'm not going to give a fullexplanation, but this is kind of
a pointer for people who care.
Um there is a special kind ofCD, DVD, Blu-ray disc where
you're not recording onto uhorganic dye, you're recording,
(03:25):
you're actually etching ontominerals.
Um the minerals are rated tohold your data for 10,000 years.
The plastic that is the disc isonly rated for a thousand years.
Uh pretty much almost any uhnormal uh DVD, CD, Blu-ray
(03:48):
player can play these, um, and alarge fraction of sp of these
ordinary writers can write them.
Uh so the tool to write them isfairly cheap.
The discs themselves are veryexpensive, and they come in
sizes uh 4.7 gigabytes, justlike a DVD ROM drive, uh just
(04:11):
like a DVD ROM 25, 50, and 100gigabytes.
So yeah, they're not terabytesize like some things, but they
are gonna last for you at leasta thousand years, and all I care
about is maybe them lasting 20.
They are called M disks.
Um, and uh there are places tolearn more about them.
(04:35):
Um, it's a little hard toidentify which disks are the
right ones for you.
Uh a clue is how much they cost,but you do need to do a little
bit of research.
So that's where you shouldstart.
Okay.
Okay.
Jim (04:47):
That's a good point.
And you mentioned a thousandyears and ten thousand years.
That's for the fancy kinds ofmaterial you're talking about.
What are just typical DVD uh uhdiscs good for?
I mean the the you know the onesthat I buy at uh Best Buy aren't
gonna last that long.
Well, not long.
How a year, ten years?
Wolf (05:08):
If they last 10 years, you are incredibly lucky.
Um they ought to last at leastfour or five years, um, but
they're not nearly as good as weused to think they were.
Jim (05:21):
Yeah, so if you really have data that you want to store
reliably for long term, like athousand years or ten thousand
years, use these uh whatchallow,M drive?
M disks.
M disks, yeah.
You use those.
Okay, thanks.
Uh if you have uh any feedbackfor us on any of these episodes,
(05:41):
uh if you just want to suggesttopics, uh if you just want to
say hi, uh feedback atruntimearguments.fm will uh get
to us.
So please send us some feedbackand I'll repeat that information
at the end of the show, and it'sgonna be in the show notes uh
because we'd love to hear fromyou.
(06:01):
That's how we know people areactually listening.
So let's get on to the topic,eh?
Wolf, you ready?
Wolf (06:09):
First, let me say uh what I'm going to talk about.
Um I have four specific pointsthat I'm gonna mention here as
the uh the principal focus ofthis episode, but underneath it
all is the money.
Um every single thing you writeor run costs money somewhere,
(06:35):
somehow.
And almost certainly you want tospend as little of it as
possible, and for that spend,you want to get the most out of
it as possible.
Um, this is true for yourself,but it's really true for your
job job.
If you're running something inthe cloud or for a boss or
millions of people are runningit.
(06:57):
Anyway, here are uh the here isthe intro to what I'm gonna talk
about.
This whole episode is aboutmaking your programs better.
And when I say better, I mean alot of things, but better is a
good summary.
Um there's lots of things insideyour program that individually
(07:20):
can be improved.
Um, and a thing we're gonna lookat is which behaviors inside
your program can or should weoptimize.
Um we're gonna look at why is itimportant, and we're gonna look
at how do we actually do it.
Um so that's a list of thethings we're gonna talk about.
(07:46):
Okay.
The actual words I'm gonna saytoday are about programming, but
the underlying ideas um actuallyapply to a lot of things.
Uh almost any kind of processcan be optimized.
Almost any kind of process costsmoney.
(08:08):
Almost any kind of process um isgoing to yield benefits if you
examine it and figure out whereit can be improved.
So it might be hard to extractfrom the specific things I say
here, but the underlying uhtruth of this episode applies
(08:30):
much more broadly thanprogramming.
Um the uh I'm I I wrote downwords to specifically say about
this.
Uh a person is creatingsomething that will be useful to
others, and money is thelimiting factor.
(08:51):
It costs money to pay thatperson, it costs money to use
their creation, that money mightbe because of time, it might be
because of specific requirementslike location or needed
services.
Um this stuff is fundamental tocoding, but even if you're not a
a coder, um, it's gonna apply toyou.
(09:16):
So, what can we optimize?
Um couple top-level things.
Uh, programmer time and effort.
Um, that's probably a thing youcare about when you're doing it
on your own for your own self.
Um, and the questions uh uhabout that include things like
what environment is it going torun in?
(09:36):
Is it gonna be um some GUI thingon a desktop?
Is it gonna be inside a phone?
Is it gonna be on the commandline?
Is it gonna be a remote job inthe cloud?
Is it gonna be inside anembedded device?
All of those have differentcriteria about how you're going
to do your job, and that meanshow hard it's going to be for
(09:59):
you.
Um what languages are you, andmaybe you're in a team, uh, so
your team too, what languagesare you good at?
What languages are really goodat solving this specific
problem?
Um, and there's lots of ways toevaluate that.
Uh usually it's about whatlibraries are available in that
(10:22):
language.
Uh, but for some languages, it'sabout the actual features of the
language.
Does your intended environmentforce a specific language on
you?
If you need to hire people, whatlanguage uh choices lead to
enough available people for youto hire?
(10:43):
A lot of these choices areforced.
Jim (10:46):
So why does this all matter?
Uh, who measures this and howwould you track it?
I I've never heard of anybodyreally caring that much about
this.
Um what I do know is that speedis important, uh, especially for
big companies that do lots andlots of processing.
Wolf (11:04):
Uh so who measures it and and and who tracks it?
Programmers usually don't.
Managers almost always do.
And weirdly, it's usuallymanagers that do things like
pick the language you're gonnause and the database that's
gonna hold your data and do thehiring, decide which people know
(11:26):
the right things to work on thisproduct.
Exactly the wrong people to doall of this choosing.
Um an example, uh, we have avery good friend who's an
engineer at Facebook.
Uh he's uh maybe more than anengineer, you know, uh a leader
of engineers, but but as anengineer.
(11:48):
And he recently mentioned thatthey have he's in charge of a
specific library, uh, andmillions of servers are using
this library.
And when his team does somethingthat is one half of one percent
slower, people scream.
(12:10):
Uh because at that scale, a halfa percent of a million servers
is 5,000 additional serversthey'll need to add just to keep
up.
Jim (12:20):
That's uh that's a lot of servers.
Uh it's hard to think in termsof a million servers, and really
he's talking about multimillionsof servers.
Uh so yeah, half a percent,that's a huge deal for them.
Wolf (12:34):
Yeah, big numbers, um, I feel like I ought to be able to
visualize them.
Uh, but we did that episode oncrypto where we were talking
about really, really hugenumbers, and I sort of gave up
and said, well, this issomething I'd have to write
down.
Um but uh back to where I was,which is other things to
(12:55):
optimize for.
Um and one of the big ones istime to solution.
In other words, the speed of theprogram.
This is the thing you're usuallythinking about.
Um and it this one isn't reallya choice.
It isn't forced in the same waythat the language and
architecture might be, uh thoughthere may be requirements.
(13:17):
For instance, if you're writingthe OS for an a Formula One car,
um that OS has to be real time.
Um or you might be solving someproblem in a spaceship, and that
problem you need an answer inunder a minute.
Jim (13:34):
This right here, this sounds like the stuff people
really want to know about, so uhkeep going.
Wolf (13:40):
Yeah, this is this is the big one.
This is the one we're gonna lookdeeper into.
Um I just want to mention twoother less likely things, at
least these days.
Um, memory and storage.
Uh that's one thing, even thoughI use two words.
Um it might might be veryunimportant for you doing a
(14:03):
personal program on a personalmachine that happens to have a
lot of resources.
Um I have a laptop of my own,um, and it has so much more
memory and so much more uh SSDspace than anything I had when I
was young.
Um I can't even believe it.
It's astounding to me.
(14:25):
Um and finally, uh a thing tooptimize, believe it or not, is
output.
Um, some kinds of output arealready what you want.
For instance, uh your program isan image converter, fine.
If the program is going to be acommand line tool, what does it
(14:45):
print?
If it uh if it uh logs things,what's in the logs?
Are the logs super big?
Are they helpful to the user?
Is it helpful to you, theprogrammer?
Are they ephemeral?
Do they only live in theconsole?
Or do you make files?
Do you manage those files?
Do you delete old ones?
(15:06):
Um do you rename them?
Do you compress them?
Umput is a thing that can beoptimized.
Um influence the speed of yourprogram.
And uh one importantcharacteristic of output, um,
especially logs, is when youhand your program to the end
(15:30):
user, the person who's gonna runit, um the logs is really almost
all the kind of telemetry youwill be able to have.
They're gonna have a log fileand they're gonna send it back
to you.
They're gonna pick out whatproblem it was.
Is the log file concise enoughto find the error?
(15:52):
That's a problem for them andyou.
So that's a thing that can beoptimized.
Jim (15:56):
So, like Wolf, I I've got a computer that's just incredibly
fast.
Uh fast processor, lots ofmemory, lots of disk.
Um for most people, I thinkthey're in that boat.
They just don't really care uhthat that their program is slow.
Um I at this point with myhardware, I can't imagine
(16:18):
writing a program for the typesof things that I do that would
even get near the limits.
Um but why don't you t tell usmore about uh about about this?
Wolf (16:30):
Well, on your own personal machine, writing your own
personal code, solving your ownpersonal problem, I I think
you're absolutely right.
Probably none of this matters.
Especially programs that onlyyou are going to run, that
you're gonna run uh at the speedof a human being, not at the
speed of how much data you have.
(16:52):
Um the thing you are optimizingfor when you write a program
like this is your your ownenjoyment.
Um there's almost no way on yourown personal machine for a
program you write to spendenough money for you to care.
(17:15):
Um your code is essentially freebecause you've already paid for
the computer.
You're not gonna hire anyone.
Uh you're gonna use yourfavorite language, so that's
gonna be easy.
There's no barriers except yourown personal time uh that you
have available for this project.
If this were for your work work,or running in the cloud, or on
(17:38):
an Arduino, or in a phone, or ona Raspberry Pi, or on a
supercomputer where you had totimechare, or calculating
blockchain blocks fast enough tobeat all the other miners, or
writing a finance app that hadto be absolutely reliable, or a
stock trading app that needed tomake the fastest possible
(17:58):
trades, then you're you're gonnaoptimize for things other than
enjoyment.
Um I I've I've just listed uh alot of the possibilities a
minute ago.
But which of these are gonna beimportant?
Um and that is very situationdependent.
(18:19):
Um and everything is atrade-off.
Uh I'm gonna list a couple ofthe trade-offs because it's not
just that you optimize forspeed.
Um it is you uh pick speedversus memory and storage.
That's a trade-off you have tomake.
(18:41):
If you make it faster, almostcertainly you're gonna use more
memory and storage.
Um that's that's kind of a rule.
Um programmer time versus almostanything else.
Um how many users are gonna runthis program?
How many times are they gonnarun it?
(19:03):
How often are they gonna run it?
Um output, I already talkedabout, seeing enough to debug
versus not overwhelming theuser.
Um and what resource are youactually spending money on?
Sometimes things are free, likewhen you own the laptop you're
(19:25):
running this program on, uh,time might be free.
Hardware is free because it'salready paid for, it's a son of
cost.
Um usually you're optimizing forspeed and you're just accepting
or ignoring everything else.
Um that's the thing people arealways talking about.
That's the first thing peoplethink of when you say the word
(19:47):
uh profiling or optimization tothem.
Um that's what they want to makebetter.
Um but whichever of these thingsis most important to you, um,
that's gonna be where.
Wherever the money is going.
Continuously measuring thatparticular thing is job one.
Making it faster or take less uhbandwidth or memory or that's
(20:12):
not job one, that's job two.
Measuring it is job one.
Um this uh i profiling, figuringthese things out, it's gotta be
part of your regular process.
Um it should be in your CI C D.
You should have everything setup to profile on your own
(20:34):
personal development machine.
You should do that um every timeyou push.
If you're gonna push somethingbrand new, it should not slow
the program down.
Um that's a thing.
Uh so let's let's focus onspeed, because that's the main
(20:55):
thing.
It's almost always the mostimportant.
And I'm going to tell you athing that old programmers say
to young programmers all thetime and have for many, many
decades.
Um because you have to be an oldprogrammer before you learn this
(21:18):
particular thing is true.
And that is this.
You're smart, and you're good atthe programming language you're
using, and you know youralgorithms, so you are almost
certain that you can tell bylooking at the source code for
your program where it spends itstime.
(21:41):
Don't feel bad, everybody thinksthat.
You can't.
This is the thing programmersmost commonly want to optimize,
and you absolutely can't knowwhat to do without measuring.
I know you think you can.
I know you do, but really, youare wrong.
(22:02):
If you could easily see whereyour program is slow, wouldn't
it be fast already?
Um, sure, you can see the stuffthat's obviously stupid.
Um often though, the slow thingsyou are looking at don't
actually have a gigantic uhimpact on the big picture.
Sometimes slow, simple, andcorrect is better than fancy,
(22:26):
fast, buggy, and hard tounderstand.
Um you're starting up yourapplication, stuff that happens
at startup happens one time.
If you have a startup that takes20 minutes, yeah, there's some
stuff in there you're gonna needto fix.
Um but if you do something inyour startup that takes five
(22:48):
seconds and could have takenhalf a second, is that something
you need to optimize?
It might be incredibly stupid.
Just looking at it could makeyou sick.
Is that something you need tooptimize?
It almost certainly isn't.
Um, you're gonna use tools uh todo this measuring.
(23:12):
Um I'm gonna talk about and I'mnot gonna mention too much about
particular OSs uh throughout therest of this, because it's
basically the same jobeverywhere you go, and there is
a basic hierarchy of answers.
But um, if you're a Windowsprogrammer, Visual Studio, um
(23:34):
the big one, not VS Code, uh,has the tools in it to measure
the stuff you want to know.
That's where you're gonna start.
If you're a Linux user, there'sa program called Perf.
Perf is king.
Everybody knows about perf.
Perf is actually great.
It might be one of the bestsolutions.
Unfortunately, it's Linux only.
(23:55):
If you're using a Mac and it'suh recent and you're running Mac
OS on it, um you're gonna wantto use uh a GUI application
called Xcode Instruments.
Um it's good, it's a littlecomplicated, and whatever
compiler you're using needs todo some help with it.
(24:18):
It's absolutely easy to use iton Swift or Swift UI, or I
assume Objective C.
Um I've been doing some Rustprogramming, and on the Mac
there's a package a crate thatyou can use that will make your
Rust programs easy to examine inXcode instruments.
(24:40):
Um anyway, you're gonna use sometool like one of those.
More important than the tool isat least two kinds of
information that the tool cangive you.
The most important of those twothings is cumulative time in
(25:01):
functions.
Um it'll usually come back toyou as some kind of table, and
it will be sorted either by timeor maybe by function name.
It might be hierarchical, itmight show you the call stack,
it might be sortable.
Um this all depends on theprogram you use to collect it.
It might be as simple as a CSVfile.
(25:23):
Um, it might be as complicatedas a web page that can do all
kinds of things.
Uh I'll give you an example.
There's a tool called FlameGraph.
Uming the cumulative time isimportant.
Um because, for instance, youcan see, oh, my program spends
(25:43):
50% of the time in such and sucha function, and then there will
also be a number that says howmuch of that time belongs
actually to that function thatappears at the top of the
hierarchy, and how much of itbelongs to the things you're
(26:04):
calling.
Jim (26:06):
Um you're talking about these tools, perf, uh, uh Visual
Studio, Xcode, uh, Xcodeinstruments.
Um it seems to me, I I've notused perf, um but it seems if
I'm thinking about this right,uh it's it's a great tool if
you're writing like in a Clanguage or Rust or something
(26:26):
that's compiled down to machinecode.
Um you and I both do a lot ofprogramming, not at that level.
I I do a lot of Perl, you do alot of Python.
Do these tools help you there?
Wolf (26:40):
It depends on what kind of thing is happening at the place
um your program needs help.
Um a way to start, just likewhen you start debugging, um,
you start with prints, printf's,and and you advance to logs.
(27:01):
Um and then later maybe you usea graphical debugger and there's
breakpoints and and uh all kindsof things like that.
Um a way to start is, and Ididn't talk about the second
thing that these profilers giveyou, but here's an easy thing
for you to get um that is reallygonna help you.
(27:22):
A thing that almost certainly isgoing to surprise you is how
many times certain functions arecalled.
Uh, and it's easy to uh stick alittle bit of debug code inside
that function at the beginningand maybe at the end uh to just
count.
Um sometimes you have functionsyou expect to get called a
(27:47):
hundred, a thousand, howevermany times, and it only gets
called once, and that'ssurprising to you.
Sometimes you have a functionthat you expect to be called
three times.
Maybe you're concatenatingstrings.
I'm not even gonna get into justhow bad concatenating strings
can be, but it can be really badif you're not doing it the smart
(28:08):
way for your language.
It's different in differentlanguages.
But you think that this functionyou're using that helps you
concatenate strings is onlygonna be called three times, and
uh when you measure, you seeit's called 10,000 times.
That's a big deal.
That means if that functiontakes more time than you thought
(28:31):
it should, the result, theimpact of that uh is magnified.
Jim (28:37):
Okay, so again, we're you and I work at at higher level
languages.
I I do a lot in Perl.
Not not as much as I used to,but I have written an awful lot
of Perl.
And Pearl's got a profile calledcalled NYTprof.
It's fantastic.
It it gives me the the thecounts of the how many times uh
uh functions are run, uh howlong it takes to run each
(29:00):
iteration, uh cumulative, howlong uh how much time that
function took, you know, mighthave run 10,000 times.
It produces a flame graph sothat I can see where the
hotspots are, you know, where Icould spend my time.
I'm guessing there's somethinglike that for Python.
Wolf (29:19):
There is.
There's a module called CProfile, or maybe it's called C
Profiler, and I think theremight be a pure Python one
that's called Profiler orProfile.
Um, and they do all the thingsthat you just said, and they are
super helpful.
Jim (29:35):
So you don't have to get down to the level of using prof
uh uh uh perf or xcodeinstruments to do this kind of
stuff.
Usually your language, at leasta dynamic language uh like like
what you and I are using, uh,will provide that.
You think Java offers somethinglike that?
Wolf (29:48):
I gotta imagine Java's got Java uh has been, at least um
since the late 90s and early2000s, one of the most used
languages.
Um that sheer popularity meansthere must be a solution there.
Jim (30:06):
I don't happen to be a Java programmer, so yeah, it just
seems like uh you know perfmight be the best for Linux, uh,
and Visual Studio might be mightbe the best for Windows, but I I
think something at the level ofthe language you're working with
is something worth looking at.
Wolf (30:23):
Uh, I think that's a great observation, and yeah,
absolutely true.
Um Okay.
So a problem with uh runtimearguments is it's an audio
podcast.
So I can't show you uh aprofiler, any of the profilers
(30:44):
I've mentioned, actually doingits thing.
Uh and profiling is veryspecific to the platform
language kind of application andso on.
So even if you could see, Idon't think I could do a
demonstration that wasmeaningful to everyone.
Um and that makes me sad.
Uh because I I want to show youhow to do this.
(31:05):
Um I'm gonna talk about it.
Jim (31:07):
When you play with these tools, it's really, really
eye-opening and and kind of uhfascinating the the information
that they tell you about youryour your code.
Wolf (31:17):
Absolutely.
Um I will say that these toolscan be very fine-grained, and
that's not how you want tostart.
Um you want to start at the top.
Where does my program spend themost time?
Is it accessing the databaseoverall?
(31:39):
Is it running the algorithmsthat it's using to calculate the
answer overall?
Um you want to start at a verymacro level.
And when I say macro, I don'tmean like C or Lisp.
I mean like microscopes andmacroscopes.
I guess there isn't a thingcalled a macroscope, except in
the science fiction novel.
Jim (32:00):
Anyway, hang on just a second.
One more thing.
I'm I'm sorry to keepinterrupting you.
We talked about these languages.
You know, there's a there is alanguage I use a lot.
Every day I'm using it, and alot of times I'm spending uh uh
uh making it faster, and that isuh SQL.
Um SQL's got a profiler, atleast uh in Postgres it does.
It's called uh SQL Explain, andI think it's a standard SQL
(32:22):
thing.
So I imagine it's it's there foruh MySQL and and uh SQL Server
and all the rest.
Uh SQL Explain uh we'll showyou.
Wolf (32:31):
And explain is so important.
Oh my gosh.
Because you can see if it usedthe wrong order, if it looked at
every row instead of using anindex.
Yeah.
Huge mistakes.
Jim (32:44):
Yeah, well, there's a that's a great measure and
measuring tool um that'savailable.
If you can get your query into afile and you can run SQL Explain
on it, boy, if you could readthe output, because it's not
easy.
It's it's fairly dense theinformation they give you.
But I you know, I can takequeries that take many, many
minutes to run and get them downinto like the 10 millisecond
(33:05):
range just by running SQLExplain and and seeing that, oh,
I'm not using an index.
I thought I was.
Uh I'm I'm doing this sub-querywhere I should be doing a join
instead.
Uh running Explain on that,huge, huge win.
Wolf (33:22):
I I agree.
Uh Explain does give you all theinformation.
Uh, I do want to point out thatum you must understand how
databases work for you for youto make good on the information
explained gives you.
Jim (33:42):
Sure, sure.
Wolf (33:43):
Like you need to see, oh, this one isn't using the index I
wanted uh and and know whythat's bad.
Um so um uh a lot like uh one ofthe things I'm gonna talk about
is um certain languages are goodat certain operations.
Um and when a language is goodat that thing, you should do it
(34:08):
the language way.
Um works a certain way, knowshow to do things a certain way,
um you want to use it that wayso it will do the best job
possible for you.
Um and that means you need toknow a little bit about it.
Jim (34:25):
Sure, sure.
Wolf (34:28):
Okay, let me give you some rules, and I'm gonna give you
these rules in order for makingyour code faster.
Um, and it all starts withprofiling, uh, so that you can
actually see where the time, andwhen I say time, what I'm really
saying is money, uh, is actuallygoing.
(34:48):
And start by looking at the mostexpensive hunk of code.
You're gonna solve theseproblems one hunk of code at a
time, um, and you're gonna go inthe order of actual cost.
That's what you're gonna do.
And here are the operations toto apply.
(35:09):
The first thing you need to knowbefore anything, before worrying
about making the code fast, isthe code right?
Is it simple?
Is it correct?
Is it testable?
It is a way easier job to speedup a simple and correct program
(35:29):
than it is to fix bugs in aspeedy, overly complicated
program.
That's number one.
Um so far we haven't gotten tothe part where we're speeding
things up.
Uh you need to know that theserules are more important than
the actual optimization.
Uh you don't start with typingcode.
(35:50):
Alright.
The second thing is for somehunk of code, do you actually
need to do it at all?
Is this piece of code necessary?
Is the job it doing contribuingcontributing to the answer?
Uh because nothing is fasterthan not doing it at all.
(36:11):
Um maybe if you have a piece ofslow code, you could speed it up
by 50%.
That that could be giant.
That could be huge, maybe.
Um but if you didn't have to dothat job at all, you would be
taking out of the total time ofyour program the entire 100%
time occupied by that uselesspiece of code.
(36:33):
So, that's the second thing thatyou're gonna use to optimize
code.
Um the third thing, we alreadytalked a little bit about seeing
obvious stupid stuff.
Did you make in this piece ofcode stupid mistakes?
Um, for instance, using thelanguage wrong?
(36:53):
Are you doing by hand somethingthe language already knows how
to do better?
Are you calculating things everytime through a loop?
Um, when it's the same answerfor that particular thing every
time and you could have hoistedthe calculation out of the loop.
Maybe the compiler catchesthings like that, maybe it
(37:15):
doesn't.
Um you should know better.
You should probably write thatcode better.
If you're in the most expensivepiece of code, as we already
agreed you would be, then you'reat a place where maybe hoisting
it matters.
Um there's different ways toconcatenate strings.
In Python, for instance, if youtake five individual strings,
(37:41):
maybe some are actual literalstrings and some are variables,
um, and you use the plus signand you add them together, um,
doing it that way is gonna makethe most possible allocations
because it's gonna process thoseplus signs one at a time, and
it's going to each time add twostrings total together, and when
(38:05):
it does that, it's maybe goingto have to allocate more space.
Certainly in the case of plainplus instead of plus equals,
it's gonna make a brand newallocation for the result of the
addition of those two strings.
You using the plus sign isabsolutely the slowest way.
You know what you should bedoing in Python?
(38:26):
You probably should be making anF string that mentions variables
and expressions inside of it.
That all gets evaluated at onetime with one single allocation,
it's exactly what you want, andit's the fastest possible thing.
That's a way Python is better atmaking strings than you might be
(38:48):
if you just did it on your own.
So know things about yourlanguage.
Jim (38:52):
Yeah, that would be like uh like using Sprintf in C or Perl
or languages like that, right?
Where you get this formattedstring with uh a bunch of
variables after it, and it doesthe replacement for you and
generates the string once.
Wolf (39:08):
That's exactly right.
Alright.
Next down on the list, and thisis an important one, and it's
probably the first one youthought of.
Um, listener, that's who I'mtalking to.
Are you using the rightalgorithm?
(39:28):
And when you pick the algorithmfor what you're doing, um, this
is not opening up a computerscience textbook and finding the
fastest sort.
Um, the right algorithm stronglydepends on the actual data you
are operating on.
(39:49):
And the data you test withshould be a good simulation of
your actual data.
Fancy algorithms might befaster, but that might only be
true if.
Your data is big enough.
The corollary to this is you aremost certainly not Google.
You are not Netflix.
You are not Facebook.
(40:09):
They have different data.
They have different problems.
They need different solutions.
Write your code for the problemsyou need to solve right now.
If you solve a problem you don'thave, it's not going to solve
the problem you do have.
The next stage is are you usingthe right compiler or
(40:36):
interpreter options if thishappens to be applicable in your
situation?
Things that will remove asserts,unroll loops, hoist expressions,
eliminate dead code, use tailrecursion, where that's a thing
that would help, specializeflows for the specific types
that pass through them in moredynamic languages.
(41:00):
Your compiler can do things tomake your code faster.
It's not going to make your codethree times as fast, but it
might make your code 30% faster.
That's a win.
Jim (41:13):
Alright, you you just threw a whole bunch of words out there
that you know I read I read theman page for the compiler.
You look at the the man page forGCC, and it talks about the dash
capital O uh optimizing uhsetting.
And you know, you can do dash001, dash 02, dash 03 to enable
(41:35):
different kinds ofoptimizations.
And they talk about things likelike you just said, uh uh loop
unrolling, uh hoisting.
What are those things?
What what does it mean to unrolla loop?
Wolf (41:48):
Um so sometimes what you have is a really tight loop.
It's small, it just does a fewthings inside it.
Um when you do that, um it canbe that the machinery of doing a
test that takes you back to thebeginning of the loop is more
(42:12):
work as a fraction of the totalwork one iteration through the
loop is doing than you want tospend.
Your loop costs too much becauseit's looping.
Unrolling a loop means takingthe thing that loop does, its
guts, the middle part, the thingthat's not the test, and instead
(42:34):
of looping over that, um,putting that thing ten times or
twenty times or a hundred times.
Uh it depends on exactly whatyou want.
It's not a loop anymore.
I mean it is, it's just a reallybig loop.
Uh because at the bottom or inplaces in the middle, it's gonna
try to exit or go back to thetop.
(42:55):
Um, it is still going to bechecking for conditions, but
it's faster than what it wasbefore, and it's longer and
flatter.
It's not a loop anymore, it'sunrolled.
Jim (43:10):
So that's what unrolling a loop is.
Okay, so so that particularexample, uh, as programmers, we
want to write loops because itrepresents, it expresses what
we're trying to do in a smallamount of code.
And it's, I mean, you couldflatten it out yourself, but
that kind of makes alousy-looking program, right?
Uh, but if you can you want towrite something simple.
(43:32):
Yeah, you want to write it as aloop and let the compiler unroll
it for you uh to make it fast byusing the the optimization
setting.
Uh that's right.
Yeah.
Okay.
Wolf (43:44):
And I'm gonna toss in here, unroll it if it helps.
Jim (43:49):
Yeah.
Wolf (43:50):
Yeah, yeah.
Which you can't do.
Jim (43:53):
Sure.
Yeah, yeah, the compiler will uhunroll it if it helps.
So uh there's other things.
Uh hoisting expressions, uheliminate dead code.
That's that that's kind of kindof easy to see.
But what is hoistingexpressions?
Wolf (44:08):
Okay, so in this uh hoisting expressions
particularly applies inside aloop, and there are more
complicated versions wherethings loop over a function, and
that function itself uh can becan do this thing.
But here's what hoisting means: it means that you are (44:25):
undefined
calculating something uh insidea loop, so you calculate this
thing 50 times or 100 times, butthe actual thing you're
calculating uh doesn't change.
It's the same answer every timethrough the loop.
Um and sometimes uh we're nottalking about a loop, we're
(44:49):
talking about some interestingkind of expression that appears
five times in your long variablecalculation.
Um hoisting means pulling it outof the thing where it's
calculated multiple times, justcalculating it once, and then in
(45:10):
all those places, so every timethrough the loop, or in the five
places it was used inside yourgiant calculation, or things of
that nature, just use the onepre-calculated value that you
made.
So the programmer didn't put anyextra code outside above the
loop.
(45:31):
Um they just wrote whatever wasnatural, but the compiler
figures out it's the same, pullsit out of the flow of the loop,
now it only happens once insteadof a thousand times, um, and you
just use the answer.
That's hoisting.
Jim (45:48):
Cool.
Cool.
Um I don't know.
Do you dare talk about tailrecursion?
I I I I I don't understand whatthat means.
Wolf (45:58):
Tail recursion, I I want to talk specifically about
specific types, but let me hiton tail recursion because it's
more broadly applicable.
Um when your program isexecuting, usually what happens
is it grabs a hunk of space onthe stack.
Ordinarily in a running program,there's two different places for
(46:20):
memory there's the stack and theheap.
You allocate things on the heapwhen they're gonna live for a
long time, they stick around.
They might live longer than apattern.
Right, exactly.
The stack gets a new block everytime you start a function, and
every time you end a function,that block gets thrown away.
(46:42):
If your function nests verydeeply, your stack will use a
lot of blocks.
And the stack is typically muchmore limited than the heap.
You can run out of stack.
Especially if you're doingrecursion.
Um, recursion is when a functioncalls itself.
(47:02):
There are lots of obviously uhrecursive problems like
factorial and stuff like that.
Um, it turns out it's it'stypically easy for a programmer
to turn something that'srecursive into something that's
just a loop, but let me explainhow tail recursion interacts
with this.
When you write a function thatis recursive, so it would make a
(47:30):
brand new block on the stackevery time it calls itself, and
it could be very deep andconsume all your stack, if it
turns out that that place whereit calls itself is the almost or
is the very last thing in thefunction, then instead of
(47:54):
waiting for all of thoserecursive calls, maybe a
million, who knows, to come backbefore it throws away the stack
block for this particularexecution of that function, it
can throw away that block beforeit calls into the code.
Jim (48:14):
That's that's why it's doing that magic for you.
Wolf (48:18):
Exactly.
And what that means is insteadof allocating a million blocks
on the stack, it it can do itwith about one.
Or whatever however your codehappens to be structured.
So tail recursion, super helpfulin certain kinds of programs.
Um let me talk about um uh uh spspecializing for given types.
(48:41):
Um is this applicable and whathappens?
Um there are dynamic languages.
Perl is one, um, Python is one,there are others.
Um it's not that variables don'thave types in these languages,
they do.
When you have a variable, theremight be an integer in it.
(49:04):
And the runtime knows it's aninteger.
It it does integer things to it.
And when you add two integerstogether, there's code it
executes to do that.
If your language is dynamic,then when you enter the function
that is going to do the additionof these variables, it doesn't
(49:28):
know yet that that variable hasan integer in it.
Um every single time you evercall this function it's an
integer.
There's a special kind of uhcompilerslash interpreter called
a specializing compiler or aspecializing interpreter, and
(49:49):
what that does is um it rhappens as you are running your
program.
It doesn't do this the firsttime you call the function, it
does it after watching thefunction get called a thousand
times.
Um it notices that over thatthousand times, or you know,
(50:11):
some number, over that thousandtimes, um almost always it was
an integer.
And then instead of testing andum calling the correct function
um and maybe that function test,all that stuff, it can go
directly to adding two integerstogether.
(50:32):
Um if it sees that's the thingyou do the most, it can make
your program be really good atdoing that thing.
Um and that is called thespecializing uh compiler or op
or uh interpreter.
Um sometimes jits do that uhjust in time compilers.
(50:53):
So um the the latest version ofthe Python interpreter, um I
think 13, 3.13 can do that.
Uh it's absolutely in 3.14.
It I don't think it's ready yet,um, but it's a it's a big win um
because Python is one of thesedynamic languages.
(51:14):
Um Python does have typing.
Uh I think Python typing is areally great addition to the
language, um, but it's all aboutstatic analysis.
You run a tool on your Pythonprogram.
Uh you don't run your programand have stuff check at runtime.
That doesn't happen.
(51:36):
What happens is um you run MyPyon it, or Pyre, or Pyrefly, one
of these um special typecheckers that looks at the type
annotations and um does a bunchof math and proves your program
is keeping the promises you madewith these types.
(51:56):
Um anyway, let me keep going, ifthat's okay.
Do you have more questions?
No.
Okay.
Um so uh can this, this is thenext piece of the solution to
making your code go faster.
This is the next lower-downstep.
(52:17):
Can this particular hunk of codebe done faster using a
specialized library?
Um for instance, um in Python,uh there's lists.
Uh if you want a matrix orsomething, um you have to use
lists of lists.
That's not as good as you couldwant.
(52:39):
Python isn't as good atmanipulating numbers in lists of
lists or combining whatever.
It turns out there are uhlibraries that, depending on
your specific requirements, um,can do these things much faster.
So if you want to deal with uhbig arrays of numbers, maybe you
(53:01):
want to use numpy.
NumPy is a Python library, youcall it from Python, but inside
uh numpy is written in C.
Uh Pandas, I think it might bewritten in C.
Polars, I know, is written inRust.
Um they're fast, they're superfast, but they present to you a
(53:23):
Python interface.
So you don't even know you'rerunning Rust.
Uh but that particular problemthat you're solving with an
array is now solved 50 timesfaster.
So sometimes the right answer isnot doing it yourself, but using
a library.
And finally, the absolute lastpossible choice, the thing you
(53:48):
want to avoid at all costs, butsometimes you have to do it.
Um, and that is can you rewritethis particular section in a
lower level language?
Um there are so many costsassociated with this.
Um for instance, uh Python is alanguage that optimizes for
(54:13):
programmer time.
Um with Python, you can writemuch faster uh than you can in
many other languages.
But maybe you're at the partwhere you've tried everything
else and you need to write somelittle piece of it in a lower
level language, um, and let'sjust pick Rust for this example.
(54:35):
Rust, for most people anyway, isharder to write than Python.
It's gonna take longer, it'sgonna take more effort.
Uh Rust is pretty portable, butif this was a thing you did in
assembly, um, which God helpyou, um portability is a
problem.
Is your team ready to deal withthis language?
(54:58):
Can anybody else on the wholeteam do it except you?
Uh, or understand it, or find abug in it, or fix it.
So using a lower language isyour absolute last option.
But sometimes you have to takethe last option.
Jim (55:16):
Well, you you you do what you gotta do, right?
To make it fast, if if uh ifthat's what you're trying to
shoot for.
Uh so I so wow, that's a lot ofstuff you covered.
Uh that's neat.
Can can you boil it down andgive us uh the key takeaways
from this?
Wolf (55:33):
I absolutely can.
I I am so sad that I can't showyou visually examples of each
one of these things uh because Ithink that would make such an
impact.
Um, but I think I can boil itdown.
Um the first thing is you gottaunderstand what you're
(55:55):
optimizing for.
Um is it speed, is it memory, isit storage, is it you the
developer?
What is it?
Um and almost certainly the realtruth behind that is what is
costing you money or or whateveryou're using as a system of
exchange.
(56:16):
The second thing is, once youknow what you're optimizing for,
is measurement.
Measurement is absolutely jobone, because that's the only way
to know where the money isactually going.
You should be measuring a lot,more than that.
It should be part of CI C D.
(56:36):
You should run it beforepushing.
Everyone should be doing it.
Measurement might be even moreimportant than testing, and
don't get me wrong, testing isvery important.
Um, when I worked on Mozilla,our build servers did timing.
If your commit slowed somethingdown, that commit was considered
(56:58):
a bustage, that's the word weused.
And people weren't allowed tocommit after that.
The tree, as we called it,turned red.
The entire job of anybody whocould help was to fix that
bustage.
Slowing the program down wasbad.
(57:19):
Real bad.
Don't do it.
Um, so measurement, job one.
Use the profiler for twoimportant jobs.
To see if the whole thing isfaster or slower, so you know
when it's time to look deeper,and second, to dive into the
actual execution and locate theparts you need to improve.
(57:44):
Um it's all about the money.
That's a point I need to makeall on its own.
You need to write, this is thisis your job as a programmer.
Write clear, simple, and correctcode, and you'll know it's
correct by testing.
Um write code that says what itmeans.
(58:12):
Write code that when you readit, tells you what it does.
Umly then can you actuallyoptimize it.
There's an old adage, I think itmight be from Don Knuth.
It might be from somebody else,but the uh the adage is
premature optimization is theroot of all evil.
(58:36):
Um, everything you do tooptimize something is almost
certainly going to make it morecomplicated, harder to
understand, harder to fix,easier to have bugs, and that is
exactly the opposite of yourjob.
So don't do it.
Um you need to understand thereal, actual, true data you will
(59:01):
be operating on.
Your tests should have data thatlooks like that.
Your algorithms should know howto work on that data.
When you do big O notation,usually it's capital C plus big
O something.
Capital C is the constant forsetting up that particular
(59:25):
algorithm.
Complicated algorithms have realbig C's.
Easy algorithms have small C's.
If you have five items and youwant to sort them, you don't
want an algorithm with a big Cbecause big C is going to take
over the entire time of sorting.
A bubble sort might be fine forfive items.
(59:47):
I'm not gonna tell you what'sright for you, but easy data can
be solved with easy answers.
Um and finally, um, you Youdon't know just by looking at
the source what actually costsyou the most money.
Yes, you can see where stupidthings happen.
(01:00:09):
But even for those, knowingwhich actually matter, knowing
which stupid thing actually hasan impact, which stupid thing
costs you money, knowing thatrequires measurement.
So write clear code, measure it,and then worry about everything
(01:00:31):
else.
Um those are the takeaways.
Jim (01:00:36):
Wow, that's a lot of great stuff.
You know, uh, we've been doingthis podcast for a couple of
months now, and uh this is thisis maybe my favorite.
This was so much fun talkingabout, and uh I learned stuff,
and I hope that our listeners uhlearn uh from this uh and I hope
they enjoyed as much as as wedid uh doing this one.
(01:00:57):
Um don't forget uh we have shownotes if you're podcast uh
player.
Uh uh it should show the shownotes.
If not, we have a website.
Um if you go to uhruntimearguments.fm, point your
browser there, uh you'll see thefull episode list and you'll see
show notes and and all that funstuff.
(01:01:18):
Uh send us feedback.
Uh you can contact us again atfeedback at runtimearguments.fm.
We'd love to hear the feedbackuh because we want to we want to
do the best we can and yourfeedback uh lets us do that.
So uh thank you very much.
Um please uh tune in again nexttime.
(01:01:39):
And um uh Wolf, thanks.
Wolf (01:01:43):
Uh my pleasure.
This is so important to me.
I really hope something I saiduh makes a difference for
somebody and they spend lessmoney.
Jim (01:01:53):
I I'm sure it will.
Thank you.
Welcome back to another episode of Runtime Arguments.
We are really excited abouttoday's episode.
Wolf is going to be talkingabout uh uh code performance and
where does the money go?
We'll be talking aboutprofiling, measurement, and
optimization.
Uh we are we are so grateful forthe listeners out there.
(00:26):
Uh they're providing greatfeedback.
We're enjoying doing this.
We're just gonna keep on doingit as long as you guys enjoy it.
Um this particular episodestarted uh the idea for it came
when I uh was doing the researchfor the WebAssembly episode.
That was several episodes ago.
Uh we were trying to useWebAssembly and trying to make
(00:47):
it fast and trying to compare itwith other languages, and the
idea came up that boy, wouldn'tuh wouldn't an episode on
optimization and measurement uhreally be a nice episode?
And Wolf was all over that.
He jumped right on it andstarted doing some research.
And he's I don't think he had todo a lot of research.
He knows this stuff really,really well.
(01:08):
I I think this one is probablypretty easy for him.
So that's what we're gonna betalking about today.
But before we do, uh I have justa very little bit of feedback
from our previous episode.
Uh that episode was on uh uhWolf, what was that episode?
The last one we did.
Wolf (01:26):
File systems.
Jim (01:27):
File systems, yeah.
How could I forget?
Because I did that one.
Anyway, we have a little bit offeedback on that.
Uh first of all, most of theresearch uh on that episode I
can't take credit for.
Uh, most of it was done by myfriend Ron.
Uh he and I took a long roadtrip up to Winnipeg uh back in
May, and maybe it was earlyJune.
(01:49):
Um to pass the time, Ron wasresearching file systems because
he was trying to figure outwhat's the best file system to
use for a server he was settingup.
So he was reading man pages outloud to me, and we were
discussing, and we learned allkinds of stuff about file
systems, and that's really whatwent into uh the previous
episode.
So if you haven't listened to ityet, go back and find it.
(02:11):
Uh it's it was just a couple ofweeks ago.
Uh we talked all about filesystems and stuff.
So that was pretty cool.
Uh, one of the things I failedto mention during that episode,
we did talk about uh theimportance of checksums in file
systems, newer file systems likeBDRFS, or that's BTRFS.
Um, they have something calledchecksums, and that's kind of
(02:35):
important because file systemsand hard disks and and any kind
of uh uh persistent storage cansuffer from something called bit
rot, and that is the the datacan degrade, uh especially for
magnetic uh material.
Uh the data can actuallydegrade, uh uh and so checksums
(02:55):
will help you detect that and insome cases correct for that.
So I wanted to make sure that uhI got that out there.
Wolf (03:02):
So that's kind of that actually that actually makes me
want to say something.
Uh I'm not going to give a fullexplanation, but this is kind of
a pointer for people who care.
Um there is a special kind ofCD, DVD, Blu-ray disc where
you're not recording onto uhorganic dye, you're recording,
(03:25):
you're actually etching ontominerals.
Um the minerals are rated tohold your data for 10,000 years.
The plastic that is the disc isonly rated for a thousand years.
Uh pretty much almost any uhnormal uh DVD, CD, Blu-ray
(03:48):
player can play these, um, and alarge fraction of sp of these
ordinary writers can write them.
Uh so the tool to write them isfairly cheap.
The discs themselves are veryexpensive, and they come in
sizes uh 4.7 gigabytes, justlike a DVD ROM drive, uh just
(04:11):
like a DVD ROM 25, 50, and 100gigabytes.
So yeah, they're not terabytesize like some things, but they
are gonna last for you at leasta thousand years, and all I care
about is maybe them lasting 20.
They are called M disks.
Um, and uh there are places tolearn more about them.
(04:35):
Um, it's a little hard toidentify which disks are the
right ones for you.
Uh a clue is how much they cost,but you do need to do a little
bit of research.
So that's where you shouldstart.
Okay.
Okay.
Jim (04:47):
That's a good point.
And you mentioned a thousandyears and ten thousand years.
That's for the fancy kinds ofmaterial you're talking about.
What are just typical DVD uh uhdiscs good for?
I mean the the you know the onesthat I buy at uh Best Buy aren't
gonna last that long.
Well, not long.
How a year, ten years?
Wolf (05:08):
If they last 10 years, you are incredibly lucky.
Um they ought to last at leastfour or five years, um, but
they're not nearly as good as weused to think they were.
Jim (05:21):
Yeah, so if you really have data that you want to store
reliably for long term, like athousand years or ten thousand
years, use these uh whatchallow,M drive?
M disks.
M disks, yeah.
You use those.
Okay, thanks.
Uh if you have uh any feedbackfor us on any of these episodes,
(05:41):
uh if you just want to suggesttopics, uh if you just want to
say hi, uh feedback atruntimearguments.fm will uh get
to us.
So please send us some feedbackand I'll repeat that information
at the end of the show, and it'sgonna be in the show notes uh
because we'd love to hear fromyou.
(06:01):
That's how we know people areactually listening.
So let's get on to the topic,eh?
Wolf, you ready?
Wolf (06:09):
First, let me say uh what I'm going to talk about.
Um I have four specific pointsthat I'm gonna mention here as
the uh the principal focus ofthis episode, but underneath it
all is the money.
Um every single thing you writeor run costs money somewhere,
(06:35):
somehow.
And almost certainly you want tospend as little of it as
possible, and for that spend,you want to get the most out of
it as possible.
Um, this is true for yourself,but it's really true for your
job job.
If you're running something inthe cloud or for a boss or
millions of people are runningit.
(06:57):
Anyway, here are uh the here isthe intro to what I'm gonna talk
about.
This whole episode is aboutmaking your programs better.
And when I say better, I mean alot of things, but better is a
good summary.
Um there's lots of things insideyour program that individually
(07:20):
can be improved.
Um, and a thing we're gonna lookat is which behaviors inside
your program can or should weoptimize.
Um we're gonna look at why is itimportant, and we're gonna look
at how do we actually do it.
Um so that's a list of thethings we're gonna talk about.
(07:46):
Okay.
The actual words I'm gonna saytoday are about programming, but
the underlying ideas um actuallyapply to a lot of things.
Uh almost any kind of processcan be optimized.
Almost any kind of process costsmoney.
(08:08):
Almost any kind of process um isgoing to yield benefits if you
examine it and figure out whereit can be improved.
So it might be hard to extractfrom the specific things I say
here, but the underlying uhtruth of this episode applies
(08:30):
much more broadly thanprogramming.
Um the uh I'm I I wrote downwords to specifically say about
this.
Uh a person is creatingsomething that will be useful to
others, and money is thelimiting factor.
(08:51):
It costs money to pay thatperson, it costs money to use
their creation, that money mightbe because of time, it might be
because of specific requirementslike location or needed
services.
Um this stuff is fundamental tocoding, but even if you're not a
a coder, um, it's gonna apply toyou.
(09:16):
So, what can we optimize?
Um couple top-level things.
Uh, programmer time and effort.
Um, that's probably a thing youcare about when you're doing it
on your own for your own self.
Um, and the questions uh uhabout that include things like
what environment is it going torun in?
(09:36):
Is it gonna be um some GUI thingon a desktop?
Is it gonna be inside a phone?
Is it gonna be on the commandline?
Is it gonna be a remote job inthe cloud?
Is it gonna be inside anembedded device?
All of those have differentcriteria about how you're going
to do your job, and that meanshow hard it's going to be for
(09:59):
you.
Um what languages are you, andmaybe you're in a team, uh, so
your team too, what languagesare you good at?
What languages are really goodat solving this specific
problem?
Um, and there's lots of ways toevaluate that.
Uh usually it's about whatlibraries are available in that
(10:22):
language.
Uh, but for some languages, it'sabout the actual features of the
language.
Does your intended environmentforce a specific language on
you?
If you need to hire people, whatlanguage uh choices lead to
enough available people for youto hire?
(10:43):
A lot of these choices areforced.
Jim (10:46):
So why does this all matter?
Uh, who measures this and howwould you track it?
I I've never heard of anybodyreally caring that much about
this.
Um what I do know is that speedis important, uh, especially for
big companies that do lots andlots of processing.
Wolf (11:04):
Uh so who measures it and and and who tracks it?
Programmers usually don't.
Managers almost always do.
And weirdly, it's usuallymanagers that do things like
pick the language you're gonnause and the database that's
gonna hold your data and do thehiring, decide which people know
(11:26):
the right things to work on thisproduct.
Exactly the wrong people to doall of this choosing.
Um an example, uh, we have avery good friend who's an
engineer at Facebook.
Uh he's uh maybe more than anengineer, you know, uh a leader
of engineers, but but as anengineer.
(11:48):
And he recently mentioned thatthey have he's in charge of a
specific library, uh, andmillions of servers are using
this library.
And when his team does somethingthat is one half of one percent
slower, people scream.
(12:10):
Uh because at that scale, a halfa percent of a million servers
is 5,000 additional serversthey'll need to add just to keep
up.
Jim (12:20):
That's uh that's a lot of servers.
Uh it's hard to think in termsof a million servers, and really
he's talking about multimillionsof servers.
Uh so yeah, half a percent,that's a huge deal for them.
Wolf (12:34):
Yeah, big numbers, um, I feel like I ought to be able to
visualize them.
Uh, but we did that episode oncrypto where we were talking
about really, really hugenumbers, and I sort of gave up
and said, well, this issomething I'd have to write
down.
Um but uh back to where I was,which is other things to
(12:55):
optimize for.
Um and one of the big ones istime to solution.
In other words, the speed of theprogram.
This is the thing you're usuallythinking about.
Um and it this one isn't reallya choice.
It isn't forced in the same waythat the language and
architecture might be, uh thoughthere may be requirements.
(13:17):
For instance, if you're writingthe OS for an a Formula One car,
um that OS has to be real time.
Um or you might be solving someproblem in a spaceship, and that
problem you need an answer inunder a minute.
Jim (13:34):
This right here, this sounds like the stuff people
really want to know about, so uhkeep going.
Wolf (13:40):
Yeah, this is this is the big one.
This is the one we're gonna lookdeeper into.
Um I just want to mention twoother less likely things, at
least these days.
Um, memory and storage.
Uh that's one thing, even thoughI use two words.
Um it might might be veryunimportant for you doing a
(14:03):
personal program on a personalmachine that happens to have a
lot of resources.
Um I have a laptop of my own,um, and it has so much more
memory and so much more uh SSDspace than anything I had when I
was young.
Um I can't even believe it.
It's astounding to me.
(14:25):
Um and finally, uh a thing tooptimize, believe it or not, is
output.
Um, some kinds of output arealready what you want.
For instance, uh your program isan image converter, fine.
If the program is going to be acommand line tool, what does it
(14:45):
print?
If it uh if it uh logs things,what's in the logs?
Are the logs super big?
Are they helpful to the user?
Is it helpful to you, theprogrammer?
Are they ephemeral?
Do they only live in theconsole?
Or do you make files?
Do you manage those files?
Do you delete old ones?
(15:06):
Um do you rename them?
Do you compress them?
Umput is a thing that can beoptimized.
Um influence the speed of yourprogram.
And uh one importantcharacteristic of output, um,
especially logs, is when youhand your program to the end
(15:30):
user, the person who's gonna runit, um the logs is really almost
all the kind of telemetry youwill be able to have.
They're gonna have a log fileand they're gonna send it back
to you.
They're gonna pick out whatproblem it was.
Is the log file concise enoughto find the error?
(15:52):
That's a problem for them andyou.
So that's a thing that can beoptimized.
Jim (15:56):
So, like Wolf, I I've got a computer that's just incredibly
fast.
Uh fast processor, lots ofmemory, lots of disk.
Um for most people, I thinkthey're in that boat.
They just don't really care uhthat that their program is slow.
Um I at this point with myhardware, I can't imagine
(16:18):
writing a program for the typesof things that I do that would
even get near the limits.
Um but why don't you t tell usmore about uh about about this?
Wolf (16:30):
Well, on your own personal machine, writing your own
personal code, solving your ownpersonal problem, I I think
you're absolutely right.
Probably none of this matters.
Especially programs that onlyyou are going to run, that
you're gonna run uh at the speedof a human being, not at the
speed of how much data you have.
(16:52):
Um the thing you are optimizingfor when you write a program
like this is your your ownenjoyment.
Um there's almost no way on yourown personal machine for a
program you write to spendenough money for you to care.
(17:15):
Um your code is essentially freebecause you've already paid for
the computer.
You're not gonna hire anyone.
Uh you're gonna use yourfavorite language, so that's
gonna be easy.
There's no barriers except yourown personal time uh that you
have available for this project.
If this were for your work work,or running in the cloud, or on
(17:38):
an Arduino, or in a phone, or ona Raspberry Pi, or on a
supercomputer where you had totimechare, or calculating
blockchain blocks fast enough tobeat all the other miners, or
writing a finance app that hadto be absolutely reliable, or a
stock trading app that needed tomake the fastest possible
(17:58):
trades, then you're you're gonnaoptimize for things other than
enjoyment.
Um I I've I've just listed uh alot of the possibilities a
minute ago.
But which of these are gonna beimportant?
Um and that is very situationdependent.
(18:19):
Um and everything is atrade-off.
Uh I'm gonna list a couple ofthe trade-offs because it's not
just that you optimize forspeed.
Um it is you uh pick speedversus memory and storage.
That's a trade-off you have tomake.
(18:41):
If you make it faster, almostcertainly you're gonna use more
memory and storage.
Um that's that's kind of a rule.
Um programmer time versus almostanything else.
Um how many users are gonna runthis program?
How many times are they gonnarun it?
(19:03):
How often are they gonna run it?
Um output, I already talkedabout, seeing enough to debug
versus not overwhelming theuser.
Um and what resource are youactually spending money on?
Sometimes things are free, likewhen you own the laptop you're
(19:25):
running this program on, uh,time might be free.
Hardware is free because it'salready paid for, it's a son of
cost.
Um usually you're optimizing forspeed and you're just accepting
or ignoring everything else.
Um that's the thing people arealways talking about.
That's the first thing peoplethink of when you say the word
(19:47):
uh profiling or optimization tothem.
Um that's what they want to makebetter.
Um but whichever of these thingsis most important to you, um,
that's gonna be where.
Wherever the money is going.
Continuously measuring thatparticular thing is job one.
Making it faster or take less uhbandwidth or memory or that's
(20:12):
not job one, that's job two.
Measuring it is job one.
Um this uh i profiling, figuringthese things out, it's gotta be
part of your regular process.
Um it should be in your CI C D.
You should have everything setup to profile on your own
(20:34):
personal development machine.
You should do that um every timeyou push.
If you're gonna push somethingbrand new, it should not slow
the program down.
Um that's a thing.
Uh so let's let's focus onspeed, because that's the main
(20:55):
thing.
It's almost always the mostimportant.
And I'm going to tell you athing that old programmers say
to young programmers all thetime and have for many, many
decades.
Um because you have to be an oldprogrammer before you learn this
(21:18):
particular thing is true.
And that is this.
You're smart, and you're good atthe programming language you're
using, and you know youralgorithms, so you are almost
certain that you can tell bylooking at the source code for
your program where it spends itstime.
(21:41):
Don't feel bad, everybody thinksthat.
You can't.
This is the thing programmersmost commonly want to optimize,
and you absolutely can't knowwhat to do without measuring.
I know you think you can.
I know you do, but really, youare wrong.
(22:02):
If you could easily see whereyour program is slow, wouldn't
it be fast already?
Um, sure, you can see the stuffthat's obviously stupid.
Um often though, the slow thingsyou are looking at don't
actually have a gigantic uhimpact on the big picture.
Sometimes slow, simple, andcorrect is better than fancy,
(22:26):
fast, buggy, and hard tounderstand.
Um you're starting up yourapplication, stuff that happens
at startup happens one time.
If you have a startup that takes20 minutes, yeah, there's some
stuff in there you're gonna needto fix.
Um but if you do something inyour startup that takes five
(22:48):
seconds and could have takenhalf a second, is that something
you need to optimize?
It might be incredibly stupid.
Just looking at it could makeyou sick.
Is that something you need tooptimize?
It almost certainly isn't.
Um, you're gonna use tools uh todo this measuring.
(23:12):
Um I'm gonna talk about and I'mnot gonna mention too much about
particular OSs uh throughout therest of this, because it's
basically the same jobeverywhere you go, and there is
a basic hierarchy of answers.
But um, if you're a Windowsprogrammer, Visual Studio, um
(23:34):
the big one, not VS Code, uh,has the tools in it to measure
the stuff you want to know.
That's where you're gonna start.
If you're a Linux user, there'sa program called Perf.
Perf is king.
Everybody knows about perf.
Perf is actually great.
It might be one of the bestsolutions.
Unfortunately, it's Linux only.
(23:55):
If you're using a Mac and it'suh recent and you're running Mac
OS on it, um you're gonna wantto use uh a GUI application
called Xcode Instruments.
Um it's good, it's a littlecomplicated, and whatever
compiler you're using needs todo some help with it.
(24:18):
It's absolutely easy to use iton Swift or Swift UI, or I
assume Objective C.
Um I've been doing some Rustprogramming, and on the Mac
there's a package a crate thatyou can use that will make your
Rust programs easy to examine inXcode instruments.
(24:40):
Um anyway, you're gonna use sometool like one of those.
More important than the tool isat least two kinds of
information that the tool cangive you.
The most important of those twothings is cumulative time in
(25:01):
functions.
Um it'll usually come back toyou as some kind of table, and
it will be sorted either by timeor maybe by function name.
It might be hierarchical, itmight show you the call stack,
it might be sortable.
Um this all depends on theprogram you use to collect it.
It might be as simple as a CSVfile.
(25:23):
Um, it might be as complicatedas a web page that can do all
kinds of things.
Uh I'll give you an example.
There's a tool called FlameGraph.
Uming the cumulative time isimportant.
Um because, for instance, youcan see, oh, my program spends
(25:43):
50% of the time in such and sucha function, and then there will
also be a number that says howmuch of that time belongs
actually to that function thatappears at the top of the
hierarchy, and how much of itbelongs to the things you're
(26:04):
calling.
Jim (26:06):
Um you're talking about these tools, perf, uh, uh Visual
Studio, Xcode, uh, Xcodeinstruments.
Um it seems to me, I I've notused perf, um but it seems if
I'm thinking about this right,uh it's it's a great tool if
you're writing like in a Clanguage or Rust or something
(26:26):
that's compiled down to machinecode.
Um you and I both do a lot ofprogramming, not at that level.
I I do a lot of Perl, you do alot of Python.
Do these tools help you there?
Wolf (26:40):
It depends on what kind of thing is happening at the place
um your program needs help.
Um a way to start, just likewhen you start debugging, um,
you start with prints, printf's,and and you advance to logs.
(27:01):
Um and then later maybe you usea graphical debugger and there's
breakpoints and and uh all kindsof things like that.
Um a way to start is, and Ididn't talk about the second
thing that these profilers giveyou, but here's an easy thing
for you to get um that is reallygonna help you.
(27:22):
A thing that almost certainly isgoing to surprise you is how
many times certain functions arecalled.
Uh, and it's easy to uh stick alittle bit of debug code inside
that function at the beginningand maybe at the end uh to just
count.
Um sometimes you have functionsyou expect to get called a
(27:47):
hundred, a thousand, howevermany times, and it only gets
called once, and that'ssurprising to you.
Sometimes you have a functionthat you expect to be called
three times.
Maybe you're concatenatingstrings.
I'm not even gonna get into justhow bad concatenating strings
can be, but it can be really badif you're not doing it the smart
(28:08):
way for your language.
It's different in differentlanguages.
But you think that this functionyou're using that helps you
concatenate strings is onlygonna be called three times, and
uh when you measure, you seeit's called 10,000 times.
That's a big deal.
That means if that functiontakes more time than you thought
(28:31):
it should, the result, theimpact of that uh is magnified.
Jim (28:37):
Okay, so again, we're you and I work at at higher level
languages.
I I do a lot in Perl.
Not not as much as I used to,but I have written an awful lot
of Perl.
And Pearl's got a profile calledcalled NYTprof.
It's fantastic.
It it gives me the the thecounts of the how many times uh
uh functions are run, uh howlong it takes to run each
(29:00):
iteration, uh cumulative, howlong uh how much time that
function took, you know, mighthave run 10,000 times.
It produces a flame graph sothat I can see where the
hotspots are, you know, where Icould spend my time.
I'm guessing there's somethinglike that for Python.
Wolf (29:19):
There is.
There's a module called CProfile, or maybe it's called C
Profiler, and I think theremight be a pure Python one
that's called Profiler orProfile.
Um, and they do all the thingsthat you just said, and they are
super helpful.
Jim (29:35):
So you don't have to get down to the level of using prof
uh uh uh perf or xcodeinstruments to do this kind of
stuff.
Usually your language, at leasta dynamic language uh like like
what you and I are using, uh,will provide that.
You think Java offers somethinglike that?
Wolf (29:48):
I gotta imagine Java's got Java uh has been, at least um
since the late 90s and early2000s, one of the most used
languages.
Um that sheer popularity meansthere must be a solution there.
Jim (30:06):
I don't happen to be a Java programmer, so yeah, it just
seems like uh you know perfmight be the best for Linux, uh,
and Visual Studio might be mightbe the best for Windows, but I I
think something at the level ofthe language you're working with
is something worth looking at.
Wolf (30:23):
Uh, I think that's a great observation, and yeah,
absolutely true.
Um Okay.
So a problem with uh runtimearguments is it's an audio
podcast.
So I can't show you uh aprofiler, any of the profilers
(30:44):
I've mentioned, actually doingits thing.
Uh and profiling is veryspecific to the platform
language kind of application andso on.
So even if you could see, Idon't think I could do a
demonstration that wasmeaningful to everyone.
Um and that makes me sad.
Uh because I I want to show youhow to do this.
(31:05):
Um I'm gonna talk about it.
Jim (31:07):
When you play with these tools, it's really, really
eye-opening and and kind of uhfascinating the the information
that they tell you about youryour your code.
Wolf (31:17):
Absolutely.
Um I will say that these toolscan be very fine-grained, and
that's not how you want tostart.
Um you want to start at the top.
Where does my program spend themost time?
Is it accessing the databaseoverall?
(31:39):
Is it running the algorithmsthat it's using to calculate the
answer overall?
Um you want to start at a verymacro level.
And when I say macro, I don'tmean like C or Lisp.
I mean like microscopes andmacroscopes.
I guess there isn't a thingcalled a macroscope, except in
the science fiction novel.
Jim (32:00):
Anyway, hang on just a second.
One more thing.
I'm I'm sorry to keepinterrupting you.
We talked about these languages.
You know, there's a there is alanguage I use a lot.
Every day I'm using it, and alot of times I'm spending uh uh
uh making it faster, and that isuh SQL.
Um SQL's got a profiler, atleast uh in Postgres it does.
It's called uh SQL Explain, andI think it's a standard SQL
(32:22):
thing.
So I imagine it's it's there foruh MySQL and and uh SQL Server
and all the rest.
Uh SQL Explain uh we'll showyou.
Wolf (32:31):
And explain is so important.
Oh my gosh.
Because you can see if it usedthe wrong order, if it looked at
every row instead of using anindex.
Yeah.
Huge mistakes.
Jim (32:44):
Yeah, well, there's a that's a great measure and
measuring tool um that'savailable.
If you can get your query into afile and you can run SQL Explain
on it, boy, if you could readthe output, because it's not
easy.
It's it's fairly dense theinformation they give you.
But I you know, I can takequeries that take many, many
minutes to run and get them downinto like the 10 millisecond
(33:05):
range just by running SQLExplain and and seeing that, oh,
I'm not using an index.
I thought I was.
Uh I'm I'm doing this sub-querywhere I should be doing a join
instead.
Uh running Explain on that,huge, huge win.
Wolf (33:22):
I I agree.
Uh Explain does give you all theinformation.
Uh, I do want to point out thatum you must understand how
databases work for you for youto make good on the information
explained gives you.
Jim (33:42):
Sure, sure.
Wolf (33:43):
Like you need to see, oh, this one isn't using the index I
wanted uh and and know whythat's bad.
Um so um uh a lot like uh one ofthe things I'm gonna talk about
is um certain languages are goodat certain operations.
Um and when a language is goodat that thing, you should do it
(34:08):
the language way.
Um works a certain way, knowshow to do things a certain way,
um you want to use it that wayso it will do the best job
possible for you.
Um and that means you need toknow a little bit about it.
Jim (34:25):
Sure, sure.
Wolf (34:28):
Okay, let me give you some rules, and I'm gonna give you
these rules in order for makingyour code faster.
Um, and it all starts withprofiling, uh, so that you can
actually see where the time, andwhen I say time, what I'm really
saying is money, uh, is actuallygoing.
(34:48):
And start by looking at the mostexpensive hunk of code.
You're gonna solve theseproblems one hunk of code at a
time, um, and you're gonna go inthe order of actual cost.
That's what you're gonna do.
And here are the operations toto apply.
(35:09):
The first thing you need to knowbefore anything, before worrying
about making the code fast, isthe code right?
Is it simple?
Is it correct?
Is it testable?
It is a way easier job to speedup a simple and correct program
(35:29):
than it is to fix bugs in aspeedy, overly complicated
program.
That's number one.
Um so far we haven't gotten tothe part where we're speeding
things up.
Uh you need to know that theserules are more important than
the actual optimization.
Uh you don't start with typingcode.
(35:50):
Alright.
The second thing is for somehunk of code, do you actually
need to do it at all?
Is this piece of code necessary?
Is the job it doing contribuingcontributing to the answer?
Uh because nothing is fasterthan not doing it at all.
(36:11):
Um maybe if you have a piece ofslow code, you could speed it up
by 50%.
That that could be giant.
That could be huge, maybe.
Um but if you didn't have to dothat job at all, you would be
taking out of the total time ofyour program the entire 100%
time occupied by that uselesspiece of code.
(36:33):
So, that's the second thing thatyou're gonna use to optimize
code.
Um the third thing, we alreadytalked a little bit about seeing
obvious stupid stuff.
Did you make in this piece ofcode stupid mistakes?
Um, for instance, using thelanguage wrong?
(36:53):
Are you doing by hand somethingthe language already knows how
to do better?
Are you calculating things everytime through a loop?
Um, when it's the same answerfor that particular thing every
time and you could have hoistedthe calculation out of the loop.
Maybe the compiler catchesthings like that, maybe it
(37:15):
doesn't.
Um you should know better.
You should probably write thatcode better.
If you're in the most expensivepiece of code, as we already
agreed you would be, then you'reat a place where maybe hoisting
it matters.
Um there's different ways toconcatenate strings.
In Python, for instance, if youtake five individual strings,
(37:41):
maybe some are actual literalstrings and some are variables,
um, and you use the plus signand you add them together, um,
doing it that way is gonna makethe most possible allocations
because it's gonna process thoseplus signs one at a time, and
it's going to each time add twostrings total together, and when
(38:05):
it does that, it's maybe goingto have to allocate more space.
Certainly in the case of plainplus instead of plus equals,
it's gonna make a brand newallocation for the result of the
addition of those two strings.
You using the plus sign isabsolutely the slowest way.
You know what you should bedoing in Python?
(38:26):
You probably should be making anF string that mentions variables
and expressions inside of it.
That all gets evaluated at onetime with one single allocation,
it's exactly what you want, andit's the fastest possible thing.
That's a way Python is better atmaking strings than you might be
(38:48):
if you just did it on your own.
So know things about yourlanguage.
Jim (38:52):
Yeah, that would be like uh like using Sprintf in C or Perl
or languages like that, right?
Where you get this formattedstring with uh a bunch of
variables after it, and it doesthe replacement for you and
generates the string once.
Wolf (39:08):
That's exactly right.
Alright.
Next down on the list, and thisis an important one, and it's
probably the first one youthought of.
Um, listener, that's who I'mtalking to.
Are you using the rightalgorithm?
(39:28):
And when you pick the algorithmfor what you're doing, um, this
is not opening up a computerscience textbook and finding the
fastest sort.
Um, the right algorithm stronglydepends on the actual data you
are operating on.
(39:49):
And the data you test withshould be a good simulation of
your actual data.
Fancy algorithms might befaster, but that might only be
true if.
Your data is big enough.
The corollary to this is you aremost certainly not Google.
You are not Netflix.
You are not Facebook.
(40:09):
They have different data.
They have different problems.
They need different solutions.
Write your code for the problemsyou need to solve right now.
If you solve a problem you don'thave, it's not going to solve
the problem you do have.
The next stage is are you usingthe right compiler or
(40:36):
interpreter options if thishappens to be applicable in your
situation?
Things that will remove asserts,unroll loops, hoist expressions,
eliminate dead code, use tailrecursion, where that's a thing
that would help, specializeflows for the specific types
that pass through them in moredynamic languages.
(41:00):
Your compiler can do things tomake your code faster.
It's not going to make your codethree times as fast, but it
might make your code 30% faster.
That's a win.
Jim (41:13):
Alright, you you just threw a whole bunch of words out there
that you know I read I read theman page for the compiler.
You look at the the man page forGCC, and it talks about the dash
capital O uh optimizing uhsetting.
And you know, you can do dash001, dash 02, dash 03 to enable
(41:35):
different kinds ofoptimizations.
And they talk about things likelike you just said, uh uh loop
unrolling, uh hoisting.
What are those things?
What what does it mean to unrolla loop?
Wolf (41:48):
Um so sometimes what you have is a really tight loop.
It's small, it just does a fewthings inside it.
Um when you do that, um it canbe that the machinery of doing a
test that takes you back to thebeginning of the loop is more
(42:12):
work as a fraction of the totalwork one iteration through the
loop is doing than you want tospend.
Your loop costs too much becauseit's looping.
Unrolling a loop means takingthe thing that loop does, its
guts, the middle part, the thingthat's not the test, and instead
(42:34):
of looping over that, um,putting that thing ten times or
twenty times or a hundred times.
Uh it depends on exactly whatyou want.
It's not a loop anymore.
I mean it is, it's just a reallybig loop.
Uh because at the bottom or inplaces in the middle, it's gonna
try to exit or go back to thetop.
(42:55):
Um, it is still going to bechecking for conditions, but
it's faster than what it wasbefore, and it's longer and
flatter.
It's not a loop anymore, it'sunrolled.
Jim (43:10):
So that's what unrolling a loop is.
Okay, so so that particularexample, uh, as programmers, we
want to write loops because itrepresents, it expresses what
we're trying to do in a smallamount of code.
And it's, I mean, you couldflatten it out yourself, but
that kind of makes alousy-looking program, right?
Uh, but if you can you want towrite something simple.
(43:32):
Yeah, you want to write it as aloop and let the compiler unroll
it for you uh to make it fast byusing the the optimization
setting.
Uh that's right.
Yeah.
Okay.
Wolf (43:44):
And I'm gonna toss in here, unroll it if it helps.
Jim (43:49):
Yeah.
Wolf (43:50):
Yeah, yeah.
Which you can't do.
Jim (43:53):
Sure.
Yeah, yeah, the compiler will uhunroll it if it helps.
So uh there's other things.
Uh hoisting expressions, uheliminate dead code.
That's that that's kind of kindof easy to see.
But what is hoistingexpressions?
Wolf (44:08):
Okay, so in this uh hoisting expressions
particularly applies inside aloop, and there are more
complicated versions wherethings loop over a function, and
that function itself uh can becan do this thing.
But here's what hoisting means: it means that you are (44:25):
undefined
calculating something uh insidea loop, so you calculate this
thing 50 times or 100 times, butthe actual thing you're
calculating uh doesn't change.
It's the same answer every timethrough the loop.
Um and sometimes uh we're nottalking about a loop, we're
(44:49):
talking about some interestingkind of expression that appears
five times in your long variablecalculation.
Um hoisting means pulling it outof the thing where it's
calculated multiple times, justcalculating it once, and then in
(45:10):
all those places, so every timethrough the loop, or in the five
places it was used inside yourgiant calculation, or things of
that nature, just use the onepre-calculated value that you
made.
So the programmer didn't put anyextra code outside above the
loop.
(45:31):
Um they just wrote whatever wasnatural, but the compiler
figures out it's the same, pullsit out of the flow of the loop,
now it only happens once insteadof a thousand times, um, and you
just use the answer.
That's hoisting.
Jim (45:48):
Cool.
Cool.
Um I don't know.
Do you dare talk about tailrecursion?
I I I I I don't understand whatthat means.
Wolf (45:58):
Tail recursion, I I want to talk specifically about
specific types, but let me hiton tail recursion because it's
more broadly applicable.
Um when your program isexecuting, usually what happens
is it grabs a hunk of space onthe stack.
Ordinarily in a running program,there's two different places for
(46:20):
memory there's the stack and theheap.
You allocate things on the heapwhen they're gonna live for a
long time, they stick around.
They might live longer than apattern.
Right, exactly.
The stack gets a new block everytime you start a function, and
every time you end a function,that block gets thrown away.
(46:42):
If your function nests verydeeply, your stack will use a
lot of blocks.
And the stack is typically muchmore limited than the heap.
You can run out of stack.
Especially if you're doingrecursion.
Um, recursion is when a functioncalls itself.
(47:02):
There are lots of obviously uhrecursive problems like
factorial and stuff like that.
Um, it turns out it's it'stypically easy for a programmer
to turn something that'srecursive into something that's
just a loop, but let me explainhow tail recursion interacts
with this.
When you write a function thatis recursive, so it would make a
(47:30):
brand new block on the stackevery time it calls itself, and
it could be very deep andconsume all your stack, if it
turns out that that place whereit calls itself is the almost or
is the very last thing in thefunction, then instead of
(47:54):
waiting for all of thoserecursive calls, maybe a
million, who knows, to come backbefore it throws away the stack
block for this particularexecution of that function, it
can throw away that block beforeit calls into the code.
Jim (48:14):
That's that's why it's doing that magic for you.
Wolf (48:18):
Exactly.
And what that means is insteadof allocating a million blocks
on the stack, it it can do itwith about one.
Or whatever however your codehappens to be structured.
So tail recursion, super helpfulin certain kinds of programs.
Um let me talk about um uh uh spspecializing for given types.
(48:41):
Um is this applicable and whathappens?
Um there are dynamic languages.
Perl is one, um, Python is one,there are others.
Um it's not that variables don'thave types in these languages,
they do.
When you have a variable, theremight be an integer in it.
(49:04):
And the runtime knows it's aninteger.
It it does integer things to it.
And when you add two integerstogether, there's code it
executes to do that.
If your language is dynamic,then when you enter the function
that is going to do the additionof these variables, it doesn't
(49:28):
know yet that that variable hasan integer in it.
Um every single time you evercall this function it's an
integer.
There's a special kind of uhcompilerslash interpreter called
a specializing compiler or aspecializing interpreter, and
(49:49):
what that does is um it rhappens as you are running your
program.
It doesn't do this the firsttime you call the function, it
does it after watching thefunction get called a thousand
times.
Um it notices that over thatthousand times, or you know,
(50:11):
some number, over that thousandtimes, um almost always it was
an integer.
And then instead of testing andum calling the correct function
um and maybe that function test,all that stuff, it can go
directly to adding two integerstogether.
(50:32):
Um if it sees that's the thingyou do the most, it can make
your program be really good atdoing that thing.
Um and that is called thespecializing uh compiler or op
or uh interpreter.
Um sometimes jits do that uhjust in time compilers.
(50:53):
So um the the latest version ofthe Python interpreter, um I
think 13, 3.13 can do that.
Uh it's absolutely in 3.14.
It I don't think it's ready yet,um, but it's a it's a big win um
because Python is one of thesedynamic languages.
(51:14):
Um Python does have typing.
Uh I think Python typing is areally great addition to the
language, um, but it's all aboutstatic analysis.
You run a tool on your Pythonprogram.
Uh you don't run your programand have stuff check at runtime.
That doesn't happen.
(51:36):
What happens is um you run MyPyon it, or Pyre, or Pyrefly, one
of these um special typecheckers that looks at the type
annotations and um does a bunchof math and proves your program
is keeping the promises you madewith these types.
(51:56):
Um anyway, let me keep going, ifthat's okay.
Do you have more questions?
No.
Okay.
Um so uh can this, this is thenext piece of the solution to
making your code go faster.
This is the next lower-downstep.
(52:17):
Can this particular hunk of codebe done faster using a
specialized library?
Um for instance, um in Python,uh there's lists.
Uh if you want a matrix orsomething, um you have to use
lists of lists.
That's not as good as you couldwant.
(52:39):
Python isn't as good atmanipulating numbers in lists of
lists or combining whatever.
It turns out there are uhlibraries that, depending on
your specific requirements, um,can do these things much faster.
So if you want to deal with uhbig arrays of numbers, maybe you
(53:01):
want to use numpy.
NumPy is a Python library, youcall it from Python, but inside
uh numpy is written in C.
Uh Pandas, I think it might bewritten in C.
Polars, I know, is written inRust.
Um they're fast, they're superfast, but they present to you a
(53:23):
Python interface.
So you don't even know you'rerunning Rust.
Uh but that particular problemthat you're solving with an
array is now solved 50 timesfaster.
So sometimes the right answer isnot doing it yourself, but using
a library.
And finally, the absolute lastpossible choice, the thing you
(53:48):
want to avoid at all costs, butsometimes you have to do it.
Um, and that is can you rewritethis particular section in a
lower level language?
Um there are so many costsassociated with this.
Um for instance, uh Python is alanguage that optimizes for
(54:13):
programmer time.
Um with Python, you can writemuch faster uh than you can in
many other languages.
But maybe you're at the partwhere you've tried everything
else and you need to write somelittle piece of it in a lower
level language, um, and let'sjust pick Rust for this example.
(54:35):
Rust, for most people anyway, isharder to write than Python.
It's gonna take longer, it'sgonna take more effort.
Uh Rust is pretty portable, butif this was a thing you did in
assembly, um, which God helpyou, um portability is a
problem.
Is your team ready to deal withthis language?
(54:58):
Can anybody else on the wholeteam do it except you?
Uh, or understand it, or find abug in it, or fix it.
So using a lower language isyour absolute last option.
But sometimes you have to takethe last option.
Jim (55:16):
Well, you you you do what you gotta do, right?
To make it fast, if if uh ifthat's what you're trying to
shoot for.
Uh so I so wow, that's a lot ofstuff you covered.
Uh that's neat.
Can can you boil it down andgive us uh the key takeaways
from this?
Wolf (55:33):
I absolutely can.
I I am so sad that I can't showyou visually examples of each
one of these things uh because Ithink that would make such an
impact.
Um, but I think I can boil itdown.
Um the first thing is you gottaunderstand what you're
(55:55):
optimizing for.
Um is it speed, is it memory, isit storage, is it you the
developer?
What is it?
Um and almost certainly the realtruth behind that is what is
costing you money or or whateveryou're using as a system of
exchange.
(56:16):
The second thing is, once youknow what you're optimizing for,
is measurement.
Measurement is absolutely jobone, because that's the only way
to know where the money isactually going.
You should be measuring a lot,more than that.
It should be part of CI C D.
(56:36):
You should run it beforepushing.
Everyone should be doing it.
Measurement might be even moreimportant than testing, and
don't get me wrong, testing isvery important.
Um, when I worked on Mozilla,our build servers did timing.
If your commit slowed somethingdown, that commit was considered
(56:58):
a bustage, that's the word weused.
And people weren't allowed tocommit after that.
The tree, as we called it,turned red.
The entire job of anybody whocould help was to fix that
bustage.
Slowing the program down wasbad.
(57:19):
Real bad.
Don't do it.
Um, so measurement, job one.
Use the profiler for twoimportant jobs.
To see if the whole thing isfaster or slower, so you know
when it's time to look deeper,and second, to dive into the
actual execution and locate theparts you need to improve.
(57:44):
Um it's all about the money.
That's a point I need to makeall on its own.
You need to write, this is thisis your job as a programmer.
Write clear, simple, and correctcode, and you'll know it's
correct by testing.
Um write code that says what itmeans.
(58:12):
Write code that when you readit, tells you what it does.
Umly then can you actuallyoptimize it.
There's an old adage, I think itmight be from Don Knuth.
It might be from somebody else,but the uh the adage is
premature optimization is theroot of all evil.
(58:36):
Um, everything you do tooptimize something is almost
certainly going to make it morecomplicated, harder to
understand, harder to fix,easier to have bugs, and that is
exactly the opposite of yourjob.
So don't do it.
Um you need to understand thereal, actual, true data you will
(59:01):
be operating on.
Your tests should have data thatlooks like that.
Your algorithms should know howto work on that data.
When you do big O notation,usually it's capital C plus big
O something.
Capital C is the constant forsetting up that particular
(59:25):
algorithm.
Complicated algorithms have realbig C's.
Easy algorithms have small C's.
If you have five items and youwant to sort them, you don't
want an algorithm with a big Cbecause big C is going to take
over the entire time of sorting.
A bubble sort might be fine forfive items.
(59:47):
I'm not gonna tell you what'sright for you, but easy data can
be solved with easy answers.
Um and finally, um, you Youdon't know just by looking at
the source what actually costsyou the most money.
Yes, you can see where stupidthings happen.
(01:00:09):
But even for those, knowingwhich actually matter, knowing
which stupid thing actually hasan impact, which stupid thing
costs you money, knowing thatrequires measurement.
So write clear code, measure it,and then worry about everything
(01:00:31):
else.
Um those are the takeaways.
Jim (01:00:36):
Wow, that's a lot of great stuff.
You know, uh, we've been doingthis podcast for a couple of
months now, and uh this is thisis maybe my favorite.
This was so much fun talkingabout, and uh I learned stuff,
and I hope that our listeners uhlearn uh from this uh and I hope
they enjoyed as much as as wedid uh doing this one.
(01:00:57):
Um don't forget uh we have shownotes if you're podcast uh
player.
Uh uh it should show the shownotes.
If not, we have a website.
Um if you go to uhruntimearguments.fm, point your
browser there, uh you'll see thefull episode list and you'll see
show notes and and all that funstuff.
(01:01:18):
Uh send us feedback.
Uh you can contact us again atfeedback at runtimearguments.fm.
We'd love to hear the feedbackuh because we want to we want to
do the best we can and yourfeedback uh lets us do that.
So uh thank you very much.
Um please uh tune in again nexttime.
(01:01:39):
And um uh Wolf, thanks.
Wolf (01:01:43):
Uh my pleasure.
This is so important to me.
I really hope something I saiduh makes a difference for
somebody and they spend lessmoney.
Jim (01:01:53):
I I'm sure it will.
Thank you.
Host
Jim McQuillan and Wolf
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by Audiochuck Media Company.
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!