All Episodes

March 24, 2025 39 mins
★ Support this podcast on Patreon ★
Mark as Played
Transcript

Episode Transcript

Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Jonathan Hall (00:00):
This show is supported by you, our listener.
Stick around till after the newsto learn some more about that.
Unless you've already listenedto us before, then you won't
learn anything because we'vesaid it a thousand times.

Shay Nehmad (00:09):
But the we will say cool things about the potential
meetups. So it's so worthlistening.

Jonathan Hall (00:23):
This is Cup of Go for Friday, 03/21/2025. Keep up
to date with the importanthappenings in the Go community
in about fifteen minutes ortwenty minutes or so per week.
I'm Jonathan Hall.

Shay Nehmad (00:33):
And I'm Shay Nehmad.

Jonathan Hall (00:34):
Shay, how are you settling into the new life in a
new country?

Shay Nehmad (00:38):
The table at the Airbnb has a thing that doesn't
allow me to connect themicrophone stand because it's
just a tiny teeny tiny the frameis like a teeny tiny bit too
wide, so I'm still recordingwith the earphones even though
I'm looking at the table infront of me right now and it has
my like unwrapped sound card andeverything disconnected, but I'm

(01:03):
missing this like one centimeterof of like the clamp to connect
everything. So I'm recordingwith the AirPods. Sums up my
experience, I think, so

Jonathan Hall (01:14):
far. Okay.

Shay Nehmad (01:15):
Fair enough. Close, but no cigar. Yeah. Probably in
a month, I'll get a lot moresettled in, looking for places
right now, etcetera, etcetera.How are you settling into your
new place?

Jonathan Hall (01:26):
Yeah. We're at week number two, I think, at the
new house. Finally havefurniture in most of the house.
I haven't set up my office yet.Really, I'm still on a flimsy
plastic table on my laptop, butI'm getting there.
I need to set up the homenetwork and run things like that
first.

Shay Nehmad (01:41):
Well, if you have problems with this IPv6 zones,
like, the while setting up thatnew network, I could recommend
our last episode. So we have alot of things to talk about in
this show. Jonathan, how aboutyou take us on the road with
something we already discussedin the past, the stats package.
So some mathy math.

Jonathan Hall (02:01):
Yes. Yes. So we this proposal that we talked
about three or four episodes agohas been accepted. The proposal
is to add a new xexpstatspackage. So this isn't the
standard library.
It's sort of the the adjacentcenter library, the experimental
center library. We're gonna adda number of functions to this
package for general statisticalpurposes. So this is going be

(02:21):
functions like mean, median,standard deviation. And when we
talked about this on the showbefore, one comment we got on
the Cup of Go Slack channel,when I do a mean calculation and
a standard deviation calculationof some of these, I want them
together. So having to do themas two separate functions as the
original proposal sort ofimplies is maybe
counterproductive because youdon't want to have to loop over

(02:43):
those values twice.
You'll be happy to know that thefinal accepted proposal has a
mean and standard deviationfunction that returns both while
iterating over it only once.

Shay Nehmad (02:53):
Oh, nice.

Jonathan Hall (02:55):
Yeah. So keep in mind, this is still
experimental. So it's not gonnabe part of the official center
library yet. Although typicallythese packages sort of meant to
to come into the standardlibrary in the future. So it may
make it the standard library ina year

Shay Nehmad (03:08):
or two. And once it grows into the standard library,
or even if it doesn't, do youexpect like existing projects? I
mean, you know, developers areneeded to run standard
deviation. What have they doneso far? Sounds like they've been
sitting around and waiting forthis proposal.

Jonathan Hall (03:22):
Yeah. So they've just been not writing any code
that needs means or standarddeviations, obviously. Because
it was impossible. No. They werejust implementing this
themselves.
So this is just You

Shay Nehmad (03:33):
you would implement it yourself? I would grab, like,
GoMath package.

Jonathan Hall (03:37):
Or or you could do that too.

Shay Nehmad (03:39):
Like a third third party.

Jonathan Hall (03:41):
Yeah. The thing is that this does it with
generics, although it's ifyou're used to go generics, this
will make sense. It uses thetilde float 64 type rather than
any numerical type. So it onlyworks on values that are at the
core float 60 four's. But that'sstill an improvement because if
you have some custom type that'sbuilt on float64, you'd have to

(04:04):
write your own version of thisfunction every time or do some
conversion before you can callit.
So this will be a slightimprovement in that regard.

Shay Nehmad (04:10):
I'm wondering who like, this seems relevant for
web projects that aren't likemath focused, but do need a tiny
bit of that for, I don't know,caching or just displaying like
a super simple dashboard.Because I can't imagine someone
who like heavily uses, I don'tknow, GoNum, right? Like the the

(04:31):
pure Go with some assembly mathpackage that has floats and
graphs and integrate and complexnumbers and linear algebra and
all that and being like, oh, allI used out of this huge package
was just the mean function. Itmight be the case. Like I I'm I
might be misreading thesituation, but to me this feels

(04:53):
like either you decide, okay, Gois a math language and let's
put, you know, Mode and Kendalland Hellsinger math whatevers
and deal with matrixes as anative type or not do it at all.
So I guess the the reasoningbehind it is there's a lot of
usages for very normal mean andvariance instead of deviation,

(05:15):
and very little usage to allthat, you know, extra math y
stuff.

Jonathan Hall (05:19):
I don't know that this is saying there's no usage
or little usage for the otherstuff, but I I think my my take
on this is more that this is astarting point to see if we like
it, if it serves a usefulpurpose, and then we can expand
from there if it does.

Shay Nehmad (05:31):
I think more math in the standard library is
useful as workload like, nativeworkloads that do, like, heavy
math stuff are are more needed.

Jonathan Hall (05:40):
You know, there's more AI and machine learning and
things like that. Mhmm.

Shay Nehmad (05:43):
Like, having people write papers in Go instead of
Python would be better foreveryone, especially the the
poor undergrads trying toinstall Python on their
university machines and crying.

Jonathan Hall (05:52):
Oh, so sorry for you those guys.

Shay Nehmad (05:55):
Yeah. I have a friend who like a mathematician
friend, he's like researchingsuper advanced stuff and he
calls calls me. He's still inIsrael time, so we have, like, a
twenty four hour, you know, loopback until we can talk about
this stuff. But he, sends me amessage. I can't figure this
out.
Why am I, why am I wailing onthese servers? I can't install

(06:15):
Python. It's the wrong version.It can't find the packages.
What's going on?

Jonathan Hall (06:19):
You know what would solve that? If they would
just rewrite Python in Go.

Shay Nehmad (06:22):
Oh, rewriting in Go. That's a good idea according
to that's a good segue.According to Andres Heisenberg,
from Microsoft. If I'm notmistaken, the Python guy, the
Python BDFL is also at Microsoftright now.

Jonathan Hall (06:37):
He's just having a party. Don't know.

Shay Nehmad (06:39):
Yeah. I I know he was there for a while. They're
having a party of languagedesigners. So the TypeScript
programming language, questionmark, I'm not even sure it's a
programming language, it's justa type system bolted onto UI,
like, driver framework forbrowsers, which for some reason
the world decided should be thenumber one language. It's a if

(07:03):
you don't do TypeScript, it'sjust a type system bolted onto
JavaScript with a lot offeatures.
And it's become very very commonto the point where I think if
you start a new JavaScriptproject, unless you have a very
specific need, you'll probablystart it in TypeScript as well.
All the major frameworks supportit, etcetera, etcetera, both for

(07:24):
back end and front end. Asprojects written in these
languages have grown, forexample, Versus Code itself,
which is a project by Microsoft,the TypeScript compiler, now
compiler is a loaded term here,but I think they have decided on
compiler and not transpiler justbecause of how many things it
does. The TypeScript compilerhas gotten pretty slow because

(07:45):
it was written in TypeScriptcompared to native languages
which compilers tend to bewritten in. As projects have
grown and grown, these codebases sort of gotten out of
control in the amount of time ittakes them to compile, and some
customers have told Microsoftthat they they like run into
basically, they run intolimitations, memory limitations

(08:06):
or runtime limitations, wherethe Versus Code code base, which
is a million and a half lines ofcode, takes a minute and a half
to compile, like seventy sevenseconds.
That's pretty bad. That's like alot of time. Mhmm. The Go team
are in the midst. They haven'tfinished it yet, but they
already have a repo that'sthat's working and people can
can use it.

(08:27):
That's called TypeScript seven,which is TypeScript in Go. It's
very much under activedevelopment, but you can try to
build it and and use it and evenrun the so the TypeScript
compiler has two interestingparts, the compiler and the
language server. Right? Thething where if you open your
your editor, gives you thelanguage server protocol, which

(08:49):
gives you all the features thatyou want as a developer, like go
to definition and go todeclaration and hover gives you
like type, extra typeinformation. All these features
in your editor are powered by athing called the LSP, the
language server provider, orprotocol, sorry, language server
server protocol.
So they have an implementationof that working and it's working
very fast, but they don't havefull feature parity yet. I think

(09:15):
they're aiming towards middle ofthis year to get to a full
feature parity. Other thingsthey want is to get the memory
footprint down because theyhaven't looked at that at all.
The big title is that it's 10times faster. Compiling Versus
Code with the new TypeScriptcompiler that's written in Go

(09:35):
takes seven seconds instead ofseventy seven seconds, which is,
yeah yeah, much much better.
You know, in their mind,obviously, they're also looking
forward, and Andres has lined uplike the thing things like
giving AIs who are generatingTypeScript code immediate
feedback on whether their codeis actually type safe. Because

(09:56):
I'm working with TypeScript.When you work with Go and LLMs,
you almost never get things thatdon't, like, stupidly don't
compile. Because there aren't alot of patterns in the language
where you could takeincompatible types and, like,
jam them, next to one another inthe training set. I I've had the
experience

Jonathan Hall (10:15):
of I I certainly have a Copilot telling me to use
variable names that don't existoccasionally or or or field
names that don't exist. But butit's not a problem because my
editor notices immediately and Ican fix it.

Shay Nehmad (10:27):
So a, you notice it immediately, but also I think
the the feedback is is very isvery fast, like you said. So
your editor noticed immediately,you noticed it immediately. In
TypeSuit it's a little moreinsidious. Can take my editor
like two seconds to figure itout. But they're looking even

(10:49):
like more forward than that andsaying, if I have such a fast
language server, maybe I can getthe language server feedback
directly into the LLM and likeonce it generates some code,
immediately give it the feedbackand if it doesn't add up, like
you don't have to go through theloop of you, Jonathan, looking
at the problem and then saying,Now, obviously they could do it

(11:11):
just for Go development, but Ithink the main market is Python
and TypeScript for them.
So now the TypeScript is muchfaster. Know, you could give it
directly to the LLM as well. Youknow, it's aligning with the
general Microsoft vibe of AIeverything. I don't know if
you've used the, you know, thenew Copilot, which is Office,

(11:31):
but you literally have AIs onevery single button. Yeah.
It's something's going on there.But anyway, they're working on
this compiler. It the editor ismuch faster. The server is much
faster. This impacts developerexperience for a very large
share of developers, so I thinkit is a relatively big deal.
And, yeah, TypeScript six is inJavaScript, and TypeScript seven

(11:54):
is native.

Jonathan Hall (11:55):
Yes. I wanted to ask about that. Isn't isn't the
current version five pointsomething?

Shay Nehmad (11:59):
Yeah. So they're gonna release, six, which is
going to be compatible withseven in terms of features
because when you port all thethings to go, something some
behaviors will change, likedeprecation. So if you don't
wanna move forward at all, youcan stay with TypeScript five,
and the 5.9 is coming soon, andthat's gonna be a JavaScript
based codebase. Then they'regonna release version six, which

(12:21):
is going to include somebreaking changes and
deprecations to align with thenative codebase, with the Go
codebase. So if you upgrade tosix, upgrading to seven will be
very easy.
If you stay with five, upgradingto seven will be very hard. For
sure, once seven is out andstable, you'll want to upgrade
to seven because it's 10 timesfaster.

Jonathan Hall (12:39):
So six will come out sooner than seven, but is
expected to be equivalent or orwhat?

Shay Nehmad (12:45):
Yeah. Behavior equivalent. It's just gonna be
written in the JavaScript andnot in TypeScript. And not in
Go. Sorry.

Jonathan Hall (12:51):
So, like, if if if this were Go where the
language is defined by a spec,then six and seven would be the
same version number?

Shay Nehmad (12:57):
Exactly. Okay. The community response, or I guess
the general this has been prettybig news, like, all the normal
content creators like Fireshapeand everybody's been talking
about it. One of the bigquestion was, why go? And
immediately, once you open, forexample, the I I opened their

(13:18):
Discord, which I don't tend todo a lot.
Mhmm. And, know, one of thequestion is, why was Go chosen
over functional languages likeElixir or Haskell? There's a big
post called why Go? Like,there's a huge discussion with
tons of comments and replies,which was eventually locked and

(13:39):
people you know, some commentsare like, oh, was learning Rust,
oops, I guess I should learn Go.Like, people are drawing really
the wrong conclusions from thisdiscussion.
I think Go is a is what they'resaying is there are multiple
criterias for what we needed andcurrency support was one of them
because, you know, they theywanted to parts that were, you

(14:02):
know, you could paralyze, theywanted to paralyze. But the most
important part, they want tokeep everything as compatible as
possible. And because Go is avery simple language and very
similar to TypeScript, it's notlike extremely different, like,
I don't know, Clojure, right, orlike Lisp based. It it looks
same ish, which is, by the way,while a lot of people can start

(14:24):
with JavaScript or Python, thenrather easily move to Go, it
doesn't require any fundamentalrethinking. It has a garbage
collector, right?
You don't have to worry aboutmemory allocation. So porting a
huge code base is a lot easierto from TypeScript to Go than,
say, from TypeScript to Rust. Ithink that's the main claim they
they've had. I I wonder, and youprobably have an opinion about

(14:47):
this because you get into fightswith, other people on LinkedIn
all the time. Why is this such atouchy topic for so many people?
Like, why aren't people justhappy that it's 10 times faster
and they're like, no, but whydidn't you use language x? Like,
why didn't you use Elixir? Whydidn't you reduce Rust?

Jonathan Hall (15:03):
Well, I I could speculate. I I think people get
attached to language. So I mean,just flip it around a little
bit. Suppose they had chosen sayRust, the Go community have
taken that as a slight?Probably.
Will they care in a year? Idoubt It's kind of like anytime
anybody says, hey, I like X. Ifyou're on team X, then you feel

(15:24):
good. And if you're on team Y,feel like, but why don't you
like Y? Point is, at the end ofthe day, we're trying to be
productive and run businesses.
And so I just recently learnedthat Pandoc, have ever used
Pandoc before?

Shay Nehmad (15:37):
Yeah, of course.

Jonathan Hall (15:38):
You know what language it's written in?

Shay Nehmad (15:40):
No. What would you guess? Python.

Jonathan Hall (15:43):
Python? No. I I would have guessed C or C plus
plus because I've been using itfor years.

Shay Nehmad (15:47):
Oh, it's forever. Actually, it's pretty old. Yeah.
Yeah. Yeah.
Now that I think about it.

Jonathan Hall (15:51):
Yeah. It's old, right? It's written in Haskell.

Shay Nehmad (15:53):
Oh, cool.

Jonathan Hall (15:54):
Yeah. That's cool. I mean, like, I I don't
know Haskell. I don't I don'tcare. But like, wouldn't, I
would have assumed it wassomething else.
And I don't care what languageit's in because I use it and it
works. And I think it's going bethe same thing with TypeScript
in a year or two. Nobody's goingto remember that they chose Go
over Rust, over Elixir, overwhatever. It says, it works
fine. It's written in Go justlike Docker is.
Nobody cares that Docker isn'twritten in Rust or whatever. I

(16:14):
mean, is there a matter offighting about it in public?
Because it works. Nobody reallycares. It's a silly thing.
Take it slightly per some peopletake it slightly personally when
when you prefer x over y, but itdoesn't really matter.

Shay Nehmad (16:28):
Yeah. There is a video by the I think it's called
fellow engineer or distinguishedengineer. I don't know, the
title that means engineer that'sbetter than you at Microsoft.
Ambraves. Where he goes intoslightly more detail about why
Go and what are the benefits,and also shows some demos.
I think if this is if you workin TypeScript like in production

(16:51):
like I do, you're just likesitting around and waiting for
this to get released, the momentit's gonna get released you're
gonna be very happy, that's it.If you're more involved, like
you, for example, maintain someTypeScript libraries or maintain
some compilers or just wanna getinvolved, this is like the time
to get involved in this projectbecause it's not yet feature
complete, so there are stillfeatures to develop. And I

(17:13):
assumed that there was a ton ofbugs and a ton of optimizations
that people could like put in.And I've seen seen a lot of
people on Twitter like, hey, Icontributed this like tiny
change request that improved theTypeScript compiler performance
by seven times because it findslike this specific part of the
type checking, whatever. But Ithink right now it's very ripe

(17:35):
for like low hanging fruits ofimprovements and and also just
features that are to bedeveloped.
And it's not like an open sourceproject by some guy. It's, like,
supported by Microsoft. I'mpretty sure it is going to
happen, and they they are goingto see it through. So I'm I'm
just very hyped about thisproject. Like, the impact on
developer experience is gonna begreat.
Obviously, people are going touse JavaScript and TypeScript

(17:58):
instead of, native languages fortheir projects, and it makes a
lot of sense to do it in a lotof business contexts. So making
the experience better there, Ithink it's better for for
everyone other than the factthat it might allow companies
to, you know, continue tomaintain their TypeScript back
end slightly longer once theirproducts have, like, become

(18:18):
successful instead of investingresources into actually making
them native and then actuallymaking them fat. Small price to
pay.

Jonathan Hall (18:24):
This won't affect runtime speed, right? Only
compilation speed.

Shay Nehmad (18:27):
Yeah. The you know what? It's I I wanna be careful.
They want it to be equivalent. Ithink that they are going to
there are going to be somebreaking changes.
So it's going to compile to,like, you know, slightly
different code. There might bean impact on runtime performance
that's unintended or, you know,if they found some slight

(18:48):
improvements like they do inbetween compilers, but nothing
like major. Because end of theday, it's still JavaScript
running on the, you know, veight engine. Like, it's not
there's not gonna be hugedifference in performance.
However, I think once thecompiler is in Go, it it might
be easier to look at other partsof that are slow, including the

(19:11):
runtime itself and maybeimproving those.
But the JavaScript runtime isso, like, difficult and and
convoluted and has a lot. Itit's not optimized for, like,
performance in the same way thatyou and I are thinking about it,
of, like, oh, I wonder how manyrows this DP could ingest
foreshadowing foreshadowing. SoI I don't expect this to impact

(19:31):
runtime performance too much,definitely not negatively.
Maybe, you know, just because alot of people are pouring
through the code that comes out,and many of these people might
be new to, you know, to thisconcept because they're coming
from Go and not from JavaScript.They might come up with
interesting idea.
You know what I mean? But thisshouldn't impact Runtime

(19:52):
performance. It should be thesame. Yeah. And it's it should
come out this year.
So go get involved. Awesome.

Jonathan Hall (19:58):
So I don't know, Shai, if you saw this story
that's been getting lots ofattention on Reddit as 1,200
stars or likes or or upvotes,whatever they're called on
Reddit.

Shay Nehmad (20:09):
Yeah. I've seen it on r3dd1t. That's the site
you're talking about. Right?Well, if people if people wanna
see it later, they can find iton cupmirogo.dev, which is our
site.
Right. I got an email sayingthat's our site. That's our
site. Right?

Jonathan Hall (20:24):
It must be. So the whole the whole thing is, we
talked about this actually a afew episodes ago, in episode 98,
which incidentally was theepisode before episode 100 due
to an off by one error on ourpart.

Shay Nehmad (20:36):
We're we're gonna we're gonna call that one as you
know what? I think we might havemissed one episode and then we
rebroadcasted another one. So ifyou count, like, the pre tax
episodes, then it's a hundred,but post tax it's 99.

Jonathan Hall (20:54):
Yeah. Okay. That's a stretch, but we'll take
it because we need to fix thisoff by one error. Anyway, the
point is typo spotting, splicechain attacks, fun stuff. So
there's been a whole slew ofpackages reported where people,
nefarious actors have beentrying to typosquat them, trying
to make their fork of a projectlook legitimate, and then adding

(21:14):
nasty code that runs bashscripts and stuff like that when
you run it.

Shay Nehmad (21:18):
Yeah. Just to clarify,

Jonathan Hall (21:20):
not the code that runs bash not all code that run
runs bash scripts is nefarious.It's fine to call OA system. Are
you sure about that?

Shay Nehmad (21:27):
Well, you know, I don't love it, but sometimes you
need to do it. The problem isthey're downloading like a a
ransomware that encrypts allyour files under documents,
which is less nice.

Jonathan Hall (21:38):
So, yeah. I don't know. It's a moral of story
here. I guess be careful aboutyour your imports. Verify when
you pull in a new package thatit's the one you want and not a
fake copy.

Shay Nehmad (21:49):
So how would you verify verify when you open a
new Git when you open a GitHubpage? What are you looking for
to see that it's not a fake one?

Jonathan Hall (21:57):
Well, I I typically look for things like
this isn't specifically to vetfake forks, but I look for a
number of stars, how many issuesthere are, how recently the
issues have been, you know, arethey getting responded to,
things like that. But that'sthat's more just for like, is
this a quality thing that'sbeing updated and maintained?

Shay Nehmad (22:15):
So so the craziest thing about this malicious fork
from from Ariga is that theycopied the GitHub project and
then faked stars. Yeah. Like,they created a few accounts
weeks ago and then filled itwith stars. And now the repo is
down, by the way, so good jobeverybody who reported it. So

(22:36):
you can't really take a look atit.
Seems like it's been happening.The same users have been doing
it for other projects too, whichis very concerning. It seems
like it's not just a veryspecific attack. I don't know,
like, number of stars is is iseasily gameable. You can just
create bot accounts and createstars.

(22:56):
So it's sort of I don't know.It's like buying really good
honey. You gotta know thebeekeeper. Otherwise, they might
just put syrup in your thing.You know what I mean?
Like, I don't know. I know theAriga guys because they came to
get give a talk there from TelAviv. I met them in person. So
if I would import something fromAtlas and I wouldn't see Ariga
in the URL, then like somethingmight pop off in my brain. But I

(23:18):
don't know, like, literallygotta just make sure that you're
looking at the original fork.
And GitHub certainly doesn'tmake it easy to understand if
you're looking at the originalfork or something or not.

Jonathan Hall (23:27):
Yeah. Well, and then if you don't know the
company and you're just like, Ineed a thing that does does x y
z, and now here, here's fouroptions. How do you choose one?
And how do you make sure thatthe one you choose is legitimate
and not not malicious? I I don'tknow the answer, honestly.

Shay Nehmad (23:41):
So there are a few good answers. Right? One of them
is set up Dependabot or Snyk or,like, any other

Jonathan Hall (23:48):
Sure.

Shay Nehmad (23:48):
Like, security thing that scans your
vulnerabilities. But that comeswith its own set of problems. A,
you trust them to find it, andb, they they if you use I don't
know. I haven't done it in Go ina while, but doing it for front
end projects, could tear yourhair out. Even you could tear
out your hair.
Jonathan, which I know it's verydifficult. But you get so many

(24:11):
yeah. You you would grow it outjust to tear it out because you
get so many false positives justspend all your day chasing.

Jonathan Hall (24:18):
You know, I am I'm doing some front end work
the last few weeks, and NPM isconstantly telling me about
vulnerabilities with no fixesavailable. Like, what what do I
do with that?

Shay Nehmad (24:27):
Mhmm. So the reason it's selling you is because
they're if you are representingsomeone who has contractual
obligation to look at thesethings and maybe mitigate them
in a different way, you know,you could remove the library
entirely or just look at thespecific code and see if it's
reachable or not. Right?

Jonathan Hall (24:45):
Yeah. Right.

Shay Nehmad (24:46):
Like, it might point you to a vulnerability in
a function that you're notcalling. So you could say, oops,
I looked at it, but it's fine.Anyway, it's very Sisyphean to
to worry about this stuff. Andthese attackers sure aren't
making it easier. But it it wasan interesting type of squatting
attack we just saw.

Jonathan Hall (25:04):
Yes.

Shay Nehmad (25:05):
And and I assume they're not I assume they're
only gonna get moresophisticated. But good on the
Auriga guys for not just fixingtheir Hypo Squadding thing, but
actually doing the investigationand looking at the X-ray users.

Jonathan Hall (25:16):
That was cool. Yeah. Yeah. That was cool.

Shay Nehmad (25:20):
One last thing on our docket for today is also a
shout out for a past episode. Idon't know which one that is,
but the episode where weinterviewed Matt Topol.

Jonathan Hall (25:33):
Matt, welcome to the show. Would you tell us a
little bit about who you are andwhat you do for Arrow, and then
we'll talk about the project.Great to be here. As mentioned,
Matt Topol. Currently work forVoltron Data, and my primarily
just work on the Apache Arrowlibraries in general.
It's my day to day job now.

Speaker 3 (25:53):
I joined Voltron Data last year in part because I

Shay Nehmad (25:57):
So, yeah, this was episode four a long time ago,
which is probably why I didn'tremember the episode number. But
I guess he's the real Capagoalumni at this point, if it's
been with us for for a couple ofyears. So there's this really,
if you're into data engineeringand you're like the person at
your company who's in charge of,oh, so the RocksDB has to go

(26:19):
into Avro has to go into Kafka,but the Kafka is multi topic, so
we have to invent a new lie anew way of logging blah blah
blah. If you know what I justsaid, you know who you are.
There's a really, fun blog poston the, Aero Apache, blog list
by Luis Alaine.
I I hope I'm saying that right.I don't actually know what these

(26:42):
letters mean. Why are why arethey why are there two dots
above the I? What does thatmean? No idea.
Like, whom not? So there's ablog post about how the author
have had this, like, veryconvoluted data pipeline with
you mentioned, Jonathan, thatyou you, like, see these words

(27:03):
and you just glaze over all thedata engineering terms.

Jonathan Hall (27:06):
I feel like my my mother watching Star Trek, you
know.

Shay Nehmad (27:09):
She does not, like, fully getting it. Yeah. That's
fair. There are just so manywords and and protocols and
names, but things like Avro,Parquet, Kafka, Arrow, it it
gets a bit overwhelming.Overall, the blog post is about
how the author wrote a library,which is aptly named Quackfka,

(27:31):
like quacking in Kafka.

Jonathan Hall (27:33):
Quackfka. Okay.

Shay Nehmad (27:35):
Yeah. It's a it's a really good name, which allows
you to stream things from Kafkainto DuckDB using generics,
using protobuf, which autogenerates, an arrow schema. And
the reason you'd want an arrowschema is because arrow is
commoner and not row based, isbetter for analytics. I wouldn't
suggest actually reading thearticle if you're into it
because I enjoyed it very much,but I have a sense that if the

(27:58):
sentence I just said bored you,you're not gonna enjoy listening
to me trying to explain it. Theunderlying thing is they used Go
to do it and it's faster.
So, you know, what TypeScriptlearned, also they learned.

Jonathan Hall (28:11):
So I read the article and I I was entertained
by it, I didn't understand mostof it. I was entertained because
it's so full of memes and jokes.Like, it it felt a little bit
like reading the script of astand up routine

Shay Nehmad (28:24):
Yeah.

Jonathan Hall (28:25):
About a topic that I've never you know, you
know, watching, you know, Idon't know, My Big Fat Greek
Wedding as a non Greek and nothaving any idea what the Greek
culture references are, butstill thinking it's funny. For
sure.

Shay Nehmad (28:35):
I overall, they try to insert thing into a DB for
analytics. And for analytics,you wanna do Kulminar, which is
Arrow. And because of all the Goimplementation and Matt's
Topol's specific contributionson suggesting to use the ADBC,
which is, you know, one of thethings he's been working on as

(28:56):
part of the maintainers of Go,you know, the Go and Arrow. It's
just an API that allows you toinsert tons of data into the DB
really fast, which was thebottleneck for this blog poster.
Like, they wanted to shove asmany rows into the DB they
could.
And it it solved, like, otherinteresting data engineering
problems, like the small rowsproblem, etcetera etcetera. So

(29:17):
if it sounds interesting, werecommend you read it. We're not
gonna go into it much more. Bothof us are not super experts on
data engineering. So You

Jonathan Hall (29:24):
know, every time you say data engineering, I
think of Nguyen Tsung.

Shay Nehmad (29:29):
You think of what?

Jonathan Hall (29:30):
He's the guy who invented data in Star Trek. If
you have to explain the joke,it's not funny.

Shay Nehmad (29:35):
Oh, no. I don't I I'm not a Star Trek. I'm not a
Trekkie. I'm just like your mom,I guess. Cool.
Let's move on to a quick adbreak. Welcome to our ad break.
Let's do it quick this time. Ifyou want to support the show,

(29:57):
there's a really easy way to doit, you can give us money. We
use Patreon for that.
One important note, if you useApple, don't use Apple for
Patreon because the Apple takesa cut, just use the web version.
I don't think I think I wentover our entire list and I
haven't seen anyone doing thatright now, so good on everybody.
But if this is your week whereyou're like, oh, Jonathan and

(30:17):
Shy and all their hijinks, Ihave to contribute $8 a month so
they can continue editing theirshow. Trust me, you want us to
continue editing the showbecause the amount of bad takes
left on the editing room flooris pretty big and you want us to
keep hosting the episode so youcan download them, etcetera
etcetera, you can pay us $8 permonth using Patreon and we'd
really appreciate it. Thank youto all our beautiful current

(30:39):
Patreoners, Patreoners, patronsfor doing that.

Jonathan Hall (30:44):
Yeah. Patrons. There's the word.

Shay Nehmad (30:46):
Yeah. If you wanna reach us, for example, find the
Patreon link or our Swag Storelink. You can go to cupogo.dev.
Recently, there have been someshipping issues if you've been
impacted by those. Moshe, I'mtalking about you, don't worry,
we're on it.
We're trying to figure it out.You can go to cupogo.dev. If you
wanna talk to us, you can go tocup o go on the Gopher Slack, or

(31:08):
you can email us at news atcupogo. Dev. That is news at
cupogo.
Dev. Another way to support theshow which we really appreciate
is for you to leave a review onSpotify or Apple Podcasts or,
Pocket Casts or just whereveryou listen to this podcast right
now, to sort of help us climbthe charts. The numbers have
been very good to us recently. Ithink people like the show. They

(31:31):
might have listened to, youknow, the episode with, Lane and
been like, oh, they these guysactually know what they're
talking about.
And now that I'm back on, maybethe the the number is gonna go
down again. But, people like theshow. They seem to like the
show. They seem to enjoy it.We've recently crossed 640
subscribers, which is a weirdnumber because what do podcast

(31:51):
subscribers even mean?

Jonathan Hall (31:52):
640 ought to be enough for anyone though.

Shay Nehmad (31:55):
So we really appreciate all of all of y'all
for listening, and if you canshare the show with a friend or
a colleague or rate us onwhatever podcasting thing you're
you're listening to, we'd reallyappreciate it. One final thing
for this ad break is that we areplanning, to do some, to meet
you in the in the meet space asopposed to the virtual space. So

(32:15):
you had in mind KubeCon.

Jonathan Hall (32:17):
Right? Yeah. So it turns out this year, KubeCon
will be in Atlanta, which isjust down the street from where
I'm now living. So if you'recoming to KubeCon and want to
meet up, I don't know if I'mgoing go to the conference,
haven't decided yet. Depends onwhat's happening in my life at
that time, but I'll certainly bein the area.
So we could certainly meet up inthe evening sometime during
KubeCon and share some beers orcoffees or whatever. So if

(32:39):
you're coming to KubeCon, givegive me a shout out. It's
November. There's still plentyof time to make those plans.
More immediately, it sounds likeyou might be meeting some of
our, lovely listeners.

Shay Nehmad (32:49):
Yeah. So there's a group called San Francisco and
the Gopher Slack. And since Itold people, about the fact that
I moved to, the Bay Area, Andyand Akshay have narrowed it down
to the twenty eighth, '20 eighthof May. So I need to check if I

(33:13):
have something there.Apparently, if not, we'll we'll
do a meetup, probably in at myoffice, like a Go meetup.
So we'll start arranging it. Wewill we might use Eventbrite. We
might just do a normal thing.But if you're in the Bay Area
and you wanna meet up with meand Andy and Akshay and some
other gophers, hopefullyhopefully, it sounds like the

(33:34):
three of us, we could do it.Last time we did a a live
meetup, it was in Amsterdam, andyou couldn't make it, Jonathan,
which was very unfortunate.

Jonathan Hall (33:43):
It was very strange too, but yes.

Shay Nehmad (33:45):
But we did a live episode, live episode recording,
so we could definitely do thatagain because that was a lot of
fun. But probably we'll do amore conservative, just like
meet up, a couple of talks,maybe one of the talks will be
the live episode. So, yeah, markit down in your calendars
tentatively and join the SanFrancisco, channel on the Go For

(34:06):
Snack, if that's relevant. Andreach out, like, if there'll be
a lot of people, maybe you couldeven find a sponsor and do the
proper meet up thing.

Jonathan Hall (34:12):
Since you mentioned that, I I actually
looked and there's actually anAtlanta channel on the same
Slack. So I'm now a member ofthat. I wasn't before.

Shay Nehmad (34:18):
That's good. Reach out to your Go communities. You
know what? If there's one actionitems, you know, f the Patreon,
f the Swag store, go join yourlocal Gopher group. If there
isn't a meetup on the calendar,start one.
You can do it you can do it thethe Jonathan and Shy meetup.
Alright. That's enough for thead break.

Jonathan Hall (34:37):
Yes. Let's move on to a quick lightning round,
and then we'll wrap up the show.

Shay Nehmad (34:45):
Lightning round. My first thing for the lightning
round. How are your A

Jonathan Hall (34:52):
little bit tired.

Shay Nehmad (34:53):
Right? How would you feel about me removing
support for them?

Jonathan Hall (34:56):
I'm not sure how I feel about that. Let me go I
don't know. I can't think ofanything clever to say that.

Shay Nehmad (35:03):
Maybe well, maybe if you just put him out the
window, then we'll removesupport for Windows slash R.

Jonathan Hall (35:09):
That's just about as lame as what I thought I'd
do.

Shay Nehmad (35:12):
So I I wasn't even aware that Windows can run on
ARM. Although, that does makesense because, like, you know,
every combination of everyarchitecture, every operating
system, someone has to run.There are a lot of people on
this planet. Windows doesn'tsupport a thirty two thirty two
bit ARM anymore, as Microsofthas noted. Very Microsoft heavy

(35:33):
episode, this, this one.

Jonathan Hall (35:36):
Mhmm.

Shay Nehmad (35:37):
And, you know, they maintain the person who
maintains the GoWindows port issaying the the ARM port is
really really difficult tomaintain and they propose to
totally remove it. There havebeen some people in the in the
comments very nicely, like,saying, hey, I do use it and I
would love keeping it alive, butthe Go's team was like, hey,

(36:01):
you're a very niche community.You keep it alive yourselves.
Good luck. We're not doing itanymore.

Jonathan Hall (36:06):
So I have two quick things I wanna talk about.
They're actually closely relatedprojects. The first one is one
I've been using for severalyears, and I recently became the
co maintainer of it. Theoriginal maintainer moved on to
bigger and better things. Soit's a package originally
developed by Datadog, but nowmaintained by me and a guy in

(36:28):
Japan, I think.
It's called Go TXTB, Go TXTB.What it does is it lets you, if
you're running tests against adatabase, MySQL, Postgres,
SQLite, probably any databasereally. It starts a transaction,
runs your tests and then rollsback the transaction. So any
writes you do during the test ordeletes or whatever aren't
persistent. Of course, that doespreclude running tests that use

(36:51):
transactions or make multipleconnections, etcetera, etcetera.
There are some limitations, butfor many, many unit tests, it
can be a nice way to make yourtests faster. So check that out.
And then the related project,I'm gonna just talk about at the
same time here before I let youtalk.

Shay Nehmad (37:06):
And just because we're on the typo squatting
episode, I really wanna pointout that when you say Datadog,
notice the small break there,it's a hyphen. It's not actually
Datadog, the analytics company,it's Datadog, which seems to be
different. We're speculativelythinking it's related because it
seems related, but it's not theofficial Datadog org on GitHub,

(37:29):
is without a dash. Exactly. Sotake that into consideration.
I don't know how much maliciouscode you've introduced since
you've become co maintainer.

Jonathan Hall (37:37):
You'll and and you won't know. I'm gonna keep
that Yeah. Well well

Shay Nehmad (37:41):
My code is obfuscated on the best days.

Jonathan Hall (37:43):
The other project I wanna mention is actually
quite related and one I actuallyuse more often. It solves a
similar problem. It's calledpgtestdb. This is under
github.com/pgtestdb. It solvesthe exact same problem, but in a
more robust way if you're usingPostgres.
So what it does is it runs allyour migration scripts, then it

(38:03):
creates a new database from yourPostgres database. Postgres has
this nice capability to likecloned databases, which is
almost instantaneous, this ispopping on the right thing. And
then run your test against that.This is better than transactions
for a number of reasons. One,you can make multiple
connections to the same databasewithout, which you can't do with
transactions and it's faster andit lets you do full support for

(38:26):
transactions.
So if you have a test that needsto run a transaction, maybe
commit it, run anothertransaction, etcetera, etcetera,
you can do all that with this.And then it just throws that
database away when your testsexit. So two packages I want to
mention there to help make yourdatabase tests more self
contained. And I think that'sall I have for today's talking
round.

Shay Nehmad (38:43):
Very, very cool. I basically had to reimplement
this for my current stackbecause I guess languages that
are not in Go can't have fastthings. But I'm just looking at
this long English. I wish Icould use it. Well, seriously,
it's pretty powerful, man.
Alright. That's all we have foryou this week. Lots of things
left in the backlog for nextweek. Again, if you can connect

(39:04):
us to anybody on the TypeScriptmigration project that's
actually working on it fromMicrosoft, if you know anyone,
we would really appreciate it.We would wanna interview them.
Also, you have any interestinginterviewees in general, we're
both super swamped right now,which is why we can't schedule
interviews. But if someone maybein The US time zone, that will
be even better, wants to come onthe show, we'd really appreciate

(39:25):
it. This episode came out a bitlate, and we're moving our
normal recording times to Fridayas well. So I think in general,
the episodes are gonna come outlike during the weekend or maybe
even start of the week fromstarting from now. Yeah, I think
that's it.
Program exited. Goodbye. Programexited. Goodbye.
Advertise With Us

Popular Podcasts

On Purpose with Jay Shetty

On Purpose with Jay Shetty

I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!

Stuff You Should Know

Stuff You Should Know

If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.

Dateline NBC

Dateline NBC

Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com

Music, radio and podcasts, all free. Listen online or download the iHeart App.

Connect

© 2025 iHeartMedia, Inc.