December 11, 2024 • 54 mins
In this episode of Ruby Rogues, we talk with Ufuk about how Shopify made the transition to using Sorbet and about the benefits they felt they received from implementing it. Ufuk also reveals a little bit about how Shopify transitioned to fully remote and about how that will be the default moving forward.
Picks
Become a supporter of this podcast: https://www.spreaker.com/podcast/ruby-rogues--6102073/support.
Picks
- Luke - https://github.com/asdf-vm/asdf
- John - Walmart Grocery Pickup
- Dave - https://www.amazon.com/s?k=Thin+Clients
- Dave - Apple ARM MacMini
- Ufuk - TCP/IP Illustrated, Vol. 1: The Protocols (Addison-Wesley Professional Computing Series)
Become a supporter of this podcast: https://www.spreaker.com/podcast/ruby-rogues--6102073/support.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:06):
Welcome to Ruby Roags today. On our panel this week
we have Dave Kimura everyone, and we have Luke stutters
Hi and I'm John Epperson. And for a guest this week,
we have case Leo. Luke Yes, Hi, welcome. Would you
tell us what you're famous for, what you do, just
(00:26):
a little bit about you so we can get kind
of going here.
Speaker 2 (00:29):
Sure.
Speaker 3 (00:29):
I'm currently working at Shopify on the Ruby and Rail's
infrastructure team. I've been at Shopify for about a year
and a half and since I've joined, I've been first
on the Rails upgrade project, and then I quickly switched
over to the Sorbey adoption team and so we spearheaded
the adoption of Sorbey across our monolith and across our
(00:53):
other code.
Speaker 2 (00:54):
Basis as well. And now I'm slowly transitioning to being
the team lead or the Ruby part of the team.
So the team will.
Speaker 3 (01:04):
Be responsible for contributing to MRI and contributing to Sourbet,
basically solving and maintaining and building on the.
Speaker 2 (01:14):
Open source Ruby foundations.
Speaker 3 (01:16):
I've been in the software development industry for over twenty years.
Speaker 2 (01:20):
I've worked with very statically type dynamic.
Speaker 3 (01:23):
P time languages across that time. Initially I was working
in a startup. I was one of the family employees
where we were developing voice XML based applications, and we
built a voice XML platform, and then we did like
various networking things voice so variety, text to speech, speech recognition,
(01:44):
et cetera, et cetera. I had a short stint in
between for a couple of years doing startup acceleoration, and
then went back to developing software and so apped me
in a nutshell.
Speaker 1 (01:55):
Wow, So there were there have been a couple of
talks that you've given it that I know about. I
don't know if you've given more than that, but.
Speaker 3 (02:03):
Yeah, I have one talk at Ruby COMF twenty nineteen
last year, and I also submitted a Rails comp twenty
twenty the couch edition, the remote one.
Speaker 2 (02:14):
I also had a talk there.
Speaker 1 (02:15):
Yeah, sorry about the confusion. One was on sorbet and
one was on the network. I know that I, particularly,
for one, am interested in hearing about your adoption at Surbey.
I think that's super interesting to me. Anything that you
particularly want to note about either of those things that
you're super interested in yourself.
Speaker 3 (02:33):
I think we can maybe talk about the network stack
video and the talk from Rails comp near the end
of the show if we have time, because I think
most people would be interested in the sorbe adoption and
how a company with a code based as large as
Shopify has been successfully adopting it. I'm presuming that's what
most of the listeners would be interested in, so we
(02:54):
can get startled with that if you want.
Speaker 4 (02:56):
How large is the Shopify code? Because Shopify moved forty
one billion dollars of merchandise in twenty eighteen, forty one
billion would have been how many dollars per line of
code does that work out to?
Speaker 2 (03:15):
I haven't calculated that.
Speaker 3 (03:17):
I don't know, But like in case some of the
listeners aren't familiar with the company, and maybe I should
just give a quick intro.
Speaker 2 (03:26):
So what Shopify does? So.
Speaker 3 (03:28):
Shopify is a leading global commerce company that provides tools
to start, grow, market and manage a retail business of
any site. So it started off in two thousand and
six codebased that was built on Rail's pre one point zero,
so it was the time of Rails when it was
(03:48):
still being shared as zippiles and it's been growing ever since.
It started as an e commerce platform and has grown
into being a multi touch general commerce platform, so it
provides solutions both for e commerce and also in store commerce,
in person commerce, so it has many solutions.
Speaker 2 (04:08):
Now.
Speaker 3 (04:08):
It supports more than one million merchants now and throughout
its lifetime it supported one hundred and seventy two billion
USB in total sales for all the merchants. Of course,
that's lopsided to the recent here, So the figure that
that you quoted is probably correct. Look like, I don't know,
(04:30):
off the top of mind.
Speaker 4 (04:31):
Calls it from Wikipedia, so it's almost certainly wrong.
Speaker 2 (04:36):
It sounds believable.
Speaker 3 (04:38):
Then, As for the code base, it's most probably the
largest Ruby on Rails code base in.
Speaker 2 (04:43):
The world right now.
Speaker 3 (04:45):
It has about thirty thousand Ruby files in our core monolith.
So I'll keep referring to Core. So when I say Core,
that's our main monolith that provides almost all of the
shop off solutions.
Speaker 2 (05:00):
Even though we now.
Speaker 3 (05:01):
Have some other services that operate outside of Core, most
of our code base and most of our operation is
actually still being handled by Core, which is our hour monolith.
And that's about thirty thousand Ruby files. And on each push,
on each deploy there are about one hundred and fifty
(05:22):
thousand tests that run to ensure that everything is still same.
So that's the size of the code base, that's the
size of the platform.
Speaker 5 (05:31):
And that's crazy. I think, you know, a lot of
us would dream to get to that kind of scale. Honestly,
I think it would be a nightmare. Not only do
you have the super huge codebase to maintain, but then
just the infrastructure alone, and what that would look like
would is probably be really crazy. Can you speak into
the infrastructure, you know, are you guys using a very
(05:53):
simple infrastructure or has it kind of gone from simple
to like very elaborate.
Speaker 3 (05:58):
So I've only been at the for about a year
and a half and most of the infrastructure has been
in place before I joined, and I'm also still learning
about some of the infrastructure that we use. But like
I can tell you, it's a kubernkkis deploy so everything's
virtualized in Docker containers and deployed to Kubernets pods, and
(06:21):
there are like some load balancing solutions, and we also
have this concept of pods. Pods are these logical units
that host a group of certain merchants in one location
with their own basic infrastructure. So that's like one way
of sharding the customer database and the customer load across
(06:42):
different machines. I'm pretty sure that, like others from the company,
have presented at various conferences talking about that infrastructure part,
but that's not necessarily my strongest point, so I don't
want to really go much more in depth about that,
even though the team that I'm working on, the Rails,
the Ruby and Rail's Infrastructure team, is a part of
(07:06):
the product line in the company called Kernel. Kernel is
basically responsible for ensuring that the platform is off and running,
so it's responsible for all the infrastructure problems, but also
it includes Developer Acceleration, which my team is a part
of as well, which provides solutions for the developers within
(07:26):
the company so that they can do their work better,
they can produce solutions faster, and also we provide solutions
for the infrastructure part of the team as well.
Speaker 1 (07:36):
Yeah, I definitely remember more than one talk at Rails
comp throughout the years on how you guys infrastructure changed.
And I remember sitting at the table with some Shopify
people one year who were talking me through how they
were moving from Vagrant to Docker. So yeah, I do
remember the story changing. I'm not expert enough to talk
about it by any means. But that's been crazy to
(07:56):
watch from the outside. Yes, so all right to get
down into sorbet. What maybe from a high level talk
about what you guys did to make survey happen at shopify.
Speaker 3 (08:07):
Sure again, maybe I should give a little bit more
context as to what sorbet is, who developed it. What's
used for sorbet is gradual type checker for Ruby that
was developed by the fine folks at Strip. So sorbet
provides you the tools to add type checking information to
(08:27):
your Ruby code base, and it gives you both static
typechecker and a runtime type checker to ensure that your
type assumptions are makes sense and are valid. The static
type checker is the most important part of sorbe. The
static type checker is a tool. It's an executable built
using C plus plus. It has its own Ruby parser,
(08:49):
so it's able to parse and understand Ruby and in
some ways simplify the Ruby language, so it can do
those kinds of static analysis without necessarily running your code.
Well I shouldn't say necessarily.
Speaker 2 (09:03):
It doesn't run your code at all, So in that way,
it's a static type.
Speaker 3 (09:07):
Checker and it's built to be blazingly fast, so it
can parse, understand, and type check our whole code based
the code based core code base that I've been talking
about of about thirty thousand files in a matter of
like ten fifteen seconds. And it was built with intentionally
(09:30):
to be fast and to be as safe as possible.
And when you're developing, sorbet was open sourced about a
year ago, so it was June twenty nineteen when it
was open sourced, but it was being developed by Stripe
for about I think two years before that, And by
the time I joined Shopify, we'd already gotten on the
(09:54):
early release beta program where they were doing like code
sharing with Shopify, so we were able to start our
adoption earlier than when it was initially open source. Then
afterwards Stripe also got like a few other companies that
were interested into the public data program as well. But
so that's why our story is a little bit different,
(10:17):
because we ended up solving some of the problems that
people who might want to adopt sorbe don't have to
solve right now because it's a much more mature product.
But back when we were trying to adopt it on
our code base, it was mostly a tool that was
built with the Stripe code based in mind. And I
(10:39):
can't talk too much about what Stripe codebase is like
because I don't know, but from our understanding is it
doesn't use RAILS. It tries to use as little meta
programming as possible. So they're like really big on intention
revealing code and not so much meta programming. So Sorbet
(10:59):
was the right tool for them, and they were developing
to solve their own solutions obviously initially, but because ar
codebas is RAILS, which already has a lot of meta programming.
So meta programming brings in a lot of dynamic methods
that can only be seen when you're running the code right,
So there are lots of methods that you don't see.
(11:23):
For example, on active record models, you don't see all
the attributes right because the attributes are created at runtime.
All the methods that relate that are related to attributes
are created that runtime, after RAILS has had a chance
to introspect your database levels. Obviously, that's a big problem
for Surbey because it looks like the code tries to
(11:43):
understand and sthatically and it says, oh, this.
Speaker 2 (11:46):
Model has no methods. So in another part of the codebase.
Speaker 3 (11:51):
When you're trying to do shop dot something Serb says, oh,
like shop doesn't have this method, so that ends up
being a problem.
Speaker 2 (12:00):
So one of the first things we contributed.
Speaker 3 (12:03):
To the Sorbeko disk was a way to have the
static type checker understand some of the NATA programming concepts.
So our initial idea was that we could write scripts
in Ruby and so they could basically execute them so
that the scripts could generate what methods would exist if
(12:26):
the code had ran, but it would still be a
static type checker. It kind of tided us over that
solution for a while, but it ended up slowing down
the type checking process a lot, but at least it
allowed us to do our own initial adoption at the time.
Another problem as we were adopting was that all the
types that are coming from our GEM dependencies, So our
(12:50):
core monolith has about four hundred GEM dependencies, which is
already like a huge maintenance problem, right, like we need
to maintain them to be on the latest versions and everything.
Speaker 2 (13:03):
But it also means there are.
Speaker 3 (13:05):
Lots of classes, modules, constant definitions coming from those gems.
But Survey doesn't go and look for class definitions inside
of gem source of it just looks at your code base,
so it says, oh, you're using this foo class here,
or like this food constant here, but I have no
(13:26):
idea what foods. So it needs to be told what
foo is, what kind of methods it has, what kind
of subconstances it has, what type of a constant it is?
Speaker 2 (13:36):
Is it a class, is it a module, what's superclass?
What are the mixings, et cetera, et cetera. So we
needed a tool that would just take.
Speaker 3 (13:45):
A GEM somehow, introspect it, and then generate an RBI file.
So it's called an RBI file that's a Ruby interface file.
It's almost the same thing as a c header file.
So it's just the definitions of these things that I
just said, is it a class, is it a module?
What's the superclass? Mixing the methods and its arguments. So
(14:07):
it just generates those without any of the implementation. So
we had to build a tool and we open sourced
it afterwards as tapioca.
Speaker 2 (14:16):
That does that. It takes your GEM file, it discovers
all the gems, it loads them into memory.
Speaker 3 (14:22):
Then it does runtime reflection on all the types that
can find from all those gems, and then it tries
to understand what the shape of all those types are
and it generates that into a set of rbiphads, which
then so they can read, and then it then it
knows about food coming from the gem flo or something.
Speaker 1 (14:45):
So just a quick follow up there, So sure you
guys have that or you have this problem in shopify
because you have a bunch of gems. Did the people
they created this gems right?
Speaker 4 (14:54):
Yeah?
Speaker 1 (14:54):
Stright? Yes, I don't know why I couldn't think of
it for a second. Did stric not have any gems? Depends? Then?
Speaker 2 (15:00):
No, they did. So basically they had a script that would.
Speaker 3 (15:04):
Kind of do the same thing that they were using internally,
but it wasn't part of the Sorbet.
Speaker 2 (15:09):
Tooling at the time.
Speaker 3 (15:10):
Okay, So they shared the initial skeleton of that script
with us, and then we ended up turning that into
the tool that's now Tabioka, and then the Storbe team
at Stripe they actually also took that script and then
built it into the sorbet in It script, which does
something very similar. It also can look into gems and
(15:34):
load types and generate ruby interfestiles for you. So that
ended up being a part of the tool, the Sorbet tool,
but we took a slightly different route than that tool,
and so that's why we're actually using Tapyoka internally rather
than the Sorbet in it tooling.
Speaker 1 (15:53):
Okay, what else did you encounter?
Speaker 2 (15:55):
Yeah, so the other thing.
Speaker 3 (15:57):
So going back to the DSL problem, it turned out that,
you know, making Sorbet run Ruby scripts as it's trying
to make sense of your code.
Speaker 2 (16:08):
Based statically was a huge slowdown.
Speaker 3 (16:12):
So we basically stopped doing it, and then also realized
that the same approach, even though it has some disadvantages,
but the same approach of generating Ruby interface files for
all the meta programming, would be a much better fit
for our problem. So recently we've been working on that problem.
(16:33):
There are other solutions in the same area as well.
One of them is, for example, the gem called sorbet
Rails that's built by people at the Chans Zuckerberg Initiative.
So they had the same problems of Sorbey not understanding
some of the meta programming that's coming from rails again,
like all the column related methods on models, all the
(16:56):
association related methods on models, all the the class methods, sorry,
all the instance methods in mailers that are turned into
class methods, all the helpers on your controllers, they're all
happening at run time in rails. So Sarbe rails was
the first attempt to actually build a gem that could
(17:18):
generate Ruby interface files for all those things, so that
Sarbe can look at the Ruby interface file. So you
have your shop model or like shop dot irb file
which hosts your shop model, but you also have this
shop dot rbi file that's generated for you, which contains
all the method definitions that would be on the shop
(17:39):
model dynamically. So having that, Sarbe can say, oh, yeah, okay,
I know a shop as this name method, So if
you call shop dot name, then yes, I know what
that is. And even better if you can also create
signatures for it, so if you can tell it what
parameters it takes and what the return type is, then
(18:01):
you can also keep typing those things as well, so
you can build on.
Speaker 2 (18:05):
Those types as well.
Speaker 3 (18:06):
So survey rails was the initial solution to this, but
we also realized that we had other meta programming DSL
concerns within our code base that obviously wasn't addressed by
Sarbe rails so we wanted a broader framework, so we
built one in house, and we actually are in the
process of folding that also into Tapioca, so that Tapioca
(18:28):
becomes this once with army tool of generating RBIs either
from gems or from all the DSL meta programming that's
going on across your code based and initially we've again
targeted the same RAILS meta programming concepts, so we borrowed
and built on some of the solutions from Sarbe Rails,
(18:51):
but we've also built DSL generators for other common gems
across our code base, and we intend to grow that
set based on contributions from the community. So we have
a generator for an active record type store frozen record
identity cash that's also a Shopify gem, So like Rails
(19:11):
plus a few gems will already be addressed with Tapioca,
and that was the biggest pain point stopping us from
increasing our typing even further across our code based.
Speaker 1 (19:22):
Okay, so you guys went through all of this working
effort to get survey working at Shopify. So what did
you get from that? Like, what was your value? What
was you guys's value proposition? As best you understand it.
Speaker 2 (19:35):
Yes, great question.
Speaker 3 (19:37):
So the obvious headline thing for SORBET is to obviously
to ensure that your code is type safe, right, so
that you're not making silly mistakes, that you don't have
a reference to references to classes or modules across your
code based that are no longer there. We've found many
(19:58):
instances of that across our co based through the use
of survey for example, or that you're not calling methods
on constants that don't actually have those methods, or that
you're not providing arguments to those methods that they don't
actually accept.
Speaker 2 (20:14):
So all of those things.
Speaker 3 (20:16):
If you're working on a normal RUT code based all
of those concerns, you can only discover them at.
Speaker 2 (20:22):
One time, so you either need to.
Speaker 3 (20:24):
Have extensive testing to ensure that all those edge cases
are executed on your CI or you discover them in production.
So the initial goal is obviously was to reduce those
types of trivial errors as much as possible. Unfortunately, in
our initial adoption phase we couldn't validate or invalidate that hypothesis,
(20:49):
but we did other things to.
Speaker 2 (20:51):
Compensate for that.
Speaker 3 (20:52):
So there's a very understandable reason for why we couldn't
conclude on that hypothesis if we were able to reduce
trivial errors. Is because our core monolith does more dynamic
things then the stuff that we have in our code base.
So it kind of like serializes objects to the database
and then read serialized objects from the database, et cetera,
(21:15):
et cetera, And there's no way aesthetic type checker can
like prevent you from, you know, de serializing an object
that definition for which has mistakenly even remote from the
codebas Right, So suppose you had like a user class
and there was a user object serialized somewhere, and then
someone removes the user class, and now someone tries to
(21:37):
de serialize the user object, and then boom, your application fails,
right because the user class doesn't exist anymore, so you
can't deserialize it. So we obviously can't prevent those kinds
of errors. So there are like other dynamic things going on,
And like I said, Core is such a large code base,
and there's like so much variance across like its failure
(22:01):
modes that it's hard to get the signal that we
were looking for among all that background. So then we
said we actually need a smaller code base to test
our hypothesison to see if we're able to move the
needle in that type safety in any direction. So after
(22:21):
we did the initial adoption on Core, we turned our
attention to a much smaller and more opinionated code.
Speaker 2 (22:29):
Base within the company, and that's a tool that we
call DEV.
Speaker 3 (22:33):
So DEV is a tool that every developer at Shopify
has on their computer and it's the tool that every
developer uses during their daily work workflows. So DEV is
a command line tool. You use it to clone Shopify
imposteres then to you only invoke dev up and it
(22:56):
knows based on a configuration file with an repo. It
knows which dependencies to install, which commands to execute to
bring the application up.
Speaker 2 (23:07):
And it also gives you different.
Speaker 3 (23:09):
Shortcuts like DEV tests or DEV type check, which was
something we added to run like various various things. So
DEV is a very opinionated code base because it's a
developer tool. It's a command line tool. Everyone uses it
throughout the day, so it needs to be fast. It
has for that reason. It has no external dependencies, so
(23:31):
it doesn't use any any gems or anything. If it
needs some other code, it vendors them in. So it
was like the ideal code base, and the team that's
working on that codebase is a small team that's been
working on the same codebase for a long time. So
we talked to that team and we told them that
we were interested in adding type annotations to the majority
(23:54):
of the dev code base and to see, you know,
what kind of an impact we could get on type safety.
Speaker 2 (24:01):
That was a project that took like a couple.
Speaker 3 (24:03):
Of months because it's especially hard to add typing to
code that you already have because it means you need
to understand how data is flowing throughout the code because
you have some method that ends up calling some other
method and then combining the results from that other method
(24:25):
by the result of another method call or something. So
you have these chains of method calls that you need
to understand what types they return to understand what types
your method in question is returning.
Speaker 2 (24:37):
So it becomes like an.
Speaker 3 (24:39):
Arduous process to fully type and existing code based. So
that took a couple of months, but we approached that
very methodically. So we took a look at the ways
dev fail, and we took a look at all the
ways that dev fails that's related to type related problems.
So any exception that dev was raising. There were always
(25:02):
like all being captured anyway, So any exceptions that was
raising that were type errors, no method errors or argument errors.
Speaker 2 (25:12):
We collated that. So we drew graphs and everything.
Speaker 3 (25:15):
So that was before we started this experiment, and we
drew the same graphs for after we finished this experiment,
and our findings are interesting. So we were actually able
to completely eradicate namers because that's the lowest hanging fruit.
So the basic premise that Sorbe gives you is to
(25:37):
ensure that all the constants are resolved somehow so that
you don't have any constants that Sorbe.
Speaker 2 (25:44):
Doesn't understand in your codebase.
Speaker 3 (25:46):
So and that's what a namer is, right, Like you
refer to food, but food doesn't exist, so you.
Speaker 2 (25:52):
Get a namer.
Speaker 3 (25:53):
And if you're not doing like const get or something,
it's something dynamic in your code base. Just static analysis
is in to eradicate all the name error problems.
Speaker 2 (26:03):
And then we also looked at, like I said, type errors.
Speaker 3 (26:06):
There were a few type airs left still in the
depth code base, but we did trace them back to
files that weren't marked type true. And maybe I should
just explain what type true, type foles all those levels
are so sorbe is a gradual type checker, so you
don't have to convert your HOLD code base to be
(26:26):
typed overnight, but you can opt into as much of
sorbe as you want. So when I said, you know,
we adopted sorbe in our core code base, of course
I'm not saying like we, you know, added types across
the HOLD code based, but we.
Speaker 2 (26:42):
Enabled all the developers to add those type signatures and
to have their code checked against those type of signatures.
So the same thing in depth.
Speaker 3 (26:51):
So we did add types to the majority of the
DEPV code base, but not to all of it, and
we were able to track the type errors to the
parts of the based that.
Speaker 2 (27:00):
Were where the files were marked.
Speaker 3 (27:02):
Type false and type fole says, do not type check
any of the method calls in this file. Only check
that like the constants resolved to constants that you know,
it does the basic type checking, whereas in type true
the file is type checked for all the method cads
and for all the parameters that are being passed correctly,
(27:23):
and if the methods have signatures, that all the methods
have the correct types.
Speaker 1 (27:28):
Yeah, it's just going to say, it sounds like you
were able to test more or less your your premise, right,
which is that exactly we're going to be able to
eliminate all of these name errors.
Speaker 3 (27:36):
Yes, so the name errors is the lowest hanging front
because you take a type false on top of your
file and run sorbey over it, and it will tell you, oh,
you have this constant that you're referring to here in
this file, but I have no idea what it is.
Speaker 2 (27:51):
So it's either coming from somewhere else in your codebase
that Sorbe isn't processing, or it really doesn't exist, so
you just, you know, remove your users. So nay or
was simple one.
Speaker 3 (28:03):
But we were able to reduce type errors as well,
and we also realized that we were able to reduce
argument errors and no method errors as well, though not
to the same extent, because even though you're opting into types,
you can there are still situations where when you call
a method, the method doesn't have a signature. Sarbe's best
(28:24):
guess is it could return anything that's represented as a
key on typed in sarbe. That's the equivalent of any
in type script. If you know you've worked with typescript
so far, so it's a type that can be anything,
so you can call any methods on it with any parameters.
Speaker 2 (28:40):
So basically SARBA stops doing type checking past that point.
Speaker 3 (28:44):
So a ton typed always returns a theon type when
you call a method on it and everything, so that
kind of lessens the amount of strict typing that you're doing.
You can often to hire types strictness levels in your
files to prevent so you can often to type strict
which will tell you if you're doing this unsafe calls
(29:06):
and it will say, oh, you're actually calling into a
method that doesn't have a type signature, so you know
there's something wrong here, so go and add a type
signature to this other method that you're calling to make
sure that this file has a stricter type checking. So
we didn't go that far in the depcode BIS. We
wanted to see how much we could get with simple typing.
(29:27):
So there were still a few argument errors or no
method errors that leaked in, but we're able to reduce
those kinds of errors as well. But that's not the
only thing we did. We also looked at if running
type checks on CI had any effect on the CI's
success right, So were there, like any false positives that
(29:48):
Sorbey was failing on that were breaking CI built or
vice versa. Were there any test failures that Sorbey wasn't
catching and we realized that they or they didn't have
an effect either way, So it wasn't causing superious superiors
failures on CI, nor was it really missing anything. So
(30:09):
there weren't really that many test failures that Sorbe also
didn't catch.
Speaker 2 (30:13):
But we also talked to the team, so I think
that's one of the most interesting things we've done.
Speaker 3 (30:18):
We talked to the team both before we started this
experiment individually and talked to them after we finished this
experiment to see so before we wanted to see how
much they knew about Sorbe, how they felt about Sorbey.
Were they enthusiastic or did they have questions about its
utility or not, So we compiled those and then after
(30:40):
we finished this experiment, we also asked them if they
were using survey time checking on their machines, if you know,
they were happy adding signatures to the code based or
if they had problems having to deal with fixing signatures
when they were factorings. So we were able to also
get a qualitative feedback on how the team and how
(31:01):
the developer happiness was impacted or not, and we actually
heard good things about that. So there was one person
on the team who really didn't like signatures. They found
it like really distracting. One of the things about Sorbet's
signatures is because Sorbet is not integrated with the language. Obviously,
Ruby as a language doesn't support types, so you need
(31:24):
to add a signature annotation on top of your method definitions,
and that signature annotation is also Ruby goat. So you
do like a sig block, So you say sig and
then you start a block, and then you say params,
and then you declare your parameters and their types, and
then you do a dot returns and then you declare
(31:45):
the type of your return and then you can do
more things in there as well. But obviously sometimes for
a two or three line method, your signature can end
up being five or six lines if it's doing something
like non trivial, or if your types have like some
gender sort something. So some people actually find that two
of their posts and sometimes distracting, so they say that
(32:08):
it makes it hard to actually see the code. They
end up seeing like all these signatures, but that person
on the team actually developed like a few villim configs
to de emphasize the signatures. So they basically turn the
opacity on the signatures so that they look like comments,
which they kind of are right like, because that's the
(32:29):
same thing with yard definitions. So when you add you know,
type annotations in yard comments, that's.
Speaker 2 (32:35):
The same thing. And then someone at the company also
built a visual studio code extension to do the same thing.
Speaker 3 (32:43):
I think it's called bisig, So it turns the opacity,
it parises the signature blocks, and then turns the opacity on.
And I've actually been using it and I'm also enjoying
it because I don't necessarily need the.
Speaker 2 (32:56):
Signatures to be in front of me all the time.
Speaker 3 (32:58):
But apart from that, we heard that developers actually liked
the safety guarantees that static typing gives them as they're
doing refactoring. For example, So for example, if they're removing
a class or a module from the CLUD based they're
almost guaranteed to know that when they run the standard
type checker that it will complain if there's like any
(33:20):
dangling usages of that class or module. So they really
liked those guarantees and they didn't really mind writing the
signatures in the first place. And it's a really easy
ramp on as well on the signature syntax. So we
actually heard good things from developers on working on the cloud.
Speaker 5 (33:39):
Based so has used in sorbe made you guys write
better code?
Speaker 3 (33:44):
Yes, that's also another one of our findings. We also
talked to three different teams that who were working in
core and they were one of the early adopters of
sorbet in the better parts of the Colde base. So
we have this componentization thing going on in our core,
in our monolith.
Speaker 2 (34:02):
Because it's a huge cod base, otherwise it becomes mayhem.
Speaker 3 (34:06):
So there's the components and some components worth your early
adopters of typing. So we did interviews with team members
from those teams as well to understand how they felt
about those early initiatives, and one of the things that
we ended up hearing over and over was that adopting
types led to better code design, which initially was interesting
(34:31):
for me, but that seems to be the biggest benefit
from adopting static or gradual typing rather than you know,
runtime type safety.
Speaker 2 (34:41):
And so like my.
Speaker 3 (34:42):
Opinion is a little bit swayed now because too often
when we're working on simple solutions, we reach out for
like these bags of values like arrays or or even
like most of the time hashes, right, So we just
like reach for a hash, and then we keep passing
those hashes around, But then it's never documented what the
(35:03):
keys are, like if let alone, if the keys are
symbols for strengths, Like, how many times did we assume
that we could access some hash using symbols?
Speaker 2 (35:13):
It turns out to be that it was indexed by strengths.
Speaker 3 (35:16):
Right, So as you're like, when your codebase is small,
that's fine, I'm not judging anyone who's doing that. But
as your codebase is growing, you need people two more easily.
Speaker 2 (35:27):
Onboard onto your team. And if they're fixing something, and
if their focus is on let's say one method or
one file, you don't want them to go hunting around seeing, oh,
where's this hash.
Speaker 3 (35:40):
Actually being coming from? Where was it created initially? What
are all the parameters that are in this hash? So
that's not like a very efficient way of working at
that scale. So at that scale, it's better to give
a name to that, to turn that into something like
a structure or like create class for it or something
(36:01):
to basically start using types. Right, once you turn something
that was a hash into a strup or a class
or a module, you give a name to it.
Speaker 2 (36:10):
You give a type to it, so you start.
Speaker 3 (36:13):
Using types, and those types actually convey more information than
passing around hashes would, so it leads to actually better
code design, and we've heard that over and over again from.
Speaker 2 (36:25):
Various people within the company.
Speaker 3 (36:27):
But it also leads to evergreen documentation as well. So
if you're looking at a method and it has a
signature on top of it, and it says these are
the parameters that it takes off these types, and it
returns this type, then you don't need to go and
look anywhere else. Basically, if you know what those other
types are as well, you don't need to go look
anywhere else to all the colors of this method or whatever.
(36:51):
It's just evergreen documentation. And the reason why it's ever
green is because if somebody changes the method signature, they
have to change the signals you're on top of the
method as well, otherwise you will have like the static
type checker breaking in CI, right, So it has to
be evergreen people need to keep it up to date,
and that makes it very easy to onboard new people
(37:13):
to your code base, or to your team, or to
your company as well. So we we heard from the
blopers that those were all great benefits of adopting types,
and like I said, we didn't adopt types across the
whole code base, but we're allowing people to adopt those
types gradually as much as they want. And our team
(37:34):
is also helping people within the company in their adoption process.
So sometimes we help them, we review their prs, sometimes
we actually pair with them to make that adoption easier,
or we just answer questions in a.
Speaker 4 (37:48):
Slack channel when there's that I hate types, sure, spent
years try to get away from them. The only reason
I still program in ruviews because it just doesn't have
That's the kind of is why I like it. It's
there's freedom from types. So my question is, what's what's
wrong with Ruby? What's going with Ruby? Because this is
(38:11):
obviously not just a Shopify project. This is something that
Stripe also looking at. So there must be some kind
of common problem that led you both to say, we
can improve our product, we can improve our developer experience,
we can find problems, sooner we can have this kind
of really kind of pick up on productivity by introducing
(38:34):
this system on top of Ruby. And it's a pretty
clean system. There's there's a link to a little what
do you call it, a sandbox, a playground where you
can try it out. And what I really like about
it is you said, is that you don't have to
go through your whole code base and start typing in
types for everything. If you've got something of mistake that
(38:56):
you keep making in a project, then you can just
kind of drop the seeing on that problem, class that
problem method, and then no one will ever have an
excuse for breaking it again. So it's a real it's
a real hammer to kind of drop on that. But
what's your what's your opinion? I know you talked a
(39:16):
lot about the developers and how they feel about introducing
typing and what they got from it. This is something
you must feel very passionately about. Where do you stand
on Ruvian types?
Speaker 3 (39:28):
Okay, so personally, just before I joined Shopify, I was
working on code basis that were built by Typescript, and
actually Typescript was I think there is something wrong with me,
and I'll come to that. So Typescript was I was
at the time working at a web agency, but I
was on the team that were doing like cutting edge
(39:49):
two weeks prints that were like trying to see if
we could make certain things work, and I was we
were mostly working on JavaScript, and I consciously chose to
tried Typescript to see.
Speaker 2 (40:01):
How it would work.
Speaker 3 (40:02):
And I Typescript coupled with visual Studio code was a
joy to use, and I started loving it for all
the code completion and all the type things and you
don't look at the documentation and a lot anyway, Like
I was really enjoying Typescript before I joined Shopify, and
I had already heard about Sorbet, and like Sorbe got
(40:23):
me similarly excited because it has it still has the
promise of doing the same thing that Typescript did for JavaScript,
which was not to destroy the language as a language,
but to add features on top of it that people
could opt into if they wanted or needed more strict typing.
Speaker 2 (40:46):
And it's again this.
Speaker 3 (40:47):
Whole thing because typescript is one hundred percent JavaScript plus
some other things. The difference with typescript is it compiles
into JavaScript, so it does not type checking at run time.
Whereas with sorbe, it's built inside of Ruby, so all
the sorbet annotations and everything are also Ruby constructs, so
(41:08):
those Ruby constructs can also do type checking for type assertions.
Speaker 2 (41:12):
At run time.
Speaker 3 (41:13):
So that's the kind of difference. But looking back, I
kind of see myself having almost exclusively worked in code
bases that had some types somewhere. So I'd worked with
DELFI for a while, and then C plus plus D sharp,
and then, like I said, typescript. But before Typescript, we
(41:34):
were working on a PHP code base that was using
type pins. So that was a five point three code
base if I remember correctly, which had type ins, and
we were really relying on type ins. So the project
that started before I joined and it was built like that,
and I ended up feeling like that was a good
(41:55):
way to work because it was preventing us from shooting
ourselves in the foot. Even though that was still doing
type checking at runtime, it was still failing fast, right,
So that's like one of the principles that we software
developers use a lot, this ability to fail fast.
Speaker 2 (42:13):
So static typechecking really gives you that. So it allows you.
Speaker 3 (42:17):
To fail fast either in so base case, you just
run the static type checker and within a couple of
seconds it shows you if there's like some assumptions in
your code based but don't check out. And that's way
better than waiting on the CI for.
Speaker 2 (42:32):
A couple of minutes. You can just run this tool
constantly on your machine right right.
Speaker 4 (42:37):
So this is where it kind of seems to differ
in terms of develop a tool to running a test suite.
This is this is not really the same as running
a series of tests. This is something which you can
use kind of almost kind of running continuing the black
in the background in official studio code.
Speaker 1 (42:58):
Is that how you use it?
Speaker 3 (42:58):
Yes? So, so ARBA actually has an LSP mode. So
the Sorbe binary, that's the C plus plus static checker binary.
If you download the Sorbe static gem or the Sorbae
JAM which pulls it in, you can actually do SIBTC
DASH dash LSB which turns on the LSP mode. So
(43:18):
LSP stands for the Language Server Protocol. I think I'm
not so sure about the acronym, but it's developed by
people at Microsoft. As they were developing visual studio code,
they developed that specification. So it allows different languages to
integrate with different editors, and LSP is the language that
(43:39):
sits in between, so Sorbe has this LSB server mode,
and they're also working on an official Visual Studio Code
extension that you can install that would bring all that
richness into Visual Studio Code. And I'm one of the
early testers of that inside of Shopify, and I've been
using it for a while now, and it's actually integrated into.
Speaker 2 (44:03):
Your editors, so it gives you.
Speaker 3 (44:05):
The best auto complete that you can ever find, way
better than all the other editors can do. But it
also shows you all the errors within your editor. That's
a great way to work because you get instant feedback
on your code changes. I don't want to give too
long an answer, but just to address the second part
(44:25):
of your question is like when would.
Speaker 2 (44:27):
You reach for a tool like this? So scale is
very important.
Speaker 3 (44:30):
So when you have, like in our code base, tens
of different components and thirty thousand different files, then it
becomes really important to have better contracts between all those
different moving parts, and what better contract than types?
Speaker 2 (44:48):
And we've actually realized that types really work.
Speaker 3 (44:50):
Well for that, and I think that was the initial
impathus for why Stripe started working on this as well,
and I'm not saying that all Ruby codes should be
typed or that everyone should reach for typing as soon
as possible. Obviously it's a matter of needs and scale,
but I also feel like that it's the right tools
for the scale that Shopify is in right now, and
(45:13):
the teams that are really relying on types are getting
a really good return for their investment.
Speaker 4 (45:19):
So obviously we're just coming out of the COVID nineteen time,
and I understand that Shopify has changed their working practices.
What's it been like at Shopify over the last few months.
Speaker 3 (45:35):
Actually, Shopify's response to this whole pandemic has been really
impressive because senior executives at Shopify were aware of where.
Speaker 2 (45:46):
The whole pandemic situation was going really early.
Speaker 3 (45:49):
On, and they were really quick in taking measures to
ensure that people working for Shopify are sick. So that
was like a very early on so they basically closed
all the offices before like most countries were aware of
what even was going on. So for me personally, I
(46:12):
can't quite talk for those office closures because I've always
been working remotely for Shopify, so I live in Cyprus.
I worked for Shopifi remotely from here, and our whole
Ruby and Rail's infrastructure team has been distributed has been
a distributed team as well, So we had a couple
of people working in the offices in Ottawa, someone in Montreal,
(46:36):
one person in Poland, one person in France, UK, Cyprus.
Speaker 2 (46:39):
So all over the world.
Speaker 3 (46:41):
So our whole team, dynamics and everything didn't obviously change
that much. But obviously, as everyone's feeling it, starting to
work remotely during the pandemic was a huge burden to
a lot of people because they weren't ready to work
like that.
Speaker 2 (46:57):
Also, it wasn't a normal.
Speaker 3 (47:00):
Working remotely set up either, right because schools are also closed,
everything else is closed. You're trapped in this house and
you can't go out, and or you're also trying to
do work, so it's not necessarily the normal working from
home situation. But like you said, our senior executives looked
at the situation going forward. So first of all, Shopify
(47:23):
isn't reopening any of their offices before the end of
this year, whatever happens, because the outlook isn't that great.
It doesn't look like this pandemic is over anytime soon,
so there's no point in like opening offices and making
people feel safe and going there just to then like
close them again. But also the company has, based on
(47:45):
this experience that we've had for the last couple of months,
the company has also.
Speaker 2 (47:48):
Decided to be digital by default.
Speaker 3 (47:51):
Which doesn't it is digital already, but it wasn't digital
by default, right, So that's that's the extension. So what
used to be the case is, for example, if there
were people in the office and if there was a meeting,
so if there were like ten people in the office
and three people joining remotely, then the ten people in
the office would be huddled up in a meeting room
(48:13):
and then the three people would you know, join in
on a call remotely. But that isn't necessarily the best
way of working because the people who are joining that
call remotely don't get the same experience that those ten
people in the room get. So the digital by default
kind of changes that. And they say, we're not closing
(48:33):
our offices, but they're saying the offices won't be the
same offices, so they'll be revamped to accommodate this new
digital by default future. But basically then in that future
we'll get to experience it, obviously, but in that future,
those ten people will also individually join that same meeting
(48:53):
as well, so everyone will be digital in the meeting
by default, rather than having, you know, ten people collocated
somewhere and then three people remote. So that's the biggest difference,
because the decision there was very simple, because you either
either do fully collocated or you do fully remote. In
(49:14):
between is this hybrid thing that doesn't work that well.
Speaker 2 (49:18):
For either party.
Speaker 3 (49:19):
Right, Like, you have a few people that are co
located in an office people that are remote, but then
you need to have all your communication digital to ensure
that the remote people have the same context as the
people who are co located. But people who are collocated,
they if they have like chats that they have in
between and they don't report them back, then you have
(49:41):
information and symmetry, et cetera, et cetera. That's the worst
way to work. I've also worked remotely, being the only
remote person on a team that was co located before
not at Shopify, and that wasn't a horrible experience because
my team was very understanding of my situation. But it's
still not the best way to work when all your
teammates are all in the same office and you're the
(50:03):
only one remote. So the future for Shopify is digital
by default for everyone. So even if you're in the office,
you're still communicating with everyone digitally. That will be the
primary mode of communication and most of our work will
not be done from offices anyway.
Speaker 1 (50:21):
Sweet, then let's roll on into picks then, Dave, would
you start us off with picks this week?
Speaker 2 (50:27):
Yeah?
Speaker 1 (50:28):
Real quick.
Speaker 5 (50:28):
If people want to follow some of the things that
you're doing online, where should they go and look all.
Speaker 2 (50:33):
The engineering work that we're doing.
Speaker 3 (50:35):
We have a Shopify engineering blog that mostly gets a
new blog post once a week, I think, or the
Shopify Engineering Twitter account they can follow.
Speaker 2 (50:46):
And I should also.
Speaker 3 (50:47):
Say that like, Shopify is still hiring and we've been hiring.
Actually we ramped up are hiring across the pandemic as well.
So if anyone's interested in a position at Shopify. Also,
now that we're digital by default, we're more open to
remote people. They can go to selfify dot COM's last
Careers or our career stage on our website and they
(51:09):
can be a part of the team so that they
can follow the developments from the inside.
Speaker 1 (51:12):
Awesome.
Speaker 5 (51:13):
Yeah, and I'll go ahead and kick us off with
some picks. So my first pick is the whyse then clients.
So I've been getting my kids ready for the digital
learning and the upcoming school year, and instead of giving
them each a six hundred dollars computer or something, I
have deployed a thin client set up at home, which
(51:34):
is probably way overkill for most home networks and stuff,
but I would much rather replace a one hundred dollars
all in one thin client than a six hundred dollars
computer and modern. So that's my first pick, and second
pick is I on this call. Actually, I got the
announcement that I got the Apple TDK, so the Developer
(51:59):
Transition Kit with the ARM based processor. So I will
be releasing some drinking Ruby videos on developing on ARM
based processors soon.
Speaker 4 (52:09):
That's pretty cool.
Speaker 1 (52:10):
Nice. I'll be pointing some people that I know are
super interested in that to you.
Speaker 3 (52:14):
Actually, yeah, we're really interested in that on my team
as well, because the people are doing for Ruby and
MRI work, I want to make sure that both of
those platforms run really well on on your arm jet.
Speaker 1 (52:26):
Look, what do you have for us this week? Well?
Speaker 4 (52:28):
To tie in with bu Hook's talk at Rail's twenty
twenty Couch Edition which talk is called peeling away the
layers of a network stack. It's a good talk. My
pick is the evergreen tcp ip Illustrated, a big book
(52:49):
of how networking works. If you want to learn how
to perform hilarious office pranks like art poisoning, if you
want to get kind of get to get to grits
with networking, this is a great book to really trid
into detail. So there we go. Tcpip Illustrated recommended to
me many years ago by a man from Florida, and
(53:12):
boy was he right.
Speaker 1 (53:13):
It is a great read. Awesome. So I have a
couple of picks for this week. One is actually something
that's been out for quite some time now, but I
really wasn't introduced to it until very recently. It's called ASDF.
It's just if you're familiar with RBM or RVM or
any other version manager or any language. ASDF is more
(53:34):
or less a similar thing, except that it's across It's
really just cross language, cross tool kind of thing, right,
like they have plugins for things like postgrass and my sequel.
I don't really see a lot of value with postgrass personally,
but I did see I was trying this out on
a project that have my sqel and switching between versions
is legit. So yeah, I mean basically with this. The
(53:58):
thing about this tool is it's really for making it
so that you can set up your development machine to
switch between versions of Python and Ruby and Node and
your database that you have, and you know, like a
whole bunch of different tools. And I mostly use Docker
these days, so this really isn't like a thing that
(54:19):
I'm going to use all the time. But I do
have a couple like outlying projects that are just fun
and so I explicitly I was introduced to this, I
was like, all right, I'm gonna try this out. It's legit.
So I don't know that I'm ready to completely give
up our VM for this yet, mostly because I do
almost all Ruby work, but it's definitely taken some mind
(54:39):
share up in my brain at this point, so it's
pretty cool. Definitely recommend checking that out.
Welcome to Ruby Roags today. On our panel this week
we have Dave Kimura everyone, and we have Luke stutters
Hi and I'm John Epperson. And for a guest this week,
we have case Leo. Luke Yes, Hi, welcome. Would you
tell us what you're famous for, what you do, just
(00:26):
a little bit about you so we can get kind
of going here.
Speaker 2 (00:29):
Sure.
Speaker 3 (00:29):
I'm currently working at Shopify on the Ruby and Rail's
infrastructure team. I've been at Shopify for about a year
and a half and since I've joined, I've been first
on the Rails upgrade project, and then I quickly switched
over to the Sorbey adoption team and so we spearheaded
the adoption of Sorbey across our monolith and across our
(00:53):
other code.
Speaker 2 (00:54):
Basis as well. And now I'm slowly transitioning to being
the team lead or the Ruby part of the team.
So the team will.
Speaker 3 (01:04):
Be responsible for contributing to MRI and contributing to Sourbet,
basically solving and maintaining and building on the.
Speaker 2 (01:14):
Open source Ruby foundations.
Speaker 3 (01:16):
I've been in the software development industry for over twenty years.
Speaker 2 (01:20):
I've worked with very statically type dynamic.
Speaker 3 (01:23):
P time languages across that time. Initially I was working
in a startup. I was one of the family employees
where we were developing voice XML based applications, and we
built a voice XML platform, and then we did like
various networking things voice so variety, text to speech, speech recognition,
(01:44):
et cetera, et cetera. I had a short stint in
between for a couple of years doing startup acceleoration, and
then went back to developing software and so apped me
in a nutshell.
Speaker 1 (01:55):
Wow, So there were there have been a couple of
talks that you've given it that I know about. I
don't know if you've given more than that, but.
Speaker 3 (02:03):
Yeah, I have one talk at Ruby COMF twenty nineteen
last year, and I also submitted a Rails comp twenty
twenty the couch edition, the remote one.
Speaker 2 (02:14):
I also had a talk there.
Speaker 1 (02:15):
Yeah, sorry about the confusion. One was on sorbet and
one was on the network. I know that I, particularly,
for one, am interested in hearing about your adoption at Surbey.
I think that's super interesting to me. Anything that you
particularly want to note about either of those things that
you're super interested in yourself.
Speaker 3 (02:33):
I think we can maybe talk about the network stack
video and the talk from Rails comp near the end
of the show if we have time, because I think
most people would be interested in the sorbe adoption and
how a company with a code based as large as
Shopify has been successfully adopting it. I'm presuming that's what
most of the listeners would be interested in, so we
(02:54):
can get startled with that if you want.
Speaker 4 (02:56):
How large is the Shopify code? Because Shopify moved forty
one billion dollars of merchandise in twenty eighteen, forty one
billion would have been how many dollars per line of
code does that work out to?
Speaker 2 (03:15):
I haven't calculated that.
Speaker 3 (03:17):
I don't know, But like in case some of the
listeners aren't familiar with the company, and maybe I should
just give a quick intro.
Speaker 2 (03:26):
So what Shopify does? So.
Speaker 3 (03:28):
Shopify is a leading global commerce company that provides tools
to start, grow, market and manage a retail business of
any site. So it started off in two thousand and
six codebased that was built on Rail's pre one point zero,
so it was the time of Rails when it was
(03:48):
still being shared as zippiles and it's been growing ever since.
It started as an e commerce platform and has grown
into being a multi touch general commerce platform, so it
provides solutions both for e commerce and also in store commerce,
in person commerce, so it has many solutions.
Speaker 2 (04:08):
Now.
Speaker 3 (04:08):
It supports more than one million merchants now and throughout
its lifetime it supported one hundred and seventy two billion
USB in total sales for all the merchants. Of course,
that's lopsided to the recent here, So the figure that
that you quoted is probably correct. Look like, I don't know,
(04:30):
off the top of mind.
Speaker 4 (04:31):
Calls it from Wikipedia, so it's almost certainly wrong.
Speaker 2 (04:36):
It sounds believable.
Speaker 3 (04:38):
Then, As for the code base, it's most probably the
largest Ruby on Rails code base in.
Speaker 2 (04:43):
The world right now.
Speaker 3 (04:45):
It has about thirty thousand Ruby files in our core monolith.
So I'll keep referring to Core. So when I say Core,
that's our main monolith that provides almost all of the
shop off solutions.
Speaker 2 (05:00):
Even though we now.
Speaker 3 (05:01):
Have some other services that operate outside of Core, most
of our code base and most of our operation is
actually still being handled by Core, which is our hour monolith.
And that's about thirty thousand Ruby files. And on each push,
on each deploy there are about one hundred and fifty
(05:22):
thousand tests that run to ensure that everything is still same.
So that's the size of the code base, that's the
size of the platform.
Speaker 5 (05:31):
And that's crazy. I think, you know, a lot of
us would dream to get to that kind of scale. Honestly,
I think it would be a nightmare. Not only do
you have the super huge codebase to maintain, but then
just the infrastructure alone, and what that would look like
would is probably be really crazy. Can you speak into
the infrastructure, you know, are you guys using a very
(05:53):
simple infrastructure or has it kind of gone from simple
to like very elaborate.
Speaker 3 (05:58):
So I've only been at the for about a year
and a half and most of the infrastructure has been
in place before I joined, and I'm also still learning
about some of the infrastructure that we use. But like
I can tell you, it's a kubernkkis deploy so everything's
virtualized in Docker containers and deployed to Kubernets pods, and
(06:21):
there are like some load balancing solutions, and we also
have this concept of pods. Pods are these logical units
that host a group of certain merchants in one location
with their own basic infrastructure. So that's like one way
of sharding the customer database and the customer load across
(06:42):
different machines. I'm pretty sure that, like others from the company,
have presented at various conferences talking about that infrastructure part,
but that's not necessarily my strongest point, so I don't
want to really go much more in depth about that,
even though the team that I'm working on, the Rails,
the Ruby and Rail's Infrastructure team, is a part of
(07:06):
the product line in the company called Kernel. Kernel is
basically responsible for ensuring that the platform is off and running,
so it's responsible for all the infrastructure problems, but also
it includes Developer Acceleration, which my team is a part
of as well, which provides solutions for the developers within
(07:26):
the company so that they can do their work better,
they can produce solutions faster, and also we provide solutions
for the infrastructure part of the team as well.
Speaker 1 (07:36):
Yeah, I definitely remember more than one talk at Rails
comp throughout the years on how you guys infrastructure changed.
And I remember sitting at the table with some Shopify
people one year who were talking me through how they
were moving from Vagrant to Docker. So yeah, I do
remember the story changing. I'm not expert enough to talk
about it by any means. But that's been crazy to
(07:56):
watch from the outside. Yes, so all right to get
down into sorbet. What maybe from a high level talk
about what you guys did to make survey happen at shopify.
Speaker 3 (08:07):
Sure again, maybe I should give a little bit more
context as to what sorbet is, who developed it. What's
used for sorbet is gradual type checker for Ruby that
was developed by the fine folks at Strip. So sorbet
provides you the tools to add type checking information to
(08:27):
your Ruby code base, and it gives you both static
typechecker and a runtime type checker to ensure that your
type assumptions are makes sense and are valid. The static
type checker is the most important part of sorbe. The
static type checker is a tool. It's an executable built
using C plus plus. It has its own Ruby parser,
(08:49):
so it's able to parse and understand Ruby and in
some ways simplify the Ruby language, so it can do
those kinds of static analysis without necessarily running your code.
Well I shouldn't say necessarily.
Speaker 2 (09:03):
It doesn't run your code at all, So in that way,
it's a static type.
Speaker 3 (09:07):
Checker and it's built to be blazingly fast, so it
can parse, understand, and type check our whole code based
the code based core code base that I've been talking
about of about thirty thousand files in a matter of
like ten fifteen seconds. And it was built with intentionally
(09:30):
to be fast and to be as safe as possible.
And when you're developing, sorbet was open sourced about a
year ago, so it was June twenty nineteen when it
was open sourced, but it was being developed by Stripe
for about I think two years before that, And by
the time I joined Shopify, we'd already gotten on the
(09:54):
early release beta program where they were doing like code
sharing with Shopify, so we were able to start our
adoption earlier than when it was initially open source. Then
afterwards Stripe also got like a few other companies that
were interested into the public data program as well. But
so that's why our story is a little bit different,
(10:17):
because we ended up solving some of the problems that
people who might want to adopt sorbe don't have to
solve right now because it's a much more mature product.
But back when we were trying to adopt it on
our code base, it was mostly a tool that was
built with the Stripe code based in mind. And I
(10:39):
can't talk too much about what Stripe codebase is like
because I don't know, but from our understanding is it
doesn't use RAILS. It tries to use as little meta
programming as possible. So they're like really big on intention
revealing code and not so much meta programming. So Sorbet
(10:59):
was the right tool for them, and they were developing
to solve their own solutions obviously initially, but because ar
codebas is RAILS, which already has a lot of meta programming.
So meta programming brings in a lot of dynamic methods
that can only be seen when you're running the code right,
So there are lots of methods that you don't see.
(11:23):
For example, on active record models, you don't see all
the attributes right because the attributes are created at runtime.
All the methods that relate that are related to attributes
are created that runtime, after RAILS has had a chance
to introspect your database levels. Obviously, that's a big problem
for Surbey because it looks like the code tries to
(11:43):
understand and sthatically and it says, oh, this.
Speaker 2 (11:46):
Model has no methods. So in another part of the codebase.
Speaker 3 (11:51):
When you're trying to do shop dot something Serb says, oh,
like shop doesn't have this method, so that ends up
being a problem.
Speaker 2 (12:00):
So one of the first things we contributed.
Speaker 3 (12:03):
To the Sorbeko disk was a way to have the
static type checker understand some of the NATA programming concepts.
So our initial idea was that we could write scripts
in Ruby and so they could basically execute them so
that the scripts could generate what methods would exist if
(12:26):
the code had ran, but it would still be a
static type checker. It kind of tided us over that
solution for a while, but it ended up slowing down
the type checking process a lot, but at least it
allowed us to do our own initial adoption at the time.
Another problem as we were adopting was that all the
types that are coming from our GEM dependencies, So our
(12:50):
core monolith has about four hundred GEM dependencies, which is
already like a huge maintenance problem, right, like we need
to maintain them to be on the latest versions and everything.
Speaker 2 (13:03):
But it also means there are.
Speaker 3 (13:05):
Lots of classes, modules, constant definitions coming from those gems.
But Survey doesn't go and look for class definitions inside
of gem source of it just looks at your code base,
so it says, oh, you're using this foo class here,
or like this food constant here, but I have no
(13:26):
idea what foods. So it needs to be told what
foo is, what kind of methods it has, what kind
of subconstances it has, what type of a constant it is?
Speaker 2 (13:36):
Is it a class, is it a module, what's superclass?
What are the mixings, et cetera, et cetera. So we
needed a tool that would just take.
Speaker 3 (13:45):
A GEM somehow, introspect it, and then generate an RBI file.
So it's called an RBI file that's a Ruby interface file.
It's almost the same thing as a c header file.
So it's just the definitions of these things that I
just said, is it a class, is it a module?
What's the superclass? Mixing the methods and its arguments. So
(14:07):
it just generates those without any of the implementation. So
we had to build a tool and we open sourced
it afterwards as tapioca.
Speaker 2 (14:16):
That does that. It takes your GEM file, it discovers
all the gems, it loads them into memory.
Speaker 3 (14:22):
Then it does runtime reflection on all the types that
can find from all those gems, and then it tries
to understand what the shape of all those types are
and it generates that into a set of rbiphads, which
then so they can read, and then it then it
knows about food coming from the gem flo or something.
Speaker 1 (14:45):
So just a quick follow up there, So sure you
guys have that or you have this problem in shopify
because you have a bunch of gems. Did the people
they created this gems right?
Speaker 4 (14:54):
Yeah?
Speaker 1 (14:54):
Stright? Yes, I don't know why I couldn't think of
it for a second. Did stric not have any gems? Depends? Then?
Speaker 2 (15:00):
No, they did. So basically they had a script that would.
Speaker 3 (15:04):
Kind of do the same thing that they were using internally,
but it wasn't part of the Sorbet.
Speaker 2 (15:09):
Tooling at the time.
Speaker 3 (15:10):
Okay, So they shared the initial skeleton of that script
with us, and then we ended up turning that into
the tool that's now Tabioka, and then the Storbe team
at Stripe they actually also took that script and then
built it into the sorbet in It script, which does
something very similar. It also can look into gems and
(15:34):
load types and generate ruby interfestiles for you. So that
ended up being a part of the tool, the Sorbet tool,
but we took a slightly different route than that tool,
and so that's why we're actually using Tapyoka internally rather
than the Sorbet in it tooling.
Speaker 1 (15:53):
Okay, what else did you encounter?
Speaker 2 (15:55):
Yeah, so the other thing.
Speaker 3 (15:57):
So going back to the DSL problem, it turned out that,
you know, making Sorbet run Ruby scripts as it's trying
to make sense of your code.
Speaker 2 (16:08):
Based statically was a huge slowdown.
Speaker 3 (16:12):
So we basically stopped doing it, and then also realized
that the same approach, even though it has some disadvantages,
but the same approach of generating Ruby interface files for
all the meta programming, would be a much better fit
for our problem. So recently we've been working on that problem.
(16:33):
There are other solutions in the same area as well.
One of them is, for example, the gem called sorbet
Rails that's built by people at the Chans Zuckerberg Initiative.
So they had the same problems of Sorbey not understanding
some of the meta programming that's coming from rails again,
like all the column related methods on models, all the
(16:56):
association related methods on models, all the the class methods, sorry,
all the instance methods in mailers that are turned into
class methods, all the helpers on your controllers, they're all
happening at run time in rails. So Sarbe rails was
the first attempt to actually build a gem that could
(17:18):
generate Ruby interface files for all those things, so that
Sarbe can look at the Ruby interface file. So you
have your shop model or like shop dot irb file
which hosts your shop model, but you also have this
shop dot rbi file that's generated for you, which contains
all the method definitions that would be on the shop
(17:39):
model dynamically. So having that, Sarbe can say, oh, yeah, okay,
I know a shop as this name method, So if
you call shop dot name, then yes, I know what
that is. And even better if you can also create
signatures for it, so if you can tell it what
parameters it takes and what the return type is, then
(18:01):
you can also keep typing those things as well, so
you can build on.
Speaker 2 (18:05):
Those types as well.
Speaker 3 (18:06):
So survey rails was the initial solution to this, but
we also realized that we had other meta programming DSL
concerns within our code base that obviously wasn't addressed by
Sarbe rails so we wanted a broader framework, so we
built one in house, and we actually are in the
process of folding that also into Tapioca, so that Tapioca
(18:28):
becomes this once with army tool of generating RBIs either
from gems or from all the DSL meta programming that's
going on across your code based and initially we've again
targeted the same RAILS meta programming concepts, so we borrowed
and built on some of the solutions from Sarbe Rails,
(18:51):
but we've also built DSL generators for other common gems
across our code base, and we intend to grow that
set based on contributions from the community. So we have
a generator for an active record type store frozen record
identity cash that's also a Shopify gem, So like Rails
(19:11):
plus a few gems will already be addressed with Tapioca,
and that was the biggest pain point stopping us from
increasing our typing even further across our code based.
Speaker 1 (19:22):
Okay, so you guys went through all of this working
effort to get survey working at Shopify. So what did
you get from that? Like, what was your value? What
was you guys's value proposition? As best you understand it.
Speaker 2 (19:35):
Yes, great question.
Speaker 3 (19:37):
So the obvious headline thing for SORBET is to obviously
to ensure that your code is type safe, right, so
that you're not making silly mistakes, that you don't have
a reference to references to classes or modules across your
code based that are no longer there. We've found many
(19:58):
instances of that across our co based through the use
of survey for example, or that you're not calling methods
on constants that don't actually have those methods, or that
you're not providing arguments to those methods that they don't
actually accept.
Speaker 2 (20:14):
So all of those things.
Speaker 3 (20:16):
If you're working on a normal RUT code based all
of those concerns, you can only discover them at.
Speaker 2 (20:22):
One time, so you either need to.
Speaker 3 (20:24):
Have extensive testing to ensure that all those edge cases
are executed on your CI or you discover them in production.
So the initial goal is obviously was to reduce those
types of trivial errors as much as possible. Unfortunately, in
our initial adoption phase we couldn't validate or invalidate that hypothesis,
(20:49):
but we did other things to.
Speaker 2 (20:51):
Compensate for that.
Speaker 3 (20:52):
So there's a very understandable reason for why we couldn't
conclude on that hypothesis if we were able to reduce
trivial errors. Is because our core monolith does more dynamic
things then the stuff that we have in our code base.
So it kind of like serializes objects to the database
and then read serialized objects from the database, et cetera,
(21:15):
et cetera, And there's no way aesthetic type checker can
like prevent you from, you know, de serializing an object
that definition for which has mistakenly even remote from the
codebas Right, So suppose you had like a user class
and there was a user object serialized somewhere, and then
someone removes the user class, and now someone tries to
(21:37):
de serialize the user object, and then boom, your application fails,
right because the user class doesn't exist anymore, so you
can't deserialize it. So we obviously can't prevent those kinds
of errors. So there are like other dynamic things going on,
And like I said, Core is such a large code base,
and there's like so much variance across like its failure
(22:01):
modes that it's hard to get the signal that we
were looking for among all that background. So then we
said we actually need a smaller code base to test
our hypothesison to see if we're able to move the
needle in that type safety in any direction. So after
(22:21):
we did the initial adoption on Core, we turned our
attention to a much smaller and more opinionated code.
Speaker 2 (22:29):
Base within the company, and that's a tool that we
call DEV.
Speaker 3 (22:33):
So DEV is a tool that every developer at Shopify
has on their computer and it's the tool that every
developer uses during their daily work workflows. So DEV is
a command line tool. You use it to clone Shopify
imposteres then to you only invoke dev up and it
(22:56):
knows based on a configuration file with an repo. It
knows which dependencies to install, which commands to execute to
bring the application up.
Speaker 2 (23:07):
And it also gives you different.
Speaker 3 (23:09):
Shortcuts like DEV tests or DEV type check, which was
something we added to run like various various things. So
DEV is a very opinionated code base because it's a
developer tool. It's a command line tool. Everyone uses it
throughout the day, so it needs to be fast. It
has for that reason. It has no external dependencies, so
(23:31):
it doesn't use any any gems or anything. If it
needs some other code, it vendors them in. So it
was like the ideal code base, and the team that's
working on that codebase is a small team that's been
working on the same codebase for a long time. So
we talked to that team and we told them that
we were interested in adding type annotations to the majority
(23:54):
of the dev code base and to see, you know,
what kind of an impact we could get on type safety.
Speaker 2 (24:01):
That was a project that took like a couple.
Speaker 3 (24:03):
Of months because it's especially hard to add typing to
code that you already have because it means you need
to understand how data is flowing throughout the code because
you have some method that ends up calling some other
method and then combining the results from that other method
(24:25):
by the result of another method call or something. So
you have these chains of method calls that you need
to understand what types they return to understand what types
your method in question is returning.
Speaker 2 (24:37):
So it becomes like an.
Speaker 3 (24:39):
Arduous process to fully type and existing code based. So
that took a couple of months, but we approached that
very methodically. So we took a look at the ways
dev fail, and we took a look at all the
ways that dev fails that's related to type related problems.
So any exception that dev was raising. There were always
(25:02):
like all being captured anyway, So any exceptions that was
raising that were type errors, no method errors or argument errors.
Speaker 2 (25:12):
We collated that. So we drew graphs and everything.
Speaker 3 (25:15):
So that was before we started this experiment, and we
drew the same graphs for after we finished this experiment,
and our findings are interesting. So we were actually able
to completely eradicate namers because that's the lowest hanging fruit.
So the basic premise that Sorbe gives you is to
(25:37):
ensure that all the constants are resolved somehow so that
you don't have any constants that Sorbe.
Speaker 2 (25:44):
Doesn't understand in your codebase.
Speaker 3 (25:46):
So and that's what a namer is, right, Like you
refer to food, but food doesn't exist, so you.
Speaker 2 (25:52):
Get a namer.
Speaker 3 (25:53):
And if you're not doing like const get or something,
it's something dynamic in your code base. Just static analysis
is in to eradicate all the name error problems.
Speaker 2 (26:03):
And then we also looked at, like I said, type errors.
Speaker 3 (26:06):
There were a few type airs left still in the
depth code base, but we did trace them back to
files that weren't marked type true. And maybe I should
just explain what type true, type foles all those levels
are so sorbe is a gradual type checker, so you
don't have to convert your HOLD code base to be
(26:26):
typed overnight, but you can opt into as much of
sorbe as you want. So when I said, you know,
we adopted sorbe in our core code base, of course
I'm not saying like we, you know, added types across
the HOLD code based, but we.
Speaker 2 (26:42):
Enabled all the developers to add those type signatures and
to have their code checked against those type of signatures.
So the same thing in depth.
Speaker 3 (26:51):
So we did add types to the majority of the
DEPV code base, but not to all of it, and
we were able to track the type errors to the
parts of the based that.
Speaker 2 (27:00):
Were where the files were marked.
Speaker 3 (27:02):
Type false and type fole says, do not type check
any of the method calls in this file. Only check
that like the constants resolved to constants that you know,
it does the basic type checking, whereas in type true
the file is type checked for all the method cads
and for all the parameters that are being passed correctly,
(27:23):
and if the methods have signatures, that all the methods
have the correct types.
Speaker 1 (27:28):
Yeah, it's just going to say, it sounds like you
were able to test more or less your your premise, right,
which is that exactly we're going to be able to
eliminate all of these name errors.
Speaker 3 (27:36):
Yes, so the name errors is the lowest hanging front
because you take a type false on top of your
file and run sorbey over it, and it will tell you, oh,
you have this constant that you're referring to here in
this file, but I have no idea what it is.
Speaker 2 (27:51):
So it's either coming from somewhere else in your codebase
that Sorbe isn't processing, or it really doesn't exist, so
you just, you know, remove your users. So nay or
was simple one.
Speaker 3 (28:03):
But we were able to reduce type errors as well,
and we also realized that we were able to reduce
argument errors and no method errors as well, though not
to the same extent, because even though you're opting into types,
you can there are still situations where when you call
a method, the method doesn't have a signature. Sarbe's best
(28:24):
guess is it could return anything that's represented as a
key on typed in sarbe. That's the equivalent of any
in type script. If you know you've worked with typescript
so far, so it's a type that can be anything,
so you can call any methods on it with any parameters.
Speaker 2 (28:40):
So basically SARBA stops doing type checking past that point.
Speaker 3 (28:44):
So a ton typed always returns a theon type when
you call a method on it and everything, so that
kind of lessens the amount of strict typing that you're doing.
You can often to hire types strictness levels in your
files to prevent so you can often to type strict
which will tell you if you're doing this unsafe calls
(29:06):
and it will say, oh, you're actually calling into a
method that doesn't have a type signature, so you know
there's something wrong here, so go and add a type
signature to this other method that you're calling to make
sure that this file has a stricter type checking. So
we didn't go that far in the depcode BIS. We
wanted to see how much we could get with simple typing.
(29:27):
So there were still a few argument errors or no
method errors that leaked in, but we're able to reduce
those kinds of errors as well. But that's not the
only thing we did. We also looked at if running
type checks on CI had any effect on the CI's
success right, So were there, like any false positives that
(29:48):
Sorbey was failing on that were breaking CI built or
vice versa. Were there any test failures that Sorbey wasn't
catching and we realized that they or they didn't have
an effect either way, So it wasn't causing superious superiors
failures on CI, nor was it really missing anything. So
(30:09):
there weren't really that many test failures that Sorbe also
didn't catch.
Speaker 2 (30:13):
But we also talked to the team, so I think
that's one of the most interesting things we've done.
Speaker 3 (30:18):
We talked to the team both before we started this
experiment individually and talked to them after we finished this
experiment to see so before we wanted to see how
much they knew about Sorbe, how they felt about Sorbey.
Were they enthusiastic or did they have questions about its
utility or not, So we compiled those and then after
(30:40):
we finished this experiment, we also asked them if they
were using survey time checking on their machines, if you know,
they were happy adding signatures to the code based or
if they had problems having to deal with fixing signatures
when they were factorings. So we were able to also
get a qualitative feedback on how the team and how
(31:01):
the developer happiness was impacted or not, and we actually
heard good things about that. So there was one person
on the team who really didn't like signatures. They found
it like really distracting. One of the things about Sorbet's
signatures is because Sorbet is not integrated with the language. Obviously,
Ruby as a language doesn't support types, so you need
(31:24):
to add a signature annotation on top of your method definitions,
and that signature annotation is also Ruby goat. So you
do like a sig block, So you say sig and
then you start a block, and then you say params,
and then you declare your parameters and their types, and
then you do a dot returns and then you declare
(31:45):
the type of your return and then you can do
more things in there as well. But obviously sometimes for
a two or three line method, your signature can end
up being five or six lines if it's doing something
like non trivial, or if your types have like some
gender sort something. So some people actually find that two
of their posts and sometimes distracting, so they say that
(32:08):
it makes it hard to actually see the code. They
end up seeing like all these signatures, but that person
on the team actually developed like a few villim configs
to de emphasize the signatures. So they basically turn the
opacity on the signatures so that they look like comments,
which they kind of are right like, because that's the
(32:29):
same thing with yard definitions. So when you add you know,
type annotations in yard comments, that's.
Speaker 2 (32:35):
The same thing. And then someone at the company also
built a visual studio code extension to do the same thing.
Speaker 3 (32:43):
I think it's called bisig, So it turns the opacity,
it parises the signature blocks, and then turns the opacity on.
And I've actually been using it and I'm also enjoying
it because I don't necessarily need the.
Speaker 2 (32:56):
Signatures to be in front of me all the time.
Speaker 3 (32:58):
But apart from that, we heard that developers actually liked
the safety guarantees that static typing gives them as they're
doing refactoring. For example, So for example, if they're removing
a class or a module from the CLUD based they're
almost guaranteed to know that when they run the standard
type checker that it will complain if there's like any
(33:20):
dangling usages of that class or module. So they really
liked those guarantees and they didn't really mind writing the
signatures in the first place. And it's a really easy
ramp on as well on the signature syntax. So we
actually heard good things from developers on working on the cloud.
Speaker 5 (33:39):
Based so has used in sorbe made you guys write
better code?
Speaker 3 (33:44):
Yes, that's also another one of our findings. We also
talked to three different teams that who were working in
core and they were one of the early adopters of
sorbet in the better parts of the Colde base. So
we have this componentization thing going on in our core,
in our monolith.
Speaker 2 (34:02):
Because it's a huge cod base, otherwise it becomes mayhem.
Speaker 3 (34:06):
So there's the components and some components worth your early
adopters of typing. So we did interviews with team members
from those teams as well to understand how they felt
about those early initiatives, and one of the things that
we ended up hearing over and over was that adopting
types led to better code design, which initially was interesting
(34:31):
for me, but that seems to be the biggest benefit
from adopting static or gradual typing rather than you know,
runtime type safety.
Speaker 2 (34:41):
And so like my.
Speaker 3 (34:42):
Opinion is a little bit swayed now because too often
when we're working on simple solutions, we reach out for
like these bags of values like arrays or or even
like most of the time hashes, right, So we just
like reach for a hash, and then we keep passing
those hashes around, But then it's never documented what the
(35:03):
keys are, like if let alone, if the keys are
symbols for strengths, Like, how many times did we assume
that we could access some hash using symbols?
Speaker 2 (35:13):
It turns out to be that it was indexed by strengths.
Speaker 3 (35:16):
Right, So as you're like, when your codebase is small,
that's fine, I'm not judging anyone who's doing that. But
as your codebase is growing, you need people two more easily.
Speaker 2 (35:27):
Onboard onto your team. And if they're fixing something, and
if their focus is on let's say one method or
one file, you don't want them to go hunting around seeing, oh,
where's this hash.
Speaker 3 (35:40):
Actually being coming from? Where was it created initially? What
are all the parameters that are in this hash? So
that's not like a very efficient way of working at
that scale. So at that scale, it's better to give
a name to that, to turn that into something like
a structure or like create class for it or something
(36:01):
to basically start using types. Right, once you turn something
that was a hash into a strup or a class
or a module, you give a name to it.
Speaker 2 (36:10):
You give a type to it, so you start.
Speaker 3 (36:13):
Using types, and those types actually convey more information than
passing around hashes would, so it leads to actually better
code design, and we've heard that over and over again from.
Speaker 2 (36:25):
Various people within the company.
Speaker 3 (36:27):
But it also leads to evergreen documentation as well. So
if you're looking at a method and it has a
signature on top of it, and it says these are
the parameters that it takes off these types, and it
returns this type, then you don't need to go and
look anywhere else. Basically, if you know what those other
types are as well, you don't need to go look
anywhere else to all the colors of this method or whatever.
(36:51):
It's just evergreen documentation. And the reason why it's ever
green is because if somebody changes the method signature, they
have to change the signals you're on top of the
method as well, otherwise you will have like the static
type checker breaking in CI, right, So it has to
be evergreen people need to keep it up to date,
and that makes it very easy to onboard new people
(37:13):
to your code base, or to your team, or to
your company as well. So we we heard from the
blopers that those were all great benefits of adopting types,
and like I said, we didn't adopt types across the
whole code base, but we're allowing people to adopt those
types gradually as much as they want. And our team
(37:34):
is also helping people within the company in their adoption process.
So sometimes we help them, we review their prs, sometimes
we actually pair with them to make that adoption easier,
or we just answer questions in a.
Speaker 4 (37:48):
Slack channel when there's that I hate types, sure, spent
years try to get away from them. The only reason
I still program in ruviews because it just doesn't have
That's the kind of is why I like it. It's
there's freedom from types. So my question is, what's what's
wrong with Ruby? What's going with Ruby? Because this is
(38:11):
obviously not just a Shopify project. This is something that
Stripe also looking at. So there must be some kind
of common problem that led you both to say, we
can improve our product, we can improve our developer experience,
we can find problems, sooner we can have this kind
of really kind of pick up on productivity by introducing
(38:34):
this system on top of Ruby. And it's a pretty
clean system. There's there's a link to a little what
do you call it, a sandbox, a playground where you
can try it out. And what I really like about
it is you said, is that you don't have to
go through your whole code base and start typing in
types for everything. If you've got something of mistake that
(38:56):
you keep making in a project, then you can just
kind of drop the seeing on that problem, class that
problem method, and then no one will ever have an
excuse for breaking it again. So it's a real it's
a real hammer to kind of drop on that. But
what's your what's your opinion? I know you talked a
(39:16):
lot about the developers and how they feel about introducing
typing and what they got from it. This is something
you must feel very passionately about. Where do you stand
on Ruvian types?
Speaker 3 (39:28):
Okay, so personally, just before I joined Shopify, I was
working on code basis that were built by Typescript, and
actually Typescript was I think there is something wrong with me,
and I'll come to that. So Typescript was I was
at the time working at a web agency, but I
was on the team that were doing like cutting edge
(39:49):
two weeks prints that were like trying to see if
we could make certain things work, and I was we
were mostly working on JavaScript, and I consciously chose to
tried Typescript to see.
Speaker 2 (40:01):
How it would work.
Speaker 3 (40:02):
And I Typescript coupled with visual Studio code was a
joy to use, and I started loving it for all
the code completion and all the type things and you
don't look at the documentation and a lot anyway, Like
I was really enjoying Typescript before I joined Shopify, and
I had already heard about Sorbet, and like Sorbe got
(40:23):
me similarly excited because it has it still has the
promise of doing the same thing that Typescript did for JavaScript,
which was not to destroy the language as a language,
but to add features on top of it that people
could opt into if they wanted or needed more strict typing.
Speaker 2 (40:46):
And it's again this.
Speaker 3 (40:47):
Whole thing because typescript is one hundred percent JavaScript plus
some other things. The difference with typescript is it compiles
into JavaScript, so it does not type checking at run time.
Whereas with sorbe, it's built inside of Ruby, so all
the sorbet annotations and everything are also Ruby constructs, so
(41:08):
those Ruby constructs can also do type checking for type assertions.
Speaker 2 (41:12):
At run time.
Speaker 3 (41:13):
So that's the kind of difference. But looking back, I
kind of see myself having almost exclusively worked in code
bases that had some types somewhere. So I'd worked with
DELFI for a while, and then C plus plus D sharp,
and then, like I said, typescript. But before Typescript, we
(41:34):
were working on a PHP code base that was using
type pins. So that was a five point three code
base if I remember correctly, which had type ins, and
we were really relying on type ins. So the project
that started before I joined and it was built like that,
and I ended up feeling like that was a good
(41:55):
way to work because it was preventing us from shooting
ourselves in the foot. Even though that was still doing
type checking at runtime, it was still failing fast, right,
So that's like one of the principles that we software
developers use a lot, this ability to fail fast.
Speaker 2 (42:13):
So static typechecking really gives you that. So it allows you.
Speaker 3 (42:17):
To fail fast either in so base case, you just
run the static type checker and within a couple of
seconds it shows you if there's like some assumptions in
your code based but don't check out. And that's way
better than waiting on the CI for.
Speaker 2 (42:32):
A couple of minutes. You can just run this tool
constantly on your machine right right.
Speaker 4 (42:37):
So this is where it kind of seems to differ
in terms of develop a tool to running a test suite.
This is this is not really the same as running
a series of tests. This is something which you can
use kind of almost kind of running continuing the black
in the background in official studio code.
Speaker 1 (42:58):
Is that how you use it?
Speaker 3 (42:58):
Yes? So, so ARBA actually has an LSP mode. So
the Sorbe binary, that's the C plus plus static checker binary.
If you download the Sorbe static gem or the Sorbae
JAM which pulls it in, you can actually do SIBTC
DASH dash LSB which turns on the LSP mode. So
(43:18):
LSP stands for the Language Server Protocol. I think I'm
not so sure about the acronym, but it's developed by
people at Microsoft. As they were developing visual studio code,
they developed that specification. So it allows different languages to
integrate with different editors, and LSP is the language that
(43:39):
sits in between, so Sorbe has this LSB server mode,
and they're also working on an official Visual Studio Code
extension that you can install that would bring all that
richness into Visual Studio Code. And I'm one of the
early testers of that inside of Shopify, and I've been
using it for a while now, and it's actually integrated into.
Speaker 2 (44:03):
Your editors, so it gives you.
Speaker 3 (44:05):
The best auto complete that you can ever find, way
better than all the other editors can do. But it
also shows you all the errors within your editor. That's
a great way to work because you get instant feedback
on your code changes. I don't want to give too
long an answer, but just to address the second part
(44:25):
of your question is like when would.
Speaker 2 (44:27):
You reach for a tool like this? So scale is
very important.
Speaker 3 (44:30):
So when you have, like in our code base, tens
of different components and thirty thousand different files, then it
becomes really important to have better contracts between all those
different moving parts, and what better contract than types?
Speaker 2 (44:48):
And we've actually realized that types really work.
Speaker 3 (44:50):
Well for that, and I think that was the initial
impathus for why Stripe started working on this as well,
and I'm not saying that all Ruby codes should be
typed or that everyone should reach for typing as soon
as possible. Obviously it's a matter of needs and scale,
but I also feel like that it's the right tools
for the scale that Shopify is in right now, and
(45:13):
the teams that are really relying on types are getting
a really good return for their investment.
Speaker 4 (45:19):
So obviously we're just coming out of the COVID nineteen time,
and I understand that Shopify has changed their working practices.
What's it been like at Shopify over the last few months.
Speaker 3 (45:35):
Actually, Shopify's response to this whole pandemic has been really
impressive because senior executives at Shopify were aware of where.
Speaker 2 (45:46):
The whole pandemic situation was going really early.
Speaker 3 (45:49):
On, and they were really quick in taking measures to
ensure that people working for Shopify are sick. So that
was like a very early on so they basically closed
all the offices before like most countries were aware of
what even was going on. So for me personally, I
(46:12):
can't quite talk for those office closures because I've always
been working remotely for Shopify, so I live in Cyprus.
I worked for Shopifi remotely from here, and our whole
Ruby and Rail's infrastructure team has been distributed has been
a distributed team as well, So we had a couple
of people working in the offices in Ottawa, someone in Montreal,
(46:36):
one person in Poland, one person in France, UK, Cyprus.
Speaker 2 (46:39):
So all over the world.
Speaker 3 (46:41):
So our whole team, dynamics and everything didn't obviously change
that much. But obviously, as everyone's feeling it, starting to
work remotely during the pandemic was a huge burden to
a lot of people because they weren't ready to work
like that.
Speaker 2 (46:57):
Also, it wasn't a normal.
Speaker 3 (47:00):
Working remotely set up either, right because schools are also closed,
everything else is closed. You're trapped in this house and
you can't go out, and or you're also trying to
do work, so it's not necessarily the normal working from
home situation. But like you said, our senior executives looked
at the situation going forward. So first of all, Shopify
(47:23):
isn't reopening any of their offices before the end of
this year, whatever happens, because the outlook isn't that great.
It doesn't look like this pandemic is over anytime soon,
so there's no point in like opening offices and making
people feel safe and going there just to then like
close them again. But also the company has, based on
(47:45):
this experience that we've had for the last couple of months,
the company has also.
Speaker 2 (47:48):
Decided to be digital by default.
Speaker 3 (47:51):
Which doesn't it is digital already, but it wasn't digital
by default, right, So that's that's the extension. So what
used to be the case is, for example, if there
were people in the office and if there was a meeting,
so if there were like ten people in the office
and three people joining remotely, then the ten people in
the office would be huddled up in a meeting room
(48:13):
and then the three people would you know, join in
on a call remotely. But that isn't necessarily the best
way of working because the people who are joining that
call remotely don't get the same experience that those ten
people in the room get. So the digital by default
kind of changes that. And they say, we're not closing
(48:33):
our offices, but they're saying the offices won't be the
same offices, so they'll be revamped to accommodate this new
digital by default future. But basically then in that future
we'll get to experience it, obviously, but in that future,
those ten people will also individually join that same meeting
(48:53):
as well, so everyone will be digital in the meeting
by default, rather than having, you know, ten people collocated
somewhere and then three people remote. So that's the biggest difference,
because the decision there was very simple, because you either
either do fully collocated or you do fully remote. In
(49:14):
between is this hybrid thing that doesn't work that well.
Speaker 2 (49:18):
For either party.
Speaker 3 (49:19):
Right, Like, you have a few people that are co
located in an office people that are remote, but then
you need to have all your communication digital to ensure
that the remote people have the same context as the
people who are co located. But people who are collocated,
they if they have like chats that they have in
between and they don't report them back, then you have
(49:41):
information and symmetry, et cetera, et cetera. That's the worst
way to work. I've also worked remotely, being the only
remote person on a team that was co located before
not at Shopify, and that wasn't a horrible experience because
my team was very understanding of my situation. But it's
still not the best way to work when all your
teammates are all in the same office and you're the
(50:03):
only one remote. So the future for Shopify is digital
by default for everyone. So even if you're in the office,
you're still communicating with everyone digitally. That will be the
primary mode of communication and most of our work will
not be done from offices anyway.
Speaker 1 (50:21):
Sweet, then let's roll on into picks then, Dave, would
you start us off with picks this week?
Speaker 2 (50:27):
Yeah?
Speaker 1 (50:28):
Real quick.
Speaker 5 (50:28):
If people want to follow some of the things that
you're doing online, where should they go and look all.
Speaker 2 (50:33):
The engineering work that we're doing.
Speaker 3 (50:35):
We have a Shopify engineering blog that mostly gets a
new blog post once a week, I think, or the
Shopify Engineering Twitter account they can follow.
Speaker 2 (50:46):
And I should also.
Speaker 3 (50:47):
Say that like, Shopify is still hiring and we've been hiring.
Actually we ramped up are hiring across the pandemic as well.
So if anyone's interested in a position at Shopify. Also,
now that we're digital by default, we're more open to
remote people. They can go to selfify dot COM's last
Careers or our career stage on our website and they
(51:09):
can be a part of the team so that they
can follow the developments from the inside.
Speaker 1 (51:12):
Awesome.
Speaker 5 (51:13):
Yeah, and I'll go ahead and kick us off with
some picks. So my first pick is the whyse then clients.
So I've been getting my kids ready for the digital
learning and the upcoming school year, and instead of giving
them each a six hundred dollars computer or something, I
have deployed a thin client set up at home, which
(51:34):
is probably way overkill for most home networks and stuff,
but I would much rather replace a one hundred dollars
all in one thin client than a six hundred dollars
computer and modern. So that's my first pick, and second
pick is I on this call. Actually, I got the
announcement that I got the Apple TDK, so the Developer
(51:59):
Transition Kit with the ARM based processor. So I will
be releasing some drinking Ruby videos on developing on ARM
based processors soon.
Speaker 4 (52:09):
That's pretty cool.
Speaker 1 (52:10):
Nice. I'll be pointing some people that I know are
super interested in that to you.
Speaker 3 (52:14):
Actually, yeah, we're really interested in that on my team
as well, because the people are doing for Ruby and
MRI work, I want to make sure that both of
those platforms run really well on on your arm jet.
Speaker 1 (52:26):
Look, what do you have for us this week? Well?
Speaker 4 (52:28):
To tie in with bu Hook's talk at Rail's twenty
twenty Couch Edition which talk is called peeling away the
layers of a network stack. It's a good talk. My
pick is the evergreen tcp ip Illustrated, a big book
(52:49):
of how networking works. If you want to learn how
to perform hilarious office pranks like art poisoning, if you
want to get kind of get to get to grits
with networking, this is a great book to really trid
into detail. So there we go. Tcpip Illustrated recommended to
me many years ago by a man from Florida, and
(53:12):
boy was he right.
Speaker 1 (53:13):
It is a great read. Awesome. So I have a
couple of picks for this week. One is actually something
that's been out for quite some time now, but I
really wasn't introduced to it until very recently. It's called ASDF.
It's just if you're familiar with RBM or RVM or
any other version manager or any language. ASDF is more
(53:34):
or less a similar thing, except that it's across It's
really just cross language, cross tool kind of thing, right,
like they have plugins for things like postgrass and my sequel.
I don't really see a lot of value with postgrass personally,
but I did see I was trying this out on
a project that have my sqel and switching between versions
is legit. So yeah, I mean basically with this. The
(53:58):
thing about this tool is it's really for making it
so that you can set up your development machine to
switch between versions of Python and Ruby and Node and
your database that you have, and you know, like a
whole bunch of different tools. And I mostly use Docker
these days, so this really isn't like a thing that
(54:19):
I'm going to use all the time. But I do
have a couple like outlying projects that are just fun
and so I explicitly I was introduced to this, I
was like, all right, I'm gonna try this out. It's legit.
So I don't know that I'm ready to completely give
up our VM for this yet, mostly because I do
almost all Ruby work, but it's definitely taken some mind
(54:39):
share up in my brain at this point, so it's
pretty cool. Definitely recommend checking that out.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
My Favorite Murder with Karen Kilgariff and Georgia Hardstark
My Favorite Murder is a true crime comedy podcast hosted by Karen Kilgariff and Georgia Hardstark. Each week, Karen and Georgia share compelling true crimes and hometown stories from friends and listeners. Since MFM launched in January of 2016, Karen and Georgia have shared their lifelong interest in true crime and have covered stories of infamous serial killers like the Night Stalker, mysterious cold cases, captivating cults, incredible survivor stories and important events from history like the Tulsa race massacre of 1921. My Favorite Murder is part of the Exactly Right podcast network that provides a platform for bold, creative voices to bring to life provocative, entertaining and relatable stories for audiences everywhere. The Exactly Right roster of podcasts covers a variety of topics including historic true crime, comedic interviews and news, science, pop culture and more. Podcasts on the network include Buried Bones with Kate Winkler Dawson and Paul Holes, That's Messed Up: An SVU Podcast, This Podcast Will Kill You, Bananas and more.
The Joe Rogan Experience
The official podcast of comedian Joe Rogan.