All Episodes

November 10, 2023 46 mins

In this episode, Jason and Chris welcome guest, Jorge Manrubia, a Lead Programmer at 37signals in Spain known for his contributions to Ruby on Rails.  Today, Jorge shares insights into his background, role at 37signals, and contributions to open source projects.  He discusses his experiences, including the importance of learning from rejection and the value of experience in job interviews.  The conversation dives into Jorge’s work on Active Record Encryption and Console1984, and Jorge touches on the development of Turbo, with a particular focus on enhancing user interface fidelity in calendar applications using morphing. Also, they discuss the challenges of using Turbo Streams for complex updates and the benefits of using libraries like morphdom or Idiomorph for simplifying the update process. Jorge also gives us a glimpse into the upcoming release of Turbo 8, so press download to find out more! 

Honeybadger
Honeybadger is an application health monitoring tool built by developers for developers.

Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.


Mark as Played
Transcript

Episode Transcript

Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:00):
This is remote, ruby.
Have you any remote ideas tothe meaning of the word?
Good morning, christopher.
Good morning, it's a Thursdaymorning, which is a rare
recording time for us.

Speaker 2 (00:14):
Rare.
This is one of six we'rerecording this week.
When this comes out, it couldeither be November of 2023, or
it could be Christmas of 2024.
I don't know, we are Andrewlists this morning, but we have,
I think, a lot of funconversation to get into.
Today.

(00:34):
We're joined by Jorge Manrubiaand we are going to talk about
turbo, we're going to talk aboutrails, we're going to talk
about Ruby.
Who knows what else.
We'll get into you, but first Iwould like to ask if you would
mind maybe just giving briefintroduction into where you work

(00:55):
, how you started working there,things like that.
I think it'd be interesting tohear.

Speaker 3 (00:59):
Well, first of all, thank you so much for having me
in the podcast.
I'm a big fan of this oneregular listener, so I'm
thrilled to be here.
So yeah, my name is JorgeManrubia.
I work as lead programmer at 37signals.
I live in Valencia, spain, andin terms of open source, I'm the
main author behind activerecord encryption and also I

(01:21):
recently present a new featurein turbo that will arrive in
turbo eight, which is using morething for smoother page updates
and what else.
I like to write a lot, so I'mwriting many technical articles
on 37 signals that side and alsoin my own personal side.

(01:42):
So yeah, that's me.

Speaker 2 (01:45):
I meant to, in the introduction, say that if you're
listening and you haven't readsome of the technical articles
that Jorge has written, youshould, because they are
insightful but they're alsofascinating.
Looks at how you and 37 signalsapproach code and I think
people really love that type ofcontent.

(02:05):
I love reading it.
So thanks for writing thatstuff.

Speaker 3 (02:08):
Thanks so much.
I appreciate it.
The reception for thosearticles has been a super nice
surprise for me.
A bunch of people have reachedout.
They have written me likeplenty of emails with questions
or with showing theirappreciation and yeah, it's been
like a wonderful thing.
I wasn't expecting like so manyinteresting interactions with
folks around the world becausethose articles.
So thanks so much for your kindwords.

(02:29):
I appreciate that.

Speaker 2 (02:31):
So you work at 37 signals.
I would be kind of curious tohear, maybe, how you got into
Ruby and how that led intoworking for the company that
extracted rails.

Speaker 3 (02:42):
The thing is that both 37 signals and rails were
in my radar, I think, sincerails started, especially since
version two.
Back then I was into the Javaenterprise.
I was working for the ITdepartment of the social
security office in Madrid, spain, if you believe it or not.
So there was like all aboutenterprise Java there.

(03:04):
I don't know if you're familiarwith okay, with a very good old
times of Java enterprise.
Anyway, rails was launched likean answer to a lot of madness
going on in the Java world inthose days and got my attention
immediately.
Then, around 2010 or somethinglike that, I started my own

(03:26):
company, my own product company,using rails, which was called
Sendan.
Sendan was getting things donetoo.
That never truly gained muchtraction to make a business out
of it, but it served me to learnrails for good, because so far
it has been like a side thing.
But when you build somethingreal that you need to operate

(03:46):
with real customers is like whenyou really learn the technology
.
After that, I got to work forall the companies.
The thing is that 37 signals wasalways in my radar, especially
when I became experienced enoughwith rails so that I started to
feel that I had a chance ofgetting to work there.
I tried many times.
I tried to apply I don't knowmaybe five times, five different

(04:07):
times, just when there wereopenings and also when there
were no openings.
I actually wrote a blog postback in the day which is I think
it's called how I Got Hired by37 Signature, by Basecamp or
something like that.
It was not like, oh my God, Iwant to do this.
I'm going to do that in alinear way.
It was more like trying andfailing.

(04:28):
I remember once I wrote like asuper length cover letter.
It was like I don't knowmultiple pages, six, seven,
eight pages.
I remember seeing a post fromDavid Stoneware asking people
please be concise in your coverletters.
So it was like, oh okay, sothey don't want like a lot of
text.
I tried to adjust what I wasdoing and eventually I was

(04:51):
lacking in 2019, I think, whenthere was an opening for the
security infrastructure andperformance team, and once there
were a series of interviews andI got finally hired.
There are like all sorts ofstories, but having to try
several times is not rare among37.

Speaker 2 (05:08):
Signature employees.
I appreciate you sharing that.
I love the honesty of like Itried this multiple times Too
often in my career like I getone rejection I'm like okay,
well, I'll never work there, Cannever apply again.
I also love, too, that youmentioned sometimes I applied
when there was openings andsometimes there weren't, because
I actually think that'sactually really valuable advice

(05:30):
for people looking for jobs,because my job at Podia I
emailed the CEO and was like hey, are you hiring?
And he's like, nope, but I'lllet you know the next time we
are.
And then it was a month later.
So yeah, there's a lot ofopportunities.
I think it's really cool thatyou shared that.

Speaker 3 (05:45):
Sure, I mean, I've been rejected like several times
in my career differentcompanies and if you take that
like in a constructive way,there are valuable lessons like
there.
Going through selectionprocesses is something that you
can get experience at so thatthe fourth time you are
interviewing in a for a big oran important company and you are
nervous, like you do thatseveral times, things get better

(06:06):
in terms of how you behave andlet's for most people, because
there are people that are soamazing that they just go for it
and make it in the firstattempt, but for most people
like I think that failing andlearning is like a much more
realistic approach.
Right.
So that was definitely my case,yeah.

Speaker 2 (06:23):
Yeah, and I think I was a lot.
Once I got rejected from likemy dream company at the time and
they were like you'd be a goodcultural fit, but you don't have
enough experience.
And I was like, what do youmean?
I don't have enough experience,like I didn't get it.
And then I've worked for Podiafor five years now and I'm like,
oh, I get it.
I had no experience.
Sometimes the timing is justnot right.

(06:43):
Yeah, there's a lot to learnfrom that.
So two things we can probablyspend hours on Chris mentioned,
before we started recording andyou mentioned the beginning
active record encryption and assomeone who is now heavy user of
active record encryption, Iwould love to kind of dig into
that.

Speaker 1 (07:03):
So just to set some context, like, we have used
encryption in active recordbefore, with Adder encrypted and
lockbox and other things.
But when your solution toencryption came out in Rails, it
was really unique because itwas like hey, we know that
you're going to apply thisretroactively and you've got

(07:24):
data in your database youprobably should be encrypting.
Maybe it's OAuth API tokens orsomething that you're just
storing in plain text, butthey're kind of like passwords,
so maybe you should encryptthose.
I thought it was really cool tosee that baked in as part of
the solution, which was awesome.
So I was curious what were yourrequirements when you started
working on it and what did youlearn along the way that led you

(07:47):
to that as the solution there?

Speaker 3 (07:50):
So when I joined 37 Signals the end of 2019.
So two weeks after joining 37Signals, I went to a meetup in
the Chicago office.
They were celebrating like acompany meetup there.
There was like the first time Iheard about the encryption
thing, because David HenryHansen in the opening talk they

(08:11):
normally deliver Jason and himhe mentioned something like my
wife asked me are your employeesgoing to be able to check my
email?
So it was a very simplequestion like that.
So he really was interested.
It was an example, obviously,but he was really interested in
raising the bar of security whenit comes to accessing data.
Just because email because thiswas in the context of launching

(08:34):
the email service by 37 signalsso email can contain health
information, information inrelation to your relationship
with other people, politicalaffiliation, whatever you want
like the most sensitive topicsyou can think of can be in email
.
So email is very sensitive,okay.
So that was the first time andI was like enjoying the

(08:54):
infrastructure team 37 signal.
So I got just assigned to theproject, which for me, was like
quite a huge challenge.
Well, it was a pretty bigproject Really.
There were no experts inencryption in the company and
certainly I was not an expert inencryption at all.
So what I did first was liketrying to study like all the
gems that were out there doingthe same there were a few so to

(09:19):
see how they did things and Ithink I learned like good things
from most of them, even if Ididn't find like quite the gem
that we wanted to use.
Then I went to work on all theprojects in the company.
I didn't start with this rightaway, even if it was in my radar
.
So first I created a prototype,we discussed the prototype
internally, we learned what wewanted to build thanks to that

(09:42):
prototype.
Then I rebuilt another versionof the library, because first I
was using tables, likeadditional records, to store the
keys.
So in the current active recordencryption we are storing the
keys with the encrypted payload,but in the initial version I
was using like additional keysin the database, which was like
a point of freedom in the designand it brought like a bunch of

(10:04):
complications at several levels.
So anyway, the thing is thatwhen we were ready to launch
encryption, we were alreadyusing hay internally, like for
our email.
So we started using hay withoutthe emails being really
encrypted, like just an internalthing, so really had like that
constraint of we need to encryptlike this information and we
need to keep things working.

(10:25):
And that was like the first bigchallenge technical challenge
that I remember.
Then there were others, butthat was like the seed for
making the library flexibleenough internally to support
that.
Then there was somethinginteresting because we hired
before going public.
We hired like a cryptographyexpert, like a security company

(10:47):
who brought a woman who was anexpert in cryptography, an
actual expert, and when shereviewed implementation there
was something that got herattention as a major flaw.
I think it was like I wasfixing the initialization vector
in combination with some AES256encryption mode.

(11:08):
It was completely like makingthe encryption almost useless.
But she was like agitatedbecause she knew about
encryption.
It was a big flaw and I wasokay, the fix was easy.
But the problem was that we hadalready data encrypted with the
previous encryption scheme.
So again, there was like a bigchallenge, a big technical
challenge, but out of there,like I could leverage on that

(11:30):
flexibility, we had to even makethe mechanism more general.
So right now it's I think it'sthe only Rails encryption
solution there is out there thatsupports like multiple
encryption schemes so you canactually change the different
properties of the schemes you'reusing over time and the system
keeps working so that you canquery all data and things like

(11:52):
that, and things are going towork.
So all those fancy featureswere out of pure necessity, out
of finding ourselves in asituation where we needed to fix
that for good.
So yeah, that's kind of theinterest story there.

Speaker 2 (12:06):
I love hearing that because to me it was like, oh
wow, they just really thoughtabout every possible like edge
case, like up front.
So I also think it's reallycool that you brought in an
expert to like review it andthen you open source it.
How much more could you ask foras a consumer of the library?
This has been tested by someonewho is an expert in that field.

(12:27):
That's amazing.

Speaker 3 (12:29):
Yeah, that was a call that Jeremy there from Rails
Core and 37 Signature Meet theremade, that call of bringing an
expert.
And I think it was an excellentcall because I was living like
how do you call it?
Like how much do you ignore andyou live a happy life?
I was there, I was happilyignoring like a lot of our
cryptography and I tried toeducate myself.
But cryptography is like thisbody of knowledge which is huge

(12:51):
and you can just dedicate yourcareer to that.

Speaker 2 (12:54):
That's amazing.
Yeah, my first foray intoencrypting data.
I can't remember the gym I use,but I had a lot of boilerplate.
It wasn't lockbox, but I hadseparate files with keys and all
this stuff and I was so scaredthat I was going to mess
something up and lose access tomy data forever and shut down

(13:14):
the app.
And so then when lockbox likewhen I found lockbox I think
everything Andrew Kane does islike gold and so when I found
that, I was like, oh, perfect,this works.
And then when it got baked intoRails, I was like, yeah, one
less dependency and it worksjust incredibly well.
I love that.
I don't have to do anythingspecial to the column.
I love that I just say a,encrypt this field and then I

(13:37):
don't think about it anymore.
It's one less thing I worryabout.
It's wonderful.
So, yeah, thank you for yourwork on that.

Speaker 3 (13:43):
Sure, I appreciate that Thanks.

Speaker 1 (13:45):
Yeah, the single column thing is really nice
because in same thing with fileuploads, in the past it was like
, hey, you want to do one fileupload, then it's four fields.
You want to do two or three onthe same model, then have fun.
Here's eight or 12 differentcolumns you got to add, and
having that all in a singlecolumn where it can be

(14:06):
serialized, this makes sense.
Why wouldn't we just do thatand then have it all contained
in one place and it's justeasier to manage.
It's just nice.
Recently, while I was partiallydoing the 256 thing for action
text embeds or whatever, but wehad encrypted stuff as well and
the Shaw one and now therotations, for that is like

(14:29):
super easy and that's like avery nice thing.
I feel like Jason in the pastdoing encryption stuff.
If I screw this up and weencrypt stuff and something
breaks, I might lose all of thisdata, which would be horrible,
and I just don't really feelthat fear anymore either.
We're not like you're saying.

(14:49):
Crypto experts can spend theirentire career on it.
It's very math heavy.
If you've ever had to debugsomething that goes wrong with
open SSL or whatever, good luck.
You'll end up with a crazystack trace and have to use.
Like I dug into S trace and waslike looking at all the internal
stuff that happens there andtrying to debug some random

(15:10):
thing and it's yeah, it's aquite a lot of stuff to learn.
It's crazy, along the sametimeline where you working on,
because you had talked aboutyou're using hey internally, so
you started encrypting stuff.
But also to David's wife'squestion about hey, can you read

(15:30):
our emails?
One of the things that Iremember, like you initially
released, was the console 1984gem, which I love, and really
the value that, like hey,anybody that's in production
looking at the Application onthe console probably should be
telling us why they're there andkeeping track of what they're
doing.
And I was just curious it wasthat around the same time.

(15:53):
Oh, we need to also Not justencrypt the data but also do
some audit, logging and stufflike that around who's doing
what.

Speaker 3 (16:02):
Yeah, yeah, definitely.
So right after the encryptionwas in place, like I started
working on both console 1984 andwell, firstly was console 1984
because but first we were justRegistering like the console
locks and trying to prevent likeaccess to encrypted data.

(16:23):
I got a first version of 1984like very quickly done, like
maybe Three or four days, but itwas incredibly nice, so it was
very easy to circumvent.
But it was kind of working.
It was like essentiallyoffering like at the crypt
command so that you can accessthe encrypted data and you could
.
Also you were in a encryptedmode by default, so you wouldn't

(16:45):
see Encrypted data by default.
That was working.
It was like connected with ourlogging pipeline.
First we were collecting likethe logs internally in our well,
we use Kibana internally.
So we were like collecting ourlogs in our login pipeline and
exploiting them through Kibana.
So let's say, the auditingsystem was like a custom

(17:08):
dashboard in Kibana.
Like for the first version webuilt so far, this was like an
internal gem when we were torelease the whole thing.
I created like all this 1984which is like a very basic admin
interface for auditing theselogs and instead of emitting
those logs through the standardoutput or through the logging,

(17:31):
whatever login system you haveconfigured.
Instead of doing that, westarted to store them in the
database.
So we built, like this tool forauditing those logs.
And then I worked on severalimprovements to make the system
Harder to circumvent, becauseit's easy that Essentially the
problem with Ruby console isthat by design is a system that

(17:51):
it lets you execute arbitraryRuby code and Ruby is incredibly
Flexible, right.
So it's hard to try to cover allthe ways you can try to Skip
the controls we set in place,but it's still.
I'm pretty happy.
I mean, there are no knownissues that I know of about how
to do that, but I'm sure someonewith expert enough can.
I mean, I wouldn't be surprisedif someone Discover new ways of

(18:14):
doing that, but I always saythat it's a good baseline.
Obviously, maybe it's not goingto if you have like very
malicious actor with access toyour console, to your production
console in Rails, and thatperson knows what they're doing
and maybe it's not likebulletproof, but it's a little
way better than not havinganything in place, right.

Speaker 1 (18:34):
So, yeah, if they've got production rails console
access, they've got Probablyprivileges to do almost anything
they want, but it's one ofthose that's really nice to be
like.
Well, we need to do customersupport and Want to just make
sure that people who are doingthat are Not going around and
doing malicious things orwhatever.

(18:56):
I think what was it?
Vercel had an employee that waslike a customer's domain was
too similar to theirs and sothey had done stuff that they
had access to that theyshouldn't have or whatever.
And this is a great tool forthat.
Where it's like, hey, productionconsole is Necessary evil, we

(19:16):
don't want you to have to dothis ever, really, but we can
provide a way to make that a lotmore, you know, just monitored
or audited and keep that safer,and I think that's a really nice
Thing, because that I mean assoon as it came out, I was like,
oh yeah, why wouldn't we usethis in default for every
application we run?
Because we might as well.

(19:37):
And it was like when I wasworking by myself, I was like I
don't need to audit myself, buthaving other people that you
work with, yeah, makes perfectsense, and it felt like
something that I Would want toinclude like out of the box, on
everything.

Speaker 3 (19:53):
Once you get used to work with, like with encrypted
data, it feels weird, like notdoing it.
After a while it starts feelingwith why am I seeing like this
piece of customer information,even if it's not sensitive by
how a customer names to do, forexample, it's not my business,
even if it's nothing, it's likesensitive data, it's not.
I shouldn't be seeing it.

Speaker 2 (20:12):
Let me ask you one quick question.
Are you currently using oneservice for uptime monitoring,
another service for air tracking, another service for status
pages and yet another service tomonitor cron jobs and
microservices?
Paying for all of thoseseparately may be costing you a
lot of money.
If you want to simplify yourstack and lower those bills,

(20:35):
it's time to check out honeybadger.
Honey badger combines all ofthose services into one easy to
use platform.
It's everything you need tokeep production healthy and your
customers happy.
Best of all, honey badger isfree for small teams and setup
takes as little as five minutes.
Get started today.
Honey badgerio that's Www.

(20:56):
Wwwhoneybadgerio.

Speaker 3 (21:01):
And also I wanted to mention that another gem that we
released, which was also out ofpure necessity, is mass
encryption, which is a libraryfor encrypting data in mass
using active record encryption,and that came out of having to
encrypt, like all the base campdatabase, billions of records
there.
Initially we only supportedlike encrypting records one by

(21:23):
one and with this library isgoing to use jobs and the jobs
are going to use absurd SQLstatements so that you're going
to update a bunch of records1000 records at a time,
something like that so it's wayfaster.
So if someone wants to encryptlike a large database, that's
definitely like the gem to keepin mind.
Yeah.

Speaker 1 (21:44):
I don't think I even knew that existed.
That's super handy.
Yeah, a lot of old applicationswill have lots of things that
you probably want to go encryptand probably millions of records
, and opening the Rails consoleor running a rake task that
doesn't one by one probablytakes a ridiculous amount of
time.
So this sounds really awesome.

Speaker 3 (22:04):
I don't remember the numbers, but the numbers for
base camp was where we did it.
It's like we wouldn't neverfinish if we did one record at a
time.

Speaker 1 (22:11):
Yeah, so you were working on that.
What point did you startworking on the turbo stuff that
you announced at Rails world?

Speaker 3 (22:21):
Oh yeah, well, that was this year.
Actually, so far I've beendoing like infrastructure work
37 signals, so mostly internalprojects and also supporting
infrastructure features forexisting applications.
So last year I asked thecompany that I would like to do
some product work if possiblebecause in my career I've done
like product work places and Iwas kind of missing that, like

(22:43):
working with designers with userfacing features in the front
end and such.
I had a lot of front endbattles in my career.
I was kind of missing those andI was like I was given like
this wonderful opportunity ofworking on the new product that
the company was talking about.
We was the calendar, the highcalendar.
So that's how everythingstarted this year.

(23:07):
Like I started.
I was in sabbatical in January.
I started on February on thecalendar.
So there was like an HTMLprototype showing several
screens for the calendar withthe principal designer working
on a Scott Atom principaldesigner working on the product
had been exploring ideas withsome HTML prototype in Rails.

(23:27):
So what I did was starting toanimate some screens and
rendering some screens for thecalendar and I was using the
approach I would recommendanyone to use when you are
starting a race applicationwhich is relying on regular
turbo dry behavior, which isusing Rails as it's meant to be

(23:49):
used, and relying on regularfull page body replacements,
which is what you get for free.
So I was doing essentially I'mgoing to update or create a new
record, I'm going to read itback, render the full screen,
and I'm going to be done.
And I was moving pretty quicklyusing that approach.
But the thing is that the UIfidelity we wanted wasn't there,

(24:10):
obviously because in a calendarwell, I think you're familiar
with calendar applications,right, chris?
I've seen that you have someexperience there right?

Speaker 1 (24:20):
Yeah, for anybody that doesn't know that, I tried
to teach myself how to write aRuby gem and made a little
calendar gem and then publishedit.
It didn't work or wasn'tcomplete, and then people filed
issues and so I was like, well,I guess I'll start maintaining
this.
And now it's got.
I think it's the most popularcalendar in Rails and I've

(24:42):
really never had a need to useit myself.

Speaker 3 (24:47):
That's a pretty cool story.
So yeah, I mean, in a calendarin general, the responsiveness
bar is high, like you want it tobe responsive.
It's not like a regularapplication in the sense that
you have this graphical canvasyou want to act on and you want
to see things happening in aresponsive way.
The fidelity, the UI fidelitybar is high in a calendar.

(25:09):
So with the default turbo drivefull page body replacement
implementation, it was nice butit wasn't good enough for what
we wanted.
So for a while I was likethinking about this problem
because our initial idea waslike green fidelity using
regular turbo, meaningturboframes and turbo streams,

(25:31):
which were like the tools thatturbo offers for partial updates
that feel really great.
So the problem there is that therendering in a calendar is very
complex.
Like it's way more complex that, for example, in hey.
So in hey, when you are listinglike your list of emails, you
are just placing like one emailand one email under the other,

(25:53):
so you're going to render like alist of rectangles.
That's kind of easy to do in acalendar is things are more hard
.
Things are way harder, like youneed to place things in the
proper day, for example.
Imagine that you are renderinglike a regular calendar grid.
You need to place things in theright day.
If it's an event, for example,an event can take like a bunch

(26:13):
of days, or maybe one day ormaybe multiple months how one
event is rendered can affectother events.
That's just talking aboutevents.
This new product is going tohave like a bunch of other
things happening which are goingto be kind of novel features.
So the rendering was reallycomplex and I spent a lot of
time like doing the rendering.
So when I went to the serverand say, okay, now I'm going to

(26:36):
imagine I'm going to, I thinkone of the first thing I did was
creating new events.
We are going to create a newevent in the calendar.
I'm going to create a new event.
Now I'm going to do ready readback to the screen after
submitting the form, and thatready read back is the rails way
of reusing all that work youput in the initial rendering of

(26:57):
the screen.
It's like a great answer Okay,I'm going to reuse all that.
So that felt amazing to me.
Like from a programming pointof view I was like, oh my God,
this is what I want, just nothave to care about which cells
do I need to update forreflecting this new event.
That could be like maybe onecell or maybe I don't know the
whole screen and beyond.
So that felt amazing from theprogramming point of view in

(27:19):
terms of responsiveness.
It wasn't feeling as great whenI was trying to think how to do
that with Turbo Streams.
Again, all that complexity thatthe calendar presented in the
initial rendering directlytranslated into the partial
updates.
Because you're going to imagineupdate and event, you can use a

(27:41):
regular replace, prepend,delete, whatever a regular Turbo
Stream action.
To do that you need a lot ofRuby code to account for all the
conditions that edit operationis going to have in the screen.
I had to do that for everyelement, events and the rest of
elements we were rendering inthe calendar for every
perspective, because we havedifferent timeframes over the

(28:03):
same data For every differentview in the calendar.
It was like an explosion ofvery complex partial updates.
It was feeling like a pardon inmy mind, pardon that I kept
thinking about that.
How are we going to do this?
Because I wasn't seeing anappealing route.
Anyway, I started to look foralternatives.

(28:23):
Phoenix Live View from theLXRiown has been a solution that
has always fascinated mebecause of how different it is
to how regular web programmingworks and how original it is and
how responsive it feels in thedemos, I decided to see.
Okay, let's see how this worksinternally.

(28:44):
Internally, phoenix Live View islike the programming model.
It's quite a departure from howregular web programming works.
So it's based on a persistedweb socket connection.
That is going to be stateful.
It's going to keep the state ofthe connection where you are
interacting with the screen.
It uses that state forcalculating the difference

(29:07):
between the new rendered contentand what you have in your
screen.
It can do that because it'sstateful and it's going to
calculate the difference as avery efficient payload.
It's going to transmit thatpayload over the web socket.
The end of it is going to applythat payload to reflect the
change in the DOM.
Okay, so to do that it was usinga library called Morphdom,

(29:30):
which that was how Morphdomentered my radar.
To be honest, it was analyzinghow Phoenix Live View did things
At the end of the pipeline.
So I wasn't as interested onthe web socket programming model
, on the persisted connectionprogramming model from Phoenix
Live View, because it's anapproach that has trade-offs for
sure and we kind of.
It's a nice thing that ingeneral we love to embrace how

(29:53):
the web works in general, likehow HTTPs are an stateless
protocol and browsers arewonderful pieces of software
optimized to render HTML and toperform HTTP requests.
So we want to leverage that asmuch as possible.
So the Phoenix Live Viewapproach wasn't directly
something we could translatedirectly and we were not

(30:15):
interested in translating thatdirectly to what we were doing.
But the Morphdom bit of it wasvery interesting because it was
a Morphdom, it was a standaloneJavaScript library.
It was doing Morphdom over theDOM directly, which was kind of,
I think back in the day it wasa novel thing because, as you
know, first React popularizedMorphdom by virtual DOMs,

(30:39):
because initially I think theywere the first ones popularizing
like Morphdom, like React, theywere using this technique where
they would create like virtualDOM in memory of the page and of
the new content and it's goingto calculate the difference in
memory and it's going to applythe difference, calculate it in
memory over the actual DOM, andReact proved that that was way

(31:02):
more performant than theprevious approaches.
Eventually browsers catch up interms of DOM manipulations, but
the idea of calculating thedifference between the current
state and the new state you wantto reach was there and Morphdom
was an implementation of thatidea, but using the actual DOM.
So with Morphdom you could takelike a DOM tree and to say,

(31:24):
okay, I want to render this newtree of elements, calculate the
difference and apply that overthe actual DOM.
It was a simple small library.
I grabbed the library, Icreated a patch version of Turbo
using that library in ourcalendar product and I was like
blown away.
I had to make some adjustments,but I was blown away about the

(31:46):
improvements in precisely thatscenario, the scenario where you
are making a change in a page,the server says red, red back
and you see like the changereflected.
So in that scenario theimprovement was very noticeable
and I was super excited when Isaw that.

Speaker 1 (32:03):
I was just going to say you were talking about
rendering a new event in acalendar.
When you think about a calendar, if you were to do that with
Turbo Streams and I'vedefinitely built some more
complex things in Turbo Streamsthat are really it felt the same
friction as you're describing,where it's like well, first we
probably have a flash messageyou want to insert at the top of

(32:26):
the page, so that's got to beone Turbo Stream action that
goes to this specific div andthen, when it's like a new event
, you have to know what kind ofevent it is.
Is it a single day event?
Is it going to be inserted intoall 30 days of the calendar or
seven of those days?
Is it going to wrap around, isit not?

(32:46):
If it's one of those, that is amultiple day thing, it probably
needs to be before the currentday items.
If you're prioritizing how theyget inserted, like it just
becomes.
Basically, you're duplicatingthe logic where I think a lot of
single page apps might tend todo the same thing, where you've
got some logic server side torender, and I think that's why

(33:10):
they're doing the server siderendering of React and stuff now
, or it's use the same logic onboth the client and the server.
But with doing that in TurboStreams it's like we have to
render the page out normally andhave these sections anyways
when you load it for the firsttime.
So the server has to know howto do that and it's templates.
But then if we want to do thatdynamically when you create one,

(33:32):
we've got to recreate all thatlogic in Turbo Streams and it
feels like why are we doing this?
And the morphing really solvesthat problem because you just
have the same process.
You just say, hey, render thepage just like you would do
normally.
If there's a flash message, itjust gets included in your
layout just like it normallywould.

(33:53):
You don't do anything special.
And then you sort of geteffectively what all those Turbo
Streams would have done toupdate the page, but just using
the exact same process, becauseyou have the diffing with
Morphdom or IdioMor for any ofthose libraries.

Speaker 3 (34:09):
Yeah, you explained that wonderfully well, that's
exactly it.
So the thing is that with TurboStreams you need to manually
create the partial updates,accounting for all the logic and
conditions that are neededthere, and with morphing you can
reuse, like the initialrendering completely, and lead
to an algorithm, the calculationof the partial updates as a

(34:31):
high level.
I see it like that too.
You are delegating that to abusiness software.
Of course, ladies, it's notlike there are trade-offs to pay
, because with morphing, firstof all, the response from the
server is going to be moreexpensive to generate, because
you are re-rendering everythinginstead of just a little bit of
code, that or the little bit ofHTML or whatever you want to
update there, and you need totransmit that payload that is

(34:53):
going to be higher also.
So there is like someadditional network latency.
It's like a miracle that it'simpossible not to use.
You need to pay in sometrade-offs, but the trade-offs,
I think, are fantastic and areworth it in most cases because
the responsiveness you get iswonderful.

Speaker 1 (35:10):
I'd love to hear some of those trade-offs and let you
noticed building it in anactual application where you're
not just like, oh, let's justadd morphdom as a feature,
you're actually using it andseeing the exact issues that
you're like we lose focus stateon these form elements, or
scroll position is wonky inthese cases, or whatever,

(35:33):
because those are the thingsthat like when you're building a
library like this.
I think it's those actual usecase applications that you're
like oh, these are the roughedges that really show where the
polish comes from.

Speaker 2 (35:45):
Yeah, and along those lines too, as someone who has
tried to use morphdom directly,even some of the surprise things
that aren't necessarily likeinput but like this, didn't
update.
Why didn't this update?
I think is interesting to you.

Speaker 3 (36:00):
Well, the first thing I'd like to mention is that
there was something that wedidn't like much about morphdom,
which is that in the way itworks, it's really picky about
IDs, about DOM IDs, so it relieson DOM IDs in order to match
elements, and the problem whenmorphing fails to match elements

(36:20):
, the problem is that what youget is not like an error.
You get, essentially trees thatyou are not expected are
replaced.
So instead of getting like asmooth update, you get like a
full imagine, like the firstmain container in your
application get replaced becausethere is a flash notice without
an ID that didn't find like amatch or whatever.

(36:41):
So those kinds of errors for mewere like a blogger, because we
were looking for a seamlessthing and that was completely
not seamless.
So there is this library fromthe HTMLX project, which is
idiomorph, which is essentiallylike morphdom, is newer, so

(37:02):
probably it was heavily inspiredby morphdom, but it uses like a
more relaxed algorithm formatching nodes and, in our
experience, like it completelyseamless.
It doesn't have that IDmatching requirement in order to
work, and we have tested itwith realistic screens, complex
screens, and it works reallywell.

(37:23):
So that was like the firstthing that I wanted to point out
, well, the two main problems wesaw in our tests and two
problems that we are going tooffer a direct answer for since
the first version.
The first of all is that whathappens when you have loaded new
content in your page and yourefresh it For example, if you

(37:45):
have paginated down in thecalendar in the car table, for
example in basecam, that theKanban feature from basecam when
you have paginated down and youhave loaded new cards and you
morph the page, you update thepage, what happens with those
new pages of content?
Because if you are, likemorphed in the body of the page,
when you reload the page, thosenew pages are not going to be

(38:09):
in the new body and they aregoing to disappear.
So that was kind of an edgecase which was like absolutely
crucial to look at a solutionfor, because it was kind of
invalidating the whole thing.
I mean, it's kind of quitecommon to have pagination or
some kind of dynamic content youwant to load in a page after
the initial load.
So that's kind of commonconstraint.

(38:31):
So what we did there wassupporting turboframes, like
leveraging turboframes.
So when we are doing a pagerefresh in the new feature that
we are going to upstream, toturbo-aid.
You can flag turboframes withan attribute.
I think it's called refreshequal reload, I think it's the
attribute.
And when you mark a turboframewith that attribute and a page

(38:54):
refresh happens, it's going toreload that turboframe.
So if you make sure that youload the new content in your
screen with turboframes and youdo that, that's going to work
well with page refreshes andthat's actually what we are
currently using in the calendarfor paginating information and
that works well and solves theproblem.

(39:14):
And the other case is thatsometimes you want to preserve a
screen state when you aremaking a page refresh.
For example, if you have openeda menu and you are acting on
some element and a page refreshhappens because you have
submitted something or whatever,you might like to keep that
menu opened.
So there is an attribute.

(39:37):
Well, we are actually reusingthe data turbo permanent.
I think there was an existingturbo attribute that we are
reusing for this, the version weare upstreaming to turbo eight.
So we are solving those twoproblems.
There is also an idiom or, forexample, it has a setting so
that when you are more in thepage, if a form control has a

(39:57):
focus, so if document activeelement equals to that control
is not going to touch it.
So to keep focus, for example,to keep that untouched this is
pretty much working in our usecases.
But I'm sure that when peoplestart using it in the wild, like
more things will pop up andmore scenarios to handle.
But with these simple things, Ithink that the system I mean we

(40:18):
have tested it in veryrealistic scenarios and it's
working very well for us.
So I'm optimistic.
It's going to work well formost folks out there, but there
will be things to refine forsure.

Speaker 1 (40:28):
Do you run into any issues with, say, like a
password manager extension thatmight inject some stuff on the
page, or like loom I know, addsbuttons to different websites.
Does that end up problematicwith the morphing?
Or is that one where it's likethe current page has these
elements, we're adding the newchanges, but that's not in there

(40:51):
?
Does it keep that stuff or doesit remove it and then maybe the
Chrome extension just re addsthat stuff?

Speaker 3 (40:58):
We use one password internally and we haven't run
into issues with it, but that'snot to mean that it might not
happen.
That's a good point.
I hadn't thought about tensions.

Speaker 1 (41:07):
to be honest, I would have to assume that they
implement sort of the stimulusstyle of monitoring the page to
work with single page apps andanything else that might do the
exact same process of not a fullpage load and then applying
their changes.
They got to be monitoring thepage and then doing something
dynamic, I would assume.

(41:28):
So.
Probably, if you do run intoissues, it's more like an issue
with their implementation, notso much the idiom or Diffing or
anything I would guess.

Speaker 3 (41:37):
I would expect that also.
You can replace a form that'sjust in a regular Ajax request
in a page.
You would be like so I rememberan issue we found in our tests.
There is a scenario like whenyou're morphing a page, stimulus
controllers are not going toreload by default because you
are not touching the DOM unlessit changes.
That's a good thing because ifyou're thinking like in the

(41:59):
TurboStreams updates,equivalence is the same.
So we found that stimuluscontrollers to reconnect if
nothing has changed in the page.
But we found some cases wheresome stimulus controllers were
on the connect.
When connected they weremodifying the element under the
hood, maybe adding someattribute or something like that

(42:19):
.
So one heuristic we are usingis that when you are morphing a
DOM node meaning when the DOMnode has changed and you are
updating it with idiomorph, ithas an stimulus controller it's
going to reconnect the stimuluscontroller so that you make sure
that it gets connected andconnected again.
So in the use cases we found inour code base that was solving

(42:40):
the issue.
So that was like an heuristicthat was working well for us.
Maybe we might need to introducemore fine grained controls
there to say, okay, you want toreconnect, you don't want to
reconnect.
But we are really trying tofind a very seamless, happy path
and fight hard before deviatingfrom that because we want to

(43:00):
keep things simple, like mydream or my vision for this is
we are going to enhance thedefault turbo drive behavior
without making that a parameterconcern.
That's what I would love thatyou updated to turbo eight and
your screens started to behavebetter when you submitted forms
and so without having to doanything.

(43:20):
So probably reality is going tobring some adjustments over
that vision.
That's still like an importantvision for us to make this
seamless.

Speaker 1 (43:30):
Yeah, I can imagine you have a set of tabs.
The user changes to the secondtab but it refreshes.
Server side doesn't know you'reon tab number two, so it might
re-render with the attributebeing like select the default
tab or something.
And then you'd have toreconcile that in some way when
you refresh the page.
But those hopefully becomeseamless and then you sort of

(43:54):
eliminated the whole spaghetti Iguess you could call it of the
turbo streams, trying tointeract with 12 different
things on a create event orsomething that would be just the
good old redirect to the showpage or whatever, and the rest
of it's taken care of for you.

Speaker 3 (44:11):
Yeah, and this is meant to work so that the URL
should reflect, like the pageyou're seeing.
So if there are, like manyscreenestate changes that are
not reflected in the initialload and you're keeping those
stayed in the page, you willprobably need to, depending on
those, you're probably going touse, like this data turbo
preserve tag in creative ways sothat those get respected with

(44:36):
one more thing.
But yeah, that's the idea,that's the idea.

Speaker 1 (44:41):
Yeah, Awesome.
Well, unfortunately we're outof time.
We've probably talked foranother couple of hours, but
yeah, this is super exciting.
Is there anything you want toshare before we wrap up here or
drop a release date for Turbo 8or anything?

Speaker 3 (45:01):
No, we don't have a release date.
I mean we are still working onthe full requests.
The full requests are opened inthe repo, so I invite so we can
go try it.
Yeah, you can go to try and toget feedback.
We have already collected likesome good feedback on the
approach.
The other side of this are likebroadcasting changes which we
can discuss now, but there islike another exciting

(45:22):
simplification that this isgoing to bring to Turbo.
I wrote an article,depthsettlesignalcom, so you can
check the article theredescribing the whole approach if
you are interested.
We don't have a release date.
We are working on it.
It should be like a matter ofweeks at least for getting this
merge like few weeks, not a lotof weeks and then Turbo 8 should

(45:43):
happen like shortly after that,but no release date.
Sorry about that and thanks somuch for inviting me.
I was going to point out thatsomeone wants to reach out.
I have a webpage which isJorgeMandruviacom and you can
reach me by email.
I love email discussions.
So, jorge, at heycom is myemail.
Someone wants to reach out?
Yeah, that's my pleasuretalking to you folks.

Speaker 2 (46:06):
Yeah, it was fantastic having you, and maybe,
yeah, maybe, when Turbo 8 dropsor something, you can come back
on and we can catch up and dothis again.

Speaker 1 (46:13):
Sure, yeah, it'd be cool to see what we talked about
today and then what's evolvedin the final version of it.
So, yeah, nice, looking forwardto it.
Well, thanks for joining us.
This is a blast and we willtalk to you soon.
Thank you so much, folks, thankyou.
Advertise With Us

Popular Podcasts

Dateline NBC

Dateline NBC

Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com

24/7 News: The Latest

24/7 News: The Latest

The latest news in 4 minutes updated every hour, every day.

Therapy Gecko

Therapy Gecko

An unlicensed lizard psychologist travels the universe talking to strangers about absolutely nothing. TO CALL THE GECKO: follow me on https://www.twitch.tv/lyleforever to get a notification for when I am taking calls. I am usually live Mondays, Wednesdays, and Fridays but lately a lot of other times too. I am a gecko.

Music, radio and podcasts, all free. Listen online or download the iHeart App.

Connect

© 2025 iHeartMedia, Inc.