July 21, 2023 • 35 mins
So... what is a Staff Engineer anyway? What even is the job description? In this episode, we’re joined by Abhishek Mistry–Staff Engineering Leader at Hudson River Trading–to discuss exactly that. What is this job ICs aspire to? How is it different from management? How do you influence and lead people when you actually have no real authority? Is it better to go broad or to go deep? How does the Staff Engineer scale their impact and provide, like Tom Brady, tremendous value above replacement? Abhishek has been in the tech industry for many years. He has tons of insight, and he’s genuinely a very fun guy to talk to.
PlugsJammText.com: text-to-screen software to use at your next party.
ResourcesLifeLabsLearning: classes on influential communication
The Pragmatic Programmer, by David Thomas and Andrew Hunt
JavaScript Jabber Podcast, on Top End Devs
Refactoring, by Martin Fowler
Functional Programming Simplified, by Alvin Alexander
The Staff Engineer’s Path by Tanya Reilly
Get in touch!Facebook: /themainthread
Twitter: @themainthread
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:10):
Welcome back to the main thread where we discuss the journey from senior engineer to staff engineer.
So what is a staff engineer
anyway?
What even is the job description? In this episode
We're joined by Abhishek Mistry, engineering leader at Hudson River Trading to discuss exactly that.
What is this job that ICs aspire to?
How is it different from management?
(00:31):
How do you influence and lead people when you actually have no real authority?
Is it better to go broad or to go deep?
How does the staff engineer scale their impact and provide, like Tom Brady, tremendous value above replacement.
Abhishek has been in the industry for many years.
He has tons of insight and he's genuinely a very fun guy to talk to.
I hope you enjoy the episode.
Let's get into it.
(00:54):
Welcome back to the main thread.
Everybody.
I am your host,
Brian Ogilvie along with my co-host Alex Gaiser.
And we are joined today by a Abhishek Mistry.
I just have to go on record and say that that is one of the coolest names ever to have been uh to have been coined.
Abhishek and I have been friends for a number of years since our kids were in preschool together.
I'm excited to have him on the podcast today.
(01:14):
Um, Abhishek, Do you wanna give us a little,
um,
just run down who you are,
where you came from,
all of that stuff.
Um And thanks Brian,
thanks for having me on.
So my name's Abhishek.
I uh I had like kind of a,
a little bit nontraditional background.
I guess that's maybe a theme for this,
this podcast,
the hosts here.
Uh and myself.
Uh But no,
I,
I started my career at JP Morgan and I was a research analyst working in finance.
(01:38):
I was,
you know,
like in a role that was somewhat technical but it wasn't,
it wasn't actually a software role.
It was a little bit more of a data science kind of role.
Um But I had a computer science background and,
and at the same time,
I kind of had a little bit of a side hustle uh building software and selling DJ software,
which is kind of a interesting side gig.
And,
yeah,
after eight years of that,
I was kind of,
(01:58):
you know,
I wanted to do something in the tech space.
Uh I really wanted to like,
try to start my own business.
And so,
so I left that job and,
and did my own,
like,
little tech startup for a couple of years,
made a,
you know,
a software product that,
that still sells today actually.
And that was a that was a fun gig.
It was an interesting gig.
I worked a lot,
learned a lot,
(02:18):
uh learned a lot of different technologies um As you know,
like you work on a startup.
That's,
that's kind of like what you do.
And I did that for,
for a couple of years.
I had a great time with it.
Um And then,
then it was time to move on and then,
um then I got into kind of like merge my interest and got into automated trading.
So the last six years call it,
I've been been working in automated trading.
(02:40):
I was at Two Sigma which is a quantitative hedge fund for three years.
And I worked on a team that traded sort of retail order flow and then um then moved over to a company called Hudson River Trading three years ago.
And I've been at HRT since. The first couple of years at HRT
I was an IC.
And so I built a lot of,
you know,
data infrastructure and worked on a lot of tooling and back testing tools and,
(03:04):
you know,
other sorts of projects for um a mid frequency trading strategy.
And then I moved over to leading a team.
Um you know,
now,
now I think we're nine people and this team basically builds kind of like UIs for trading purposes.
So some of it's for kind of monitoring our,
our,
our existing automated trading,
some of it's for kind of manual or semi-systematic strategies where humans,
(03:27):
you know,
need to be in the process because of various reasons,
like maybe the market is not fully automated or something like that.
And then there's,
you know,
some other sort of tooling for,
for certain research purposes,
like you can view jobs that you've submitted in a,
in a,
you know,
sort of distributed research environment and get understand kind of what's going on.
Are there any actions you need to take?
Did you make a bug somewhere?
(03:48):
And,
and you,
you take a look at it um provides a lot of like visibility.
Um I think the thing that our team does that's a little bit different from other teams is that we usually have some web component to our,
our software.
So a lot of people build various command line tools and processes that run and it's all like,
you know,
like there's a lot of low latency C++ and things like that.
I think for us,
(04:08):
we have a lot of back end tooling,
but the front end is,
is,
is a website.
So that's,
that's,
that's where it's a little bit different.
So yeah,
that's,
that's a little bit about me.
Awesome,
man.
Thanks for that.
That's,
that's great.
Um Yeah,
cool.
Like we are definitely a um a non-traditional background group here today.
So this is great.
Those who don't remember Alex and I were both uh like coding bootcamp grads before we got into our careers and had wholly different lives beforehand.
(04:34):
We've worked on some cool things,
but I don't think we've got cool,
uh DJ software,
cool projects under our belts.
So,
that's great.
I know I've never done,
never done DJ software.
I've actually had the pleasure of being at a bar where your,
um,
where your photo sharing app was deployed and,
and it was really cool,
it really enhanced the,
the whole experience.
It was great.
So we,
we talk about this podcast a lot about um about trying to grow to staff engineer,
(04:58):
trying to uh you know,
have impact as a staff engineer,
but we wanted to kind of take a step back and just to define (05:01):
what even is a staff engineer?
It seemed like that was a topic that you're excited to talk about Abhishek.
So um I just love to kick that right over to you and say,
uh you know,
in your quick definition,
like,
what would you say?
What is a staff engineer?
Yeah,
I mean,
it's a great question because I feel like I struggled with like understanding this question and the answer for many years.
(05:23):
I,
I think like,
what the way I think about it is like,
OK,
what's a senior engineer?
I think a senior engineer is someone who you can give project and they will sort of make it happen,
right?
Maybe it's,
you know,
they,
they scope it out,
you know,
plan out,
you know,
the Jira or,
or,
or like,
you know,
sort of milestones,
things like that.
They may lead junior engineers in this effort,
(05:45):
delegate parts of it,
talk to stakeholders,
et cetera.
But it's all sort of like a project that they're sort of given and,
and,
and then they sort of run with it.
Right.
I think a,
a staff engineer to me is sort of like,
it's a step beyond that.
It's kind of like,
like,
I think of it in terms of like in sports people talk about like value above replacement,
right?
(06:06):
Like you,
you know,
you get like Tom Brady is an amazing quarterback and then like,
you take your average quarterback and you just replace him with Tom Brady,
you know,
then you go to win the Super Bowl,
right?
Like that's kind of like you've done this experiment,
we've seen that,
that,
that's true,
right?
So,
so,
um you know,
like,
so clearly to so clearly Tom Brady has tremendous value above replacement.
(06:28):
And so like when I think about staff engineer,
it's sort of like,
is this person giving,
like delivering much more value than just,
you know,
some other senior engineer that I could hire from,
you know,
uh uh Meta or,
you know,
some place like that,
right?
So,
so that's kind of like what,
what I think is a staff engineer is like someone who's adding a lot of value.
So I,
what I,
I wanna dig in a little bit more about on that,
(06:51):
providing more value.
Because I think one of the mistakes that a lot of juniors make as they're looking to grow their careers and,
and reach sort of beyond that senior level is when you're a junior,
you,
you know,
you're mostly blocked by your ability to learn and execute in certain technology,
whatever it is that you're doing at your company and the more efficient you become at using that technology,
(07:11):
the more productive you are.
You ship more code,
you,
you learn better design patterns,
et cetera,
et cetera,
you get more done.
But there's a ceiling on that,
right?
Like you get,
you're gonna get promoted to a senior engineer are usually by the strength of your technical abilities.
And at a certain point,
you've maxed out how productive you as an individual can be.
And in order to provide more value,
you can't just simply work longer hours or write more code,
(07:33):
right?
Like that is diminishing returns at this point.
So the kind of value you're talking about,
what do you,
what do you see that as?
Yeah,
I mean,
I think it's different for different people too,
right?
Like some people are really like some people are really deep technically and they understand and nuances of,
you know,
maybe it's,
you know,
you're working on embedded systems and you really understand memory layouts and CPU instructions like better than,
(07:56):
than your peers.
And and,
you know,
you can solve a technical problem that other people can't solve.
Right.
And that's one way to kind of be,
be that sort of staff engineer.
It's like,
you know,
other people are struggling with something and you,
like,
jump in and,
you know,
you figure it out pretty quickly.
It's,
it's one way,
it's,
it's often a difficult way because that,
that,
you know,
sort of like years and years and years of expertise and
(08:17):
maybe a lot of people don't get there.
Another way is some people are just,
like,
really good at being broad,
understanding,
you know,
across an organization or,
or a company or,
or multiple teams or something like,
kind of how things fit together,
right?
So,
like that breadth and understanding of,
like all these different parts helps them to then navigate and,
(08:39):
and come up with like really awesome things that other people might not have done,
right?
This might be like,
you know,
I'm building some new tooling.
Like I,
I did this recently where,
you know,
my team had,
had sort of like a bunch of different log files in different places and it was super messy and it was oftentimes hard to find out what,
what,
er,
where errors were like,
you know,
it's like,
oh,
this thing broke,
(08:59):
I don't,
I don't even know where,
where to look for the errors,
right.
So,
so let's clean this up.
Like I want to build,
build some tooling to clean this up.
Right.
And so,
um,
but rather than just,
just working on cleaning up that,
that,
that logging for,
for my team,
then I went and talked to some other teams that were having similar problems and understood their needs and like how some of their needs might be a little bit different than our needs.
(09:20):
Right.
But then,
like,
maybe we can build some common tooling that'll work across multiple teams and,
and,
and everyone benefits.
I mean,
obviously our team will benefit and we,
you know,
sort of accomplish that goal.
But then other teams will,
and,
and in order to do something like that,
you have to understand a little bit of what,
you know,
what people are doing and like how everything sort of ties in and fits together and,
and that can be,
you know,
(09:40):
pretty,
pretty hard to do,
you know,
there's different ways,
like,
like I said,
sort of like two ways I,
I immediately thought of is like going really deep or going really broad.
Yeah,
we talked about this before.
It seems like a lot of times like your typical tech lead has to usually be very broad in their knowledge and maybe get very,
very deep in a couple of uh couple of strata, right?
(10:02):
but have a,
a pretty decent understanding of a lot of things and a very deep understanding of a few things.
So that actually begs a question that I think is very a very distinctly different thing between a senior engineer and a staff engineer.
Because one thing that really seem that makes seems to be much more important is something like institutional knowledge and organizational knowledge at the staff level.
(10:22):
Whereas it seems like in order to solve that sort of question,
you need to really have a deep and broad understanding of the organization that you were working in itself rather than.
Whereas if you're working as a senior engineer,
you may only be working on one part and you can actually move between companies or between teams more easily because you,
the specific technologies that you're working with are,
you know,
(10:43):
less important than like the specific problem that you're working on.
Whereas from what you're describing,
you need to understand like the broad strokes of the organization itself.
And I don't know,
it's just something that I think is,
I'm curious if that's what you found and as somebody who wants to move on to that level,
how do you cultivate that sort of knowledge?
Absolutely.
I think that's,
yeah.
No,
that's a great question.
(11:04):
I think that's,
I think you're 100% right there.
I,
I this is where,
like,
you know,
a lot of times people sort of make the comment that engineers only stick around for like 2 to 3 years on a,
at a particular job or like,
you know,
it's sort of like after a couple of years it's time to move on.
Um And I think,
like,
you know,
if you're an early career engineer,
like that's,
(11:24):
that's a pretty reasonable strategy,
you know,
you get exposed to a lot of different things,
you see things at different companies,
you understand what your preference are,
preferences are and then sort of like where you shine and,
and,
and,
you know,
that kind of helps you,
but at a certain point it's,
you know,
it is a little bit detrimental to,
to sort of jump around like that because I think what we,
(11:44):
we're kind of getting at here is you have to do really impactful work.
And in order to do really impactful work,
you need to understand a lot about your organization or your company or,
you know,
just different moving parts and it's,
it's hard to do that if you're jumping around a lot,
I think one way that you can kind of show impact that without having a ton of organizational context is having context on similar problems,
(12:10):
right?
So,
like,
if you've like,
I don't know,
you build Google search and then like Microsoft hires you to work on Bing,
there's probably a lot of knowledge that you have that,
that you can apply.
I mean,
you know,
obviously you have like IP considerations and stuff like that,
but like still like you,
there's a lot of techniques and things like that,
that,
that you can sort of bring to the table,
you know,
like,
so that,
that,
that's,
that's one way you can kind of bridge that.
(12:30):
But I think,
you know,
it's a little bit harder,
it's definitely a little bit harder.
And so,
like,
coming that,
you know,
like this is why companies are a little bit reluctant to hire people at high levels because it's just like there's so much organizational,
you know,
uh knowledge that,
that,
you know,
makes it easier to have like a ton of impact.
So,
yes,
absolutely.
I think,
(12:51):
I think building up that,
that context to,
to take on those impactful projects and become a,
a staff engineer like that's that,
that,
that's sort of a little bit of a requirement there.
And to follow up on your example,
to follow up on your example with moving from search,
from like Google search to uh to like bing or something like that,
maybe you can solve the technical problem but and it kind of maybe is jumping ahead a little bit,
(13:13):
but it seems like a lot of the skills are not technical skills,
they're organizational skills and people skills and those don't necessarily transfer over if you're really good at getting something done at Google.
But if you go over to Microsoft,
understanding the levers that you need to pull in order to get something done may not translate.
And it may be,
even though you may might have might be the best in the world of getting a search engine put together you might suffer compared to somebody who just knows who to talk to,
(13:41):
you know?
Yeah.
Yeah.
I mean,
you're not even just knowing who to talk to but,
like,
having relationships with those people.
Right.
Like,
I mean,
there's,
there's so much that I've gotten done in my career just because,
like,
hey,
like,
you know,
it's like,
hey,
this guy,
like,
you know,
it's somebody that,
you know,
you,
you've talked to,
you,
maybe you've worked on projects with uh or at least,
you know,
gone out to lunch with and,
(14:02):
and such,
like,
I,
I don't know,
there,
there is an aspect of,
of things that,
like,
you know,
we're all humans.
Like,
I,
I think,
you know,
we,
we sort of like,
you know,
we,
we live in code and like live and breathe these,
like,
you know,
sort of autonomous,
like,
you know,
automated things,
right?
But at the same time,
like,
people are people.
If you're asking somebody to sort of,
you know,
(14:22):
go along with some design or,
you know,
you're asking for feedback on some,
some tool that you built and,
you know,
the person's like,
super busy,
like,
it's not really top,
you know,
like,
it's not really the thing that they're,
like,
personally interested in at that point or,
you know,
like,
maybe they got some deadline to work on or whatever.
Right?
And,
and,
and if you have a relationship with that person and then it's a totally different conversation. Then,
(14:43):
then they're much more open to,
you know,
kind of helping you out and maybe,
maybe they take like,
you know,
10 minutes,
15 minutes,
whatever,
take a look at whatever you're,
whatever you're doing and,
and get back to you.
And that's,
that's the kind of thing that,
like,
you know,
you,
you,
you need to,
to build those relationships. And those relationships take time to build,
you know,
it's very hard to come in at a,
at a,
at a very senior level and work at,
(15:04):
at,
at that,
in that scope with that many relationships required when you haven't had time to build them.
Yeah, absolutely, and I've seen this where,
like,
you know,
uh somebody came in,
you know,
and it's like,
you know,
top dog at this other firm and like,
you know,
they come in,
they have some idea and they're like,
"Oh,
we should do this,
this and this," and people are like,
"No."
you know,
like,
like,
like,
you know,
(15:25):
like who are you,
like,
you know,
like who is this person dictating?
Like how we should do things like,
you know,
like we're not doing this.
I do want,
I want to really come back to that exact subject in a minute because it's about like how you have influence as it engineer versus as an executive.
But uh but one thing I want to do first is I,
I feel like we talk a lot about the differences between staff is like when you're growing to a senior level,
(15:47):
it's very much about scaling yourself scaling your abilities.
But when you're getting to that staff level,
it's,
it's largely about scaling through others,
right?
These relationships that we've been talking about. Using,
using a large network of a lot of different people to get something very complex done.
And in some ways it seems like the job,
uh,
of being an effective staff engineer is,
(16:10):
is fundamentally different from what the job is of being a successful junior or senior uh senior engineer.
And I,
I wonder sometimes like are the skill sets that you build as you work from junior to senior level,
all of those skills that you work on and develop?
Are those still valid and useful at the staff level?
Or do you have to learn a fundamentally different group of skills in order to operate effectively at that next level?
(16:32):
I mean,
I,
I,
I would say they're still necessary,
you know,
you still have to be able to write code,
you still have to be able to find bugs and things like that.
I just think that like the job becomes a lot more than that,
right.
So it's a whole different set of skills that you need on top of that and,
and you know,
you're gonna be writing code,
you know,
if your staff engineer,
like you are expected to write code in most places,
(16:53):
but the code you're gonna be writing is probably more technically demanding than what other people are doing.
Or maybe,
maybe it's relatively simple stuff.
But at the same time,
like,
you don't have a lot of time,
like,
your,
your time is being spent on so many different things.
And if you want to put out this diff,
you know,
it better be something that you can do relatively quickly.
(17:13):
Uh And that's where those,
those skills come in handy.
Is that like you've done this so much,
you have like a certain mastery over,
over the process of,
of architecting code testing it, going through code reviews,
you know,
all that stuff that you can then spend less time on that and it frees up more time for you to,
you know,
scope out some important idea or get feedback for on something or,
(17:35):
you know,
like just different things that you'll,
you'll need to do that are that are outside of just writing code.
Yeah,
it seems like uh these days,
most of the code that I write tends to be,
as you mentioned on something that's pretty complex that it's not just straight out of the box like,
oh,
I'm gonna,
I'm gonna just make this React component or this API call or um you know,
build this simple server,
(17:55):
right?
It's like you,
you've,
you've,
you've got a product that's running and you've discovered now that you've scaled beyond your,
your query or your rate limit,
that kind of thing and you need to sort of rearchitect how you batch things or,
you know,
pre aggregate data,
that kind of stuff and it takes time to sit and think and explore a couple of ideas.
And by the time you really get around to writing code,
it may not be that much code,
but it's,
it's the right code and,
(18:17):
and it's,
it solves a very complex problem.
I've noticed that definitely like my lines of code,
like go down when I'm working on problems like that,
even though it's,
it may be just as impactful or probably more impactful as the code I was writing before when it was just like you've got a task, pull down
the task, there's a very well defined path in order to accomplish it,
get it done,
put it up for review,
(18:37):
land it move on.
Yeah,
absolutely.
I mean,
I think,
I think some of the most impactful work I was talking to one of the,
one of the other engineers at,
at HRT about this.
Uh you know,
the idea of like a request for comment,
this is something that some companies are really,
really good at, some companies take to the extreme, and then some companies don't really do much of. I think like a lot of times like the value like the most impactful thing you can do is just create an RFC,
(19:01):
right?
Like it's not even the code,
like none of the code is actually all that complicated or,
or important,
but it's the,
it's the act of putting together all the thoughts,
you know,
collecting feedback from people and coming up with a good design and building that document.
That's the,
that's the hard part,
right?
And that's the part that's,
like,
really impactful and valuable and then,
and then anyone can just sort of take that and,
(19:22):
and,
you know,
sort of run with it.
Yeah.
So a lot of what I've been working on lately has been building out a fundamental source of truth query layer that brings together data from a lot of different systems that have very few (if any) standards between them.
And uh in order to build that out,
you know,
I need to fundamentally understand the the needs of my consumers,
(19:43):
right?
The people who need to query this,
which is,
you know,
the vuln management, privacy, security people versus uh the people who are actually the SMEs in these different systems that I'm trying to aggregate data from.
I can't be a subject matter expert on every possible system that that Meta has running,
right?
So,
so much of it is,
is like you said,
these RFCs where I'm gonna write down,
what I think is probably a good architecture of how I'm gonna get data out of one system.
(20:06):
And what I think is the important data to get out, how I'm gonna normalize it so that the rest of the world can consume it.
And then I send it out to the people who really know and they say,
well,
yeah,
you know,
uh you,
what you think is a unique identifier in this system actually isn't because of X Y Z.
You need to think about this other thing,
right?
And that's the only way to actually come to the right solution is by engaging all of these different folks who really know how different systems work.
(20:30):
And that does become a much larger part of my job now is is just getting that alignment.
Yeah,
absolute alignment.
I love,
I love that.
You said that word.
Like I think that's,
that's such a like key part that people often miss.
I mean,
I don't know how it is with you,
Brian,
but like,
I,
I definitely feel like there are,
there are points in,
in,
in my past where I work some things that I thought were important,
(20:51):
but management didn't sort of like see them as important, and like that was such a common pitfall.
And,
and I think that's like a key differentiator of staff engineers is that issue of alignment.
I think we've all experienced that from time to time.
Yeah,
I like or I think something is really important because it feels that way viscerally to me, and I work on a solution and people are like "Uh, great,
nice."
Uh And then when it comes time for performance review,
(21:13):
I'm trying to talk about the impact of what all the work I did and I look at that thing and I go,
"You know,
like I enjoyed working on that,
but I don't think it had the broad impact that I imagined it would and I probably should have worked to align what I was gonna work on with my,
not just my own manager but his skip, my director,
(21:33):
like making sure that what I was working on was what they feel is impactful and,
and then,
you know,
putting metrics in place beforehand to verify whether or not it's having the impact
I thought it would."
Yeah,
I think that's,
I think it's so important and,
and it's definitely something that I've missed in the past and I,
I,
I think it's great to have ideas and,
and be able to like,
proactively find things and that's part of the staff being a staff engineer,
(21:55):
but you can't just like,
work on things that,
that you think are,
are worthwhile.
It's really like,
what is management value and,
and it is,
you know,
sometimes that's not gonna be the same thing and,
and you might think like,
oh,
these guys are clowns like management,
you know,
full of idiots or whatever.
Uh And oftentimes,
you know,
that there is some truth to that,
but like,
a lot of times there,
you know,
that's not the case.
(22:16):
It's just that like,
and has a different perspective of,
you know,
all the different projects that are going on different incentives and maybe like the thing that you're interested in is actually,
is actually important and impactful.
It's just not the most important and impactful thing in that.
Org at that point in time,
it might be something that right now I,
I,
I kind of want,
you know,
people to work on this X Y Z other thing and then we'll come back to your thing and maybe,
(22:40):
you know,
six months or a year or whatever.
And that's,
that's something that like everyone should be kind of like,
just be humble about it and kind of like,
take the,
take that perspective.
Like you can,
you can come up with ideas,
you can find evidence and I'm,
I'm a big fan of prototyping and finding data and things like that.
Like I think these are things that don't take that much time,
you know,
spend,
you know,
a couple of hours like prototyping something,
(23:01):
go show people and go talk to your manager,
go talk to,
to their manager to your point,
Brian,
like,
you know,
couple levels or so like,
you know,
and see what people think and,
and maybe they,
maybe they just say,
"Yeah,
this is great,
Brian,
like,
go ahead,
you know,
like we want you to do this right now."
But they might also say like, "This is cool,
but,
you know,
we'll probably look at this at some point in the future," and,
(23:23):
and that,
that's not a knock on you like,
that's not a knock on your idea.
It's just that,
you know,
there's all these other important things to work on and ultimately it's a business.
Right.
It's not,
you know,
we're not doing this because we just like,
I mean,
hopefully you like what you're doing and I,
I definitely like what I'm doing,
but at the same time,
like,
you know,
the point of a company is to,
to,
to then,
you know,
(23:43):
deliver value to,
to customers.
And so you can't lose sight of that fact.
Uh As,
as we're,
as we're sort of picking projects and,
and working on things.
Yeah.
Um I've got a project myself that you mentioned like an Infra Management project that,
that I am very passionate about,
but it's been on ice for a year while I've worked on some other things that are,
are higher priority for the org.
And um yeah,
(24:04):
I mean,
it,
that's just kind of life,
right?
You're,
you're hired by a company to,
to increase their revenue or lower their bottom line or whatever it is that,
you know,
your company needs you to do, and alignment with those company priorities is what makes an effective staff engineer,
Alex,
I saw you mentioned that you had a question here that I really want to get into around um around how is being a staff engineer fundamentally different from being a manager at a similar level because there's so much overlap between when it comes to coordinating between different pieces.
(24:33):
Um And a,
a good manager certainly does that.
But what,
what are the differences between a good staff engineer and a good manager?
Yeah,
I mean,
I think that there's definitely a lot of overlap there,
right?
Like,
I think part of being a staff engineer is,
is having the ability to lead people and,
and set direction and kind of like guide projects and part of being a manager is also,
(24:57):
you know,
sort of leading people guiding projects,
setting direction.
I think,
I think where,
where it's kind of different is that,
you know,
as a,
as a staff engineer,
as an IC,
you don't have reports,
so you're expected to kind of lead people but you don't have like there's no like set authority there,
right?
Like when you're in a management role,
like the people who report to you,
(25:19):
you know,
there's a little bit of sort of like a power differential there,
right?
In,
in,
in that like you control probably their,
their compensation and,
and certainly their,
their growth and at the company and things like that,
whereas,
you know,
as a,
as an IC staff engineer,
that's not going to be the case.
So how do you then lead people?
Right?
If you know,
it's a little bit different skill set even than,
(25:40):
than,
than as a manager in that sense.
So there's overlap,
but it's,
it's also a little bit different,
like I think the way you lead people as a staff engineer is to kind of convince people that your ideas are the best ideas.
And that's where,
you know,
building relationships with people helps.
That's where,
like,
just,
you know,
being good at communication.
I think this is,
you know,
something that helped me a lot was I took some like classes around influential communication and kind of,
(26:06):
you know,
listening to people and like making sure that they feel heard and,
and building consensus.
These are like,
these are skills again,
they're not,
they're not coding and they're kind of not the,
the sort of thing.
Engineers typically like,
focus on to be honest,
but it is,
it is the thing that's like critical for,
for like,
you know,
operating at that level.
If you uh if you have any of those classes that you took that you want to recommend to listeners,
(26:29):
uh We'll definitely like,
we could put links in the show notes.
So I'd love to hear about those.
Yeah,
I'll,
I'll,
I'll try to uh dig up some URLs and,
and send them over.
Cool.
That actually dovetails with another question that we had,
which is,
are there any unexpectedly useful skills or experience that you've acquired anything that,
you know,
you wouldn't get from reading a book on being a staff engineer?
That might be surprising.
Um I think being a good listener:
(26:51):
the ability to listen to people and like,
understand what they're really asking or what they're really concerned about. And it's not like a lot of,
like,
empathizing with people and just kind of like seeing where they're at and where they're coming from.
You know,
this is the kind of thing that,
like,
a lot of people do.
I mean,
some people do this for a living,
right?
Like you're therapist.
Like,
they,
they do do this for a living.
Um,
but,
like,
for me personally,
(27:12):
I don't think that was like,
something that I,
I had built up but like to,
to have this,
this sort of like,
impactful projects and,
and,
and to drive,
you know,
change across the organization or across teams or things like that,
you kind of have to spend a lot of time just listening to people and that's,
that's something that,
like,
I think we could all do better at and,
you know,
there is,
there is a certain skill to,
(27:33):
like,
when somebody disagrees with you,
you know,
and you,
you talk to them and then,
and then they,
like,
walk away feeling like,
ok,
you know,
like this person,
you know,
sort of heard what I had to say and,
and,
and I don't think like,
this person's a jerk or something like that.
Right?
Like that's,
you know,
and maybe,
maybe you're still gonna,
like,
ultimately not gonna do what they would like,
but they don't get upset about it.
Right.
And that,
(27:53):
that's,
that's the kind of like skill that it's pretty useful and it's like it was something that,
that I,
I found kind of unexpected.
Useful at all levels. That's always gonna be helpful throughout your life.
I think you say life,
right?
Like you're talking to your,
to your partner or whatever,
right?
How to be an effective staff spouse.
(28:14):
Uh,
I,
I think my wife would define me as a senior spouse.
Not a staff spouse.
That's true.
No,
I,
I definitely have a director at home.
Um,
well,
we are,
we are getting towards,
um,
towards time.
This is a really terrific conversation and I,
I'm having a great time with this.
Um,
but I wanna give us a chance to kind of talk through,
um,
if you have any,
any like,
uh,
general recommendations,
(28:35):
things you've been reading lately,
something that's just helping you that,
that is,
um,
you know,
just something that hasn't been part of the main conversation.
Anything you want to throw out there for listeners who might want to check it out.
If you want a minute to think I'll jump in and start.
Um,
so we started a book club for my Org a few,
uh,
gosh,
about six weeks ago now and we originally intended it for juniors to learn.
(28:56):
But as it turns out a lot more seniors showed up,
uh,
which is kind of,
I think,
telling about,
uh,
where they're at in terms of their kind of interest in,
in growing themselves and learning.
But we've,
uh,
we've just been reading together And so what I wanna pick here is community learning.
So we've each week been taking a chapter of "The Pragmatic Programmer,"
reading it on our own, and then one member of the group will sort of present the chapter when we come together.
(29:17):
And,
um it's been really great.
Uh I've read the book,
you know,
I don't know how many times at this point over my career and it's it,
but hearing other people and getting a chance to discuss it with them is,
is having me look at things in a different way, and I just love this.
So if you have a chance to uh get into a group and sort of go through something and learn it together,
uh I think it is far more effective than when you do this on your own.
By the way,
I have to shout out to uh Charles Max Wood from JavaScript Jabber because reading "The Pragmatic Programmer" in a book club was totally his idea which I stole.
(29:45):
Um But uh All right,
uh Alex,
you got,
uh you got anything you wanna wanna share about?
I'm rereading uh "Refactoring" by Martin Fowler.
That's always,
that's always a good book.
It's always got something new to add every single time I read it and it always gives me a lot of useful insight.
And then I'm also reading um "Functional Programming Simplified" by Alvin Alexander.
(30:05):
It's a Scala book, and I've got it with me.
It's got the coolest cover because it's got a Lambda, and it's attacking an entire crowd of people because functional programming can be scary,
but not when you read this book.
Um Yeah,
it's a good book.
It's a good intro.
I,
I assume all those people are Java 00P developers,
they're all Java 00P developers.
They wouldn't know a functor if it,
(30:26):
you know,
came up and gave a big wet kiss.
Yeah.
Um All right.
So,
yeah,
I mean,
uh I feel like,
well, I don't know how to follow that.
That was weird.
Um Functional programming is great.
I,
I,
I actually TA'ed for a functional programming class back in college.
(30:46):
It's fun stuff.
One thing I like actually topical that I,
that I thought was interesting is uh there's a book called "The Staff Engineers Path".
I don't know if you guys have read it,
but uh that's something I would recommend it.
Um Yeah,
it's just,
it's sort of a collection of anecdotes of various people um who sort of like,
you know,
chronicled,
like some of the things they found useful or influential or,
(31:07):
you know,
kind of their journey.
Um And I,
I,
I would,
I would recommend taking a look at that book.
Yeah,
I think I,
I think I read it a couple of years ago and it was,
it was great. That's awesome. I'll check it out.
I'll put a link in the show notes as well.
Otherwise,
you know,
I spend a lot of time just like reading white papers of different things or documentation and,
you know,
trying to understand this framework versus that framework and like,
(31:27):
where,
you know,
what we should be using and,
you know,
what are the pros and cons of like using whatever the latest shiniest thing is versus like something tried and true like Tornado or whatever,
right?
Like,
I,
I don't know,
like that's uh that,
that's,
that's,
you know,
fortunately or unfortunately,
there's a lot of my time is just like figuring out like what,
what technology is out there. There's just so many different things.
(31:48):
Yeah,
I love getting a chance to explore new languages that I don't usually work in because,
um,
sometimes like the,
the conventions that they set up in that language are usually because they were trying to solve some very specific problem that other languages failed these creators in and,
um,
and learning about that process,
whether I ever end up using that language or not in production,
it's kind of nice to just sort of learn why they felt it was necessary to create it.
(32:12):
And,
and maybe I learned some paradigms that I can bring back into my daily work.
Where do you find these white papers?
Oh,
just go to different projects.
I mean,
you know,
like whatever it is you're looking at,
like,
let's say you're looking at uh we were talking about logging libraries earlier,
like,
I don't know,
you could look at things that are out there like Elasticsearch or,
you know,
um,
or,
or,
I don't know,
just different logging,
(32:33):
you know,
and,
and metrics and collection tools,
you know,
see how they're implemented.
Like,
I,
I think I remember like,
some,
a couple of years ago,
uh,
I,
I was looking into something that's kind of,
I feel like it's dead now,
like these,
uh,
noSQL databases and I was trying to understand like,
uh,
I,
I don't know,
maybe,
maybe "dead" is the wrong word,
but it's definitely like,
it was hot for a while and then now it's probably like.. They're dead to me.
(32:55):
I'll say that much.
Um So I was trying to understand it,
you know,
like I read like a bunch of white papers uh that,
you know,
Cassandra put out or MangoDB and,
you know,
like,
understand like why they're doing things a certain way or like,
you know,
what technical challenges they're trying to
solve or,
um you know,
like that,
that,
that's the sort of thing I'm talking about.
Cool.
Yeah.
If you're,
if you're not seeing the video,
(33:16):
you missed an epic eye roll from Alex,
which is really unfortunate because uh uh this is,
this is a conversation we've had a bunch in the past around.
Um You know,
I'm just like,
ah,
I don't like document databases.
They're not my thing.
Like ah just give me a SQL table.
I'm super happy.
Uh And,
and uh you know,
and the argument is like,
oh,
it has a scale.
I'm like,
well,
Facebook is pretty big.
(33:36):
We're based in SQL.
So.
But,
yeah,
it's funny.
I just think,
like,
I,
I,
you know,
like IBM has been making SQL databases for,
like,
50 years and they're used in,
like,
all these different banks and stuff like that doing,
like,
you know,
millions of transactions a day.
Right.
Like,
it's,
it's possible to do these things that,
uh yeah,
it can scale.
(33:56):
Yeah,
like that.
Yeah, it's a very boring thing,
right,
like build a SQL database that doesn't get you venture funding.
That's right.
Everybody's, no, everybody does SQL.
But have you heard about like anti SQL?
You have to use the opposite schema.
It's great.
It scales inversely.
It's wonderful.
It,
yeah,
the the,
(34:16):
the the paradigm there is no normalization whatsoever.
Um So yeah,
listen,
I think we've ragged on MangoDB enough and,
and uh and in all honesty,
like,
you know,
every technology that's out there,
it,
it exists because it solves a particular problem,
right?
And so,
you know,
again,
like not to,
not to belabor my own preferences,
right?
That's all they are.
They're my preferences.
(34:37):
Uh So every technology has,
has worth and has validity.
It's just a matter of like how you want to solve your problems.
Oh,
I,
I should mention that my views are,
you know,
the views expressed here are my own and not those of my company, Hudson River Trading. I thought I'd say that.
Thank you.
For being the adult in the room and reminding us that that is always true:
(34:57):
We are here as individuals.
So,
yeah,
we're,
we're right at time here.
Um So I,
I think we should just go ahead and call it here,
but, Abhishek, I wanna say thank you so much for your time.
This is really terrific uh explaining your perspective on,
on what makes a staff engineer at all.
So again,
thanks for coming.
Uh You're welcome back any time.
Um That was awesome.
Absolutely.
All right.
Thanks a lot,
Brian and Alex.
Uh You know,
(35:17):
I have a pleasure coming on here.
This is,
this is a great time. And uh for everyone listening,
we will catch you next time.
Welcome back to the main thread where we discuss the journey from senior engineer to staff engineer.
So what is a staff engineer
anyway?
What even is the job description? In this episode
We're joined by Abhishek Mistry, engineering leader at Hudson River Trading to discuss exactly that.
What is this job that ICs aspire to?
How is it different from management?
(00:31):
How do you influence and lead people when you actually have no real authority?
Is it better to go broad or to go deep?
How does the staff engineer scale their impact and provide, like Tom Brady, tremendous value above replacement.
Abhishek has been in the industry for many years.
He has tons of insight and he's genuinely a very fun guy to talk to.
I hope you enjoy the episode.
Let's get into it.
(00:54):
Welcome back to the main thread.
Everybody.
I am your host,
Brian Ogilvie along with my co-host Alex Gaiser.
And we are joined today by a Abhishek Mistry.
I just have to go on record and say that that is one of the coolest names ever to have been uh to have been coined.
Abhishek and I have been friends for a number of years since our kids were in preschool together.
I'm excited to have him on the podcast today.
(01:14):
Um, Abhishek, Do you wanna give us a little,
um,
just run down who you are,
where you came from,
all of that stuff.
Um And thanks Brian,
thanks for having me on.
So my name's Abhishek.
I uh I had like kind of a,
a little bit nontraditional background.
I guess that's maybe a theme for this,
this podcast,
the hosts here.
Uh and myself.
Uh But no,
I,
I started my career at JP Morgan and I was a research analyst working in finance.
(01:38):
I was,
you know,
like in a role that was somewhat technical but it wasn't,
it wasn't actually a software role.
It was a little bit more of a data science kind of role.
Um But I had a computer science background and,
and at the same time,
I kind of had a little bit of a side hustle uh building software and selling DJ software,
which is kind of a interesting side gig.
And,
yeah,
after eight years of that,
I was kind of,
(01:58):
you know,
I wanted to do something in the tech space.
Uh I really wanted to like,
try to start my own business.
And so,
so I left that job and,
and did my own,
like,
little tech startup for a couple of years,
made a,
you know,
a software product that,
that still sells today actually.
And that was a that was a fun gig.
It was an interesting gig.
I worked a lot,
learned a lot,
(02:18):
uh learned a lot of different technologies um As you know,
like you work on a startup.
That's,
that's kind of like what you do.
And I did that for,
for a couple of years.
I had a great time with it.
Um And then,
then it was time to move on and then,
um then I got into kind of like merge my interest and got into automated trading.
So the last six years call it,
I've been been working in automated trading.
(02:40):
I was at Two Sigma which is a quantitative hedge fund for three years.
And I worked on a team that traded sort of retail order flow and then um then moved over to a company called Hudson River Trading three years ago.
And I've been at HRT since. The first couple of years at HRT
I was an IC.
And so I built a lot of,
you know,
data infrastructure and worked on a lot of tooling and back testing tools and,
(03:04):
you know,
other sorts of projects for um a mid frequency trading strategy.
And then I moved over to leading a team.
Um you know,
now,
now I think we're nine people and this team basically builds kind of like UIs for trading purposes.
So some of it's for kind of monitoring our,
our,
our existing automated trading,
some of it's for kind of manual or semi-systematic strategies where humans,
(03:27):
you know,
need to be in the process because of various reasons,
like maybe the market is not fully automated or something like that.
And then there's,
you know,
some other sort of tooling for,
for certain research purposes,
like you can view jobs that you've submitted in a,
in a,
you know,
sort of distributed research environment and get understand kind of what's going on.
Are there any actions you need to take?
Did you make a bug somewhere?
(03:48):
And,
and you,
you take a look at it um provides a lot of like visibility.
Um I think the thing that our team does that's a little bit different from other teams is that we usually have some web component to our,
our software.
So a lot of people build various command line tools and processes that run and it's all like,
you know,
like there's a lot of low latency C++ and things like that.
I think for us,
(04:08):
we have a lot of back end tooling,
but the front end is,
is,
is a website.
So that's,
that's,
that's where it's a little bit different.
So yeah,
that's,
that's a little bit about me.
Awesome,
man.
Thanks for that.
That's,
that's great.
Um Yeah,
cool.
Like we are definitely a um a non-traditional background group here today.
So this is great.
Those who don't remember Alex and I were both uh like coding bootcamp grads before we got into our careers and had wholly different lives beforehand.
(04:34):
We've worked on some cool things,
but I don't think we've got cool,
uh DJ software,
cool projects under our belts.
So,
that's great.
I know I've never done,
never done DJ software.
I've actually had the pleasure of being at a bar where your,
um,
where your photo sharing app was deployed and,
and it was really cool,
it really enhanced the,
the whole experience.
It was great.
So we,
we talk about this podcast a lot about um about trying to grow to staff engineer,
(04:58):
trying to uh you know,
have impact as a staff engineer,
but we wanted to kind of take a step back and just to define (05:01):
what even is a staff engineer?
It seemed like that was a topic that you're excited to talk about Abhishek.
So um I just love to kick that right over to you and say,
uh you know,
in your quick definition,
like,
what would you say?
What is a staff engineer?
Yeah,
I mean,
it's a great question because I feel like I struggled with like understanding this question and the answer for many years.
(05:23):
I,
I think like,
what the way I think about it is like,
OK,
what's a senior engineer?
I think a senior engineer is someone who you can give project and they will sort of make it happen,
right?
Maybe it's,
you know,
they,
they scope it out,
you know,
plan out,
you know,
the Jira or,
or,
or like,
you know,
sort of milestones,
things like that.
They may lead junior engineers in this effort,
(05:45):
delegate parts of it,
talk to stakeholders,
et cetera.
But it's all sort of like a project that they're sort of given and,
and,
and then they sort of run with it.
Right.
I think a,
a staff engineer to me is sort of like,
it's a step beyond that.
It's kind of like,
like,
I think of it in terms of like in sports people talk about like value above replacement,
right?
(06:06):
Like you,
you know,
you get like Tom Brady is an amazing quarterback and then like,
you take your average quarterback and you just replace him with Tom Brady,
you know,
then you go to win the Super Bowl,
right?
Like that's kind of like you've done this experiment,
we've seen that,
that,
that's true,
right?
So,
so,
um you know,
like,
so clearly to so clearly Tom Brady has tremendous value above replacement.
(06:28):
And so like when I think about staff engineer,
it's sort of like,
is this person giving,
like delivering much more value than just,
you know,
some other senior engineer that I could hire from,
you know,
uh uh Meta or,
you know,
some place like that,
right?
So,
so that's kind of like what,
what I think is a staff engineer is like someone who's adding a lot of value.
So I,
what I,
I wanna dig in a little bit more about on that,
(06:51):
providing more value.
Because I think one of the mistakes that a lot of juniors make as they're looking to grow their careers and,
and reach sort of beyond that senior level is when you're a junior,
you,
you know,
you're mostly blocked by your ability to learn and execute in certain technology,
whatever it is that you're doing at your company and the more efficient you become at using that technology,
(07:11):
the more productive you are.
You ship more code,
you,
you learn better design patterns,
et cetera,
et cetera,
you get more done.
But there's a ceiling on that,
right?
Like you get,
you're gonna get promoted to a senior engineer are usually by the strength of your technical abilities.
And at a certain point,
you've maxed out how productive you as an individual can be.
And in order to provide more value,
you can't just simply work longer hours or write more code,
(07:33):
right?
Like that is diminishing returns at this point.
So the kind of value you're talking about,
what do you,
what do you see that as?
Yeah,
I mean,
I think it's different for different people too,
right?
Like some people are really like some people are really deep technically and they understand and nuances of,
you know,
maybe it's,
you know,
you're working on embedded systems and you really understand memory layouts and CPU instructions like better than,
(07:56):
than your peers.
And and,
you know,
you can solve a technical problem that other people can't solve.
Right.
And that's one way to kind of be,
be that sort of staff engineer.
It's like,
you know,
other people are struggling with something and you,
like,
jump in and,
you know,
you figure it out pretty quickly.
It's,
it's one way,
it's,
it's often a difficult way because that,
that,
you know,
sort of like years and years and years of expertise and
(08:17):
maybe a lot of people don't get there.
Another way is some people are just,
like,
really good at being broad,
understanding,
you know,
across an organization or,
or a company or,
or multiple teams or something like,
kind of how things fit together,
right?
So,
like that breadth and understanding of,
like all these different parts helps them to then navigate and,
(08:39):
and come up with like really awesome things that other people might not have done,
right?
This might be like,
you know,
I'm building some new tooling.
Like I,
I did this recently where,
you know,
my team had,
had sort of like a bunch of different log files in different places and it was super messy and it was oftentimes hard to find out what,
what,
er,
where errors were like,
you know,
it's like,
oh,
this thing broke,
(08:59):
I don't,
I don't even know where,
where to look for the errors,
right.
So,
so let's clean this up.
Like I want to build,
build some tooling to clean this up.
Right.
And so,
um,
but rather than just,
just working on cleaning up that,
that,
that logging for,
for my team,
then I went and talked to some other teams that were having similar problems and understood their needs and like how some of their needs might be a little bit different than our needs.
(09:20):
Right.
But then,
like,
maybe we can build some common tooling that'll work across multiple teams and,
and,
and everyone benefits.
I mean,
obviously our team will benefit and we,
you know,
sort of accomplish that goal.
But then other teams will,
and,
and in order to do something like that,
you have to understand a little bit of what,
you know,
what people are doing and like how everything sort of ties in and fits together and,
and that can be,
you know,
(09:40):
pretty,
pretty hard to do,
you know,
there's different ways,
like,
like I said,
sort of like two ways I,
I immediately thought of is like going really deep or going really broad.
Yeah,
we talked about this before.
It seems like a lot of times like your typical tech lead has to usually be very broad in their knowledge and maybe get very,
very deep in a couple of uh couple of strata, right?
(10:02):
but have a,
a pretty decent understanding of a lot of things and a very deep understanding of a few things.
So that actually begs a question that I think is very a very distinctly different thing between a senior engineer and a staff engineer.
Because one thing that really seem that makes seems to be much more important is something like institutional knowledge and organizational knowledge at the staff level.
(10:22):
Whereas it seems like in order to solve that sort of question,
you need to really have a deep and broad understanding of the organization that you were working in itself rather than.
Whereas if you're working as a senior engineer,
you may only be working on one part and you can actually move between companies or between teams more easily because you,
the specific technologies that you're working with are,
you know,
(10:43):
less important than like the specific problem that you're working on.
Whereas from what you're describing,
you need to understand like the broad strokes of the organization itself.
And I don't know,
it's just something that I think is,
I'm curious if that's what you found and as somebody who wants to move on to that level,
how do you cultivate that sort of knowledge?
Absolutely.
I think that's,
yeah.
No,
that's a great question.
(11:04):
I think that's,
I think you're 100% right there.
I,
I this is where,
like,
you know,
a lot of times people sort of make the comment that engineers only stick around for like 2 to 3 years on a,
at a particular job or like,
you know,
it's sort of like after a couple of years it's time to move on.
Um And I think,
like,
you know,
if you're an early career engineer,
like that's,
(11:24):
that's a pretty reasonable strategy,
you know,
you get exposed to a lot of different things,
you see things at different companies,
you understand what your preference are,
preferences are and then sort of like where you shine and,
and,
and,
you know,
that kind of helps you,
but at a certain point it's,
you know,
it is a little bit detrimental to,
to sort of jump around like that because I think what we,
(11:44):
we're kind of getting at here is you have to do really impactful work.
And in order to do really impactful work,
you need to understand a lot about your organization or your company or,
you know,
just different moving parts and it's,
it's hard to do that if you're jumping around a lot,
I think one way that you can kind of show impact that without having a ton of organizational context is having context on similar problems,
(12:10):
right?
So,
like,
if you've like,
I don't know,
you build Google search and then like Microsoft hires you to work on Bing,
there's probably a lot of knowledge that you have that,
that you can apply.
I mean,
you know,
obviously you have like IP considerations and stuff like that,
but like still like you,
there's a lot of techniques and things like that,
that,
that you can sort of bring to the table,
you know,
like,
so that,
that,
that's,
that's one way you can kind of bridge that.
(12:30):
But I think,
you know,
it's a little bit harder,
it's definitely a little bit harder.
And so,
like,
coming that,
you know,
like this is why companies are a little bit reluctant to hire people at high levels because it's just like there's so much organizational,
you know,
uh knowledge that,
that,
you know,
makes it easier to have like a ton of impact.
So,
yes,
absolutely.
I think,
(12:51):
I think building up that,
that context to,
to take on those impactful projects and become a,
a staff engineer like that's that,
that,
that's sort of a little bit of a requirement there.
And to follow up on your example,
to follow up on your example with moving from search,
from like Google search to uh to like bing or something like that,
maybe you can solve the technical problem but and it kind of maybe is jumping ahead a little bit,
(13:13):
but it seems like a lot of the skills are not technical skills,
they're organizational skills and people skills and those don't necessarily transfer over if you're really good at getting something done at Google.
But if you go over to Microsoft,
understanding the levers that you need to pull in order to get something done may not translate.
And it may be,
even though you may might have might be the best in the world of getting a search engine put together you might suffer compared to somebody who just knows who to talk to,
(13:41):
you know?
Yeah.
Yeah.
I mean,
you're not even just knowing who to talk to but,
like,
having relationships with those people.
Right.
Like,
I mean,
there's,
there's so much that I've gotten done in my career just because,
like,
hey,
like,
you know,
it's like,
hey,
this guy,
like,
you know,
it's somebody that,
you know,
you,
you've talked to,
you,
maybe you've worked on projects with uh or at least,
you know,
gone out to lunch with and,
(14:02):
and such,
like,
I,
I don't know,
there,
there is an aspect of,
of things that,
like,
you know,
we're all humans.
Like,
I,
I think,
you know,
we,
we sort of like,
you know,
we,
we live in code and like live and breathe these,
like,
you know,
sort of autonomous,
like,
you know,
automated things,
right?
But at the same time,
like,
people are people.
If you're asking somebody to sort of,
you know,
(14:22):
go along with some design or,
you know,
you're asking for feedback on some,
some tool that you built and,
you know,
the person's like,
super busy,
like,
it's not really top,
you know,
like,
it's not really the thing that they're,
like,
personally interested in at that point or,
you know,
like,
maybe they got some deadline to work on or whatever.
Right?
And,
and,
and if you have a relationship with that person and then it's a totally different conversation. Then,
(14:43):
then they're much more open to,
you know,
kind of helping you out and maybe,
maybe they take like,
you know,
10 minutes,
15 minutes,
whatever,
take a look at whatever you're,
whatever you're doing and,
and get back to you.
And that's,
that's the kind of thing that,
like,
you know,
you,
you,
you need to,
to build those relationships. And those relationships take time to build,
you know,
it's very hard to come in at a,
at a,
at a very senior level and work at,
(15:04):
at,
at that,
in that scope with that many relationships required when you haven't had time to build them.
Yeah, absolutely, and I've seen this where,
like,
you know,
uh somebody came in,
you know,
and it's like,
you know,
top dog at this other firm and like,
you know,
they come in,
they have some idea and they're like,
"Oh,
we should do this,
this and this," and people are like,
"No."
you know,
like,
like,
like,
you know,
(15:25):
like who are you,
like,
you know,
like who is this person dictating?
Like how we should do things like,
you know,
like we're not doing this.
I do want,
I want to really come back to that exact subject in a minute because it's about like how you have influence as it engineer versus as an executive.
But uh but one thing I want to do first is I,
I feel like we talk a lot about the differences between staff is like when you're growing to a senior level,
(15:47):
it's very much about scaling yourself scaling your abilities.
But when you're getting to that staff level,
it's,
it's largely about scaling through others,
right?
These relationships that we've been talking about. Using,
using a large network of a lot of different people to get something very complex done.
And in some ways it seems like the job,
uh,
of being an effective staff engineer is,
(16:10):
is fundamentally different from what the job is of being a successful junior or senior uh senior engineer.
And I,
I wonder sometimes like are the skill sets that you build as you work from junior to senior level,
all of those skills that you work on and develop?
Are those still valid and useful at the staff level?
Or do you have to learn a fundamentally different group of skills in order to operate effectively at that next level?
(16:32):
I mean,
I,
I,
I would say they're still necessary,
you know,
you still have to be able to write code,
you still have to be able to find bugs and things like that.
I just think that like the job becomes a lot more than that,
right.
So it's a whole different set of skills that you need on top of that and,
and you know,
you're gonna be writing code,
you know,
if your staff engineer,
like you are expected to write code in most places,
(16:53):
but the code you're gonna be writing is probably more technically demanding than what other people are doing.
Or maybe,
maybe it's relatively simple stuff.
But at the same time,
like,
you don't have a lot of time,
like,
your,
your time is being spent on so many different things.
And if you want to put out this diff,
you know,
it better be something that you can do relatively quickly.
(17:13):
Uh And that's where those,
those skills come in handy.
Is that like you've done this so much,
you have like a certain mastery over,
over the process of,
of architecting code testing it, going through code reviews,
you know,
all that stuff that you can then spend less time on that and it frees up more time for you to,
you know,
scope out some important idea or get feedback for on something or,
(17:35):
you know,
like just different things that you'll,
you'll need to do that are that are outside of just writing code.
Yeah,
it seems like uh these days,
most of the code that I write tends to be,
as you mentioned on something that's pretty complex that it's not just straight out of the box like,
oh,
I'm gonna,
I'm gonna just make this React component or this API call or um you know,
build this simple server,
(17:55):
right?
It's like you,
you've,
you've,
you've got a product that's running and you've discovered now that you've scaled beyond your,
your query or your rate limit,
that kind of thing and you need to sort of rearchitect how you batch things or,
you know,
pre aggregate data,
that kind of stuff and it takes time to sit and think and explore a couple of ideas.
And by the time you really get around to writing code,
it may not be that much code,
but it's,
it's the right code and,
(18:17):
and it's,
it solves a very complex problem.
I've noticed that definitely like my lines of code,
like go down when I'm working on problems like that,
even though it's,
it may be just as impactful or probably more impactful as the code I was writing before when it was just like you've got a task, pull down
the task, there's a very well defined path in order to accomplish it,
get it done,
put it up for review,
(18:37):
land it move on.
Yeah,
absolutely.
I mean,
I think,
I think some of the most impactful work I was talking to one of the,
one of the other engineers at,
at HRT about this.
Uh you know,
the idea of like a request for comment,
this is something that some companies are really,
really good at, some companies take to the extreme, and then some companies don't really do much of. I think like a lot of times like the value like the most impactful thing you can do is just create an RFC,
(19:01):
right?
Like it's not even the code,
like none of the code is actually all that complicated or,
or important,
but it's the,
it's the act of putting together all the thoughts,
you know,
collecting feedback from people and coming up with a good design and building that document.
That's the,
that's the hard part,
right?
And that's the part that's,
like,
really impactful and valuable and then,
and then anyone can just sort of take that and,
(19:22):
and,
you know,
sort of run with it.
Yeah.
So a lot of what I've been working on lately has been building out a fundamental source of truth query layer that brings together data from a lot of different systems that have very few (if any) standards between them.
And uh in order to build that out,
you know,
I need to fundamentally understand the the needs of my consumers,
(19:43):
right?
The people who need to query this,
which is,
you know,
the vuln management, privacy, security people versus uh the people who are actually the SMEs in these different systems that I'm trying to aggregate data from.
I can't be a subject matter expert on every possible system that that Meta has running,
right?
So,
so much of it is,
is like you said,
these RFCs where I'm gonna write down,
what I think is probably a good architecture of how I'm gonna get data out of one system.
(20:06):
And what I think is the important data to get out, how I'm gonna normalize it so that the rest of the world can consume it.
And then I send it out to the people who really know and they say,
well,
yeah,
you know,
uh you,
what you think is a unique identifier in this system actually isn't because of X Y Z.
You need to think about this other thing,
right?
And that's the only way to actually come to the right solution is by engaging all of these different folks who really know how different systems work.
(20:30):
And that does become a much larger part of my job now is is just getting that alignment.
Yeah,
absolute alignment.
I love,
I love that.
You said that word.
Like I think that's,
that's such a like key part that people often miss.
I mean,
I don't know how it is with you,
Brian,
but like,
I,
I definitely feel like there are,
there are points in,
in,
in my past where I work some things that I thought were important,
(20:51):
but management didn't sort of like see them as important, and like that was such a common pitfall.
And,
and I think that's like a key differentiator of staff engineers is that issue of alignment.
I think we've all experienced that from time to time.
Yeah,
I like or I think something is really important because it feels that way viscerally to me, and I work on a solution and people are like "Uh, great,
nice."
Uh And then when it comes time for performance review,
(21:13):
I'm trying to talk about the impact of what all the work I did and I look at that thing and I go,
"You know,
like I enjoyed working on that,
but I don't think it had the broad impact that I imagined it would and I probably should have worked to align what I was gonna work on with my,
not just my own manager but his skip, my director,
(21:33):
like making sure that what I was working on was what they feel is impactful and,
and then,
you know,
putting metrics in place beforehand to verify whether or not it's having the impact
I thought it would."
Yeah,
I think that's,
I think it's so important and,
and it's definitely something that I've missed in the past and I,
I,
I think it's great to have ideas and,
and be able to like,
proactively find things and that's part of the staff being a staff engineer,
(21:55):
but you can't just like,
work on things that,
that you think are,
are worthwhile.
It's really like,
what is management value and,
and it is,
you know,
sometimes that's not gonna be the same thing and,
and you might think like,
oh,
these guys are clowns like management,
you know,
full of idiots or whatever.
Uh And oftentimes,
you know,
that there is some truth to that,
but like,
a lot of times there,
you know,
that's not the case.
(22:16):
It's just that like,
and has a different perspective of,
you know,
all the different projects that are going on different incentives and maybe like the thing that you're interested in is actually,
is actually important and impactful.
It's just not the most important and impactful thing in that.
Org at that point in time,
it might be something that right now I,
I,
I kind of want,
you know,
people to work on this X Y Z other thing and then we'll come back to your thing and maybe,
(22:40):
you know,
six months or a year or whatever.
And that's,
that's something that like everyone should be kind of like,
just be humble about it and kind of like,
take the,
take that perspective.
Like you can,
you can come up with ideas,
you can find evidence and I'm,
I'm a big fan of prototyping and finding data and things like that.
Like I think these are things that don't take that much time,
you know,
spend,
you know,
a couple of hours like prototyping something,
(23:01):
go show people and go talk to your manager,
go talk to,
to their manager to your point,
Brian,
like,
you know,
couple levels or so like,
you know,
and see what people think and,
and maybe they,
maybe they just say,
"Yeah,
this is great,
Brian,
like,
go ahead,
you know,
like we want you to do this right now."
But they might also say like, "This is cool,
but,
you know,
we'll probably look at this at some point in the future," and,
(23:23):
and that,
that's not a knock on you like,
that's not a knock on your idea.
It's just that,
you know,
there's all these other important things to work on and ultimately it's a business.
Right.
It's not,
you know,
we're not doing this because we just like,
I mean,
hopefully you like what you're doing and I,
I definitely like what I'm doing,
but at the same time,
like,
you know,
the point of a company is to,
to,
to then,
you know,
(23:43):
deliver value to,
to customers.
And so you can't lose sight of that fact.
Uh As,
as we're,
as we're sort of picking projects and,
and working on things.
Yeah.
Um I've got a project myself that you mentioned like an Infra Management project that,
that I am very passionate about,
but it's been on ice for a year while I've worked on some other things that are,
are higher priority for the org.
And um yeah,
(24:04):
I mean,
it,
that's just kind of life,
right?
You're,
you're hired by a company to,
to increase their revenue or lower their bottom line or whatever it is that,
you know,
your company needs you to do, and alignment with those company priorities is what makes an effective staff engineer,
Alex,
I saw you mentioned that you had a question here that I really want to get into around um around how is being a staff engineer fundamentally different from being a manager at a similar level because there's so much overlap between when it comes to coordinating between different pieces.
(24:33):
Um And a,
a good manager certainly does that.
But what,
what are the differences between a good staff engineer and a good manager?
Yeah,
I mean,
I think that there's definitely a lot of overlap there,
right?
Like,
I think part of being a staff engineer is,
is having the ability to lead people and,
and set direction and kind of like guide projects and part of being a manager is also,
(24:57):
you know,
sort of leading people guiding projects,
setting direction.
I think,
I think where,
where it's kind of different is that,
you know,
as a,
as a staff engineer,
as an IC,
you don't have reports,
so you're expected to kind of lead people but you don't have like there's no like set authority there,
right?
Like when you're in a management role,
like the people who report to you,
(25:19):
you know,
there's a little bit of sort of like a power differential there,
right?
In,
in,
in that like you control probably their,
their compensation and,
and certainly their,
their growth and at the company and things like that,
whereas,
you know,
as a,
as an IC staff engineer,
that's not going to be the case.
So how do you then lead people?
Right?
If you know,
it's a little bit different skill set even than,
(25:40):
than,
than as a manager in that sense.
So there's overlap,
but it's,
it's also a little bit different,
like I think the way you lead people as a staff engineer is to kind of convince people that your ideas are the best ideas.
And that's where,
you know,
building relationships with people helps.
That's where,
like,
just,
you know,
being good at communication.
I think this is,
you know,
something that helped me a lot was I took some like classes around influential communication and kind of,
(26:06):
you know,
listening to people and like making sure that they feel heard and,
and building consensus.
These are like,
these are skills again,
they're not,
they're not coding and they're kind of not the,
the sort of thing.
Engineers typically like,
focus on to be honest,
but it is,
it is the thing that's like critical for,
for like,
you know,
operating at that level.
If you uh if you have any of those classes that you took that you want to recommend to listeners,
(26:29):
uh We'll definitely like,
we could put links in the show notes.
So I'd love to hear about those.
Yeah,
I'll,
I'll,
I'll try to uh dig up some URLs and,
and send them over.
Cool.
That actually dovetails with another question that we had,
which is,
are there any unexpectedly useful skills or experience that you've acquired anything that,
you know,
you wouldn't get from reading a book on being a staff engineer?
That might be surprising.
Um I think being a good listener:
(26:51):
the ability to listen to people and like,
understand what they're really asking or what they're really concerned about. And it's not like a lot of,
like,
empathizing with people and just kind of like seeing where they're at and where they're coming from.
You know,
this is the kind of thing that,
like,
a lot of people do.
I mean,
some people do this for a living,
right?
Like you're therapist.
Like,
they,
they do do this for a living.
Um,
but,
like,
for me personally,
(27:12):
I don't think that was like,
something that I,
I had built up but like to,
to have this,
this sort of like,
impactful projects and,
and,
and to drive,
you know,
change across the organization or across teams or things like that,
you kind of have to spend a lot of time just listening to people and that's,
that's something that,
like,
I think we could all do better at and,
you know,
there is,
there is a certain skill to,
(27:33):
like,
when somebody disagrees with you,
you know,
and you,
you talk to them and then,
and then they,
like,
walk away feeling like,
ok,
you know,
like this person,
you know,
sort of heard what I had to say and,
and,
and I don't think like,
this person's a jerk or something like that.
Right?
Like that's,
you know,
and maybe,
maybe you're still gonna,
like,
ultimately not gonna do what they would like,
but they don't get upset about it.
Right.
And that,
(27:53):
that's,
that's the kind of like skill that it's pretty useful and it's like it was something that,
that I,
I found kind of unexpected.
Useful at all levels. That's always gonna be helpful throughout your life.
I think you say life,
right?
Like you're talking to your,
to your partner or whatever,
right?
How to be an effective staff spouse.
(28:14):
Uh,
I,
I think my wife would define me as a senior spouse.
Not a staff spouse.
That's true.
No,
I,
I definitely have a director at home.
Um,
well,
we are,
we are getting towards,
um,
towards time.
This is a really terrific conversation and I,
I'm having a great time with this.
Um,
but I wanna give us a chance to kind of talk through,
um,
if you have any,
any like,
uh,
general recommendations,
(28:35):
things you've been reading lately,
something that's just helping you that,
that is,
um,
you know,
just something that hasn't been part of the main conversation.
Anything you want to throw out there for listeners who might want to check it out.
If you want a minute to think I'll jump in and start.
Um,
so we started a book club for my Org a few,
uh,
gosh,
about six weeks ago now and we originally intended it for juniors to learn.
(28:56):
But as it turns out a lot more seniors showed up,
uh,
which is kind of,
I think,
telling about,
uh,
where they're at in terms of their kind of interest in,
in growing themselves and learning.
But we've,
uh,
we've just been reading together And so what I wanna pick here is community learning.
So we've each week been taking a chapter of "The Pragmatic Programmer,"
reading it on our own, and then one member of the group will sort of present the chapter when we come together.
(29:17):
And,
um it's been really great.
Uh I've read the book,
you know,
I don't know how many times at this point over my career and it's it,
but hearing other people and getting a chance to discuss it with them is,
is having me look at things in a different way, and I just love this.
So if you have a chance to uh get into a group and sort of go through something and learn it together,
uh I think it is far more effective than when you do this on your own.
By the way,
I have to shout out to uh Charles Max Wood from JavaScript Jabber because reading "The Pragmatic Programmer" in a book club was totally his idea which I stole.
(29:45):
Um But uh All right,
uh Alex,
you got,
uh you got anything you wanna wanna share about?
I'm rereading uh "Refactoring" by Martin Fowler.
That's always,
that's always a good book.
It's always got something new to add every single time I read it and it always gives me a lot of useful insight.
And then I'm also reading um "Functional Programming Simplified" by Alvin Alexander.
(30:05):
It's a Scala book, and I've got it with me.
It's got the coolest cover because it's got a Lambda, and it's attacking an entire crowd of people because functional programming can be scary,
but not when you read this book.
Um Yeah,
it's a good book.
It's a good intro.
I,
I assume all those people are Java 00P developers,
they're all Java 00P developers.
They wouldn't know a functor if it,
(30:26):
you know,
came up and gave a big wet kiss.
Yeah.
Um All right.
So,
yeah,
I mean,
uh I feel like,
well, I don't know how to follow that.
That was weird.
Um Functional programming is great.
I,
I,
I actually TA'ed for a functional programming class back in college.
(30:46):
It's fun stuff.
One thing I like actually topical that I,
that I thought was interesting is uh there's a book called "The Staff Engineers Path".
I don't know if you guys have read it,
but uh that's something I would recommend it.
Um Yeah,
it's just,
it's sort of a collection of anecdotes of various people um who sort of like,
you know,
chronicled,
like some of the things they found useful or influential or,
(31:07):
you know,
kind of their journey.
Um And I,
I,
I would,
I would recommend taking a look at that book.
Yeah,
I think I,
I think I read it a couple of years ago and it was,
it was great. That's awesome. I'll check it out.
I'll put a link in the show notes as well.
Otherwise,
you know,
I spend a lot of time just like reading white papers of different things or documentation and,
you know,
trying to understand this framework versus that framework and like,
(31:27):
where,
you know,
what we should be using and,
you know,
what are the pros and cons of like using whatever the latest shiniest thing is versus like something tried and true like Tornado or whatever,
right?
Like,
I,
I don't know,
like that's uh that,
that's,
that's,
you know,
fortunately or unfortunately,
there's a lot of my time is just like figuring out like what,
what technology is out there. There's just so many different things.
(31:48):
Yeah,
I love getting a chance to explore new languages that I don't usually work in because,
um,
sometimes like the,
the conventions that they set up in that language are usually because they were trying to solve some very specific problem that other languages failed these creators in and,
um,
and learning about that process,
whether I ever end up using that language or not in production,
it's kind of nice to just sort of learn why they felt it was necessary to create it.
(32:12):
And,
and maybe I learned some paradigms that I can bring back into my daily work.
Where do you find these white papers?
Oh,
just go to different projects.
I mean,
you know,
like whatever it is you're looking at,
like,
let's say you're looking at uh we were talking about logging libraries earlier,
like,
I don't know,
you could look at things that are out there like Elasticsearch or,
you know,
um,
or,
or,
I don't know,
just different logging,
(32:33):
you know,
and,
and metrics and collection tools,
you know,
see how they're implemented.
Like,
I,
I think I remember like,
some,
a couple of years ago,
uh,
I,
I was looking into something that's kind of,
I feel like it's dead now,
like these,
uh,
noSQL databases and I was trying to understand like,
uh,
I,
I don't know,
maybe,
maybe "dead" is the wrong word,
but it's definitely like,
it was hot for a while and then now it's probably like.. They're dead to me.
(32:55):
I'll say that much.
Um So I was trying to understand it,
you know,
like I read like a bunch of white papers uh that,
you know,
Cassandra put out or MangoDB and,
you know,
like,
understand like why they're doing things a certain way or like,
you know,
what technical challenges they're trying to
solve or,
um you know,
like that,
that,
that's the sort of thing I'm talking about.
Cool.
Yeah.
If you're,
if you're not seeing the video,
(33:16):
you missed an epic eye roll from Alex,
which is really unfortunate because uh uh this is,
this is a conversation we've had a bunch in the past around.
Um You know,
I'm just like,
ah,
I don't like document databases.
They're not my thing.
Like ah just give me a SQL table.
I'm super happy.
Uh And,
and uh you know,
and the argument is like,
oh,
it has a scale.
I'm like,
well,
Facebook is pretty big.
(33:36):
We're based in SQL.
So.
But,
yeah,
it's funny.
I just think,
like,
I,
I,
you know,
like IBM has been making SQL databases for,
like,
50 years and they're used in,
like,
all these different banks and stuff like that doing,
like,
you know,
millions of transactions a day.
Right.
Like,
it's,
it's possible to do these things that,
uh yeah,
it can scale.
(33:56):
Yeah,
like that.
Yeah, it's a very boring thing,
right,
like build a SQL database that doesn't get you venture funding.
That's right.
Everybody's, no, everybody does SQL.
But have you heard about like anti SQL?
You have to use the opposite schema.
It's great.
It scales inversely.
It's wonderful.
It,
yeah,
the the,
(34:16):
the the paradigm there is no normalization whatsoever.
Um So yeah,
listen,
I think we've ragged on MangoDB enough and,
and uh and in all honesty,
like,
you know,
every technology that's out there,
it,
it exists because it solves a particular problem,
right?
And so,
you know,
again,
like not to,
not to belabor my own preferences,
right?
That's all they are.
They're my preferences.
(34:37):
Uh So every technology has,
has worth and has validity.
It's just a matter of like how you want to solve your problems.
Oh,
I,
I should mention that my views are,
you know,
the views expressed here are my own and not those of my company, Hudson River Trading. I thought I'd say that.
Thank you.
For being the adult in the room and reminding us that that is always true:
(34:57):
We are here as individuals.
So,
yeah,
we're,
we're right at time here.
Um So I,
I think we should just go ahead and call it here,
but, Abhishek, I wanna say thank you so much for your time.
This is really terrific uh explaining your perspective on,
on what makes a staff engineer at all.
So again,
thanks for coming.
Uh You're welcome back any time.
Um That was awesome.
Absolutely.
All right.
Thanks a lot,
Brian and Alex.
Uh You know,
(35:17):
I have a pleasure coming on here.
This is,
this is a great time. And uh for everyone listening,
we will catch you next time.
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy And Charlamagne Tha God!
The Joe Rogan Experience
The official podcast of comedian Joe Rogan.