How not to get stuck when learning web security? Louis Nyffenegger from PentesterLab

In this interview, I'm talking with Louis Nyffenegger who's been teaching people websecurity since 13 years by creating Pentesterlab - web security learning platform, as well as by giving multiple talks and guiding people through their careers.

Episode Transcript

(00:00):
Hello, in this episode I have a guest. I will interview Louis from PentesterLab

(00:05):
and he has an incredible experience because as a creator of PentesterLab he has insight into
what problems do people face, what challenges are easy, what are difficult, where people get stuck
and we all, me and you, we can use this knowledge to basically speed up our learning journey.
So hello Louis, thank you so much for joining me today and can you please tell people who don't

(00:28):
know you maybe what is your background and how did you get to the point you're currently at?
Okay, so my name is Louis, I'm the founder CEO of PentesterLab. So I started by going to
university and doing an engineering degree in computer architecture, then at the same time I
did a master in security back in France, then I was lucky enough to get an internship doing some

(00:52):
Windows kernel programming. After that I got a job starting as a security consultant, moved to
Australia, became like a Pentester and then Code Review and then AppSec and at the same time I
started doing like training at conferences and I had all this amazing content that I created for the

(01:13):
training and I was only using them like once or twice a year so I decided to launch PentesterLab
and the rest is history I guess.
The rest is history, even for me PentesterLab was one of the first resources I used for learning
and I must admit I haven't logged in there for a while but my profile, I believe at least from

(01:34):
the early days I have a lot of the flags so definitely thank you for this. These days,
do you even have the time to hack for fun or do you mostly spend time on PentesterLab?
So PentesterLab is still like a really small business and I'm lucky enough or I force myself to
keep it in a position where I can still be doing some hacking because that's what I'm really

(01:58):
passionate about. As much as it's cool to manage young people hacking or other people hacking,
I like to do Code Review, Pentest and things like that. I don't Pentest for clients but I
like to do like vulnerability research, find bugs and things like that.
What are your favorite bug classes?
I don't really have a favorite bug classes. I like complexity, I like details, I like things

(02:26):
where I'm like, oh, I didn't know that. That's really what interests me. So you can have boring
bug classes with amazing bugs or boring, amazing bug classes but the exploit is really, really
interesting. I like the details. I think all of us that do CTFs, do some labs, we all like the

(02:48):
edge cases where it's really, really difficult and it really shouldn't work but it does work
after all. I think it's definitely, especially it's fun when it happens in real life as well,
then it's super, super satisfying. Exactly.
How about from the creator of lab standpoints? Which bug classes are fun to create labs that

(03:08):
just spend a few hours and you have the lab ready? And on the other hand, which are super
hard where you have to spend a lot of time to do the whole setup to create the lab?
So little labs where it's like a simple bugs or just like a CV are pretty easy to create
because for CV, you just can find the Docker container with a vulnerable version. What is

(03:30):
really hard to create is labs around OAuth and SAML, like all these protocol where you have a
lot of little moving pieces and you need to have them just right, especially since I'm trying to
get labs where you only have one bug. So it's really easy to have like a lab with 20 bugs,
but only one bug and one way to exploit it, it's a lot harder.

(03:51):
How does the infrastructure work? Is it like Kubernetes under the hood or is it, how does it
work? Just some Docker and like some magic around it and yeah.
Okay. What do you do when there's like an RCE and you have to make sure that one user does not
affect other users?
Oh, it's just like lockdown or at the Docker level. Yeah.
Do you have separate containers for separate users?

(04:13):
Yeah.
Okay. Okay. I imagine this, when I think of creating labs, because also we do as a CTF team,
we also do, even this weekend, we organized onsite finals. So I do sometimes create labs
and two like challenges are how to make sure that different users do not affect each other. Of
course, you can just spin up instances for each user. And when you have an RCE, we are forced to

(04:39):
do this. So we do it, but also like you don't want to do it for a backlash if you don't need it
because it generates costs. So this is one challenge. And also the other challenge is
the client-side bugs where you need the interaction from the user. I think,
do you use headless browsers when you have a client-side?
Yes. A lot of Puppeteer and historically a bit of PhantomJS as well. And all of that is in AWS,

(05:07):
so in Lambda. So this way it's pretty isolated from everything else.
Yeah. But for me, it's still like, I usually just copy paste some code of the Chromium with
different Chromium options, which I admit, I don't understand 100%. And I'm worried if I
won't use a vulnerable Chrome version and be vulnerable this way. But from my perspective,

(05:29):
when I created the labs, the easiest ones are the ones where it's like a server-side bug that
does not affect other users, like a file read. Yeah. And that's why, for example,
you need both all the moving pieces, but also you need the client-side, like the fake client
visiting everything and with the right redirect and everything like that. So

(05:51):
it's challenging, but it's good fun. Yeah. Yeah, for sure. How do you, these days,
after you've modeled so many bugs, so many labs, how do you still find ideas for new labs?
Research. It's really easy to create a lot of labs. What is really hard is to create the right
amount of labs and keep people learning. And that's a big part of what I do is curation,

(06:18):
like looking at CVs, probably like 50 to 100 CVs a week, looking at, I have a bit of automation
around that, looking at bug bounty reports, looking at research, looking at all these
things and trying to feel like, okay, this is new, or this is like a variation of something,
and this variation matters. Okay. Once you have the CVE, how do you, what's the process?

(06:42):
There's no public exploit, of course. What's the process to finding out what is the bug?
So you get the vulnerable version, you get the patch version based on the CVE information,
you do a diff, and then you try to find the bug and exploit it.
Yeah. What do you do when, because some companies, when they

(07:02):
commit a fix, the comment will have like fix security bug, SSRF, whatever. But some will
intentionally name commits, like I'll have the commit message that does not say it's a
vulnerability. What are your top tips for finding out which commits are sort of hidden vulnerability
fixes? I think the main thing is to really carefully read the advisory, because in every

(07:28):
advisory, you got some tiny details that tell you like what is going wrong. And then it's just like
spending time reading source code and yeah, and trying and trying and trying.
Okay. How about, cause I know that it's the easiest when the company publishes tests for,
and the test sometimes has the exact payload. Are there more tricks similar to this one that

(07:53):
I don't know about maybe? So yeah, test cases really,
really help with that. Comments in source code, changes, looking at some function, for example,
a lot of vulnerabilities nowadays are around like people making mistakes around using substring to
compare things or they're missing a check. So they're using, they're adding an if statement

(08:14):
or they're modifying like a check somewhere. And then it's just, yeah, a lot of times.
A lot of times, yeah. Like the test cases is good, but often like you need to be lucky to like,
or you need someone to have written the test cases and yeah.
Yeah. It's, it's, I don't say it's common, but when it is, it's very satisfying.

(08:35):
And I also saw some people, there was one case, one CTF task where I spent a lot of time on
something and it was an XSS based task. And one of my friends just took a payload from
Don't Purify test cases and it works straight away. And it was like a soul, such a long payload.
And I was like, how did you come up with this? Oh, I just copied it from Don't Purify.

(08:59):
Yeah. Like good or like sometimes as well.
Well, it wasn't like it was his skill and he's, he's thinking that he came up with this idea and
I didn't, but it was, it was funny. You said you have the automation around CVEs?
Yeah.
What is this?
Oh, more like you get like a script, you give it like the Git repository, the patch version,

(09:22):
the vulnerable version, and you get like all the diff automatically and things like that. So that
way you don't have to do all that manually. Like just like five or 10 Git commands, but at least
it's done. Yeah.
Oh, okay. So then, okay. So then you still have to like see yourself.
Yeah.
Yeah.
Okay. I stopped you like automatically find out which CVEs are worth.

(09:44):
I wish I had that. So I tried to get that output from the diff to a chat GPT, for example,
it's terrible. Like giving it like the advisory and the patch, chat GPT is just like, Oh,
creating like weird finance and things like that and making it, making up things. So.
It doesn't really work.
Not yet.
No. Do you have any other automations or tools that helped you in the process?

(10:08):
I got a bit of like rules around same grabs and things like that to like look at code basis. But
for that part, not yet. I need to spend more time. It's always a balance between like
more automation, less automation and yeah.
Yeah. And then when you create automation, you have to invest more time to maintain the
automation.
Exactly.
That's, that's one of the reasons why I don't maintain my end automations and I'm just,

(10:30):
I decided I'm a manual hacker and I live with, cause yeah, I had some tools in the past and it
was consuming more time than actually saving.
Yeah.
How about authentication related bugs? Cause we will jump into, into particular bug cases
because I know you have a lot of labs on JWT and SAML. And the question is what are the attacks

(10:56):
which where you have to use custom tooling? Cause we have the tools in burp, we have extensions,
we have some other tools, but are there some, some attacks that require us to set up some,
something on our own?
For SAML, SAML Raider is pretty good, but some exploitation requires you like are not supported
by SAML Raider. Basically like as a signature wrapping attacks, not all of them are in SAML

(11:20):
Raider. So sometime you need to like write your own script. It's not that complicated. It's a bit
of XML parsing and rearranging information for JWT. I haven't looked at JWT extension because I
write my own tool and that's why, yeah, that's why I'm trying.
That's what I'm trying to teach people as well is it's good to rely on tool when you want to be

(11:44):
fast, when you want to be accurate as well. But most of the time, what you want to do,
especially when you're learning is learning to write your own tool. It may look like a waste of
time at the start and it may be, but at one point you're going to end up like, okay, this protocol,
these things, there is no tool for it. And if you don't know how to write your own tool,

(12:07):
you're just going to be stuck forever. And you're going to have to write, to wait for someone to
write the tool for you, which isn't great. And for example, there's a tool, JWT tool, I think,
that a lot of people use to create, to attack JWT. It was written by someone who use, I don't
know if it's still like a Pentest or a customer and all the tools were based on the labs and yeah.

(12:31):
And I think that's something I'm really proud of.
Yeah. Nice. Yeah. Well, actually I remember I have my own repo with Pentest or AppSource
somewhere. It's private, obviously. And I remember there were like different Python
scripts for like JWT, something. I remember it was a two digit number. So, so actually you can
just probably compile all this to one good tool, add some, some more and it's probably.

(12:54):
Exactly. And the learning you have from that, like being able to automate, like sending requests,
doing like a bit of crypto, like HMAC signature and things like that, that's going to always,
always be valuable. Like once you hit that level where like, oh, there is no tool to do that.
I need to write my own.
Yeah. Which, which is your language of choice these days, if you have to write a tool?

(13:14):
Oh, Ruby. Always, always has been, always will be like, everyone is different. And I think I,
I made a video on that actually. I'm not recommending a language to a person. I'm
telling people like, okay, try a few and see how it goes. My brain with Ruby just work. Magic.
Other people, I don't know why, Python work for them. You have to find the one that work for you.

(13:40):
And once you got that one, like go like really hard on it and get really, really good at it.
And for me, I spent too much time on Ruby, so I can't go back.
I don't use it a lot, but it was one of the first languages where I learned to create a web
application because I had it at the university. And I really liked it back then to have like
different controllers because it was generate the whole application or the files. I'm talking

(14:05):
about race, obviously. And this year I think I created the CTF challenge in Go, but I think
last year I used, I use Ruby. I use race for it because I just liked it. But then there are so
many files. If you generate the app with Rails, I'm like, and I didn't touch most of them.
Then I'm a bit afraid of what is in my application. Yeah. It's as a trade-off, you get a lot for free,

(14:32):
but you get a lot of for free. So you don't never necessarily know what the attack surface is,
but yeah, right. It's really good for productivity. Yeah. Well, there's some hacking tools
for Rails because hacking, sorry, hacking tips for race. I mean, because one of, one of the
things I know it does automatically at developers may not know about this is it adds the dot Jason

(14:54):
route implicitly. What other tricks you, you commonly use with, with phrase.
So a lot of people forget to put authentication properly. So authorization properly. So I think
it's always worth, like when you look at a Rails application to compare all the authorization
check, because often people will put it for like the show for the index, but they will forget to

(15:18):
put it in place for update or edit or delete. Mass assignment is a good example as well, where you can
add parameter to the HTTP request. And this parameter may be mapped to the object,
to the underlying object. So you use, for example, you add admin equals one in the URL
or in the post request, in the body of the request, and then you'll become admin when

(15:39):
you create an account. Yeah. A lot of things, but what else Ruby, no race removed XML processing by
default a while back. Yeah. Mostly around authorization. Yeah. I think authentication
is most of the time handled by device, which is pretty good. Yeah. There was a bug in device

(16:02):
last year in James Cuddle research. Am I correct? Yes. Yeah. Was it really exploitable in the wild?
I think it was. Yeah. Okay. Nice. Yeah. I haven't, you need the right timing and everything, but yeah.
Yeah. It's cool. Have you played with this year's research from guys from Portswiger?
Not yet. Not yet. I need to like, look if like right now I'm like deep diving into source code

(16:26):
review and like creating like a big, I created like already a big training, but trying to improve
it, improve it, improve it. So I don't do as much hacking. I do a lot of code review.
Okay. Well, there's some, you're like best code review tips that, that you give to people.
Read the source code. People just grep, grep, grep. And then they say like, oh, I suck at code

(16:50):
review. I'm like, are you actually reading code? And most of the time people are like, ah, actually
no, I'm just greping and looking at result. And that's not what code review is about.
Are you a sources to sync or sync to sources type of reviewer?
I mostly use like different approach, not necessarily like, like that source to sync
or sync. I'm trying, I like to look at sync first, but I'm more like around, like looking

(17:16):
at differences, like comparing like two code bases, doing the same thing, looking at one
functionality in depth and things like that. Yeah. Okay. Yeah. I know there's no one
definitive answer to this question. I wouldn't give a definitive answer myself, but I still ask
it to, to, to see because people, people often expect there's just one good way to do something.

(17:38):
No, like programming finds a way that works for you.
Yeah. What do you say to people that I'm, that ask another very common question is,
do you need to be able to code to be a hacker? That's a very good question. I think you don't
have to, but oh my God, you're missing something.
Yeah, for sure.
Like it's amazing. You write something and bang, you ran it. And it does, the computer does

(18:02):
whatever you tell it to do. It's amazing. Like you don't need it to be a hacker, but you're
missing on something amazing. It's like, yeah, it's like you've got this amazing part of computer
science and you're not doing it. And it's like, you don't need to be the best programmer in the
world, but like, yeah, you need to code just because it's amazing feeling.
It's also like, there are some things that you see as a, when you, when you just use the

(18:26):
code as a, as a developer that I think would be hard to pick up just with black box approach all
the time. I'm actually planning to just create my own like SAML and SSO servers to like control
absolutely everything and be able to do some attacks that I couldn't do with like existing
IDPs. So, so I'm planning to do it, but because I know also I will probably find out some things,

(18:52):
some ideas for attacks that I, I don't know about them yet. So I definitely want to do it.
That's why you need to read code as well, instead of graphing, because you want to find those
unknown unknowns, things you wouldn't think of. And that's where the bugs are. Like if you're
doing hard targets or like bug bounty on things that a lot of people have tested,

(19:15):
sure, everyone runs like run SAML radar, like three, three times and things like that.
So you need to come up with your own thing. Right.
Yeah. If you, if you were supposed, if you are supposed to create a new web application from
scratch, do you use a lot of AI for, obviously for some reproducing, some vulnerability,
do you use a lot of AI to, to, to support you writing the code?

(19:37):
A little bit for like the boiler plating part, like getting the things to work together.
But most of the time I need to write, like the interesting part, I write myself because yeah.
Okay. So, so how do you use it to boilerplate?
Mostly like to create, like, Oh, I need like a simple web application in Python
and give me like the structure and I got everything working. Or if I come across

(20:00):
an error message, like. So you can set GPT for it or something else?
Have you tried Courser?
Not yet.
I haven't, I haven't either, but it sounds really good.
Yeah. Like, yeah.
This co-pilot, but I'm the same. I just ask chat GPT. Although my conversations are quite
long with chat GPT because then I ask it to do something more and the changes. And

(20:21):
it's actually not, not bad, especially for, there is some, some point where it's,
it starts to get so complex. It does hallucinate a bit, or maybe not hallucinate,
but just make mistakes. But for starting, it's definitely, definitely nice.
Yeah. It definitely hallucinate on CV. I can tell you that like creating CV numbers
out of thin air. I just, Oh, I'm looking for a bug in VAT with VAT. Do you remember?

(20:47):
Like there is this bug that I read about, like, I don't know, 10 years ago and I can't find it.
And I asked chat GPT to try to find it for me and couldn't find it. I mean,
chat GPT made up like 10 different CVs and all of them were wrong.
Oh, that's interesting. I would think this is the perfect use case for it.
Exactly. But no.

(21:08):
Well, I think for me as a manual hacker, it's not bad. I mean, it's inevitable that AI will,
will be used for hacking and probably hacking agents or hacking assistant or whatever will,
will, will happen at some point, but I'm not complaining if it's a bit slow.
That's why you need more code review.
Yeah. What, what, what do you think are bad classes that are so complex that even the

(21:32):
hacking agents of the future will not be able to find the bugs?
That's a good question. I think you need to look at what automated scanner don't find today.
Yeah.
Like all the crypto attacks, all the interactions between components, all of those things where

(21:54):
you ever need to know like small, really small details that you can only have from code review
or things where you need to aggregate a lot of different things, like chaining vulnerabilities,
for example, would be another one where the AI may struggle a bit more than human, but yeah.
Yeah. I don't think we will lack work anytime soon.

(22:14):
No.
From learning or from observing different people learning,
what are the most common problems that, that people face?
The second one first. The second one is people don't read error messages.
People don't read error messages. Trust me on that. They just see an error and think like it

(22:35):
doesn't work. Okay. Or people assume like where, when there is an error, it mean it doesn't,
it didn't work. Let's say you exploiting like a peak old serialization bug. At one point,
the application will crash because you sent an object that is not the right object.
Yeah.
But the code execution happens before that. And people see the error message and say like, okay,

(22:56):
that didn't work. The first one is a lot of people don't know what a public IP address is.
And that's because a lot of environment, training environment are using like a jump host
where you're in the same like private network as a target. But when you're attacking like real

(23:17):
target on the internet, you need like an IP address or you need to use ngrok and things
like that. And a lot of people email support or email me saying, oh, that doesn't work,
but they're using like a private IP address for the connect back and things like that.
I remember in the early days, you had some labs as virtual machines.
Yeah.
And I remember also struggling. It was like, I didn't even know networking back then.

(23:41):
Struggling to know on which IP address I can access the web app that's inside the VM and I
can access outside the VM. I remember having problems with this and it's a hard topic for
general, but.
But I think, yeah, the problem is that people learn one way and then they don't realize that
that's not what they should be using when they go on a hard time, like on a real target on

(24:05):
the internet. And I'm pretty sure a lot of people are missing bugs because they're just like, oh,
I don't have a connect back. But yeah, because you put like your private IP address,
so it's never going to happen.
Yeah.
Where do you think people can learn these things? Because they are not directly related
to any vulnerability. Where do you think someone can learn about this stuff?

(24:25):
So that's very interesting. Back in the day, we were learning that because that's the only way we
could like install like a mail server, an HTTP server at home or a DNS server. But nowadays,
like everything is services online, so you don't need that knowledge as much.
I think you need to learn as you go and yeah, just go and learn about public and private IP
address. It's going to take like an hour, but you may find more bugs or not miss bugs at least.

(24:51):
Yeah. This is the hard part because when you have a problem, you know that you need to look
for the solution for the problem. But in the case you gave the example with somebody not getting
the connect back from the server, they may not be aware they have the problem and they may not
find out until a few years later. So this is the hard part, no?
But it's a bit easier with a Burp collaborator now, so people don't have that problem as much.

(25:19):
You kind of mentioned it a little bit, but back in the day, there were way fewer resources,
but also I guess the entry barrier was lower. Now I think there are so many resources,
it might be problematic to know which ones are the good ones.
So how do you think people have to navigate this?

(25:41):
It depends where you're at in your journey as well. It's easier and easier to find the good
resources once you know what you're looking for. At the beginning, I think you need to find
somewhere where you have good content and curation, so you don't end up doing the same
thing again and again and again. That's one thing that is hard with creating content for Pentestal
is not repeating myself, same lab again and again with 50 variations of the same thing,

(26:05):
but also trying to make it harder and harder, but not big steps, just small steps, smaller steps,
small steps all the time. I think the important things when you're learning, and I did a keynote
on that, is you don't want... So what people do usually, they say, okay, I'm going to be a

(26:26):
beginner at Linux, beginner at Windows, beginner at web, beginner at reversing.
Then once they got this basic knowledge, they move to medium, like intermediate,
intermediate web, intermediate Linux, intermediate Windows, but too hard to do all those things.
What you need to do is pick one, get really good at Windows, or get really good at Linux,

(26:50):
or get really good at reversing, whatever. And early on, get something, pick something,
and get really, really good at that thing. And in that thing, take something that is even smaller
and get even better at that thing, like Tomcat, or MySQL, or whatever, and get really, really good
at that thing. Because the lesson you're going to learn, the hard lesson, when you learn hard things,

(27:13):
then you can reproduce that on all the other things. And because that last part, getting
really, really good at something, that's when you do your own research, you can present the research,
and you are not using someone else's tool, you are not using or following someone else's research,
you are doing your own. And that's probably the hardest part. And once you know how to do that,

(27:35):
it's really easy to replicate that to other technologies. Yeah, that's really, really smart.
Yeah, because it's impossible at some point to learn everything. And
also, when you get to a certain level, once you're a beginner, there are so many resources everywhere.
But once you get intermediate or advanced, it gets harder and harder. So how to still

(28:00):
keep learning new things if we are more advanced at something? I think you need to look at
things that are around the task you want to achieve that will help you go deeper or be faster.
Let's say you want to look at SAML, you want to be really good at SAML, you need to get better

(28:20):
at code review, you need to get better at debugging, you need to get better at scripting,
you need to get better at crypto, you need to get better at XML signature, all those things
that are around SAML and will make you will enable you to find those SAML bugs.
Okay. So it's more like solving problems in time where you try to reproduce some bug and

(28:44):
you learn something else? Or would you like consciously sit and think what things around
the SAML I have to learn and only then learn these things and then learn the one in the middle?
Oh, it's more like looking at other tasks around SAML, like trying to see like,
what do I need to know to find things that maybe other people missed, or to be more efficient,

(29:09):
more accurate, or do new research on that subject. And often that's, okay, I need to debug
and like look at a debugger for hours, looking at exactly what SAML is doing,
looking at the code review for like looking at SAML source code and go like through like 10
implementations of it, and things like that. And all of that won't, are tasks that are not

(29:32):
necessarily linked to the SAML, but they will help you like go to the next level.
Yeah. So this is at this point, it's mostly just learning with the code that exists and with
our experience, and it's not no longer looking for resources online.
No, it's, that's often when people get stuck. And that's often when people move to something else,

(29:54):
because it's hard. It's not comfortable. It's, you're wasting your time because you're like,
okay, or you feel like you're wasting your time because sometimes you're going to look at
something and it's, you're not going to yield anything. You're just not going to find anything,
but you need to go through that journey to get better at one point. Otherwise you just get stuck

(30:14):
and move to like, okay, I'm going to buy another certification or I'm going to buy like a new
training. But at one point you need to start suffering on your own, I guess.
Yeah. How should people know which is the moment to learn more advanced topics from the area we're
already quite good at and where is the time to learn something new?

(30:35):
I think when you start to find that all the training you can find on one topic you're
interested in are things you already know, or like small variations of something you already
know, or the research you look at is like something, okay, I didn't find it, but I could
have found it because it's now that I see it, it's obvious. I never looked at that code base,

(31:00):
but now that I, now that I read that blog post, I think I could have found that bug.
And I think that's the point where you're ready maybe to like go to the next level
and to move to something else. I think it's, sometimes it's good to look at something else
to get better at that initial thing as well. So if you get stuck in your journey here, like

(31:21):
you're learning some old and you're like, after a while you get stuck at some old,
maybe it's time to move to something else. Maybe look at Windows Federation, for example.
Yeah.
And things like that to maybe, and then you come back to someone and you
may have like a new vision, a new way to look at things.
Yeah. How about people that think about more general topics? Let's say they feel quite good

(31:44):
with web. Of course, there's still a lot to learn because web is huge. And within web,
you have small niches, like you just mentioned some, we keep mentioning it.
How do they know if they should stick with web or maybe learn Linux, learn cloud, learn things?
Because I think a lot of people get to the point in their career where they are not sure if they

(32:05):
have to learn something new to have a better resume or they should just master one area.
So I think it depends on what interests you and what's your long-term goals as well.
If, let's say you're like at that level, you're pretty good and you tell yourself,
like, I want to be a CISO in 10 years, or I want to be a manager in two years or three years,

(32:26):
you probably want to look at other things. If your goal is, I want to be a security researcher
or I want to go to point to own or things like that, you probably want to stick to that one
thing. Yeah. Yeah, I agree. For context of bug bounty, it's probably almost always better to
specialize in as small thing as possible because you just need the very, I'm a very, very good

(32:52):
example. I'm really good at web, but I don't have a lot of like cloud experience. I have never done
Windows pen testing, not even like Linux pen testing. I maybe it's okay. I could do something,
but I've never actually done a pen test on my own. So because I always knew I want to do bug bounty
and I was just learning about more and more advanced things from bug bounty.

(33:18):
And I think for me, it works really well. But I think if I were for hypothetically,
if now I were to find a new job, my resume wouldn't look that impressive because there
would be like web skills. Okay. They would be really strong. But then probably the company
would prefer somebody that I don't know, knows G GCP and AWS and maybe big part of,

(33:40):
of like my web knowledge. A bad manager will, a good manager will understand that your knowledge
can be passed to other people in your team and their knowledge can be passed to you.
And that's easily to transfer that knowledge. That's a lot easier to transfer knowledge to
someone who is really good in one thing and having this person transferring knowledge to other people

(34:02):
than just having a team of people who are really average and just know the same thing.
Yeah. Do you think there are a lot of managers like this?
I hope so.
Really?
Yeah. Smart managers.
Yeah. I'm not planning to just, just that's the way I think about it.
For bug bounty, it completely makes sense to invest in things. If you look at
people finding good bugs, they usually spend a lot of time specializing on one or two things.

(34:27):
Yeah.
Like they get really good at OAuth. They get really good at timing issues. They like,
you need to find something that is hard. So everyone else before you gave up.
Yeah. Yeah. Especially if you get to like, uh, events with, with hacking where you just have
to compete with everyone else and you have to somehow find bugs that they didn't find

(34:49):
because otherwise it doesn't make sense. So that's when the fun starts. Cause I think it's
for bug bounty, you can still earn fairly, fairly good wage for finding pretty simple bugs,
but finding a lot of them on, on, uh, lower paying programs. But when you get to,
when you want to, to be at the events and the big public programs and things like this,

(35:12):
we start to, to like need to be more specialized and detailed and things like this.
So yeah, it definitely pays off in bug bounty to just be, be a specialist.
Definitely.
And even, even to the level, not only speaking about back classes, even specialists that
a particular program, there are a lot of people that are just specializing at one program and

(35:35):
that's a skill of its own. That's also probably fairly transferable to other things. It's not
like you learn one website or one company and then the skills from this are irrelevant, but
yeah, it's, it's a very, very, very specialized industry, I would say.
Yeah. Because when you're really good at these few things or these few programs,

(35:58):
you can think in terms of patterns and then map those patterns back to other programs,
other technology and things like that.
Yeah. I even thought about maybe just spending a lot of time on one of the, let's say password
managers and then trying the same things and just, you know, six, seven of, of, of the most
common one. And I haven't done this, but maybe I will.

(36:18):
That's one of the thing I like to do in code review. And that's one of the thing I teach is
place, but the difference, you know, like you've got two images and you need to find
like the differences between two, you take two code base, doing exact two code bases,
doing exactly the same thing. And you compare them. You just read, okay. They're doing check
one, two, three, they're going check three, four, five. Oh, it's that different. Either

(36:40):
does the order matter? What is the same? What is different? And it's a lot easier
because you don't need to know what you're looking for. You just need to compare things.
Yeah. Do you, are you doing it as a conscious process that you,
you literally have to code basis next to each other? Really?
Yeah. Oh, not, not, not necessarily at the same time, except for like small snippets.
Yeah.
But, um, like I found a bug in a play or a session injection in play. I was like taking

(37:06):
the train every day to go to work. And one day I decided, oh, I'm going to look at, uh,
five or 10 implementation of session in Java applications.
Yeah.
They all look the same except play very different. And I was like, okay, I'm going to investigate.
Yeah. And you found a bug because of this. Can you say what was the bug?
Oh, it was, um, so basically it's a science session, like Ruby on Rails. So you can't

(37:30):
modify the content of the session without modifying the signature and you don't have
the secret. So you can't forge a signature. But what you could do is you got like a key value,
key value in the session. You could inject in the value to add more elements in the session.
Also, we could just, uh, close out the bracket.

(37:50):
Close out the quotes.
Exactly. So they weren't using JSON. They were using their own format and that's what the bug
was. They were using like a, so null byte, key, colon, value, null byte, null byte, key, another
key, colon, another value. And you could inject a null byte in your username, for example, and
then add whatever you wanted.
Yeah, that's so nice. I had the CTF test quite similar to this some time ago. There was some

(38:16):
very strange, also assigned science session with some attributes, but also there were
the binary format had the length before the value. And, um, and you had to create, uh,
wait, what was how? Okay. You had to, the challenge was that the app used the same

(38:41):
private key for signing these cookies and for SSH authentication. So I had to create my own
SSH challenge that would sort of match this binary format for the session. So then the process of the
challenge of SSH would give me the valid signature that I could use in the HTTP request.
Nice.
It was so nice. I don't think I could ever find this without the access to the source code,

(39:05):
but it was so fun.
Yeah, that's, yeah. That's why I like about source code review.
What are the other bad classes that you think are quite common in the world, but are not
discovered only because most people don't have access to the source code?
Missing randomness, uh, crypto, crypto, crypto, and, um, some level of authorization.

(39:31):
It's interesting actually, because, um, so I'm looking at a lot of CV regularly
and one thing you never know is how the bug was found. Yeah. And sometimes you find,
you see bugs like, okay, if it's, if this person who found it using code review, that's amazing.
If I found it using, uh, just like fuzzing, it's pretty boring and you're like, Oh, I wish I knew.

(39:54):
Yeah. Yeah. It's a shame that, that from, uh, a lot of time, even in write-ups,
when somebody writes a blog post, we don't actually know how they discovered the bug.
I really, really like it when somebody not only describes the bug shops is really good at this or
asset node in general, they always write the steps, how they discovered it, how it actually works,

(40:14):
why it works. Then they only, then they get to the back classes also all with like code snippets.
Yeah. So it's, it's a really, really like good, good, good thing to read, but it's, it's quite
rare. No. Yeah. Like explaining some mindset, the thought process and all of that thing is really,
really valuable. Yeah. You also do read a lot of write-ups now. Yeah. So how do you sort of optimize

(40:38):
like, you know, that's a lot of write-ups are not really interesting for you.
And I assume you just have a way to like first scan the write-up to see if it's interesting or not.
I just like read it quickly. Okay. Uh, friends of mine created something named talk back and
the aggregate, uh, write-ups and things like that. Uh, like all security research and rate it like

(40:59):
automatically rate it, export like a keywords using like AI and things like that. It's really,
really good. Yeah. That's nice. And it's free. So yeah. Really? Yeah. What's the name?
Talk back, talk back. Okay. I will make sure to, to add this later to the description
because it would be really interesting for me for the case studies because I do also read the

(41:21):
write-ups when I do the case study and I usually scroll to steps to reproduce. And this is like
the, the, the most valuable thing of the writer, but this, this is really going to really help me.
Yeah, no, it's pretty good. But it's, uh, if they don't do bug bounty write-ups,
they mostly do like security research outside of that. So probably not as useful, but maybe

(41:42):
they would be happy to add it as well. Is the tool itself public or just the database?
Just, uh, data. So maybe it's not open source, but maybe I can talk with them.
You're really, really cool. What are other obstacles? You mentioned people don't read
errors. You mentioned people, uh, don't know public and private IP addresses.
What are some obstacles? Maybe not as specific ones, but in general,

(42:05):
things that just hold people back and they can't advance or can solve a challenge.
So I think a lot of people tend to give up at the same time. And that's very like, uh, applicable to
bug bounty is that it feels like everyone is trying, trying, trying, and just give up at the
same time. And I see people like struggling exactly at the same point on this challenge.

(42:30):
And there's this image of like someone digging and there's like, um, a diamond and we're just
like that close to like all the diamonds and we just give up and like turn around and go back.
And that's always something I have in mind when I do like research or like trying to find bugs,
it's like, am I as a guy, like, uh, but like turning down and like being so close. And I

(42:50):
think people need to like, okay, when they're learning, when they're trying to find bugs,
when they're exploiting bugs, they need to like, okay, take a deep breath, go for a walk,
come back and spend like two hours, five hours, two days more. And they definitely
going to find more bugs. Like everyone gave up too early. Really? Yeah. How do you know you're

(43:12):
not spending too much time on something that's not vulnerable? You may be spending too much time,
but most of the time it's worth spending too much time. Okay. Because no one else did.
Yeah. That's interesting. How do you have some sort of visualization of the data in
PentesterLab where you see where people, um, where people give up or something like this?

(43:36):
No, I don't have that, but I do, I do the support. So it's pretty easy to know,
like exactly what's the, so you know it from like people's messages. Exactly. Okay. Yeah.
That's interesting. It's like you have people like you so closely, like one type of way you're
like, and yeah, but it's hard. Like people get frustrated, people don't understand, people are
learning. So it's hard, like learning to hack is really hard. Like it's a lot of work, but

(43:59):
most of the time it's like, yeah. And that's as well, why I like to do support is like trying to
help people, but not too much because like, if you want them to remember something,
you need them to have this moment where like, Oh, I found it. Yeah.
Yeah. That's, that's always scary to me that I will miss a bug because of a typo or something.

(44:21):
Cause you know, once in a while I work on something and it doesn't work. And then after,
sometimes after an hour, for example, I notice a mistake as stupid as a typo or just the system.
Sometimes it was random. Like yesterday I wrote a writeup where somebody had SSRF
of DNS rebinding that worked one in 30 attempts. So, and I also had had this year, some bugs that

(44:47):
like, um, I had, um, it was a sort of a client side bug. Um, okay. I can't disclose too many
details, but in general I had just had the loop that would send the same request over and over
again because it would just randomly sometimes work and sometimes not with exactly the same
payload. And knowing that, um, these things happened to me and I resolved them probably

(45:12):
there were, there must be things that were similarly stupid problems or randomness problems
or my type of problems. And I didn't find out about them and I missed bugs because of this.
This is the scary part for me. I know a lot of people like you spent a lot for that.
Just keep solving like the new labs all the time or go back to all labs for two things,
speed and accuracy, like getting better, like getting more accurate and getting faster and

(45:39):
getting faster at doing the thing. And also getting faster at detecting if, uh, it's not
exploitable or they made a mistake. And again, like reading error messages is trying to like
explore behaviors. It's like, um, am I doing it wrong or it's not exploitable.
Yeah. But with, with labs and with CTFs, it's that much easier that, you know,

(46:03):
you're probably, you're doing something wrong with real world. This is the real challenge.
Yeah, definitely.
How about the challenges itself? I assume you have some stats on which are the most popular
one, like most popularly solved or attempted by people, which are the most popular.
I was the early one and the easiest one. Yeah.
About, I know that probably the, the, the free ones are the most popular by numbers.

(46:28):
But out of, let's say the, the, the paid section, what, what is something that people choose a lot?
So we have a path to tell people like that's the direction you should follow. So usually
like people give up at one point. So that's the early one obviously is a, uh, most,
like the most solved one. Um, I think in terms of like what people really enjoy, uh, is a JWT one

(46:52):
and the crypto one. Yeah. Like people look at those and it's, it feels like magic.
Yeah, it does.
Like when you crack like a CBC or an ECB or like a CBC Mac, all these like things that
are encrypted and it's like, Oh, it's just like basic 64 random crap. And you play with that.
And then you get something else and people are amazed by that.

(47:15):
Yeah. It's funny you mentioned it. I would probably say this was my favorite challenge
with like one of the CBC ones because I never saw something like this in real life. I'm not
a crypto guy and just having something like this was, was very, very fun. What's on the other hand,
are there some challenges that people skip for some reason, even though they are on the path?
Um, not necessarily because you need to finish them to have the badge. I think, um,

(47:42):
we have like a lot of code review challenges now and it's hard to do scoring for code review. So
people often get frustrated because of that. Um, because what we do is we ask people like to pin
like a file name and a line number and the type of bug. And sometimes it's not that accurate. So
it's like best effort kind of thing. So often like people get frustrated because they are reading

(48:06):
source code and it's already hard. And then like, Oh, it doesn't work. And but now people are
pretty happy. And yeah, it's not, it's not like, um, easy, like they're not,
they are easy labs, but it's not like, um, an easy thing to do to learn all of that.
It's a lot of content to go through. And yeah. Yeah. And then some challenges are, you know,

(48:27):
a few hours of work. I remember this. So to finish, are there many people that have all
the challenges solved? Yeah. Really? Yeah. Like, Oh, I'd say probably like,
I know that at least like 12 to 15 people, like if there's a challenge is out the next day,
it's solved. Like, yeah. Like loving it. Yeah. Yeah. That's nice. Congrats to you.

(48:50):
If there's you. Yeah. Also about a PentesterLab, cause, uh, you, you just, you mentioned that you
still do the customer support. So you intentionally run it as a small business.
Yeah. Like I think doing customer support is great because, um, it helps me make the content
better and understand what people struggle with. And I really enjoy hacking and things

(49:16):
like that. So I don't, I don't like, I enjoy doing like a lot of different things as well.
So that's why keeping it small really helps. And, uh, I'm having a lot of fun. I can't believe it's
my job basically. Are you still doing it alone or do you have some people?
I have people helping me with design, with, uh, looking at CVS, like mostly like the extraction
of data for CVS and things like that. Yeah. And yeah. Yeah. I look up to it because I think

(49:42):
it will be easy or a lot of people in your position, having pens to be so popular,
they would just try to scale it up and, and be perhaps more income. Cause it would be easy to
like scale it up. It'd be more popular, have large income and not, not necessarily big profit. And I
think you, you just run it to, so it's fun for you. So we don't have to have like problems of

(50:03):
big company and you would basically be a full-time manager if you did this. And I have a similar
approach at least at the moment that I don't want to make the channel. I could just hire
people and do different stuff and be big and publish videos every day and stuff like this.
And I just don't want, so, so I do definitely look up to it because, uh, I'm also happier.

(50:25):
I think doing this, I don't want to be the CEO. I know some people that became CEOs and they miss
hacking. So, um, so I'm definitely looking up to, to like your approach basically.
Yeah. I've been like a manager in a previous life and I didn't enjoy it much. So
previous life, you, you, you, you now treat being employed as the previous life.

(50:46):
Yeah. No, but, um, I think I, I have the right balance. Like it's not easy, but it's a good
balance. Yeah. What are some tips to me? Cause I'm, I'm way younger entrepreneur than you are.
So you definitely have some, some tips that you can give me, um, trying to have fun. Yeah. Because

(51:08):
it's a long, long road. So if you're just in it for like making money and things like that,
that's not going to work. So you need to keep the fun. Um, you've got to find like a lot of
people who are more successful than you, uh, doing more stuff, bigger company, more employees,
doesn't mean they are happier than you. Yeah. So that's because for diverse reasons and find

(51:31):
something that makes you happy, I think. And people like kind of feel it. I think people
like Pentestal because all people who know me know that I really enjoy what I'm doing
and I'm really passionate about it. And for example, um, doing security code review, uh,
content is not necessarily where, um, customers are, but it's something I really enjoy. And I

(51:52):
think that's what matters right now for the industry and for what people should be learning.
So if I were like trying to get more profit and things like that, I should probably do like some
blue team content or things like that. Yeah. But security code review is so good.
That's so nice. What are the, your plans for the platform for let's say upcoming year?

(52:14):
Um, a lot of code review, a lot of code, like every single language. I want to be able to
have someone who's like, okay, I got a new job as an app sec person in a Golang, uh, shop.
They only do Golang. I need to have this path and say like, okay, do those labs,
lab, lab, lab, lab, those labs. And you're ready. Okay. And, um, more presentation, more talks,

(52:40):
um, public speaking, more, uh, training. So I do like life training now. So that's,
I really enjoy that. It's good to have, like, instead of like just being in my corner,
like talking to people, it's pretty hardcore, but yeah. Like, um, as in, I did like, uh,
two days in a row and like the second, I could barely talk because it's like me talking for 12

(53:00):
hours and, uh, going for like 972 slides or something. 972 slides. Yeah. There's a lot
in like four times, three hours. So it's pretty, yeah. And, but yeah, like trying to grow,
but with the right clients, I think that's what matters and what is a great client, right client,
someone who really enjoy hacking. Okay. Like someone who really enjoy like technical

(53:24):
nitpick, technical details. Someone would spend like, uh, two days on some little details or email
me like, Oh, we, have you thought about that? That's great. Like when I started that lab,
I thought about like this other way that you don't show in the video. Like, that's amazing.
And yeah. And having fun and yeah. But like, yeah, there's so much content I want to create.

(53:45):
Like I want to do something, uh, more content for, uh, beginners because I feel like a lot of content
around beginners is not really well done. Okay. Like I want to approach it with more like a
security engineering point of view, explaining the details, like, um, people teach DNS.

(54:07):
It's interesting to teach DNS to security people, but in DNS, you have like security mechanisms in
place and explaining both security mechanism, how they work, why they are there and things like that
to build not only the knowledge, but the threat models around technology and things like that.
Okay. And same, I want to explain TLS, like the number of like infographics on TLS

(54:29):
online that are wrong, that drives me nuts. Like, yeah, like, yeah, I wouldn't,
I wouldn't draw a correct one. Yeah. That's not a problem. Don't do infographics if you can't.
Awesome. Thank you so much for joining me today. No worries. Thank you for having me like, yeah,

(54:49):
that's great. Yeah. Thank you. Uh, and thank you for listening to the episode. If you enjoyed it,
uh, we spoke a lot about code review. If you want to, if you're inspired to do more code review,
uh, check out the episode with shops that's linked in the description.
And for now, thank you for listening and goodbye.