S9E5 | Let's talk about Cypress | Mark Noonan - The Angular Plus Show

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:08):
Welcome to the Angular plus Show. We're app developers of
all kinds share their insights and experiences. Let's get started.

Speaker 2 (00:21):
Welcome everybody to another episode off the Angular plus Show. Today,
My Lovely did you see what I do?

Speaker 3 (00:27):
Did?

Speaker 2 (00:28):
Their lovely co host Brian love How are you doing today, Brian?

Speaker 3 (00:32):
I'm doing really well. I'm taking this from my car
and it's a beautiful sunny spring day here in Bend, Oregon,
and I'm just excited to be here with Mark and
get to meet Mark and learn more about Cyprus.

Speaker 2 (00:44):
Yes, so our guest today, Mark, I should have asked
that before for the last name. Let me try. I
feel a missus today, Mark Noonan.

Speaker 4 (00:55):
That's correct.

Speaker 2 (00:58):
So you're product manager at Cypress if that's correct?

Speaker 3 (01:02):
Right?

Speaker 5 (01:03):
Yes?

Speaker 2 (01:04):
So what brings you here today? How can we help you?

Speaker 4 (01:09):
Well, I can tell you a little bit about my
background and we can kind of go from there.

Speaker 3 (01:13):
Good.

Speaker 4 (01:14):
So I joined Cyprus like three and a half years ago,
and I was a front end developer before that. So
I came in at the time that Cyprus was doing
component testing and like adding it to the core Cypress app,
and so that Cypress ten release was really something I
contributed a lot to and worked on the open source side,

(01:34):
and then as time went on there there was kind
of opportunities in product management and I moved over to
like explore some PM work and I became the product
manager for our accessibility product. And that's largely what I
focused on for the last year year and a half,
which is the kind of embedded accessibility testing in Cyprus Cloud.

(01:55):
So that's where my focus has been. It's expanding a
little bit now into some of our others, but that's yeah,
it's really front end adjacent as a product manager role.
It's like I talk to people about front end code
all day long, so.

Speaker 3 (02:08):
I like it a lot.

Speaker 2 (02:10):
Okay, before we dive into Cyprus more, do you miss
front endcoding? Like I get it, you're still talking about it,
but like like do you still have that itch of like, Okay,
I would really like to write some code now.

Speaker 4 (02:24):
Yes, yes, And I'm still I can open a PR
every now and then to fix something. Then I'm like, oh,
this this CSS adjustment needs to be made, and it's
it's easy and fast and doesn't take a lot of
reviewing and I can just do that. So I miss
it a little bit, but I would say, nowhere near
as much as I thought I would. So I still

(02:47):
enjoy coding, but I can always do something like that
in my spare time, and it turns out lots of
the stuff in the PM role it's actually super appealing
to me and really fun to do.

Speaker 2 (02:57):
I was just about to ask, was it like the
the thing that I love about development is like you
have like you have like massive highs and very low lows,
all right, Like if something is working for you, you
feel like you did like this huge accomplishment, and if
something's not working, you feel like a failure. But that's
a different story. So you have those success moments very
often throughout one day. So like feeling successful in your

(03:19):
role is very easy as a developer, whereas as a
manager you need to like completely shift your focus and
like you're the most successful when your team is successful.
So was that like something that you had to get
used to or were you just like flip the switch
and you were like, Okay, I can do this.

Speaker 4 (03:35):
I think one of the reasons that like we identified
this is a good change for me was I was
always kind of pushing more towards the product side. So
even as a developer, I would bring up things like, hey,
shouldn't our product also do this? You think all developers
should have that mindset. You're in the code, Oh why
don't we have a button for this task that I
now have a microscope on top of that nobody else

(03:57):
has thought about for a year. So I love that stuff.
And then the interesting thing about the feedback on your
work as a PM is you spend a lot of
time talking with customers, at least in the model.

Speaker 2 (04:10):
That we use.

Speaker 4 (04:11):
So you go down on like, all right, cool, I
did a really hard thing and it took a little
bit less time and it works great and it's really robust,
and you go down on like that type of joy.
But then you see people who come to a call
with you and they have used your product and they
describe the exact experience you were hoping for. They're like,

(04:31):
it's intuitive. I understand this. It's easier than the thing
that I was using before, Like you know, what's our
next steps? It's not every call, but like those are
the ways where it's like, okay, we put twenty pieces
on the board and the experience came across perfectly, And
like I love that aspect of.

Speaker 2 (04:49):
It nice as much as I would love to keep
talking to you to shift from a developer to product management,
because I think that's super interesting. I think that's something
that rather I think is interesting for all and since
probably they want more here about Cypress. And from my
last experience with Cypress was probably like twenty eighteen, if
not even older, so I'm not quite up to date.

(05:10):
I think, Brian, you're a little bit more up to
date than I am, but also not like on the
latest and greatest.

Speaker 3 (05:16):
That's right. Yeah, But I've written a lot of Cypress
tests over the years, and in the past I've even
trained companies and developers on using Cypress and so kind
of around best practices and how to use the promise
like API and how to get around maybe some of
the quirks of it, but also like how to really
refine your tests and try to reduce that flakiness and
so you know, that's not something that's inherent to Cypress.

(05:39):
I think that's something that's inherent to brass or testing,
and Cypress actually does a really good job of like
minimizing that as best as it can with the retriability.
But I've been a big fan of Cypress for many,
many years and so yeah, I'm excited here to learn
even more about Cypress.

Speaker 2 (05:56):
So go ahead, Mark, assume, just assume for the sake
of clarity. You know, I have never heard of Cypress.
I have no idea what it is.

Speaker 3 (06:07):
What is it? Yeah?

Speaker 4 (06:10):
Cool? So I would tell you that there's two pieces
to it. There is an open source application to write
tests for things that run in the browser, and that
can be end to end tests or in the last
few years, component tests, so you can directly visit URLs
from some website that you're running, or you can mount
an individual component from any of the main frameworks and

(06:33):
test that piece in isolation, but using all of the
standard stuff that's available in Cypress. So there's the open
source piece, and then behind that there is a cloud
product where there's additional services and you know, stuff that
adds on top of it. And that's that's the business
engine behind the ability to have the open source products

(06:54):
like maintained and updated.

Speaker 2 (06:56):
Cool. So you described a shift more from enter and
tests to component testing. Can you tell me a little
bit more about that. What does this mean, what does
this involved, and at the end of the day, what
does this mean for me as a potential cypros user.

Speaker 4 (07:10):
So yeah, as a potential as a developer who would
use Cypress specifically, because we also have you QA people
who don't write code. They write tests mainly, and they
are usually focused on end to end tests only. But
when you're creating a component, what I would do as
a front end developer is find some way to get

(07:31):
it in the browser and use some kind of either
temporary environment or sometimes just load up the application state
and do my iterations of like build a component, go
and look in the window and see if it's working,
and do all of that in the browser kind of
side by side. And there are component tests, or at
least at the time that Cypress component testing came around,

(07:53):
there were component tests that were more like gest based
terminal based work clothes where you kind of had to
imagine it was rendering in a browser. And so Cypress
brought the component testing approach of mounting and rendering isolated
components into browser in a test environment, so you could
do more like test driven development if you want to

(08:14):
build up the state of the components. And I use
this in its alpha form at my job before Cypress,
so I was like really keen on this because I
hated different syntax for JEST and my end to end tests,
and I hated like not knowing what was happening when
I would just get problems with that. So I was
an early adopter of that and it got me way

(08:35):
more focused on Cypress. That's a sidetrack, but that's what
it does for yours a developer, is it gives you
the option of using the same exact mental model for
everything in both layers of testing.

Speaker 3 (08:47):
I'm curious, Mark, when you were using like the component testing,
were you building like highly reusable components or were you
using that just to build like I'm calling like everyday components,
like very stateful components, or just like parts of your application,
or like what was the need, like what was the
use case in the need for that at the time.

Speaker 4 (09:08):
So I worked at a fleet management company and we
were rebuilding our UI from a kind of legacy Java
server rendered application into view. So at that company we
had a few reasons to have components rendered independently. We
weren't really making a whole design system though the front
end team was fairly small, so for us, I wanted

(09:31):
to be able to build pieces of it in a yeah,
but without running that whole Java application locally and like
where I eventually live.

Speaker 3 (09:41):
Yeah. Yeah, But I also use.

Speaker 4 (09:44):
The component test for company demos, you know where you
just have like everybody out there, they're like, what is
this thing? The front end team is building months to
be in the UI? Right, but this will let me
tell you my favorite cipres's command, which is side dot pause.
So in these like large company demos, I could take
something like the fleet management tracker that shows fuel levels

(10:07):
or battery levels or something, and the component test would
get into all those different states. And so by judiciously
having pauses, I could share the screen, show the user workflow,
and then talk with everybody about this is what happens
in this condition, and then this is the next state
in the next state. And I loved that. I think
that was a really simple and useful way to share

(10:30):
our work as well as right tests as we went.
So it was good for me.

Speaker 3 (10:34):
I agree, because I think, like I've seen I and
I've done in the past something like what you're talking about,
mark where I've used like storybook for example, and you
can add like knobs or whatever, so then I can
change state and I can say, okay, well, then your
example you know, here's that fuel level indicator. I'm going
to you know, let's lower that. Let's watch that animation
or whatever it is. Okay, now let's raise it. Let's
see that now it changes colors, whatever it is, like

(10:57):
those different states and behaviors, which I think a storybook
is great and this isn't a knock storybook. But then
after I've written my storybook file or my dot story
file or whatever, like I don't have a test. Now
I got to go and write a test, right, or
I got to go write a unit or whatever. I
think what's really compelling about the story you just told us,
Mark is that you kind of got like two birds

(11:18):
with one stone there, right, Like you were able to
like use Cypress in a like component isolation to demo
and to show those different states in those different things.
And then at the end you had a Cypress test
and so then you were able to like roll that
into a test and like check the box on your
pr like wrote a test and like it's good to go.

(11:39):
So I thought that that's really smart. I've never thought
of doing that, and I just want to like applaud you.
That's a really I've never heard anybody use Cypress in
like a demo fashion. I've heard of like component testing
and lots of Cypress tests, but certainly not in the
sense of like doing a demo like that. So that's
really cool, man, that's a great idea.

Speaker 4 (11:59):
I was just so excited. And I'll tell you we
had we had storybook at that company, but for a
very narrow use case.

Speaker 3 (12:06):
So usually for like ice, like your reusable components, your
design system, your buttons, your whatever drop downs, whatever it is.

Speaker 4 (12:14):
Yeah, exactly, and those things could have Cypress tests as well.
But the reason our use case was so narrow was
we didn't have anybody else at the company to deploy
the storybook for so it was really just a way
of doing that isolated development. And so my pain as
a developer was I'm writing Cypress and to end tests,

(12:34):
I'm writing just component tests, and I'm instrumenting for storybook
with like getting the knobs and everything set up.

Speaker 3 (12:42):
Yes, yes, And so I.

Speaker 4 (12:45):
Was delighted when I saw the alpha for component testing
and I was like, oh wait, I can just start
to put things in Cyprus.

Speaker 3 (12:50):
Interesting.

Speaker 2 (12:50):
Yeah, theified So one thing I'm wondering, because the there
are various solutions for like component tests signal on the
market with like I don't think Playwright has something like that,
But there's the test component testing to some extent, and
I don't because very easily you get into, like if

(13:11):
you directly compare competitors, you get into like an apples
to apples kind of comparison. What I would like to
hear from you is what makes Cypress unique in the
way they tackle certain problems maybe, or what does Cyprus
shine it so well?

Speaker 4 (13:26):
We can talk about a few aspects of it, but
component testing specifically, the shared syntax with end to ending
component has this like hidden power that I don't sort
of like the demo workflow. I don't know how many
people exploit this.

Speaker 2 (13:41):
Well.

Speaker 4 (13:41):
Let's say you have a complicated interactive component, maybe a
multi step form with text calculations or many many different
things going on, and you write a component test that's
really exhaustive, goes through all of the permutations and make
sure they're working correctly, and it runs super fast because
your component test normally don't call the REALAPI or anything.

(14:03):
But you write this command that's like, all right, perform
these forty steps in this component based on a set
of fixture data. That's like one example of the type
of product you sell, and then when you inject that
component in real life and you want to write an
end to end test, that pattern of every step in
the workflow and how to take something as an example

(14:25):
product and perform those steps is exactly the same, and
you can directly lift it into your end to end
test and say, I want to do this one time
because end to end tests are slower and they're going
to use the REALAPI and they're going to do other things.
But like you can synchronize the end to end and
component test code in such a way that the developer

(14:45):
modifying the component test is creating this reusable helper that
will work in an end to end test, and like
things just make sense. Those worlds don't have to be
so separate. So like I really love that stacked effect.
But we can get an other details as well about
like things I like about Cypress.

Speaker 2 (15:04):
Tell me.

Speaker 4 (15:08):
So maybe the way to approach it is like when
I was learning about testing and development, Cypress was fairly
new at that point. So the approachability of Cyprus from
the documentation standpoint and their logical flow of the test
was really appealing to me. And there's definitely opinions about

(15:29):
a syncawaight versus like the confluent approach of Cyprus these chainables.
But I found that model, and like the subject of
the test that I'm holding as the commands go through,
really easy to grasp and work with. But there were
some limitations of like the model of Cyprus, most of
which have workarounds, but people listening to this might be

(15:51):
familiar with, Like the tab key is a really good example.
So Cypress keyboard events for typing are simulated, and that
comes with some problems every now and then, and one
of them is you couldn't fire native tab events. So
there have been workarounds for this, but yesterday we actually
just released the the name is side dot press, like

(16:12):
that's the new command for firing native keyboard events, and
so you can dispatch a regular tab key event in
your tests now for testing things like autocomplete and keyboard
behavior that were like you know, accessibility testing the tab order,
all of those things were a little tricky because you
had to install a plug in, and that's now solved

(16:34):
and the path is clear for other types of native
custom keyboard experiences to be tested in Cyprus. So, like,
I'm really excited about that one.

Speaker 2 (16:43):
That's actually a good thing that you brought us up
because you mentioned at the beginning that you work on
accessibility kind of things like how how can I look
like i've when I did accessibility work, I usually like
either like turned off my monitor or used like a
static analysis tool like act core. There's easland rules. There
are some like chrome plug and something like that. How

(17:04):
does Cypress fit into that, like help me fix my
mental model? Where do I put Cypress in this?

Speaker 4 (17:12):
Yeah? Yeah, there's so many approaches and philosophy around how
to do accessibility test automation, and in Cypress you can
do all kinds of things to get like a lot
of good accessibility feedback into your tests. So, taking aside
the paid product Cypress accessibility that I work on, if

(17:32):
you're just using Cypress without the cloud, we have some
recently developed docs showing how to use XCore. For example,
there is a Cypress ax package that will run this
library that does static analysis of a page. So for
people who are listening, an ax core doesn't mean anything
to them. If you've ever seen an accessibility score in Lighthouse,

(17:53):
that's ax core under the hood, at least some of
the rules. So the Historically, the thing people would do
is use the Cypress package, run a check at a
certain point in the test, maybe after a page loads,
but really you do things in the page that change
accessibility and change the DOM during the test, and optionally
you can fail if certain of the one hundred or

(18:14):
so accessibility rules start to fail. So that's a very
generic check. It doesn't really know what your application is
trying to do. It just knows any page should meet
these criteria. It's a really good way to start getting
feedback if you've got nothing. But then there's this extra
layer because it is just regular test automation, so you

(18:36):
can do things in your ordinary Cyprus regression tests that
speak to an accessibility bug you just fixed. If someone
reported like, hey, even though all your ax core tests
are passing, I still have this wrong text associated with
a button or something like that that a generic test
won't catch. You can do those types of things, and

(18:58):
one way people do that is through you using testing
library in Cypress, which is it's got you know, plugins
for all the test runners, and it's very similar to
the main syntax in Playwright. So if you like that
testing library syntax, you can do that in Cypress. As well.
There is one caveat that I have personally like as

(19:18):
a developer with the testing library approach, which is usually
based around accessible roles, where you say something like find
by roll button and you give it a name and
you get some confidence that there's accessibility there. But the
main thing people need to know who are testing that
way is that the role is not the entire ballgame

(19:42):
in terms of whether that thing is actually accessible, and
a lot of elements that have let's say it's a
div with a roll button, it will pass a get
by role check, but it will not have all of
the keyboard behavior, the focused states, the high contrast, you know,
all these things that go with that, and it's a
very subtle distinction, but it has massive implications for accessibility.

(20:05):
So I'm not against using find by role, but I
also I like an additional way to make sure, when possible,
we use native HTML elements instead of custom ones. With
the native roles, this could be a rabbit hole, So
I'll pause here to gauge like an interest level and
continuing down this path of accessibility on elements. But yeah,

(20:28):
there's a lot of talk about there.

Speaker 2 (20:30):
I went down this rabbit hole like like a couple
of years ago, I had to write my own custom
select in an accessible way, so I feel that pain
a lot. Brian helped me gauge, like, do you think
we should go further down this rabbit hole? I'm super interested.
I love accessibility.

Speaker 3 (20:46):
I'm interested because these are some of the things that
I haven't actually thought about. I didn't know that you
could use like React testing library or whatever testing library
inside of Cyprus. Maybe I've just never seen that use
case or whatnot. But yeah, I think that one of
the common things that we see with some of the
consulting work and around Cypress is just like or use

(21:08):
of selectors when it comes to like writing their tests,
which causes some brittleness. That's I think outside the realm
of accessibility. But I think like that just ties into
your comment around like, hey, we're going to actually write
this as if it's a real user and a user
and an accessible user is going to be looking for
the button to submit the form, not going to be

(21:29):
looking for whatever it is, you know, some sort of
maybe the button tag or button element, but you know,
looking taking more of that user perspective compared to a
technical perspective. But yeah, I'd love to hear it. If
you have more accessibility kind of hips and tricks around cypress,

(21:49):
I think that I would be interested in. I think
the listener would be too. So if you've got a
couple more, let's let's go into it.

Speaker 4 (21:55):
Yeah, So picking up on what you just described, the
it goes back to component testing actually, like the idea
of using test IDs for your end to end test.
Like here's a locator, right, I'm going to call it
data side or data test ID, and then I always
know how to pull.

Speaker 3 (22:12):
That element out.

Speaker 4 (22:13):
And that has a similar problem to like find by role,
but it still can be really effective. And what I
mean by problem is like it doesn't assert anything about
the accessibility.

Speaker 3 (22:23):
Of the earth does not. Yeah, it's only like that's
just a development side of things, Like it has nothing
to do with accessibility.

Speaker 4 (22:30):
So it's like explicitly generic. You can click on divs
and spans and stuff, right, Yes, However, I see the
component testing and end to end testing divide as like
the person writing the component test is probably the developer
who's responsible for the HTML output used by that component,

(22:50):
and they're responsible for the accessibility of like that initial render,
right should it if it's a button component and it's
got custom styles. Do we choose the develement and apply
the easy styles or do we do a tiny bit
of overwriting on the main button element. And so if
the component test specifies the HTML output well enough of

(23:14):
like what's supposed to fail when that component is changed,
then the end to end tests using really stable generic
test ID locators doesn't hurt anything on accessibility because that's
not really where you're trying to assert that. But there
is a bunch of other accessibility stuff that a component
test can't really assert very well because usually when you

(23:36):
provide a component test, it's data. You're stubbing out everything
it might require. So if you test if you could
pass in an area label, you get one out and
stuff like that. And so in the end to end test,
I think you still need to look at accessibility, but
you're more suspicious of the content rather than the underlying

(23:58):
HTML because people can do things in the code that
break the assumptions of the component test. So you can
nest components in weird ways. In the end to end context,
you cannot provide a required label that's supposed to feed
through and wire up. All these things are like risk
areas and so the relationship between like how much we

(24:20):
test accessibility and what exactly we think is important in
the end to end test depends on the risk of
the context. So to me, there's all of that, and
all of that is just test design. It is what
are we intending to test? What are we doing? It
applies to any framework, right including Cypress, but everything else

(24:40):
as well, Like what are we asserting about? And then
we talked about ax core, and there's like how do
you use these generic libraries and get the data? And
so you call something like an ax core Cypress X
package and you make an assertion and you get feedback
about a certain class of things. So, for example, button

(25:00):
missing a label like it's an icon to open and
close a menu, or it's a close button on a
model or something. Those things that don't have labels that
would be useful to screen reader users are they're going
to be caught by something like axcre, but only if
it's actually a button. So the divide between the semantics

(25:23):
and like what's actually intended in your tests is also
somewhat dependent on the code quality of what you're already building,
and so code review catches that manual review should catch that,
but it also it gets me into what I'm really
excited about on the Cypress cloud side, where we have
the ability to run like axcre checks basically on everything

(25:46):
in your Cypress tests. So if you just record to
the cloud, we can post process the tests like we
don't do anything during test time, and we can look
at those and combine a report from like hundreds or
thousands of different steps you took in your tests and
give you an accessibility report like here, here's every page
you tested, and here's what we saw. And I think

(26:06):
that that's really good. But the next step we're doing,
and the reason this is like top of mind for me,
is we're looking at the implications of your test code
to add to that kind of report. Because you already
wrote all of the interactions that a user does. So
you've showed us that you're clicking on divs and spans
and SFG charts and graphs and things, and you've taught

(26:27):
us that those things are supposed to be interactive. So
all of a sudden, we can add that layer of
detail to a generic scan and say here's what we
found with ax core, and then here's fifty other suspicious
elements that don't have semantics, but you're treating them like
they're interactive. So you need to go and look at
those figure out if they're buttons or links or forms

(26:48):
or what are they, and then put those into the
right semantics. And I think that that's going to be
massively revealing for some projects where the developers worked without
accessibility in mind. So I'm really excited about like that
direction of accessibility testing based on how users actually use
the application.

Speaker 2 (27:07):
Do you have anything I know this is like probably
the most German thing you can say right now, but
do you have anything for like regulations and stuff? Because
I know there's like Accessibility Act in Germany, there's something
in the US for that. There is the European Union
has some regulation that go in place next month, I think,
going two months. So is that an area that you're

(27:29):
exploring that you're considering or is that more like hell no.

Speaker 4 (27:36):
I'm like midway through writing a blog post about the
European Accessibility Act right now. So that's a note, right right.
We see the automation piece as kind of one of
maybe four threads of a really good healthy accessibility process,
and so most of the people like the transition between

(27:56):
do we use the free plug ins or do we
just want to switch on something that's in Cyprus cloud,
And just like watches, everything is usually are we a
large enough company and we have a legal compliance obligation
And the automation piece is one of the ways that
they helped meet that. It'll never be everything, but that's

(28:17):
you know, within the confines of what automation can do, right.
You can do it, you can pay and do it
really fast and really easy, or you can roll your
own and manage those pieces like more precisely. So it's
a trade off, but the legal landscape is definitely why people,
quite often white people talk to us about what they're doing.

Speaker 2 (28:38):
I bet I was particularly asking them because I had
a project for the Federal Agency of Work in Germany
and they had like a special department because obviously every
piece of software that was put out had to go
through like accessibility testing, and they actually had like blind
people test that. So it's just super interesting to them
and in person be like, Okay, I've never I'm not

(29:01):
familiar with this process. So and that's where this is
where where I came from with this question. Sorry, Brian,
I feel like a cultrol I.

Speaker 3 (29:10):
Have a I have a question mark tell us, so
you've got your you're building this product in the cloud
to give this like and I think it was interesting
just to like kind of reiterate it's like a post
processing so it's not like this additional layer of testing
or assertions or whatever you want to call it is
happening during my actual test execution time because we wouldn't
want to bump those times up anymore that we had to.

(29:30):
But during that post processing you're running this additional accessibility thing.
I guess I just have a couple of like really
dumb questions about it, like a how much does that cost?
Does that just get included with my cloud pricing? And
then be like, can you tell us a little bit
about like the secret sauce? Like how do you do that?
I mean, is that something that is like static analysis

(29:52):
or using like maybe some sort of AI or LLM
to do some of that stuff, Like what does that
kind of look like behind the scenes in order to
actually do some of the US post processing for accessibility?
And I'm just curious that the engineer and me is
wondering we.

Speaker 4 (30:06):
Can talk about how that works because and on pricing,
it's like a you know, call us and it's a
sales process. So unfortunately I don't have like published.

Speaker 3 (30:15):
Process there, you're not you know how to say, I
don't expect Yeah, yeah, but it's there. It's it's something
that I can turn on. That's cool.

Speaker 4 (30:21):
But you touched on like why is cypress Cloud even
able to do that? Because we have two products that
work in this way and they're based on the test
replay aspect of cypress Cloud. Ah, yeah, you get it now.
Like if you use cypress with video recording in the past,
which is kind of the only way to see the

(30:42):
playback of the test, you just had the video and
you couldn't really replay and inspect all the dom states.
But because now by default when you record to the cloud,
it's capturing all of the steps along the execution of
the test. We can then process and analyze those for
to pull different purposes without adding anything to the test behavior.

(31:05):
So this is unlocked. Like the accessibility processing, we literally
load up every single state that we saw in a
browser and run axe Core in an actual browser and
then merge all the reports and do this post processing step.
But we also have a coverage product which I recently
took over the product management for as well, because these
are kind of siblings. And what that does is it

(31:27):
looks at instead of the accessibility, which is like, what's
the quality of the code that we saw in the thing,
the coverage aspect is like what is tested and not
in the UI, like what's the quality of the tests?
Because we can bring back all of those application state
snapshots and show you like red and green highlights and say, hey,

(31:49):
in this run with a thousand tests you tested, here's
what I see all the time you tested the login
flow like four hundred times, right, every test is kind
of unnecessary clicking the log in button and doing all
of the log inflow, And so we can show you
that and say optimize, you know, switch to using cookies
or something, and then you use to feed.

Speaker 3 (32:09):
Up your tests.

Speaker 4 (32:10):
But also you never ever ever tested the reset password flow,
and it's going to have a red icon over the
reset password button to say, even though you did all
this other stuff, this one thing is not tested, and
so you're going to find out like I used to
find out that the reset password flow doesn't work when
a user tells you they can't reset their password. Like,

(32:33):
so that combination of stuff is all possible through like
this post processing workflow, and on those sides there is
no AI, but there is at least right now, but
there is AI coming and a little bit in beta
form for creating new tests.

Speaker 3 (32:54):
So that's the yes exactly, Yeah, yeah, that's cool.

Speaker 4 (33:01):
So good example. There's two sites to what's happening with
code jen as well. On the on the cloud side,
it's like, okay, we show you a missing test element
and you want to start off a test. There, we
can generate all of the things needed to recreate that
exact state that renders the element, so you can just
grab it over and begin your next test that way.

(33:24):
But in the Cypress app side, there's two things happening
around the open source part of it. The first is
like Cypress studio, the test creation part of Cypress is
been experimental for a long time.

Speaker 3 (33:37):
Did they kill it that we had it and then
they killed it and then they brought it back.

Speaker 4 (33:42):
In Cypress ten because we showed it was Yeah, we
made extensive changes to the UI support component testing. But yes,
even in the experimental version, like people upgraded to ten
and the same day they're like, hang on, I use
this experimental part of Cypress all the time. So bring
it back and so we did did the work to

(34:04):
make it work in the new system.

Speaker 3 (34:06):
Yeah cool, but it's.

Speaker 4 (34:08):
Going to get a lot more mature because I imagine
now it's also going to be where you can do
this more AI. It'ssisted recommended, yes patterns. So I'm not
I'm not like involved directly in that, but it's looking
really good to me. As like the both the part
that does not use AI will get improved, so just

(34:29):
everybody will have a better studio experience. It won't be
in experimental anymore, and then it's it's where the AI
stuff will start to live.

Speaker 3 (34:37):
If people have to imagine, Like I'm not close to
the metal on this, but I have to imagine just
from my experience of building stuff with AI as well
as like what I remember Cyprus Studio doing, Like it
just seems like an ll M is a great fit
there to just automate a lot of the tests, you know,

(35:00):
authoring experience. I'm sure it's not going to get it perfect,
and I'm sure I'm sure it'll have to be plugged
and fixed and all that kind of stuff, but I
would I would hope that you can get pretty close.

Speaker 4 (35:10):
The big improvement is like the understanding based on the
way you already write tests. So if you've got custom
commands for logging in or family sessions.

Speaker 3 (35:20):
Those there, Yeah, what I have that context? Yeah, right, and.

Speaker 4 (35:25):
Then you name the tests, so the ALM is aware
of the task you want to complete in the applic
tests and the applications running live in the Cypress browser
alongside you. So the feedback is illediate after every action.

Speaker 3 (35:39):
Yeah.

Speaker 4 (35:40):
So the dream is it's just like accept accept, accept,
and you just watch it kind of build itself based
on what you expect the test to do. Yes, and
as the tester, you you have responsibility for defining first
of all what's important to test, and then also stepping
in where it's like redundant or you know, providing the
feed back to Cypress. So that's a really exciting area

(36:03):
of Cypress and we expect, you know, a lot of
useful things to happen there for sure.

Speaker 3 (36:09):
Cool, very cool. You're on mute, John.

Speaker 2 (36:13):
I knew that we now touched a little bit on
the part of like, okay, what's coming, like tell us
more like that there. I've been out of the loop
a little bit anyway, So everything sounds new and fancy
to me, But what's on the roadmap for Cypress that
you can already share with us?

Speaker 4 (36:33):
So in the app itself, the Angular twenty, I don't
remember if we already touched on that. In the podcast,
we'll have will have Cypress component testing support, so we
have that date of like when that becomes official and
looking at the issues that might or may not need
to be changed around that. Yeah, so that's happening. Studio

(36:55):
Ga is definitely a big one. It's something even in
its current form, people didn't realize you could do it
Cypress without the flag, but generating test code like by
just using the UI and doing a few assertions and
things is always kind of magical. So I had on
my radar was like I want the tab key and

(37:16):
I want studio ga. Like when I started to work here,
these are the things that I thought would be really useful,
and so we had those things.

Speaker 2 (37:24):
Obviously didn't do.

Speaker 4 (37:25):
Them day one when I started here, but I'm happy
to see them created.

Speaker 3 (37:28):
I can't imagine and we have a public roadmap.

Speaker 4 (37:33):
So what I when I didn't work at Cyprus, I
used it and I didn't really like think too much
about it. So I think some people might have this
the idea of Cypress as the kind of small desktop
app that it was, you know, prior to Cypress ten,
where you just have a kind of list of specs
and you go in. But really the application itself that

(37:54):
runs locally has gotten more advanced than mature as well.
It knows how to talk to your remote cloud instance,
if you have those results there to give you local
things like that. And yeah, there is a public roadmap.
There is a GitHub like project that you can see
and that has what's happening with the app itself. So

(38:17):
a lot of the work that's happening right now is
sort of the ongoing maintenance of a complex modern web application,
where we like have testing changes, like how you drive
Firefox with this kind of new by Dye protocol compared
to the CDP that they had before. So we've shipped

(38:37):
things like that. And I think one of the other
limitations people might have been used to about Cyprus was
same origin testing that you kind of had to keep
each test within a single origin.

Speaker 3 (38:50):
I remember that, yeah, and that was that was.

Speaker 4 (38:53):
A part of how Cypress injects itself into the dom
and the permissions in everything. So in the last year,
maybe two years, we shipped the side up origin command
so that you can safely move in and out of
different parts, and so that's a typical limitation that's gone.

(39:15):
There is still something people remember around like new windows
and taps again because Cyprus runs in the browser.

Speaker 3 (39:22):
Or I frames I think, or no, right, yeah, I think.
So that's where you're going.

Speaker 4 (39:28):
Yeah, there's some workarounds for EE frames, and I know
that it's an exciting new thing that we would like
to solve on.

Speaker 3 (39:36):
The test frames were but you heard it here. First.

Speaker 4 (39:41):
The direction I was going was like, we have more
official support for like, hey, you need to test something
in a multi tab and the existing workarounds don't work
for you, which is really a puppeteer integration to go
and do some stuff in that other tab like live
and then manage back.

Speaker 3 (39:58):
Okay, cool.

Speaker 4 (39:59):
And that's an experimental release. So if people were like,
we want to do something in Cyprus and we have
this restriction we know about from like two three years ago,
most of those it's worth checking what the updated state
is because we've made progress on them and they're all
like benefit tremendously from feedback and use cases in the GitHub,

(40:20):
like being able to talk to us about the importance
of for example, pressing tab in a test that was
our most uploaded GitHub issue, and so solving that solved,
like what kind of GitHub was telling us was the
most important thing next on the roadmap, And people gave
us lots of examples of why they need it, and

(40:41):
then also why the workarounds of the plugins were starting
to fall apart, as like not necessarily doing enough so
that open source feedback gehub actions process is like phenomenal
for people who are trying stuff out, and we do
look at everything and we account for it as much
as possible in the product plan.

Speaker 2 (41:00):
I have one important question, at least it's important to me.
One thing that I love when talking to people that
work on the things. I would love to know a
feature that if you I mean kind of you're in
the position to make those kind of changes, but if
you could now press a button and say, Okay, this
is the next big thing Cyprus is going to work on,
no other constraints and everything. I know this is not

(41:23):
how the word works, but I still think it's interesting.
What would that be for you? Like, what would you
love to work on right now?

Speaker 4 (41:30):
It's funny because I haven't thought about it today and
like yesterday it was tab key.

Speaker 2 (41:36):
Yeah, you mentioned that, like those were your two big things, Well,
those are.

Speaker 4 (41:40):
The things I really wanted to see. The thing on
my mind the most is like making the accessibility process
more effortless and putting some more stuff from that into
the open source application where sort of like the studio model,
where we can make the experience of not being connected

(42:02):
to the cloud better, like give you more accessibility feedback
without plugins or something like that, and then also driving
that connection for the people who are on the platform
with like what your local app is doing as you're
testing and developing, and the things that happen in the
cloud that give you data about coverage or accessibility or
like the diff between two runs, what's changing. That's like

(42:26):
become a really important part of all this.

Speaker 6 (42:28):
So I would Yeah, my answer is probably we would
pile more accessibility fun stuff into the app to help
people fall into the pit of accessibility success by having
information without having to ask for it and know they
need to check these things.

Speaker 4 (42:46):
It's one of my favorite things about what we do now.

Speaker 2 (42:49):
I love that goal because like writing accessible applications is
really difficult and you need to learn a lot and
like readspects and stuff, and it's not fun. So if
you can shove it in people face, I think that
would be beneficial for people.

Speaker 4 (43:02):
It would be my favorite and part of the reason
I care about this so much. So I came to
Cyprus like having done a career change, but previously I
worked for seven years at a developmental disability support nonprofit.
So I'm all about independent users and accessibility and the
actual effect it has on people is sometimes incredibly outsized

(43:24):
for the effort that it is. So it's like, yeah, why,
it's why I'm so psyched about all these things we're
talking about.

Speaker 3 (43:30):
I can see your passion coming through from that experience,
that life experience. I think that's really cool. Mark. I
think one of the very short story, one of the
first times I ever saw somebody using this is many
many years ago, fifteen twenty years ago. I had a
friend who took care of a disabled man, really nice guy,
but he would navigate. He was like paralyzed from the

(43:51):
net down. I think, I think maybe on a car
accident or something, and he would navigate like his chin
and then he could move his thumb and like click,
and I remember watching him like just navigate through websites
and applications, and there were the clear distinction for me
as a very junior. I mean, I I don't think
I even had cracked into my profession. I think I
was still in my like undergrad for computer science. This

(44:14):
was a long time ago. This would have been like
two thousand and three or two thousand and four. I
know that day's me. I know, don't look at me
like that, yng Mark. I see some gray the beard
and you're probably like, yeah, boy, those were good days.
But anyways, I remember watching him and like it was
a clear and I would help him with stuff because
that was a computer guy, you know what I mean.
He would ask me to help him with stuff. Really

(44:35):
nice guy. I remember like the frustration. I remember watching
his frustration when he would go to some websites or
some things that were not accessible and he would just
kind of get like ugh, like like angry or whatever,
frustrated that it was like difficult for him to navigate
through it. And then other ones that were really easy.
I mean, he could just like fly through it because

(44:56):
the tabbing was correct and the dropdowns made sense. He
was able to like get around and stuffing like he
wasn't blind, but he was able to navigate the page
quite effectively using the little bit of inputs that he
had compared to with without like without that accessibility. And
so I mean, maybe just to like pile on to

(45:17):
your life experience, my tiny little bit of life experience
showed that, like it actually does matter and it affects
people's lives. Accessibility does. And so anything that we can do,
I think is developers to kind of keep that in mind.
Is we're building apps, because we're building apps for real people,
and some of those people have different abilities or restrictions

(45:37):
around their like how they interact with that application. And
so I just really appreciate your sharing that with us
and all of the things that you're teaching us on accessibility.
That's really cool.

Speaker 4 (45:48):
Yeah, it's a great example, and I realized we probably
didn't define accessibility up top as regards like what the
purpose of it is in software. I think most developers
are aware of the topic now. But you give a
good example of like the independence and effectiveness of someone
being able to experience and use a website based on
they have a disability. And then it's down to like

(46:10):
at least a huge chunk of it is just down
to what HTML did we ship?

Speaker 3 (46:15):
And I think about Yeah, I.

Speaker 4 (46:18):
Think about these massive machines that we kind of create,
like the whole process of serving a website and the
databases and all the things that go into it, and
we're just we're getting HTML to you at the end
of the day, and like the the sometimes the lack
of attention to the quality and nature of that HTML,
which is really how our product interacts with you and

(46:41):
is the whole purpose of what we did everywhere else
in the software development and like serving everything process is
like we might as well send you really good, high
quality HTML that works for everybody's needs because we worked
so hard to get it to you and to get
you as a customer, and like these things are really good,
it might as well be good.

Speaker 3 (47:01):
Yeah, that's a really good point. Yeah, I think that's
a call to action for front end developers, you know,
your DBAs and your you know whatever. Maybe they don't
they don't hold the keys to that, but we do
as front end developers and engineers, and so were the
ones that can focus on that.

Speaker 4 (47:16):
Yeah, the product people, the designers, like just being aware
and thinking of the implications of all that, Like those
are great conversations.

Speaker 2 (47:23):
I do have to say, I think that was a
perfect like full circle for this episode. I don't think
there's any way we can make this like any better
from here, so I think we should slowly fade out.
Thank you Mark for joining us, and well we mostly
talked about Cyprus, but I think we made a pretty
noticeable tangient also to accessibility, So thank you very much

(47:43):
for that. I really appreciate you.

Speaker 4 (47:45):
Yeah, appreciate the chance to come in and chat to you.
I enjoyed it very much.

Speaker 2 (47:49):
One last thing before we go off. What is a
good way for people to reach out to you if
they have questions about Cypress? If there are any way
if people have a stack of money floating around that
they would spend on some testing tools.

Speaker 4 (48:04):
Yeah, the first way to reach me is like LinkedIn
is perfect. Just to connect with me there and they'll
be able to do things. There's also discord where we've
got a really active discord of like fifteen thousand or
so people, maybe more now. So if you have specific
Cyper stuff and you're like I want a person that
will respond to me, you can ping me on there

(48:26):
and I will usually whether I'm supposed to or not.
In my day, I'll usually come and look at what
people send me on discord, so I try to be
helpful there, and then on the Cypress docks and Cypress
Idio as well. We have the product sections that explain like, hey,
here's the cloud thing we have now which people are

(48:47):
using for so much more than you might have expected
based on you know, a couple of years ago, even
when it was mostly about the management of tests and
you know, running them in parallel and canceling them, you know, efficiently,
and things like that. All that stuff is still there,
but it's like a much different type of experience with
I think more to offer now. So you can just

(49:09):
look at our website to check that out. I don't
have more to pile on there.

Speaker 2 (49:13):
Well, again, thank you so much. This was really great.
I really appreciate you.

Speaker 6 (49:16):
Brian.

Speaker 2 (49:16):
Any lost words, No, that's it.

Speaker 3 (49:18):
Thank you so much for coming on the show. Mark
it was a pleasure, And to the listener, go check
out Cypress if you haven't in a while. It's definitely
continues to get better and better and start writing those
end and integration tests that you've been putting up for
so long. I think you'll be thankful, And I love
that Mark talked about the component testing because I think
there's a lot of there's a lot of opportunity there,
especially as you mentioned the code reusability between your end

(49:41):
end tests and your component tests, and so yeah, check
it out cypres dot io and follow Mark on LinkedIn
and connect on the discord perfect.

Speaker 2 (49:52):
Thank you all.

Speaker 5 (49:54):
Hey, this is Preston Line'm one of the MG Champions
writers in our daily battle to crush out code. We
run into problems and sometimes those problems aren't easily solved.
Ngcomf broadcasts articles and tutorials from Angie champions like myself
that help make other developers' lives just a little bit easier.
To access these articles, visit medium dot com, forward slash, nngcomm.

Speaker 1 (50:15):
Thank you for listening to the Angular Plus show in
Chicoff podcast. We would like to thank our sponsors, the
NGCOFF organizers Joe Eames and Aaron Frost, our producer Gene Bourne,
and our podcast editor and engineer Patrick Kays. You can
find him at spoonful ofmedia dot com.

All Episodes

S9E5 | Let's talk about Cypress | Mark Noonan

Episode Transcript

Popular Podcasts

New Heights with Jason & Travis Kelce

The Breakfast Club

Fudd Around And Find Out

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}S9E5 | Let's talk about Cypress | Mark Noonan