Safer Agent Automation with GitHub Agentic Workflows

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Bret (00:00):
Say something's gone wrong in your repository, you don't wanna have

(00:02):
to get up in the morning, and say,oh my God, something's gone wrong.
Should I have coffee orshould I investigate it?
You shouldn't have to make thattrade-off because the agent should
have already investigated it for you.
So you should be able to just go toyour breakfast, have your coffee, read
the report, read the analysis, andit says, here's a possible fix for it.
Here's actually meta pool request for it.
And you go, oh yeah man,uh, we, we gotta fix that.

(00:22):
Let's get that in.
it should be there, ready for you.
the agent should be proactiveand should be immersed in a
world of cooperative agents.

Bret (2) (00:30):
Welcome to the Agentic DevOps podcast, and I am your host, Bret Fisher,
back with another episode about one of mymost exciting things that I'm working on
this year, the project that I am adoptingon GitHub, and I am trying to dig into
the weeds as much as possible becauseI think that GitHub is onto something.
Specifically GitHub Next and MicrosoftResearch, which are both the research arms

(00:54):
of both those organizations, and they'reworking together to evolve GitHub Actions
to what it would be if it was AI native.
What would it be if AI was there in asafe and reproducible way that we could
sandbox and protect in a, in a verydetailed and heavily scrutinized way?

(01:15):
And it's not what you think.
It is not simply just addingLLM prompts into GitHub Actions,
which you could do today, andyou've been able to do for years.
It's not simply just adding ClaudeCode as a step in your GitHub Action.
Those are things that already existed.
But out of GitHub Next last year,we heard about the early alpha beta
release of something called AgenticWorkflows, which is technically what I

(01:37):
would call a feature of GitHub Actions,but it's a whole website now with tons
of examples, a team working on it.
And when you really dig into the detailsof what this is, I think this is the
only way we should be doing anythingwith an AI inside of GitHub Actions.
If we are prompting in GitHubActions, they should be using

(01:58):
this tool, in my opinion.
As I dig more into this, and thenthe recent Claude Code security
concern that we had, and then allof the security concerns we've had
around GitHub Actions lately relatedspecifically not to the supply chain
per se, but specifically to workflowsthat are getting basically prompt
injected through untrusted prompt input.

(02:19):
That is one of the biggest risks anywherewe put a model, whether it's in a
chatbot or it's in our automation orit's in front of some of our systems.
If someone can put untrusted text in aplace that somehow doesn't get verified
by a trusted member of our team beforeit goes to a model, that's a risky

(02:39):
place to be, and that's part of our job.
Platform engineers, DevOps,security engineers, like we're all
very concerned about that, right?
We're maybe a little bit trepidatiouson what we should be doing with these
things today, which is automating moreof our systems because we don't wanna be
that person on the team that gets caughtputting risky AI stuff into automation.
'Cause as we know, automationcan make good things really easy.

(03:03):
It can reduce toil, but it can alsoautomate the bad things if we're not
careful, and you can sometimes automateyourself all the way into an outage.
So we have on the show thistime Don from GitHub Next.
That's their research arm full of PhDsand experts that are trying to figure
out tooling for the future of GitHub.
Personally, I just love GitHub Next.
I'm always on their website,githubnext.com, and looking at what

(03:26):
they're working on because to me it'slike reading the tea leaves of where the
big money is researching for the futureof software development life cycle.
And we also have Pele from MicrosoftResearch, who's also on the team building
this product And using it daily, whichwe get into exactly how they use it.
And I'm really excited because I feellike this system has a lot of rigor

(03:47):
to it, which is what I'm looking for.
And when I'm thinking aboutimplementing AI anywhere in my
automation, especially when it comesto CI/CD automation, possibly anything
around my code repos, I'm wantingthat to be as secure as possible.
So I look to make my steps deterministicwith traditional programming, and then
only as a last resort when I need ajudgment or a, maybe a consolidation

(04:12):
of text or a summary of text, that'swhen I consider putting in models.
And I used to just attach Claude Code,maybe build it a workflow around that,
or attach Codex or Copilot as a step,but this is something totally different.
You use command line tools to generatesomething that is a very long workflow.
You use CLI tools to create a lock fileto make sure that that doesn't change.

(04:35):
You establish rigor around makingsure that the prompts and the
things that you're trying to createwith AI are trustworthy and can be
sandboxed and protected properly.
And I feel like the depths that I'vegone into after this show that we're
gonna get into in a second, I feellike I now can say this is how I'm only
gonna be doing the AI in GitHub Actions.

(04:56):
So welcome to the show Donand Pele, and let's get into
it.

Bret (04:59):
Don tell us who you are and how you got here.

Don (05:01):
Bret, thank you so much.
the introduction is through ourmutual friend, Ben, who works as a
product director at GitHub Actions,and yeah, we are having a blast.
I work at GitHub Next, and I am nowworking on, I guess, what we'd call
ai, agentic DevOps so continuousai, agentic software, automated
agentic, software engineering,and lots of associated topics.
And yeah, I got a long background inkind of programming language design

(05:25):
and runtime design and done a lotof product delivery over the years
i- in Microsoft and other places.
A lot of DevOps along the way, andyeah, I've been at GitHub Next a few
years now, and I absolutely love it.
Yeah, we got a charter to investigatethe frontier of software development,
and there's never been a more excitingtime to Mm-hmm … be doing that.
A time where things are so nascentand things are in formation and in

(05:47):
change and, turning upside down,And you're really able to kind of
radically rethink a lot of things.
I mean, a lot of it's, turnsyour head upside down, just
how much things are changing.
But, you know, I, I'menjoying that very much.
It's

Bret (06:00):
a good moment to be here, I think.
I keep reminding myself, I don't know, acouple times a week, it feels like that
just reflect on the fact that you're inthe middle of this and that, like, when
you look back on it, these are the storieswe will tell and like, be present because.
When I look back at the cloud and thepc, the mainframe, to PC migration, and
like all the sort of large infrastructureevolutions that I've been a part of,

(06:20):
you know, when you're in the middleof it, it's, I was too young to
realize that it was of significance.
and as I've grown older, I've startedto see the patterns and so I can
sort of feel like I, this is it.
This is a thing.
It's, this is exciting.
this will not happen, thiswill not be like this forever.
and we will look back onthis as, wow, that was crazy.
so Yeah.
yeah.
Yeah.
Welcome.
I'm glad to have you here.

Peli (06:38):
my name is Philippe Duhaleu.
I'm a engineer in MicrosoftResearch in a group called
Research in Software Engineering.
We're also very interested in thingslike verification testing, uh, all
this kind of stuff over the years.
And I had been looking at, LLM Automationback in the days before agents, and
got in contact with Don in GitHub Next.

(06:59):
And, we decided to kind of worktogether on this idea of continuous ai.
This kind of started this project,and that was a year ago, roughly.
Uh, but we had been working withDon from a distance from Microsoft
on various projects It's--we've been around for a while.
Yeah.
yeah, so that's the intro.
I've been working on developertools for professionals, but

(07:19):
also for kids for a while.
I've built, K12 coding platforms.

Bret (07:24):
Nice.
Yeah.
Is there an analogy between a K12,learning platform and agent harnesses?
It is, it
is

Peli (07:30):
There's a lot of things that people don't realize in the way it's designed.
It is designed as a sandbox,just like I build the coding
infrastructure for Minecraft.
Mm. So when kids learn to code inMinecraft, we, so there are some design
patterns that, that are applicable.
Uh, you know, these agentsare finicky little monsters.
so we gotta talk a lot about kindof sandbox design and API design and

(07:50):
things we do under the hood to makeit more reliable when people say,
"I know, agents aren't reliable.
or- Right.
So we do a lot of work underthe hood to remove that.
But also we talk a lot about safety.
' cause when we have a real sandbox, youcan't just experiment You can build
castles, without destroying the world.
So there's a lot of that is baked in.
but it's made for professional, it'smade for DevOps, it's made for people
who are actions users and stuff.

(08:12):
Yeah.
But yeah, it's uh, actually, it happened,I didn't realize I, I was designing a
system like that and then after the factit's like, whoa, this kind of feels like
I've rebuilt a system that I've builtfor so many times for that environment.
yeah.

Bret (08:24):
Okay.
So it like it became apparentafter it was happening, yes.
and I, I should back up because I think,uh, we got connected because I heard
about Agentic workflows last year,when it was before, GitHub Universe.
And it was like early betaor maybe even before that.
And I think I was like requesting earlyaccess to get in because it was right up

(08:44):
my alley and I had no idea what it wasand just knew that, uh, at the time I was
building a GitHub Actions course, stillam recording videos, uh, week by week.
And I've been a big GitHub Actions fanfor, gosh, at least five or six years,
since before we had reusable workflowsand a lot of the niceties we have
nowadays ' cause I really kind of sawthat as the evolution of the everyone's

(09:05):
CI almost kind of what Jenkins representedin the industry for so long, where it
was sort of the default and I was soanxious for the, another winner to,
to replace Jenkins 'cause I was tiredof maintaining the infrastructure.
And then, you know, got onto GitHubActions, saw the advantages of having it
built right into where our code storagewas and where everything else was.

(09:27):
And that just really fit, Ithink with a lot of my clients
and a lot of my community.
And so over the years, uh,GitHub Next, which we should
probably talk about what that is.
I've always looked at GitHub Next, youknow, a couple times a year to try to see,
try to re- see the tea leaves that you allare reading and figure out where, what's
next for GitHub, what's the excitingthing that might come to fruition?
Whether it's a font, uh, fontpackage, which is my favorite fonts

(09:49):
that I use everywhere today or, somefeature in the UI of GitHub that
was considered a new experiment inhow we represent the information.
And it's been fun to watch that.
'cause y- you feel like you'rekind of seeing the skunk works
or the, like the, the deep thinkof an organization, in public.
And that's been a really cool thing thatI don't think a lot of people know about.
Obviously Microsoft research is thesame thing that's been around for

(10:10):
decades, but, um, uh, it was coolto see that so close to the coding
platforms that we're all using every day.
So last year, if I'm settingthis up correctly, there was an
announcement around GitHub Agenticworkflows as an idea, and then
you announced it at Universe,
And then since then, it, I feel like it'sbeen like off to the races, incredible
documentation, slide decks that areavailable that you all are putting out.

(10:31):
and there's a lot of information thatI think has to come along with this
because it's not an obvious evolutionnecessarily of how it's not just like
one little feature added to a workflow.
So who wants to take up the mantleof describing, elevator pitch,
the agentic workflow concept?

Don (10:49):
I mean, I'll just run through the kind of three principles, right?
One is the idea of agenticrepository automation.
You know, uh, we, the… when peopletalk about agents, I think there's been
something missing in the conversation.
Okay.
Which is about, you wantagents that are proactive.
That's what I think ofwhen I think of an agent.
I don't know where the ideacame in around the world.
That agent was something you invokedfrom your chat session, right?

(11:09):
I don't know who invented that.
because when I think of theword agent, I think of something
that kind of is there around me.
It does stuff.
It knows the, it knows of What'shappening in the world around me.
And in the context of a repository,that means it kind of knows what's
happening in the repository.
And, when I think about things like, Iwant to, refactor my code so that all
the files are under a certain size, Iwanna check, uh, my error messages to

(11:29):
check the language is age-appropriatefor a particular target audience,
or hundreds and hundreds of otherthings I wanna do in my repository.
I, those are things Iwanna do continuously.
They're not something Iwanna just do once, okay.
I don't wanna have to stand upevery day and kind of say, oh
my, we had some error messages.
Gotta re reestablishthat kind of principle.
that the error messages are allin, in good shape or whatever.

(11:50):
You know, A GitHub repositoryfundamentally is a continuous growing,
evolving, collaborative kind of space.
And it, it's got a history that's veryinteresting and long from wherever
it came from, and it's got future.
and the agents, uh, the, anythingthat happens in the repository
has to be established tohappen on a continuous basis.

(12:11):
It's gotta be able to be adaptive towhat- to the change that's happening in
the, in, in the, re- in the repository.
If you're gonna have AI workingin a pull request is also a
continuous kind of object.
You know, it's gotta kind of go with you.
the journey all the way throughthe repository's a journey.
And we've been digging, to methat's the heart of what GitHub is.
it's like that's just central.

(12:32):
That's why it's the place whereeverybody comes to work together.
It's why continuous integrationand continuous, uh, deployment,
make sense there and.
So if you're gonna do agentic working, youwanna establish it on a continuous basis.
we all know the examples.
here, continuous documentation,continuous code improvement,
continuous fault analysis,proactive, kind of fault analysis.

(12:53):
You don't wanna have to,

Bret (12:54):
say something's gone wrong in your repository, you don't wanna have
to wait, you don't wanna have to kindof get up in the morning, and say,
oh my God, something's gone wrong.
Should I have coffee orshould I investigate it?

Don (13:05):
Right.

Bret (13:05):
You shouldn't have to make that trade-off because the agent should
have already investigated it for you.
'cause you, the flow is toinvestigate a fault in CI or a
fault in your website or whatever.
there's where the signal,the fault signals are coming
from the flows are encodable.
you know, all the steps you wanna do.
You've probably written them out in yourMm …kind of documentation and so on.
So
you should be able to just go toyour breakfast, have your coffee,

(13:26):
read the report, read the analysis,

Don (13:28):
uh,

Bret (13:28):
and you know, it says, here's a possible fix for it.
Here's actually meta pool request for it.
And you go, oh yeah man,uh, we, we gotta fix that.
Let's get that in.
Right?
So it should be there, ready for you.
the agent should be proactiveand should be immersed in a
world of cooperative agents.

Don (13:44):
And when you think about that kind of vision, there's,
that's quite close to the idea of.
uh, Assistance in your repository,sort of a virtual team in a way.
They're not humans.
Okay.
We, I don't like theanthropomorphization, but they,

Peli (13:56):
yeah, it's hard
not

Don (13:56):
to though.
they're gonna

Bret (13:57):
be

Don (13:57):
It's hard.

Bret (13:58):
to, yeah.

Don (13:58):
Yeah.
And, and there's so many timesin my working life I've wanted
assistance in various repositories.
I think, there's so much, you know,I, I, there's, I don't know how to
do really good engineering uh, in,in various dimensions, in every
repository I have to kind of touch.
Uh, so bringing in thisproactive assistance.

(14:20):
I know.
And so, when you thinkabout it, where are we?
Let's make that real.
We're in, sitting in 2025,and let's make that real.
What's the ideal experience?
What's the most, let's platformfit that onto GitHub as a concept.
And you go Actions actually, yeah.
Actions has got a lot of trouble.
Yeah.
We, we, you know, you start thinking,well, we automatically generate

(14:42):
some big complex YAML, which kindof is like compilation of some
intent or something like that.
And then you go, then you realizeactually, no, I just wanna run
those amazing coding agents, whichwill also appear around that time.
I just wanna run them in actionson a kind of continuous basis.
Okay.
And what would be the idealsimple, let's make that simple.
Let's make that as simple as possible.

(15:04):
That would be Check in a markdownfile, which describes your prompts.
and It runs and you have, check and itand give some triggers like in actions.
Everything just works.
That's the kind of core idea ofGitHub agentic workflows is to make
it, to capture the simplicity, thebeautiful simplicity of GitHub Actions.
YAML, you know, people have got allsorts of complaints about GitHub

(15:24):
Actions, and YAML and so on, butit's got a massive advantage, which
is, it is damn really simple to use.
You just check in a fileand you have magic, right?

Peli (15:35):
Yeah.

Don (15:35):
And that's uh, existing's ya YAML A- and one way that people put
it, I think Ben puts it this way, isthat actions is a way for a- average
developers repository maintainers toget access to some cloud resources.
They get access to cloud computenetworking and some storage
in the context of a repo.
And they don't have to go ask thesecurity team a permission grant

(15:58):
has been given by the company thatdecides to let you use Git, GitHub,
or your open source repositories.
You get this amazing playgroundand it's not just a… It's a, it's
actually like a factory space, right?
It's actually like, it's got storage, it'sgot network, it's got so much compute, and
these days it's now we're bringing kind ofcoding agents into that kind of factory.

(16:20):
And, uh, so that's the simplicity of it.
You're running through the kind of,uh, add wizard kind of thing here,
which is kind of, uh, saying, you know,yeah, helping you, uh, set up your
tokens as well, uh, in, in the video.
And it's, it, it says, would youalso like to trigger the workflow?
And then bang, you've got your firstrepository, automation running, and, uh,

(16:41):
that thing I think is set up to run daily.
Uh, you can kind of change it to run,weekly on different kind of schedules,
uh, different triggers and so on.
And bang, you've just got, youjust made your, brought the most
powerful coding tools into yourfactory, onto your factory floor.
And the, the potential is limitless.

(17:01):
Now.
You have got all the power to automateeverything, and creative things,
analytical things, uh, problem-solvingthings, code improvement things.
everything is set up, in, in,in the GitHub repository now.
Okay.
So that's the vision repository.
Automation with the codingagents you know and love.
You can use Claude.

(17:22):
You can use copilot, CLI,you can use Gemini, CLI.
And I think we're adding some more.
codecs as well.
And, uh, we, we will take, uh,proposals for new ones as well.
So that's it.
Repository automations with thecoding agents, you know, and love
safely with strong guardrails.
We'll get onto that in GitHub Actions.
That's the formula.
Nice and simple.

Bret (17:41):
Nice.
That's a good elevator pitch.
one of the challenges I think in all of,in all, like, like just if we think of
the entire software life cycle on GitHub,I've always experienced this with every
piece of software I've implemented as sortof a DevOps ops person, is anything that
I put in, even if its goal as a productor a tool to, to automate something.
Like even putting in GitHub Actionsas a thing to help me automate

(18:04):
traditional, bef- before we had agents.
There always was an addition, the,a, a cost of toil that was added on
top of that, that I could not escape.
And when I teach GitHub Actions, a lotof, you know, it's obviously there's,
there's the day zero creation of theseYAMLs, but the day two stuff is arguably
even more important because it rarelygets discussed, like in, in getting

(18:26):
started guides and stuff like that.
And so over the years, GitHub Actionshas added functionality to, you know,
reusable workflows and templates and,the GitHub repo op opportunities there.
And there's, there's been a certain levelof trying to manage this giant beast that
we've all created when you have hundredsof repos, and now that means possibly
thousands of YAML files to manage.

(18:47):
And, uh, you know, I hate to, I don'tlike using the word at scale, I think
like it's way overused and a lot ofus aren't even dealing with scale.
We're just small teams.
but I always felt like there was thischallenge of, oh yeah, we're gonna
implement this CI tool for you, but youprobably need someone's, you know, half
a half a day, a week, maybe, or moreto, you know, fix the broken workflows,
to update the GitHub, or approve thePRs for the action upgrades to make

(19:12):
sure that the Dependabot is configuredcorrectly with all the latest stuff.
And so there's a lot ofthis sort of hidden toil.
And I was gonna throw the question to youof, do either one of you see this future
where it, it feels like there might bea possibility here where we can add.
We can add features or functionalityor, or, or solve problems without
adding additional toil, but alsomaybe that toil might be managing

(19:34):
the agents, like up updating theagent skills or the configuration,
or like, that becomes the new humantoil layer that we have to maintain.
Or do we think that this isthe, the gain is far exceeding
possibly the, the toil involved?
Um, with that, what do you,what do you think about that?
You-

Peli (19:51):
So essentially no, we compile down to action.
Yeah.
So our markdown becomes an action.
So we inherit all the toil today, Todaywe have all the toil that you have to do.
However, we're working closelywith the GitHub action team,
and we're looking at that toil.
we're looking at the sources of that,and we're looking for solution to get
rid of these, you know, the requirementto, oh, there's no way to push an

(20:12):
action over an entire org, you know?
Mm-hmm.
You always have to pushfiles, things like that.
It's like, can we do make this better?
Uh, so we're looking at theseproblems where you start to have
hundreds and thousands of repos andeverything becomes a scale problem.
so there is definitely work being therefor the action product itself to be able
to fix those oh, you mean But in general
sort of a

Bret (20:31):
deterministic level, like something that's just a feature of GitHub is a
deterministic feature of action to be just

Peli (20:37):
yeah …better at this.
But aside from that, we can go and.
Attack any toil and automate it, anything.
Because now we have, we have a, wehave the ultimate reasoning hammer
that we can just point at a problem,just go systematically on every
repo, do some reasoning, right?
You can take, in fact, any deterministictool and wrap it into, with an
agent and point it at any repoand, you know, hope for the best.

(21:00):
It might give you a result.
But, let's say your Dependabot thing, it'salways slightly different, For every repo.
But now you have an agent that actuallypretty good at dealing with that.
So you can go and, and run campaignsover your repos and, and fix complying
issues and things like that, thata deterministic tool would fail.
These agents are able to go and, and Ithink that's what you mentioned by toil.

(21:20):
Yeah.
Or I, yeah, but we also inherit Yeah…the, the limitation of actions today.
because, we're working on that,but, you know, that takes time.
but

Don (21:28):
Yeah …it's on

Peli (21:28):
our radar.

Don (21:29):
I got a different thing for the toil.
Mm. It's kind of like, yeah.
it does take time, take costs something tokind of create automations that take kind
of cost something, costs you personal timeand there's ongoing maintenance time for,
to create existing, uh, GitHub Actions.
YAML.
Couple of things kind of change a bit.
One is that because we'redealing with coding agents, it's
possible to create extremely um.
You might think of it as ambiguous,but it's actually kind of

(21:51):
general, kind of, um, workflows.
So you can kind of create, oneof my favorite workflows is to
do with, repository maintenanceand it's called Repo Assist.
and, um, and it's a multitaskworkflow and it kind of, each day
it's got like 11 tasks I think, andit kind of rolls the dice each day.
to say, Hey, what am I going to do today?
Okay.
So, because you don'twant it to do all of them.
because it, you know, Yeah … only do thefirst three, so it just rolls the dice.

(22:13):
Just to kind of getthings nice and balanced.
One of them is, for instance, tolabel issues, to just to make sure
all the issues are, can check mostrecent issues It can look at a backlog
and just make sure all the labelsare kind, you don't have to write
that out algorithmically, right?
You just kind of tell it the end statethat you want and maybe some hints about
how to get there, about what the journeyis, it has to make and, uh, you know,

(22:36):
pretty much what I just said really.
And it'll work out allthe rest of the details.
Uh, it, you know, in the sense thatthat's the prompting that will be
running each day or every third,fourth, fifth day or whatever it runs.
Uh, and, um, it sorts out the rest in anyrepository for any set of labels for any.

(22:57):
Language, you don't have towork in English, you could
be working something else.
So whereas in a traditional setting, youmight have had to configure the exact
labels to use, configure the kind of,uh, the, the, the, the kind of heuristics
to use to label issues and so on.
In this setting, it can bemade to work everything.
And that's really golden.
'cause that means you have thesevery generic workflows, which can

(23:19):
be used in many different settings.
I mean, other Yeah …tasks in that repoare things like take, uh, work through the
issues and just analyze them one by one.
not all in one run.
just do a bit of that frontier work, uh,uh, and, and do a depth investigation,
do a reproduction, and give some adviceabout what to do about this issue.
And it's sort of what you'ddo with a coding agent today.

(23:40):
You might check it out locally andinvestigate the kind of thing, but
it's all done for you, proactively.
Um, and again, it's very general, right?
You don't have, it's the sort of thingwe could never have programmed up
two or three years ago because it'samazing, you know, these things Yeah
these agents do.
But it's also very general and thatmeans the burden I use repo assist
in, I think, um, 12 different repos.
They're all different and I haven't hadto change it really between any of them.

(24:03):
So you can have these very generic,powerful, tools which help you
make progress, uh, expressed atthe right level, low enough that it
actually kinda knows what it's doing.
It doesn't just do something canned.
but general enough, it's reallyapplicable to everything.

Bret (24:16):
I'm starting to see enough teams that are tiptoeing into agents
as, in their CI, essentially, right.
P- Somehow, somewhere sticking a modeland writing a prompt in some fashion to
a model, whatever we wanna call that.
And they've, I think they, a lotof the teams that I'm working with,
see, the first experience is sort of,it's a checkbox feature in GitHub.
Like they might turn on the PR reviewagent for copilot and it's sort of an o

(24:40):
on/off thing, or you, you can opt in toeach PR and it just becomes a feature.
So it doesn't really feel, I mean, evenit's providing automation and obviously
it does, you know, it puts commentsin the PR it does all these things,
but it, it's not something that theyhave to hobby kit implement, right?
They're not writing a YAML filenecessarily on day one of that.
So I feel like that's the first phase.
And the second phase is where.

(25:00):
They wanna actually ha- you know, seeeverybody else putting Claude, code into a
CI run, or they, you know, they, I noticenowadays it's actually getting pretty
rare to look at the, uh, maintainersor, or the contributors to a repo.
And that Claude logo is alwaysthere, it seems like nowadays.
people are sort of figuring out how theycan either write code or, or review code.

(25:21):
That feels almost like, for me, thatwasn't the most interesting part to me.
So, someone who had to maintain theCI, I was always looking for things
that would, you know, automaticallytroubleshoot a failed check, right?
And try to provide an automated, uh,'cause I, I'm that person who's usually
responsible when the, the checks failbecause the dev team's gonna push back and
say, yeah, there's a configuration issue.

(25:43):
It's not our fault, blah, blah, blah.
And so we're gonna havethat back and forth there.
And I, I've, I've started to describethis, because people aren't even really
sure where to start, and I love thatthe Agentic Workflows website, the
documentation really is starting tocategorize these things into certain
areas where you're finding successand you're seeing the good metrics
coming out of tho- those results.

(26:03):
But I've often tried to describe itto them as just find a place where
there's huge human judgment involved.
that You've previously had to involve ahuman, but it wasn't a deliberate gate.
because a lot of people get nervousabout, I think, the idea of AI in their
CI because the first thing they mighteven, l- or one of their first thoughts
might be, well, I don't want it toautomatically deploy to production.

(26:24):
Mm. And to me that's like, of course.
Yeah.
That's probably the last thing I'm goingto automate Mm. with any sort of, Mm.
uh, model it, you know, that is to me,sometimes a lot of teams, it's a manual
gate we intentionally put there, andso we're enforcing a human stop point.
But there's so much other, that's likeyou mentioned, labels, uh, automatically
labeling automatically, you know, maybeapproving like low risk, patch releases

(26:45):
of Dependabot updates or something.
Like, there's probably somelow hanging fruit there.
I certainly have been a part of teamswhere we've had to implement that
cross repo l- auto labeling, even justsynchronizing back when we didn't have it.
Uh, synchronizing label names acrossrepos, you know, just a lot of sort
of silly stuff that maybe just wasn'ta feature in the product yet and we
were backfilling it with some sortof manual automation that feels

(27:08):
like the ripe, uh, opportunitiesfor, uh, people getting started.
But do you, f- do you frame it that way oflook for opportunities where there's, y-
human judgment that we have today, but itwe, it wasn't because we enforced a human
to get involved, we just didn't have a wayto automate that with a, for each loop.
Is that something that you,you frame it that way for.

Don (27:27):
Yeah, I personally like to begin with uh, a chat about what are the
problems people are having in the repo?
Like what are the actualstruggles that are happening?
Okay.
So, um, in the case of, repo assist,you know, having that chat with
myself, the problem is we've gotan issue backlog of 200 issues that
go back years and years and years.
And every time I come to thisrepo as a maintainer, I don't

(27:47):
know what to do with them.
I don't wanna close them out.
I, 'cause there's value there.
I know there are bugs there, and Idon't want to leave bugs that people
have found lying around this software.
so, my problem is one of kind ofthe burden of being a maintainer,
the guilt in a way, you kind of, youknow, it's not, I, haven't got on top.
I love to get on top of the repo.
I'd love to get it meaning getthat issue count down in the below
100, below 50, below 10, uh, and.

(28:08):
So the flow is designed to help me solvethat problem and reach where I wanna be.
So I kind of like to have thediscussions about like, what
are you trying to achieve here?
What are your goals?
What are your quality goals?
You're after, uh, you know,is performance the top thing?
That's your, your problem?
is sort of, is quality and crunchingout the bugs, the problem is integration

(28:31):
and kind of cross-repo kind of working.
That's absolutely in a lot ofsettings, a ki- kind of problem.
Making things regular acrossmultiple repos is a good example.
Uh, so once you start theconversation about, tell me
what, what's making life painful?
What's Yeah …causing you to lose sleep,then, what causes you to disengage from
the repo or, whatever I don't know, Peli,How, how do you begin conversations?

Peli (28:53):
I'm doing extreme agentic development in the way that I'm
at, what, 400 plus PRs a week?
exclusively through, I mean, a-agentic workflows is written with
agents and the challenge is how do youcreate quality software doing that?
Yeah.
And this involves many, many agents,uh, looking at the generated code,
cleaning it, adding tests, extractingspecs, generating tests from specs.

(29:17):
So there's an entire.
Intricate set of agents that are runningand that are powered by agentic workflows
that are running in the repo and that are,look, you know, it's really exploring,
okay, what is, so we can, we can generatecode at a incredible rate now, but we've
always known that creating a featurewas a tiny piece of the equation.
You had to do test plans, you had todesign features, research, you had

(29:41):
to write documentation, maintain it.
All that stuff is ripe for automation.
and where does the human fit in?
Where does the engineer comein and, you know, say no.
In my, stat, I think 20%of the PRs, I refuse them.
there's still quite a bit ofengineering involvement I do maybe
three up, three intervention per PR.
so there's still quite a bit of steeringfrom the, from me, but I have a lot

(30:05):
of tools that are agentic workflowsthat are optimizing very specific
angle of a code base, like reducingcode duplicates, fixing linter issues.
And you know, I mean these arekind of sound practices that
software engineers have been doing.
Um, Yeah.
and then we go down the rabbithole and we're like, oh, well.
We always fix something.
So now we extract linters.

(30:26):
So we infer linters fromour, our own practices.
your devs are always fixing the same bugs.
Maybe it should be a linter.
then once you have a linter, youhave something that is very scalable
in terms of compute and cost.
And then, but there's, there'ssomething we discover with the
agent they're very meta agents canhelp, agents can generate agents.
And agentic workflowsare an instance of that.

(30:47):
They, the workflows are reallyspecialized tools in my mind.
You know, they use agents andeverything's an agent, so we
have to put different names.
But I build tools, uh, and one ofthe unblocker of this experiment
is that I can go from an idea to atool in five minutes that is running
in the CI and giving me a result.
The first version will be trash, youknow, it will not work, it will crash.

(31:07):
And then there's a self-reporting loop andtwo or three iteration, you have something
that actually creating the value.
and then, you know, add more loops.
You get, you start optimizingand, and saving tokens.
But …that's essentially thekey thing is zero friction from
your idea to an automation that'sbringing value to your project.
That's paramount.
Yeah.

Don (31:25):
Yeah, the repository as the kinda agent foundry, the agent host,
the place where you can just createand deploy your automation and, um,
get it to do everything over, youknow, the fabric of the repository.
Create issues, It can add issuecomments, It can read existing issues.
It can create pull requests, itcan add to existing pull requests.

(31:46):
Those are the kind of, uh,discussions of the other elements
of the information fabric.
But that's, and it can read thesecurity reports, it can look at the
actions, it can look at the CI runs.
And so that's the fabric informationfabric you're working over.
And, um, it, it makes you look at a lotof the stuff that people are doing with
agent harnesses differently becauseyou don't really need to think about
where you put your to-to-do list.
Let's just put it in an issue, right?

(32:06):
You don't need to think, where do youput the output, the analysis of, say
you've got an agent, which is kind of,uh, checking performance every, night,
different dimensions of performance,and running through them, uh, uh, or
checking your, get your, your, yourgetting started guide, reading your
docs, and kind of running through thekind of getting started material and
making sure it's simple and makingsure everything kind of works right.

(32:26):
Those basic kind ofwalkthrough kind of things.
where does it put its output.
I don't.
wanna have to think about deployingthis to some Agent Foundry and
some other platform, right?
I I, it's just gonna run in GitHub andit's gonna write its output to GitHub then
it's gonna create an issue or add, right?
A comment to an existing issue.
Uh, or it's just gonna create a pullrequest and fix the the thing directly.

(32:47):
Depends on the design that you want.
And, th- so it's that it's, you mentionedit earlier with GitHub Actions about
how having it right there next toyour code is just a, a great thing.
And, uh, it's the same with this.
It's like right there on the informationfabric we all know and use it knows how
to use the GitHub information fabric.
really well.
It's all about issues and howto query, them, how to search,

(33:09):
about pull requests and so on.
Uh, so, um, I love that.
I lo- I love that it's operating inmy home, in my, my fact, my place
where we get work done together.
And Yeah …I love, love that Ican dig back to the actions log
and see exactly what happened.
I love Right … when, when theagents of the actions log to
work out what went wrong, right?

Peli (33:28):
Yeah.
The inherent every time you comeback, yeah, every time you come
back to the graph, to GitHub,the engineer can intervene.
The human is back in the loop.
Right.
The agent does somecomputation, creates an issue.
Uh,

Don (33:38):
th- That's right.
It gives that natural place forthe human to be in the loop.
Yeah.
And, and that's, that can beso confusing when you kind of
disassociate from the factory.
you take it to the outside, from the repo.
Uh, and you wonder where's the, h-where's the human gonna be l- in the
loop in this kind of stuff, right.
I'm drawing out.
And the answer's really simple right?
it's in the pull request.
Right?
It's okay if the, uh, the, i- yeah.

(33:58):
Okay.
First of all, it's actions.
What it does to the issue set of issueshas to be really tightly constrained.
And that's where we can startto talk about guardrails.
You know, it's not allowedto delete every issue.
It's not allowed to close every issue.
It's not allowed to comment onevery issue or randomly kind of
write ha ha ha all over the place.
It's got really strong limitsover what it's, what it can do.
It can add a comment to asingle issue, for example.
That's a super strong limit.

(34:19):
Uh, okay.
But it can do that automatically.
it's not gonna boil the oceanor whatever its expression to
let it act on the issue section.
so it's kind of n- it's a big enormousscratch pad for these agents to work on,
right?
Uh, and, th- and that again,it makes it clarifying.
'cause you know, it'snot writing to the repo.
It's not like writing a to do or goals.mdor tasks.md into the repo, like a lot

(34:39):
of people are doing, it's very temptingto use the repo as the scratch pad.
Uh, But it's not gonna do that.
Uh, uh, So it's, it's, we've got ourscratch pad that is issues and it's
got its way of proposing forward actionin the world, which is pull requests.
And, uh, or you can also proposean issue or propose a pull
request, an actual concrete change.
It's very common.
We get the, uh, agentic workflowsto create issues instead of

(35:02):
going straight to a pull request.
Because again, it, g- it's, it kindof divided into a world where there's,
it, is it, it's roughly a, like,is it gonna take one kind of check?
Yes.
Do it thing in the pull request, ordoes it kind of …offer choices?
If it's offers offering choices,then you'd better create an
issue first, because the humanreally needs some guidance.
The agent really needssome guidance about Right.

(35:23):
What's the next step?
But in that, the human inthe loop is really simple.
It is just at digesting what the agentdoes in the issue space, the comments
it adds the options it gives and actingin the pull request space to make a
actual, big, uh, actual change progressforward in, in the code towards the goals

(35:46):
that, that everyone's pursuing together.
I love that.
I lo- I love that I know whathuman in the loop m- means.
I love that.
it just very clarifying in what is,and it's a- also something I can trust.
if all it's gonna do is create apull request and just one of them
is that, it's almost certainlygoing to be useful what it creates.
So at first like, that's great, butit's certainly not, you know, I can

(36:07):
trust, I can sleep well at nightthat the thing is doing things in a
positive direction or certainly notin a significantly bad direction.
So guardrails and security,super, super important.
Do very worried about something you saidearlier, Bret, just to be controversial.
you were saying, run Claudecode or these, uh, coding agent
CLIs directly in GitHub Actions.

(36:28):
I understand the temptation of thatand, uh, in fact, the origins of GitHub
agentic workflows are in that kind ofspace, but it's really dangerous, right?
The- We call that kind of runningnaked, We're kind of Mm-hmm …running
without a security architecture orrolling your own security architecture
or attempting to do the re-analysisof the security architecture.
At every time, every workflow you do,That's really easy to make mistakes.

(36:49):
And you're being exposed to inputs.
You you've got the world'smost powerful coding tools.
And frankly, they can be used for goodand they can be used for bad, right?
And they are running in GitHub Actionswith potentially access to secrets under
the direction of your, of arbitrarypeople working up, walking up to the
repo of people feeding in information.

(37:10):
Okay?
So it's a little bit like youhire an amazing team of people.
They're sitting in an office and you'reallowing people to walk in through the
front security gate and just feed themnotes about what to do under the door.
And they read them.
They go, oh yeah, I'll do that.
You know- And they think those are
"I'll just do

Bret (37:24):
it" … just as important as the boss's notes.
Yeah.
And they'll do

Don (37:26):
any They'll just… You know, I mean, yeah, you can try and box it and
try and tell it's not so important.
And of course you do all those things,but it's still, you kind of want the
security guard at the desk, and youwant the security guard you particularly
want, I mean, you want any outwardaction, any write action from that
team, any act external action in theworld, any information sent outside.
if you're, if you're worriedabout private information leaking.

(37:49):
you want, it's gotta have a securityarchitecture, you've gotta, you, you,
must, if you're gonna run automatedcoding agents in the context of anything
sensitive whatsoever, you must havea security architecture full stop.
That's Do not leave home without that.
in any serious way.
And- And that's what GitHub-the security landscape

Peli (38:06):
is different than what people think is secure.
Mm-hmm.
It is not just running in a container.
Right.
There's a lot more threat.
These things are intelligent.

Don (38:14):
Yeah.
So for instance, running in acontainer, but in our model, we
have some very strong guarantees.
Uh, And, we really care about security.
not just because we're trying to stoppeople getting work done or anything, but
because we think the better the guardrailsyou have, the faster you can run.
Okay?
The faster the automation can go,the better the train tracks are.
The faster a train can go, themore you can ramp it up, ramp

(38:35):
up the TGV up to high speed.
because you trust the rails you're on andnothing's gonna go wrong along the way.
And that's what, that's how it actuallyworks with GitHub agentic workflows.
Now, let's just run through some ofthose guardrailing kind of things.
Perhaps one of the really big ones is thatthe agentic step, the actual coding agent,
the reasoning the… runs read-only, okay.
With just one Mm …narrow safeoutput that it's allowed to make.

(38:59):
Okay.
And that is a reallystrong thing, All right?
That it, we are not giving thesethings, write Access to MCPs.
uh, In our MCP docs, we say, if you'regonna add in extra mcps, you should not.
Give them write Accessto the external world.
You should design these safe outputsinstead, which are very tightly
controll- controlled handover points.
And when you really digestthat, it's running read-only

(39:21):
without access to any secrets.
'cause we go through our gatewayto access, uh, inside the
container and got the ga- gateway.
Those two together, huge relief.
There's no chance it's gonnaleak all your repository secrets.
Right, Uh- right.
'cause it never had access
to

Bret (39:36):
begin with.
Yeah.
It

Don (39:36):
never had access.
And it's got a firewall aroundit for network access as well.

Peli (39:41):
Yeah.
There's

Don (39:41):
still concerns about there, but they're known concerns.
you can calibrate what'sgoing on in that landscape.
And, uh, and I, it's that allows me tosleep well at night when these things run.
When we first started running codingagents and actions, I was worried that we
were creating actions that were gonna behackable, com, subvertible, compromisable,
and, and you do get those people.

(40:02):
Uh, and, but it's this very tightcontainerization, readonly, no access
to secrets and, um, and, and very narrowscope of action in the safe output.
And I, t- together these allow me, I, I'ma guy who worries, Worried about things.
as we should,
And I think that

Bret (40:20):
will- as we should.
I think that's been a hot topic.
In fact, I'm doing, uh, some workshops andI'm actually speaking at a conference this
summer around GitHub Actions security.
And I've also got a, a littleplug for a open source tool I'm
about to release called, uh, GASA,GitHub Action security assessment.
That really just kind of takes thetop 10 things that I see, or the top
dozen that teams I work with or, uh,you know, and I've been studying a

(40:43):
lot of these GitHub action, uh, supplychain attacks and trying to understand
where the core misconfigurations are.
'cause that's really whatwe're talking about Mm.
-a lot in almost all these cases Yeah.
It's really

Don (40:51):
just where's the problem in this seven-step attack.

Bret (40:54):
Yeah.

Don (40:54):
where did that go wrong?
Yeah.
Right.

Bret (40:56):
What, what step in there can we secure immediately and, and
without Mm. consequence or, you know,usually without breaking anything.
And so that's one of the, I, I, Imean- I'm trying to build a little
tool that helps people discover that.
'cause a lot of these things are justlike Mm. features of the platform that
people don't thoroughly Mm. understandthat event in a GitHub action workflow
or that particular Mm. checkbox inthe security settings of Mm. GitHub

(41:16):
Actions in their repo settings, thatthey just, they left it by default
'cause they didn't understand it.
They don't know the caveats.
And, and so I've, go ahead.
I was gonna set you up real quick Yeah.
for this and just say, Okay.
Go, go ahead.
Yeah.
Like, what if people aren't fully awareof what's what we're talking about here?
If you just add like a step in aworkflow and maybe your workflow
today, your workflow before didn'texist, and you add a new workflow to.

(41:39):
Take, to assess an issue, right?
Like you they- Issue triage.
you need, you need some automation aroundthe title, and maybe for it to, you're
thinking, I wanna have a model intuitivelys- select the label based on what it
sees in the description, and the title.
And so we've seen some m- disastrousoften cases where people just, they add

(42:00):
a step to a workflow, they put a promptin there that basically says, please read
through this, and then decide the label.
And here's my GitHub, uh, you know,workflow, my action token out of the
gate by default, which may or may nothave full privileges to the entire repo.
and you think that "Well, I'm givingit a prompt to just change the issue
or just uh, to just change the, labelof an issue or, maybe add a comment or

(42:24):
something at most, but they don't reallyunderstand the effect or the causality of
what they just did and prompt injection.
So can you maybe set that up, as like,the problem with that and how this helps
prevent that, obviously you said likeremoving secret access and all that.
but- I, I can take on

Peli (42:39):
on that.
Yeah.
looking at the token you havewith the agent, let's say
you have to write an issue.
So you're doing issue write Cancreate, delete, update any issues in
your repo at an insane rate, right?
And the title might be a prompt injectionthat changes the goal of the agent.
Suddenly the goal, the agent has beenreprogrammed to close all your issues.
Or to implant malware to all yourissues or, you know, but basically

(43:03):
there's nothing preventing the agentfrom saying, oh sure, let me go and,
you know, I'll curl 500 times andpass the token and do it, or so.
So that's the danger that theseagents are extremely powerful.
Uh, and the security posture.
And the security construct wehave, you know, when we say
read-only, it's at the token level.
It is a deterministic guaranteeby the GitHub action platform.

(43:25):
It is not based on any kind of prompting.
This is a strong guarantee that youbasically adhere to because you're
using the permission object in action.
The same thing where we say there's noco- there's no sequence in a container.
Then you kind of, some level oftrust containers and so forth.
So our security story is based onDevOps primitives that are well-known
in industry containers, and permissionscopes and things like that.

(43:49):
And that is the deterministicsecure box we built.
Yeah.
So we don't rely on agentic.
we, we have some agentic protection,but as much as possible, we wanna
build a box that is deterministically,provably safe to some extent, right?
Some guarantees.

Don (44:08):
Vibe security, Yeah.
Yeah …that's what we call it.
So when you, um, when you use GitHubagentic workflows, you write this
markdown and you write some front matter.
Okay.
It's, it's, it's lovely, right?
The front matter looks a lot like actions.
YAML.
It will be fa- familiar to people.
There's some differences, but, uh,it's pretty, pretty familiar territory.
Um, and you, you run this stepcalled, uh, GHAW Compile, okay?

(44:30):
And it can produce the lock.yaml, whichis a YAML that actually runs and that,
that compile actually, I, I, I wishwe'd chosen a different word for that.
And we might make it a synonym,which would be something like Harden.
Okay?
Mm-hmm.
'cause what you're actuallydoing is taking that prompting
and you're kind of hardening it.
You're m- you're putting asecurity architecture around it.
You're saying, I'm gonna runthat prompting, I'm gonna
run it in in a coding agent.
But that's gonna be, you know, we'regonna create the YAML, which puts it in

(44:53):
a nice box and gives it a nice, securething and puts in a threat detection
step to as a, as just a extra step.
And I think that's the right kindof model to use is what I want an
automated tool, which will just makeme feel good, safe about running
that, uh, about r- uh, running thatcoding agent in GitHub Actions.
And where that, we run that YAMLthrough several checkers as well.

(45:15):
Actionlint, and uh, zizamor and,uh, there's another one as well,
Probably- Poutine … Poutine.
and, um, Poutine, Runner

Peli (45:21):
guard.

Don (45:21):
Okay,
And

Bret (45:22):
so we run- Ooh, uh, Spell that for me.
'cause I don't know about that one.

Peli (45:24):
P-O, P-O-U, you know, like the food, like the Canadian

Bret (45:27):
fries, like the gravy Cheese and the fries.
I love, yeah, the gravy.
Yep.
Yeah.
Okay.
I'm already in.
Sign me up.
Yeah.

Don (45:32):
Yeah.
so we, run those tools, uh,they caught some things in the
YAML, uh, in our hardened, YAML.
And, um, so there's alot of goodness there.
And there's something you can hand offto the, it means your security team,
if you're doing this in an enterprise,can actually check that YAML as well.
They get to see what's going on.
They get to see the fullsecurity architecture and check
it matches up with what we say.
They get to see the exactcontainer settings and its firewall
settings and what's mounted andwhat, what's not, and so on.

(45:56):
Uh, and yeah, so yeah,the guardrails are good.
And I love the security architectureand the confidence is it gives me to
run fast, uh, with agentic automation.

Peli (46:07):
Yeah.
So back to, building educational runtimesand, you know, you think about velocity,
what this sandbox gives you is inthe strict mode, a guarantee, because
that you're not gonna leak secrets.
So you're gonna have a read-onlytoken when you're a, you know, the
agent won't see your agents, and thatyou have a very specific, you know,
precisely where the agent's gonnabe able to mutate the world, right?

(46:28):
Because all the writes are transactionaland then we validated them.
there's a layered, a number of layerof guarantees, and you're in full
control saying, I will allow youto do one issue as Don said, or "I
will allow you to open an, a PR.
Right?
So from a practitioner's point of view.
These are guar- deterministic guarantees.
Then this allows you to go wild onthe prompt side and do back star and

(46:48):
do YOLO because you're gonna YOLOinside of the container, not on your
dev box where all the secrets are.
Um, you're gonna YOLO in a containerthat has no secrets that, you
know, you can't escape unlessyou break out of the container.
But, that is insanely empowering.
because now you can try thingswithout second-guessing everything.
You can try things faster, more tools,without taking down the whole house.

(47:11):
That will take you from tryingthings very carefully with AI to
actually go, go, go, go, go muchfaster because you have safety.
So people underestimate the fact that,
To,

Don (47:22):
to give an example, uh, on, on, on it means you're gonna write
prompting of things like, okay,agent, work out the test coverage
in this repository and improve it.
Okay?
Find the big holes and assess the value,uh, and choose the highest value bits
and fill in improve the test coverage.
That means it's actually goingto in- possibly install tools

(47:42):
to be taking test coverage.
It's gonna be working out the commandline invocations to kind of do that.
you know, stuff, if this is a cthing, you know, the impossible
stuff of ever taking coverage ofa c repo or something like that.
And, uh, and it's gonnabe reading the files.
Uh, and it's gonna be, uh, it's justgonna be doing everything right.
Yeah.
And you can keep your, it meansyou can keep your prompt in general

(48:03):
and the agents will use the fullpower of the software engineering
toolkits that it's got, uh, available.
And one of the magical things as well,you're running in GitHub Actions and,
people take those VMs for granted,but, and the system side of what's
built there, 'cause that's a Sure.
I don't know how big are the images now?
Like hundreds of gigabytes or something?
Terabytes.
Oh, really?
there's a lot of secretingredients There's- …that

Peli (48:24):
make action an amazing platform.
Yeah.

Don (48:26):
Yeah.
Mm. So that means every time the agent,every time your agentic workflows, or
in fact your YAML workflows are kindof waking up, uh, or, or running.
They're running with all the worldsoftware engineering tools, kind of very,
very efficiently available and well-knowninstall locations and all sorts of things.
Yes.
And, uh, that gives them a superpowerful, um, I mean, they're just,
they're just incredible what they can do.

Peli (48:46):
So, for example, if you think about the actions and the features you
have, you know, every run is recorded.
Every run you can store artifacts.
You can, You have APIs to read them.
So we store the agent session, thesessions that are sitting on your dev
box individually, they're kind of lost.
We store them.
So we analyze them, we optimizethem, then we debug that.
Uh, so the automated agentic workflowsare primed to be optimizable,

(49:09):
debuggable because we have full history.
You run things five times, you realizeyou always do the same MCP calls.
What happens?
You tell the agent, movethese MCP calls to steps.
And guess what?
Because we're action, we cando a mix of deterministic, just
good old steps and agentic.
So let's say your agent does, youknow, give me the the 10 first pull

(49:30):
requests, and it does the GitHub MCP.
That is an agentic step.
It eats a bunch of token.
But now you move that into a GH PR callas a step, drop it into a JSON file
and let the agent do JQ bash on that.
Suddenly your agent is eating10, 10 less turns, 20 less turns.
Mm. you've moved the dial.
That's one of the interesting, these areall interesting things we've discovered

(49:52):
in actions that being able to pull thedial between deterministic and agentic.
Because in your CI we're 100%deterministic historically, And
people have gone 100% agentic.
But the truth is Yeah …it'sgonna be in the middle.
And the more you are.
Yeah.
The more you're deterministic,the cheaper it is.
The most, powerful- And the less…

Don (50:12):
Yes.
they are, uh, that's right.
So, and that's why we love being in theactions ecosystem, because those steps
can also use the full existing, GitHubActions, everything in that ecosystem.
Yeah.
And, uh, and that's,that's really powerful.

Peli (50:26):
You can see this gradient coming right in your CI, you know, 100% CI, 100%
deterministic, but now you start squeezingin a bit of agentic as much as you want.
Maybe just a little reasoning atthe end of your test run, you know?
Yeah,

Don (50:39):
yeah.
I do like to point out to those,uh, people who are really big
on, like, CI, and CD need to bedeterministic, and we absolutely
have to kind of nail that in order to'cause it is That's absolutely true.
we don't challenge that.
Okay.
We say there's a, a third area,a new area, like a third leg of a
stool that we didn't know was there.
Right, Which is like continuous ai.
Right.
Which includes some of these subjectivesteps and it's got different properties,

(51:02):
but the continuity is a big part.
The automation is what is and the kindof always aligning with the state of the
repo as it is today, which is what wekind of mean by continuity, continuous.
Um- the claim

Peli (51:12):
is bolder.
The claim, you need CI anddeterministic and fast.
This is how you're gonna tame the agents.
The agents are, you know, little monsters.

Don (51:19):
Absolutely.
It's the ultimate guide.
This is the

Peli (51:21):
golden era of CI.

Don (51:23):
Yeah.

Peli (51:23):
Yeah.
The Teams
that don't have CI willnot benefit from this.
The Boost.

Bret (51:28):
Yeah.
I've often thought or described it topeople that, uh, are asking me like,
you know, where I get started and I,I talked to them about that, you know,
you're probably not gonna rewrite yourDocker build workflow to be agentic.
that's probably not the area of focus.
I would imagine that a lot of these,the majority of these are n- net new
workflows or expanding Absolutely.
An existing workflow to dothings it couldn't previously do.

(51:50):
Not rewriting my reliable test runs.
I mean, maybe there's an AI that's gonnahelp me automate parallelism and all that,
but it's separate from maybe the run.
itself.
Yeah.
And It's

Don (51:58):
really, really important.
We get that deeply because the way oneof my friends put it is like the CI/CD.
And in fact, GitHub in many ways iswhere the grownups are in software.
Development process, right?
That that's, Yeah …you know, we're allgoing crazy about doing our coding agents
and YOLOing on our local machines andyou know, whatever kind of cra But in the

(52:19):
end, the place where you build confidencein an organization where you feel you
can deploy, where they're in your CI/CD.
and we've got to keep thatgrown-up mentality about quality.
And we want, you know, when we talkabout code improvement, we we're talking
about proposed code improvements, whichhave to get through the gates, right?

(52:39):
Yeah.
We're not talking about things that theyshould come with test improvements, right?
That kind of match that just likeyou'd expect from any pull request.
so we absolutely have to keep CID beingthe place where the grownups are and keep
the mentality that goes along with that.
Uh, as well as having a moreflexible idea about automation.
And we wanna reallyempower the DevOps people.
This is like, what it's all about is likewe feel there's just this missing piece

(53:03):
of the puzzle in the AI story, which iswhere we empower the people who run the
repositories to use AI to their benefit.
And we know some people are sufferingin the open source world from
AI coming in from third parties.
Right.
We wanna empower people.
So the maintainers and the peoplewho create the repos decide What

(53:24):
automation runs in their repositoriesfor what goals, under what c- cost,
trade-offs, what, you know, whatquality trade-offs and everything.
They're the ones who can balancethose things in the context of the
business goals or the open sourcegoals that they kind of have.
And, uh, yeah, empower them and, andnot, not, don't just make them suffer
and kind of the recipients of Make them.

(53:44):
Yeah.
the- And they'll see somany new uses for it.
That's one of the things, this is jobcreation all over the place in the sense
there's so often, so many opportunities todo work we could never have done before.
Performance optimization.
is a good one There's, there's-this is literal job or at least
work creation because the people upclose to the repos are the ones who

(54:05):
know what, where the suffering is.
where the, where the un, where theunenforced invariance, unenforced
quality, uh, the, the opportunities forimprovement, which were never explored.
Uh, and the, the legacy code, whichcan actually be brought back alive and
actually serve a, a, a role going forwardor transition to a new system or whatever.

Peli (54:26):
Yeah.

Don (54:26):
So many opportunities for work, once you get into the
right mindset and, uh, yeah.
It's, it's a golden age for DevOps people,uh, who, um, to create There's lots
to learn, but it's, it's a golden age.

Bret (54:37):
Yeah.
D- I, I have often, The more I'veunderstood the mindset behind your
creation of agentic workflows and how,you know, like my first realization
was if I start looking at this lockfile, most of this is deterministic.
Yeah.
it's relatively long.
It's not, it doesn't look anything likesomething I would write in a GitHub
action workflow, but it is reallyjust a, mostly a framework around, at

(54:58):
least when I first got started aroundcontrolling and protecting and guiding
it, it is to this very small partthat's actually a model prompt, Yeah,

Don (55:05):
yeah.
It's, it's, yes, it's, that's right.
There's somewhere in the middle there'sa invocation of a coding agent, but
you put all this apparatus around itto say, we were gonna make that safe.
We're gonna make that guardrail.
And we, and yeah.
Go on, Pavel.

Peli (55:15):
Yeah.
There is something new.
Actually, you know, if you look atagentic workflows, it's, it's a big YAML.
I mean, by this time, we supporta lot of features, but you
don't even edit this yourself.
I mean, at least for onthe, in the ideation phase.
And, you know, until you reachyour 90% done, this is gone.
This is done through an agent.
You don't actually have, youhave to come in with your intent,

(55:37):
what you're trying to achieve.
And we, we've done a lot of researchand we've done this, we have this prompt
that is gonna try to generate the bestagentic workflow for you as a starter,
but there's also a completely newexperience where as an automator, you use
an agent to design that automation, andthen you can fine-tune, you know, and,
and run the compiler deterministically.

(55:57):
But this will get you from 0 to 80%.
Without actually havingto read the documentation.
'cause you come in and say, um, youknow, you, you name your scenarios and
you, you know the keywords, you know,issues PRs, build workflow, run test.
The agent has access to your agent mdthe agent has access to all your actions.

(56:17):
So if you already have CI/CD,the agent can read your CI build,
figure out how you build yoursoftware, how you run your test.
And one thing that is great about actionis that it is baked into the LLMs.
LLMs today know very well action.
They know how to write the YAML, theyknow the entire schema, they know how
to refactor it because people have beenblogging about yeah actions forever.
Uh, so there's like this free, youknow, we have this format is actually

(56:41):
designed to be close to what theagent would expect because then you
get this magic where it just knowsit, there's no fine-tuning needed.
You can tell it to refactorthe prompt into steps.
And it's like, sure.
I mean, yeah, I, andit knows the ecosystem.
It's gonna go and pull in the rightcustom actions to, you know, to
do actions GitHub scripts, or todo checkout and all this stuff.

(57:01):
So that is also part of the magic here, isthat not only we're leveraging a platform,
but we're leveraging the fact that theplatform is already trained in the model.
Yeah.
You don't need to load askill to learn actions.
It is already in- Thisis some new product.

Bret (57:15):
Yes.

Don (57:16):
Bret- I- uh, ca- can I just share, uh, my screen briefly?
Oh yeah, sure.
Yeah, I, just wanna, just wanna show one,one thing to kind of get across why Yeah.
why I'm so excited by this.
uh- yeah,
let

Bret (57:28):
me, um,

Don (57:28):
All right hold

Bret (57:29):
on a second.
let me pull that in.
I don't have that buttonon my stream deck.
One second.
You can see it.
I can see it, but I need to put it onthe screen for, everyone else to see it.
I need to give it a guest place.
And then, let's

Don (57:43):
see.
There we go.
All right, brilliant.
I, so I just wanna briefly mentionthis, this particular workflow.
This is a, uh, this is a workflow.
You just install one of thesein your repo and it kind of
helps you maintain the repo.
Okay.
This is the thing Imentioned before, repo.
This is… and, uh, it, it'sreally simple getting started and.
it's super, and this is kind of how,how it works There's a diagram here

(58:04):
kind of selects a couple of tasks.
It reads the memory, and theseare the different tasks It
might do issue labeling for you.
It might do an issue investigationand the other things.
And you can configure this andyou can edit, you can say, add
a new task to do this or this.
And, you know, and it kind of justworks on a daily rhythm or hourly
rhythm or whatever rhythm you want.
And I kind of wr- I've written up, uh,written that up in a kind of blog post

(58:25):
and you can kind of see how it works.
but I just wanted to sh- sharethis really, which is this
report we've written, uh, onthe impact of using repo assist.
So if you kind of lookat what's on the screen.
You can probably guess where we startedto use repo assist in this particular
repository There's a number of issuesthat were open in the repository.
So this was a pretty much dormantrepository, but with a backlog, right?
It's got a… you know, I, I as amaintainer sort of stopped engaging with

(58:47):
this because I didn't actually, each ofthese issues would've taken me sort of
a, a night, probably in the traditionalway to kind of reengage with the issue.
And even if I was doing it manuallywith a coding agent, it would've
taken me significant, 20 minutes,30 minutes, an hour for each issue.
Okay.
And instead you've got the automatedAI effectively r- burning through
the backlog, commenting on it,making pull requests for it, and

(59:09):
like actively, proactively kind of.
make solving all the, all of that backlog.
And I mean, boy, it allowed me eitherto close out the backlog or actually fix
the backlog and make I think three major,some of this was feature requests as well.
So it actually kind of took therepository forward as well as implementing
features and got three major newversions of this, uh, of this component

(59:32):
out in, uh, as open source releases.
And, that repositoryis now in a good state.
Reposys continues to run.
So if there's any more, uh, it'snow running sort of on a weekly
kind of basis to as a cost control.
And if any new input comes intothe repo, uh, new issues, uh,
it will, it will start to do.
Its, its kind of magic.
It will look after that for me.

(59:53):
Uh, and of course I'm still in control.
The human's still in the loop, butyou can just get the very dramatic
difference it makes between softwarewith bugs to software that is actually
maintainable and fully usable.
And it's not just one repo.
Here's another repo.
The same workflow, uh, with,uh, here's another one.
This is a different maintainer.
So it's not just me, other,other maintainers picking it up,

(01:00:13):
a slightly different percent.
Uh, but you know, after, after a monthof sort of this thing ticking away,
uh, it, the repo's in excellent shape.
Uh, Here's another one.
This is something I co-maintainwith somebody else, uh, it's
a, slight different trajectory.
And, uh, and here's another one wherethere were good reasons to leave a lot
of, uh, feature suggestions, uh, lyingaround the repo at the end of the kind

(01:00:35):
of, uh, as it acquiesces at the end.
Yeah.
So, um, That's cool …super,super happy with how this is, uh,
this is going, uh, this one, themaintainer actually only comes back.
He said, So didn't wannawork on this full, time.
didn't wanna crunch a whole lot away.
was just kind of happy just tokind of come back to it every
few months and kind of this graphwill keep going down step by step.

(01:00:55):
So the, the report we're lookingat is, uh, just to bring it up to
the top, is the impact of automatedrepository maintenance assistance.
and you wrote a blog about this, right?
on our GitHub Next site.
And yes, there is a, uh, blog, the blogabout Reposys in general, and on my,
uh, just grabbing this is a, is a linkto our new report, from GitHub Next,

(01:01:18):
it's on our GitHub Next site as well.
Yeah, so check that out.
Uh, where was that report?
Um…

Bret (01:01:23):
Yeah.
Uh, I think to me one of the mostexciting things about all of this,
and, and we're, we've been hintingat this the whole time, is that the.
As a CI maintainer, uh, uh, a-as someonewho's m- I call it the middle gray area
of the software development life cycle,where it's post-commit of the developer,
but it's, it's pre-production running.

(01:01:43):
And everything in that middle has beenlike, I, I, r- I can remember, we, if we
go back to even 2018, I can distinctlyremember at DockerCon and at KubeCon,
we were talking about what was the nextwave of innovation, because we felt like
the container ecosystem had matured, andwe kind of knew what that looked like
and how to move things around as images.
And that was all well-defined.

(01:02:04):
And w- at the time, we were alltalking about the CI platform
as the next piece of innovation.
And there was all this discussion arounddifferent startups that were getting
funding because that was gonna be then-next opportunity for innovation.
And uh, we, there was experimentsthat, you know, GitHub Actions, uh,
workflows were probably part of that.
Like that was a part of that wave.
It didn't, it was awesome, but alsodidn't seem to always fulfill the

(01:02:25):
promise of what we were trying toinnovate on and reinvent in the CI space.
But I feel like we'refinally at this moment where.
I might just be able to doall the things I always wanted
to do to fixing the platform.
You know, the maintenance, thetoil, the backlog, the endless
backlog of things that needed to beoptimized or locked down or scanned
or improved, like documentation.

(01:02:47):
And that w-we… management always tendedto, you know, focus on the feature set.
You know, th- those of us in DevOpsare always trying to help, help
them understand and that there'smore than just adding features.
We need to maintain the system.
SREs are a thing now, so we allget this, you know, we at least get
someone in production that's helpingto optimize the production information.
But I feel like the CI platform is stillthis sort of, uh, if redheaded stepchild

(01:03:09):
is a thing we still say like, it feelslike the thing that still doesn't get the
love and nurturing that it always needed.
and- Mm. I mean- …I've lostcount of the number of places going

Don (01:03:17):
Yeah …my view, it's the center of the factory.
It's the, it is the software factorywhere all the grown-up stuff happens.
So much of the forward progresshappens beyond the- yeah.
Maybe feature development, whichmight be done by agentic, local,
or Pele actually does a whole lotin the CI It's just Everything.
He wants a feature implemented, he justwrites an issue and, uh, or, or yeah,

(01:03:38):
everything comes through it's CI system.
Amazing.
It, You know, the, the softwarefactory- So you're- …is real.
And

Bret (01:03:43):
so you're not prompting you're issuing You're issue prompting?
Uh, no.

Peli (01:03:48):
I skip the issue.
Uh, I set a promptdirectly, but a lot of it.
Are agents that create issues.
So the issue is Oh, okay …a work queue.
Right.
Mostly.
So a part of the work, which ismaintenance and code improvement,
uh, or documentation updates, wouldbe produced daily by, by workers.
but you know, what you're saying is right.
It's, you know, up until now you couldhave a sloppy CI and for example, you

(01:04:10):
could rely on a good dev team to kindof trust your, you know, they would do
the right thing, you know, the quality.
This is not true anymore.
The only thing, the only way you'regonna leverage these agent is to have
a very, very tight CI with a very goodtest suite and not just one test suite.
You need test of the testintegration test first test.
I mean, you bring it.
Yeah.
Because you need to triangulate themso that they cannot escape that box.

(01:04:32):
And then you, and then once youhave that, you can get the boost.
The boost is the cloud, your dev box.
There's only so many eyes and somany terminals you guys can handle.
So I don't know if you're, if you'rea spider, you get eight eyes and
you can maybe do 64 terminals in thecloud, I can easily run hundreds.
Okay.
So let, me- There's,there's like no compare.

Bret (01:04:50):
Yeah.
Uh, let me ask real quick, 'causeI think one of the things I love
about this, or I, I try to optimizeon this show is to change behavior.
Give people an insight that will actuallycause them to do something different
rather than just executive over-overviews.
not that we've been doing that, we'vebeen digging in the weeds, but I,
I'm, I'm actually very curious like.
if When you're on the forefront, canyou talk through what it, what your

(01:05:13):
activity looks like when you wantto create something new with the CI?
So you're, you're prompting theLLM on your local harness, right?
It sounds like it's creatingthe issue on your behalf.
You've got a bunch of automationrunning in the background.
Are you asking it to like develop thePR and then you're gonna wait for it
to tell you when the checks are ready?

(01:05:33):
Like are you even going to GitHubor are you like harness first?
Like t-talk to a little bit throughthat so that we can get an idea.
Okay.
I'm,

Peli (01:05:40):
a bit extreme.
Sure.
Um- That's what I want, I want,

Bret (01:05:43):
I want the red pill all the way.

Peli (01:05:45):
down.
So first of all, it's fully async.
Okay.
there is no discussion onmy part with the agent.
I fire and forget through, git, theGitHub cloud agent most of the time.
So, you know, you go either to github.comand you do new agentic session.
I mostly use my phone.
So I do that from the iOS app.
So I'm not waiting.
That means I can have five to10 agents running at all times.

(01:06:06):
And it's just like playingchess on multiple boards.
Now they take time.
So if I forget what do you do betweenthe agents, you think about your future.
you have more time to think or youtalk to people In fact you have more
time to talk to people because theagents are doing the work when they,
s- how is this gonna change your work?
So.
That's a very big one.
people are into the token, youknow, they look at the token
flowing down and it becomes a slotmachine, and they get addicted.

(01:06:29):
They get headaches and stuff.
They're tired.
Yeah.
Don't have, you know,just schedule the work.
let it churn for a while, come back.
so that's One big thing.
Now when I determine that there'sa pattern that I'm doing the same
stuff all the time, or there's likesomething I'm thinking in my head of an
automator, whoa, I could do that again.
You know, there's like,there's some value to that.
So the most obvious one, one of thefirst one we wrote, was like, this

(01:06:50):
agent creates a lot of duplicate code.
And it did it in kindof a, a very subtle way.
it would rewrite string functions,like string start, string, you know,
string split, but in different ways.
so I had this idea I was like, okay,maybe we look at the, we look at
the function titles and we ask theagent to bucketize them by intent, So
because the body of the function maybe s- completely different, but the

(01:07:14):
intent of the function, is the same.
so you open your phone and you say, createme a daily agentic workflow that uses some
LSP to list all the functions or regexto list all the function in code base.
Bucketize them by intent, pick thebiggest bucket and now generate
a prompt that says, remove allthese duplicates." That's it.
That's all you have to do.
Wait five minutes, youget an agentic workflow.

(01:07:34):
As a PR, you review the PR,you look at the safe outputs,
which are saved by default.
This is your first version.
probably not efficient, but it's gonnarun, it's gonna burn a lot of tokens.
It's not, it doesn't have theright MCP, you know, it's not
optimized, but it kind of works.
Yeah.
And you're like, conceptthis is real value.
Then, you know, we iterate and so forth.
And this concept of, I've got somethingthat, that annoys me in my code base

(01:07:57):
and I can get goodness I can get slightimprovement in a human consumable
way, which is basically the dailynewspaper concept is insanely good.
First of all, it's super fun.
You feel good because like, "Whoo,you know, my code is better now.
it's a new way to also handle theagent, because upstream that means
you don't need the perfect PR.
You can work in a feature branch.
You can go faster bynot doing the 17 nits.

(01:08:20):
Mm. because you have cleaners now.
You have tools that arelooking for patterns that are
known to happen with agents.
You know, they're gonna happen.
by the way, humans wereterrible at coding too.
We forgot that.
But, uh, so now you cleaneverything, human or non-human,
you clean all the patterns.
and we've been, I meanby now we have what?
27 or 50 running.
So we have also a summarizerthat looks at them.

(01:08:40):
I don't have time to look at them.
I have something thatmines them, We track them.
But it's always started from there issomething I'm doing all the time and I
wonder if the agent could actually help.
And this is important, like, I don't knowat the moment I'm writing the agentic
workflow whether it's gonna work or not,Mm. but my time, my experiment costs
me five minutes, 10 minutes, So I'm notinvesting three months to build a static

(01:09:03):
analysis tool like we used to, like,Hey, maybe this works and some of them
are just terrible or just too expensive.
You, know, or,
they're, but a lot of themare like, surprisingly good.
Then you put the right MCPS and so forth.
Uh, then we optimize.
By optimize.
I mean, think of an agenticworkflow as a concretized plan.
You've done sash plan, you paidfor opus, you burn a lot of tokens.
You've got a really good plan.
That one is now set in stonein your agentic workflow.

(01:09:25):
That means you can go fora lower model to implement.
Mm-hmm.
And then you can start splittinginto submodels and everything.
Lower your tokens.
And all of these are hyper-specializedtools because they, the clearer the
goal, the better the agent gonna be.
So these are all kind of intuitionsthat we build, that we've kind of
measured, that we use, um, so it's likea plethora, like a, an insane amount of.

(01:09:45):
Very, very specific tools.
Yeah.
Linters are a good example you know, thiskind of tools we've built over the years.
Yeah, I love- Linters have rules.

Bret (01:09:54):
My my favorite thing, and I think I might even have it in my global agents
file on my machine is always, always lint.
Uh, at the end of every edit you make,uh, or at the end of every run of an edit
you make, because I, I don't even wannas- I don't even wanna look at what your
output is unless it's passed linters and,uh, at, I think the minimum, like you
ment- you mentioned the actionlint and thezismore and, yeah, these are Yeah …like

(01:10:14):
table stakes for me, for GitHub workflows.
Uh, I was just curious real quick.
Yeah.
What is a, what doesoptimization look like?
Is that just improving prompt,it, uh, when you're So many

Peli (01:10:22):
things?
Yeah.
So you wanna have the sameperformance, the same reasoning,
at a lower cost, right?
that's really the end game.
you don't wanna degrade your performance,but also you don't wanna pay, you
cannot just pay opus all the time, Uh,so there's a lot of tricks in the bag.
And we talked about moving,turns into the step side of
the action Pre-computing data.
Also, this grounds the agent.

(01:10:43):
You do the computation, you cooksome python, you're given the pre-fed
computation with all the aggregates andsay, now reason on this and don't do any
computation, don't make up numbers, right?
So that's also a trick too.
But then other thingsare using small models.
So splitting a monolithic prompt intoa prompt plus small models, typically.
For example, if your task says, go andsummarize files, the file summary can be

(01:11:07):
done by a small agent, a subagent, Yeah.
Then comes back with the summary.
Right now, you've actually went from a 6Xmodel sonnet style, you know, 6X, to 0.3.
you've dropped by 20X your cost per token.
Um, and of course, you know,there's a prompting and we have
AB testing in the platform.
So when you run at scale over thousands ofrepos, you can start doing campaigns and

(01:11:28):
field test, prompt improvement and measurein a scientific way, in a reasoned way.
Just like, you know, think theseare really websites, right?
at the scale you're goingto run these agents.
My belief is that it's not really an evalthing, it's more like a website where you
do ab testing on features, you're going todo ab testing on prompts or model and so
forth, and measure as you're spending themoney measure whether they work or not.

(01:11:49):
It's really hard to build evalswhen you have hundreds of agents and
everything's moving all the time.
Right.
Uh, but- That's
that's

Bret (01:11:55):
a lot of eval runs in addition to the agent runs themselves.
Yeah.
But AB

Peli (01:11:59):
testing is a proven way of fielding, uh, improvements and measuring.

Don (01:12:03):
Yeah.
And in terms of, you know, thepeople on the call watching this, uh.
you know, One of the big behaviorchanges and mindset changes is
like, what's your future workgonna be in an enterprise, right?
It's not just running over one repo, butyou're gonna be the agentic maestro or a
team of agentic maestros who are able todo things at scale across tens, hundreds

(01:12:24):
of repos in say, a major, uh, a majororganization that might be about like,
uh, applying security patches, uh, ormaking judgment calls about how the impact
of what it means to roll things out.
It might be, uh, making thingsmore regular across those repos.
It might be, um, improvingthe test, uh, assessing the
test coverage across the repo.

(01:12:44):
You know, if I was the CTO of a major,um, all companies, software companies
these days, I'd kind of want a reporta summary report, up of like, what is
the, what are all the repos we have?
What are, what's the status of them all?
What, how do they Right…cluster together?
How do they, how, whattechnologies do we actually use?
And not just based on what we thinkwe use, but actually assessing,

(01:13:06):
what we actually depend on.
And so there's so much that's even oninformation reporting, kind of working,
kind of going up even before you getto kind of taking, improving all of
those different, uh, repositories.
Yeah, I, I, I, I think there's a, youknow, I've been worried for a long
time that some of these, you know,I'm in London and the there are a
whole lot of investment banks downthe road, I've worked with some of
them over the years, and I actuallyam really worried about some of them.

(01:13:26):
You know, I'm worriedabout their software.
They, they tell me they've got20,000 production systems or
something like that, right?
And It's just like,Yeah … uh, it's, it's insane.
It's, the, the, the, The softwarelegacy debt that they have across
those systems is just huge.
Luckily for them, the tools have nowcome along, which can deal with that.
But they need agentic Maestros to comeup with the workflows to get a grip
on that software complexity in uh,through summarization and action and

(01:13:50):
all sorts of other, yeah, along variousdimensions are kind of kind of working.
So we're not just talking one repo,we're talking a whole, the agentic
organization and how that actuallymaps down to actually working
with real software artifacts.
It's not just something in theoryor something kind of, Some of these
automation platforms like, uh, youknow, Asana and NAD and the other
ones, they're much more on thekind of information working side,
like working with the HR systems oryour, uh, ERP systems or whatever.

(01:14:12):
this makes it very concrete to me,like I now know what the agentic
enterprise means for the wholesoftware side of the enterprise.
Yeah, it's very real.
It's like, it, it, there's alot of work to be done to make.
To actually crank the handle onthat and auth, a lot of this becomes
auth constrained, for example, who'sallowed to do this stuff, right?
Yeah.
the ideal job in this world is where youhave maximal auth and you are trusted.

(01:14:37):
And that is what the maestro really, the,uh, the grand wizard of the enterprise.
That's the ideal job to havein this kind of situation.
You've got lots of power, lots oftokens to spend you, or can find
out what the actual business valuework, uh, to be done actually is.
And you can actually make it happennot just shout from the sidelines a-
across some organizational divide.

(01:14:57):
So yeah, you want to be part of the grow-If you're looking for a new job to, or
a new career direction you want to be,you want to be leading the conversation,
the agentic software, automated softwareconversation in your whole company.
However, that is

Bret (01:15:12):
the continuous AI czar.
maybe we'll workshop the name.
Or the group of czars.

Don (01:15:15):
title, I mean Yeah, The group.
I mean, it doesn't have tobe a single czar, but yeah.
The, the maestros, the, uh, the thoughtleaders in the company, the people who
see this in multidimensional ways, uh, whoaren't just, they're not just nutty and
evangelistic, they've grown up about it,but they can use re- repository automation
at scale for positive action across the

Peli (01:15:36):
whole.
org.
it will be the catalyst for areorganization of software production.
The way we build software will changebecause we will design new processes
between agents and humans where,you know, we've been doing the pull
request for a while, the agents arekind of, you know, kicked in the
door and starting to shake things up.
But there will be new ways tobuild software and they will.

(01:15:56):
And you know, and we are seeing, ina way we're experimenting with all
these new kind of flow, informationflow and production flow that involve
agents, human agents But, we do it fast.
We have the means to do it safely.
Yeah.

Don (01:16:08):
My, yeah, the one I'm currently, um, which is a kind
of a software factory, uh, image.
I've used that kind ofterminology quite a lot.
And, uh, when you saidlike, what's the action?
How do I, how do Istart my design process?
At the moment it's about saying, let'sbuild a factory, uh, where there's
actually lots of inputs flowing in.
They might be issues of somekind, but they might not be just

(01:16:29):
issues in a maintenance sense.
It might be like in, in the case ofGitHub, we have automated tools which
find problems in, in, in GitHub problems,in the logic of how we use our databases
N+1 problems and things similar.
So you're kind of going to get theseto flow in and then you've got a
whole automation human, a mixture ofhuman and the factory is a, is a place
where both agents and humans work.

(01:16:51):
Crucially.
Okay.
And When you think of it as afactory, then things can get
blocked at the human point.
Like the, uh, even when you get allthe automation set up, which is really,
really great, you, it can still getblocked by overwhelming the human
with too much kind of generation.
And you can either scale that backor you can increase the quality.
There may be good reasons whythey're doing that, or you can
actually turn off the wholefactory because it's not actually

(01:17:13):
serving the humans' needs properly.
Okay.
So the, the aim of the agenticmaestro is to design that human, i-e,
that, that, that agent-human factoryand make it flow, make it work.
'cause when it does flow, you get thosereally dramatic results on quality.

Bret (01:17:30):
Awesome.
I feel like it's been, this has beena good discussion around, uh, agentic
workflows because I think that's not,it's not a feature that we see in the UI
n- yet, so I feel like it's still reallyearly days in terms of getting everyone
to be aware that this thing existsand how to go about implementing it.
So I'm excited about talking more aboutit, and especially now that I, I feel

(01:17:52):
like you're giving me more reasonsto pay more attention to it because
I'm realizing that I've, even thoughI've, I've dove into some of it and
implemented some of it that I'm, I'mstill, I feel s- I'm a babe in the woods.
I'm a babe in the woodsright now, but, um, so, so
are

Don (01:18:04):
we.
Yeah.
So are we, yeah.
I wanted to ask you.
It's an exciting

Bret (01:18:07):
time the last question or the last topic before we wrap this up.
you're both, you're bo- o-one of you is GitHub Next one
of you is Microsoft Research.
You're basically bothalready thinking years out.
my assumption is that there's,there's things that are coming.
So like what, this is all brand new to us.
It's, I'm sure it's still very new to you.
Like what, what is the thing that'scoming, not that this is ever replacing

(01:18:28):
this, but like what, what are youexcited about for the rest of the year?
what else do you, is it more pl- findingmore places that this can operate or like
sussing out the real value of where theseworkflows are running in sort of m- like
maybe, maybe maybe making t- the top fivelist or the top 10 implementation list?
Like where is it that you, you'relooking to in the short term?

(01:18:50):
take this.
Anyone wanna I'll let

Don (01:18:51):
you first?
I'll

Peli (01:18:52):
let you we're still in techno preview.
Yeah.
So In a sense it's, we haven't,we're still in first gear.
so very excited to see where, wherethe product is gonna go and we're
gonna really be able to, to go out andtry to, and we have intuitions about
what's gonna happen when life scales.
We haven't really done it.
Yeah.
And very excited to actually learn.

(01:19:12):
Everything's fine when youhave one repo, 10 repo.
But we are very much looking forwardto 1000, 10,000, 100,000 scenarios.
And now looking at large scale,agentics and all these scales, economies
are gonna happen, uh, through that.
personally I think it'sa golden era of CI.
I mean, if there's one thing out ofthis discussion is yeah, stay in CI,

(01:19:32):
it's gonna get good because everybody'sgonna turn to you and say, how do I
run my agents in your, in your CI?
how do I do more agent stuff andyou know, and do all these, I heard
this and I wanna do it in yourCI and they're gonna turn to you.

Don (01:19:45):
Yeah.

Peli (01:19:45):
And they, they're they're also, they're,

Don (01:19:46):
al- they're also gonna turn up and say, Hey, can I use
OpenClaw inside the enterprise?
You know?
'cause I wanna automate my,like, uh, I wanna automate doing
my PRs inside the repository.
And it's just like, you know, maybeyou could go learn ag- get agentic
workflows because that's, that'sactually like a, you know, pre- it's
pretty safe way of doing automation.
Right.
And they're gonna turn up.
with… there's, lo- we're seeinglots of other, We already kind of
touched on the, like run the nakedcoding agents kind of approach.

(01:20:08):
And it's like, the answer to that isgo use GitHub agentic workflows, right?
'cause that's got, that'sgot a security architecture.
You can also make, there'llbe other options as well.
There'll be other security, architectures.
Yeah.
but there's an answer toa question that's there.
And, and we, we chose a continuousAI framing because it was.
It's An industry, we wanted to create anindustry neutral term that a ju- uh, just
like you've created Agentic DevOps and,they, and they're more or less, two, two,

(01:20:31):
very closely affinitized, which is great,and, and they, they're questions with
an answer, with a natural answer, whichis GitHub Actions and, G- and GitHub,
which from a product perspect- productdevelopment perspective is, I'm very
happy to have made those contributionsand where we've landed with all of that.
Uh, in terms of looking forward,there's all the kind of rollout of
this kind of agentic workflows at scaleor agentic working in the enterprise.

(01:20:54):
And that's gonna take years to roll.
the enterprise turns slowly,development teams turn slowly.
They've got their own opinionsand their own skilling.
As I said, this isn't, this is aboutas much as I use the agentic maestro
thing, the Agentic maestro also listensto the dev- developers, listens very
closely to them because they're theones up close to the coalface who
know how to maximize, they merge
the

Peli (01:21:13):
PRs that are not,

Don (01:21:15):
and they probably merge the PRs or not.
If you're really lucky as an agenticmaestro, you get to merge PRs too.
It's like, it's, it'sgood to have that power.
uh, but there's the, I'll,I'll leave you with one thing.
that's a little bit further out,uh, which is the… when we put
these workflows together, Previouslyo often they're simulating what
we'd imagine a human to do, liketest improvement or test coverage.

(01:21:36):
They're doing one thing and you canimagine getting someone in to do, improve
your tests and assigning them that job.
But nowadays we can actually get themto use multiple kind of tools and
methodologies all at the same time.
And so if we look at say, performanceimprovement, for example, uh, y- the-
this thing not only knows how to dothe profiling runs and how to write the
benchmarking kind of, tools and how todo garbage collect and optimization.

(01:22:00):
It can also go read theassembly code, right?
You know, which no- which n- wenone of us can do, Read right?
We can't interpret that andit makes good sense of that.
And so, um, you can set up, if it's, ifyou really needed to squeeze that last
2 or 3 or 5% out of, uh, a performanceout of, some say Go-based system, and
this might apply to say, GitHub orsomething, then you could actually

(01:22:21):
set the agents also optimizing, addingnew optimizations to the go compiler.
Okay?
Like it can take us a private copyof the Go compiler and make a, uh, a
kind of maker, si, you know, make itsown bots, compiler patches to that to
actually improve the register allocation.
Uh, and that's a multi-skilling thingwhere you could never find a single.
Person who could-- had allthose skills across the board.

(01:22:44):
Right.
Ta- uh, Performance optimization-The agent … it's full
of those kind of problems.
Right?
So, you know, the people who actually knowhow to make their .NET or Java garbage
collectors, the memory hierarchies anduse them re- uh, so that everything flows
really, really nicely in those systems.
They're really rare.
The people who can do that, the agentskind of know how to do that kind of work
where you can ki- you can encode code it.
So these kind of multi-skillingflows, are super interesting.

(01:23:08):
Uh, I think, that, are beyond, they're,they're a bit beyond what we, the frontier
of what we imagine these AI systems doing.
'cause they're kind of like littleteams of people or little teams of, of
cooperating kind of agents all taking.
a… It's not just critiqueor different roles.
It's actually entirely differentcompatible sort of skills, which kind
of compose together really nicely.

(01:23:29):
yeah.
So it's one idea.

Bret (01:23:31):
All right.
I think the tagline for this,this is, uh, besides that,
is another potential tagline.
'cause it, it sounds like I'vegot five different options
for the title or the tagline.
It could be, uh, GitHub Actionsis the OpenClaw for grownups.
Dude.
I gotta get, I gotta think of Yeah …allthe buzzwords I gotta put in there.
Yeah.

Peli (01:23:48):
I
gotta have the word agent somewhere.
in there.
That, that's your show.
Yeah.
your show.

Don (01:23:52):
Absolutely.
it.
You know,

Bret (01:23:54):
it's got cron, it's got A- absolutely, we can, we've got memory.
It's got, yeah.
We can, it can learn over time.
Yeah.
The addition

Don (01:23:59):
of the addition of memory to these GitHub agentic workflows, Yeah
… makes huge, huge difference to that.
'cause now they can do research.
The first task they do when theyhaven't done it before is they can
go research your code base and, andactually work out how to do all that.
All those, uh, depth kind ofengineering things, uh, that, and
kind of keep their own private noteson that, and update those notes.
Amazing.

Peli (01:24:19):
We have action cache.
I mean we mount memory on actioncaches on repo on comments in the wiki.
Plenty of places to store.
Uh, and then, you know, you've gotthese long-running, We have a practical
Auto Loop, which is the auto loop.
Think of the auto researcher ref loop,not just days, weeks, month, because
it's mounted on top of an action run.

(01:24:41):
Saves its data into a branch,then action restarts on a
branch, and then it keeps going.
So if you think about all your inner loopthat are doing route for, let's say a
day now you have the outer loop of that.
Yeah.
That's gonna run for a month.
So you can point it at uh, anentire code base and say, convert
this stuff into something else.

Don (01:24:56):
Yeah.
But, uh, Wow OpenClaw for grownups.
Yeah.
No, automation is an incredible thing.
Very empowering.
And, uh, it's just, yeah, a lot youcan do with it, but make it safe.
Make a guardrail- And, yeah,we're very happy to help provide
a basis for doing this, uh, atscale in the enterprise and with

Peli (01:25:14):
Um, we are very, we are a weird open source project, but we've closed,
we take, uh, bugs as, uh, as specs.
but we've closed so far.
See the number today, 633community bugs since we shipped.
so if you're using agentic Workflowsand you find something, uh,
run our agents on your workflowand tell it to file an issue.

(01:25:36):
And, uh, yeah, we've been, uh,running as fast as we can to
answer the needs of practitioners.
Most of the bugs we get are fromprofessional CI/CD engineers.
And this is, these are really deep GitHubaction feature that we didn't know.

Bret (01:25:51):
Yeah.
Right.
Okay.
Yeah, so- All right.
Right.
Like the, the esotericedge cases of, uh, yes.
Of GitHub Actions when someone knowsevery little nook and cranny of the Yeah.

Peli (01:26:00):
Arc Runner on GHS with something, something.
Uh, we've been looking at that andyou know, there's this thing where
the sandbox is closed by design,but we also have the hooks for the
pros, for the people who know togo and plug in the stuff they need.
GitHub apps, custom jobs, custom stepsuh, Uhashicorp step to get your secrets.
All this stuff is basically, youknow, we, we wanna fully leverage

(01:26:23):
the platform and we want the problem.
When you do a box that doesn't have theescape hatch for the pros that then people
turn up that, you know, you need to getstuff done, you turn up the security.
So we designed it.
Yeah.
bad

Bret (01:26:33):
habits.
Yeah.

Peli (01:26:34):
designed it so that you don't have to turn up security to get into
these enterprise scenarios that are,very complex CI/CD with multiple apps,
multiple security, multiple tokens, andth- things become, you know, the sample
we see on the landing page becomes waymore, you know, when you start factoring
in real life CI/CD constraints, Then itbecomes, but that is because we build on

(01:27:00):
top of action and we're just an increment.
We inherit all that goodness.
We're in all the ecosystem.
So, I mean, the call foraction is try it out.
There's something you don't like.
We've been running as fast andresponding to, uh, we only have
a backlog of 30 issues and we'veclosed 630, so we're, we're on it.

Bret (01:27:18):
That's awesome.

Peli (01:27:19):
Yeah.

Bret (01:27:19):
Well, yeah.
Your agents are on it.
You're on top of the

Peli (01:27:22):
agents.
No, the, we don't take PRs.
People run the agent on their side.
There's a full, ana- uh,full investigation on their
run with their secrets.
Then it's anonymized and thenthe agent files a generic issue.
Nice.
And that's just built in also it'sa crazy way to do all software.

Bret (01:27:39):
You're right, It's built in, it's built in, a feedback loop.
all right, so this has been awesome.
I'm so glad to have both of you here.
I'm very excited about the future of this.
I've been a heads down GitHub Actionsguy for a long time, and I feel
like This is my whole new thing.
And this episode has convince me thatI should have spent a lot more time
the last three months on it than I did.
you've convinced me and I'm excitedto get into the weeds of this.

(01:28:00):
Where can people find both of you,

Don (01:28:02):
certainly if you, if you wanna contribute ideas, uh, to the design of
GitHub agentic workflows or even justfeedback on using it or examples of using
it, there's the, the repo You can makeyour pitch give us good feedback and
find us, make a pitch for a new feature,a new coding agent or some new thing
you, you think sh- should be supported.
And it's all open source so you cankind of, uh, s- see it all and work it
all out even before it, it gets to us.

(01:28:22):
and ship on Mondays.

Peli (01:28:23):
You ship on Mondays?
Yeah.
You, c- and,
and

Don (01:28:26):
uh, for me, you can find me on LinkedIn.
I do a lot of posting on LinkedInand my blog as well, which,

Peli (01:28:30):
yeah, I'm mostly on GitHub and you can find me on LinkedIn, but Nice.
Please file an issue with your ideawhy you think we should have it.
With the agentic plan, you know, burnsome opus tokens on making your claim.
a lot of them were one-shot in BB Kitcode.
Yeah.
Awesome.
And the turnaround, we've got somethree-hour turnaround sometimes if you're
at the right moment in time betweenfiling the issue and getting a release.

(01:28:54):
this is agentic speed.

Bret (01:28:56):
Yeah.
Uh, I'm, I've been playing aroundwith GitHub mobile more and more.
And so it, it, the fact that you, that'syour workflow, that's your process
is convincing me that I need to leaninto more of that because I, I, it's
a, it's a habit that I don't have yet.
Like, I don't have the musclememory to go, I have an idea.
Let me jump into my GitHub app.
so I need to break that.
for everyone listening, it'sgithub.com/github/gh-aw, Obviously

(01:29:20):
there's a bunch of websites.
You can go to githubnext.comto see all the exciting stuff
coming out of the research.
Uh, basically just a, a bunch of, it'sa whole list of smart people that I
basically wanna invite on, all, ofthem on the show for some things.
I think I, I just saw the presentationLove it …from AI Engineer
Linden The, n- yeah, the the new,team-based agentic harness mindset.
I am like, that is my next, I am.

(01:29:41):
So I saw that demo and thought thisis exactly what I've been missing
and what my teams probably want,and, a replacement for Slack,
and like all these other things.
So I'm very excited about that one.
and I was like, immediatelywent to sign up for the beta.
I don't know if she realized that theremight be a whole lot of signups, but
when I shared it out, I got a bunch ofresponses from people going, oh, yes.
You know, not just yes,but hell yes on that.

(01:30:02):
So I'm excited to see how these two areasmerge and I can have the prompt cr uh,
crowdsourced from my humans and likewe perfect the ai- output of possibly
how these GitHub action workflowsare gonna be created by an agent.

Don (01:30:16):
Bret, Thank you for having us on.
It's been a lot of fun

Bret AI July 2025 (01:30:19):
Thanks for joining us, and I'll see you in the next episode.

All Episodes

Episode Transcript

Popular Podcasts

Hey Jonas!

Stuff You Should Know

Dateline NBC

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}Safer Agent Automation with GitHub Agentic Workflows

Episode Transcript

Popular Podcasts

.css-r6mb8g{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:1;overflow:hidden;}Hey Jonas!

Stuff You Should Know

Dateline NBC

All Episodes

Safer Agent Automation with GitHub Agentic Workflows

Hey Jonas!