April 22, 2025 • 38 mins
The Institute of Internal Auditors Presents: All Things Internal Audit Tech
In this episode, Charles King talks with Imraan Mulla about the transformative potential of AI in internal auditing. They discuss AI’s evolution from basic automation to advanced applications, the challenges of adoption, and the future of AI in internal auditing. The conversation also covers continuous monitoring, risk assessment, and the role of human judgment in an AI-driven world.
HOST:
Charles King, CIA, CPA, CFE, CIPP,
Partner, AI in Internal Controls Leader, KPMG LLP
GUEST:
Imrankhan (Imraan) Mulla,
Vice president, Analytics and Innovation, Capital One
Key Points
- Introduction [00:00-00:00:21]
- AI's Evolution in Internal Auditing [00:21-01:08]
- Challenges of AI Adoption [01:08-02:16]
- Driving AI Adoption and ROI [02:16-04:31]
- The Future of Internal Audit with AI [04:31-05:57]
- Prioritizing Investments in AI [05:57-08:10]
- Continuous Monitoring and Targeted Audits [08:10-11:05]
- Flexible Audit Plans and Risk-based Auditing [11:05-12:29]
- Analyzing Unstructured Data [12:29-14:14]
- Human Judgment and AI [14:14-16:06]
- Building a Culture of Innovation [16:06-18:11]
- Internal Audit as a Beacon for Innovation [18:11-20:48]
- Agentic AI and Its Applications [20:48-22:49]
- Final Thoughts [22:49-23:19]
The IIA Related Content Interested in this topic? Visit the links below for more resources:
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:02):
The Institute of Internal Auditors presents all
things internal audit tech.
In this episode, Charles King talks
with Emron moolah about the transformative potential
of AI In internal auditing,
they discuss AI's evolution from basic automation
to advanced applications, the challenges of adoption
and the future of internal auditing.
(00:22):
The conversation also covers continuous monitoring,
risk assessment and the role
of human judgment in an AI driven world.
Iran, I'm so excited to have this conversation with you.
I feel like you always bring so many great ideas
and sort of forward thinking
and what, what you do at CAP one
is, is really, really impressive.
It's great to be here and talking to you.
(00:44):
Uh, you always bring out the best conversation.
So excited for this one. Excellent. All right.
Well, when we were back at gam, we had a session on AI
with Anton Dam from Audit board.
And you know, one of the things that was really eye-opening
for me in that session is every time I do a session on ai,
I ask the audience, how many of you are using AI at work?
(01:08):
And historically, over the past year,
let's say it's been roughly half, maybe,
maybe a slight majority, 60% of people raise their hands.
When we were having that session,
it was three quarters at least, maybe more.
I do think like a lot of more audit shops have gone ahead
and got their auditors, uh, gen AI to sort
of the white label and just out of the box gen ai.
(01:30):
So I think definitely the conversation has changed just from
few months ago when we did this.
You know, it was, uh, maybe 20%, 30% to 60, 70%. Yeah.
We are seeing more people with Gen AI in their day to day.
What they do with it is another question, right?
Well, I think that's, that's
exactly the right question to ask.
How is it being used? Because when we ask the question about
how many of you are using this
(01:52):
for assurance related activities,
the number was a lot smaller.
And that's one of the things that we see in our work
with clients as well, is that people buy AI tools,
but really widespread adoption, the kind of adoption
that's leading to really transformational outcomes,
we're not seeing that yet.
And I know you were an early adopter
(02:12):
and you know, definitely I
consider you a leader in the space.
How are you thinking about driving adoption in your team
and making sure that you're getting some
of the ROI from these AI investments?
I think like they sort
of common pitfall in adopting AI is going to auditors
and asking them what the, how are you gonna use it, right?
(02:35):
It's similar to the conversations
that we've had on analytics, right?
You have a auditor asking an analytical person, like,
how do you want to use analytics?
It often comes down to people want to,
I always say AI is the new outsourcing, right?
You've been in there where
automation was the new outsourcing.
So now AI is the new outsourcing.
(02:55):
A lot of auditors want to outsource to AI what they find
tedious, you know, hard to do.
But that is not where true transformation is.
I think where true transformation is, is reimagining audit
and thinking if AI can do all the things
that audits could do
or doing manually, what should an audit shop do?
(03:17):
What is highest value?
What are the biggest insights we can bring to our execs,
boards, stakeholders?
And how do we reimagine how we work?
So in terms of think, how do I sort of encourage teams
to think about it as start with the end and work backwards.
Don't start with, okay, I have gen ai,
what could I use it for?
(03:38):
And don't throw everything at Gen ai.
And what ends up happening, which we, I think I see it, you,
you probably see it is most people use it
to wordsmith reports, issues
and things that they spend a lot of time doing.
True. There is some value in that,
but it does not fundamentally change the value
of the audit function.
Yeah, I sometimes think of it as the,
(03:59):
the paradox of quick wins.
You can go say, oh, we did all these things
and they, you know, had some incremental value,
but quick wins, rarely transform the function
and get these kind of radical improvements and efficiency
and effectiveness that we're looking
for from these big technology investments.
A hundred percent. I think the, it's the, it's quick, easy,
(04:22):
convenient for auditors,
but in that you miss the asking the bigger questions of
how do we truly transform the value
that audit banks to the enterprise.
Yeah. So you and I both love technology.
We both think that internal audit is, you know,
right in the middle of being disrupted.
I'd love to look out, you know, let's look out 10 years
and let's talk about what does an internal audit shop look
(04:45):
like 10 years from now?
When, when all of this agentic AI and ai
and all of this is, you know, more fully embedded into
what we're doing, how has internal
audit changed in your opinion?
How I think of it is, in 10 years from now,
internal audit functions
and everything that how an internal audit function
operates will fundamentally change.
(05:08):
Let me explain a little bit, give it a bit of color.
So everybody talks about AgTech ai.
So you, in my view,
you'll probably not have one autonomous AI doing it all.
But instead of that, what you will have is a core AI
that knows your enterprise policies, knows your
processes, knows your risk.
(05:30):
So you think of it
as this generalist expert on all things
related to your enterprise.
And what that will combine with is sort
of small language models
that are specialists at certain things.
So there's a small, small language model
that is a specialist at evaluating third party contracts.
There is another small language model
(05:50):
that's an expert at evaluating BCP
business continuity programs.
And essentially methodology will become, will include,
or methodology will include a library of prompts
and sort of AI tool calling that
when you are in a certain part of the,
either the planning process, risk assessment,
(06:10):
certain agents come back
and give you sort of, here's the key outliers,
here's the key risks, here are the key themes,
here are the key trends that we want to go and evaluate.
And auditor's job then becomes, instead of going
and collecting all these data
and doing stakeholder interviews, it becomes about
investigating and opining on, based on these data points,
(06:32):
where do we think risks are?
Where do we think our controls are in terms
of mitigating those risks?
So that's a pretty big change from where we are today.
So I wonder, you know, that
that's gonna take some investment,
that's gonna take some change management.
You know, if, if I, I came to you
and said, you've got an unlimited budget
for innovation, right?
Dollars don't matter.
(06:53):
We just want really innovative next
generation internal audit.
Talk to me a little bit about
how do you prioritize those investments?
Where do you start? What should you,
what should other internal audit leaders be focused on in
terms of bringing that future to the present?
I think like the mistake most people make is they pick the
tool and then they decide what to do with it.
(07:14):
So you pick gen ai then like, what can we throw at gen?
I think that's the wrong way to think about it.
In audit, you should always start with the risks.
What are your highest risks?
Then you find the tool or technology
or AI that is, uh, appropriate for that.
So in lot of cases,
you might be better off not using gen AI today, right?
Because the technology is not mature
(07:36):
enough to give you what you need.
It might be, uh, machine learning
or NLP that gives you more predictable, reliable outcomes
that can sort of fundamentally transform your process.
So in terms of where to start, you want to think about it,
will this be better into my process?
And does this now become the default
(07:57):
process by which we do this?
We cover this risk. If it is something
that sits on the side, that is a nice to have
that you could still do it without,
and there's nobody would know.
I think you've not realized that.
You know, one of the things that we've been doing at KPMG is
really kind of breaking down the internal audit process,
start to finish, right?
(08:17):
We have a whole process decomposition,
and we're going through every one of those tasks
and we're saying, which of these can be
technology enabled, right?
Not just gen ai, but really looking at the, the full stack
of, you know, gen AI is, is a piece,
but there's, you know, I'll say traditional AI ML kind
of solutions, basic analytics that a lot
(08:39):
of internal audit teams already have.
Automation, you know, whether it's RPA or workflow.
Looking at your audit management tools that you have
and thinking about how do you really optimize the use
of those and, and get value out of them.
And then going through and saying, well,
there are some things that we can a hundred percent,
we can automate this,
or we can use gen AI to perform this task.
(09:01):
And there's other tasks where we can use some kind
of technology to partially automate it,
and there may still be a human piece.
And then there's pieces like stakeholder engagement
that we may not ever want to automate
because those are the parts of our job
that we think are really important.
And where you, we still need a lot of human interaction.
And as we think about moving through that process, we try
(09:23):
to layer in a view of, well,
what is gonna be the biggest return on that investment?
And return may be that may be dollars return, may be, um,
a little bit more intangible, like, you know, what is going
to create more job satisfaction for our people?
What is gonna take something away
that they maybe aren't as interested in doing?
(09:43):
Or make the doing of that a more interesting, more rewarding
activity or what is gonna add value
to our other stakeholders, right?
Not people on our team, but
what are other executives gonna see
as valuable from the internal audit function in terms
of the value we deliver to them, maybe in the form
of reports or insights,
but also in terms of the interactions
(10:05):
with audit stakeholders
and reducing the amount of time they need
to spend on an audit, reducing the,
the non-value add time they need to spend gathering data
for us or answering questions for us,
rather than having a really meaningful dialogue.
And that's sort of been my approach is if you really kind
of flow chart out an audit and really pinpoint those areas
(10:25):
and just move through the life cycle of an audit, I think
that can be a really valuable exercise.
I think that is the right approach.
The only thing I would add is as you go through this step,
like I think it's important to go back to
what is the true value of performing this process, right?
Instead of getting caught in the mechanics of it,
I think the mistake, like people do hear that,
(10:46):
but then they got get caught up in the mechanics
of like step one, step two, step three.
Can we, which of these steps can we automate?
But I think that can distract the conversation away, right?
And it gives you something very quick and easy to latch onto
and solve for, but doesn't truly,
you know, up the value.
So if you're doing risk assessment, what you are trying
(11:08):
to opine on is what is the highest risk?
How, what evidence do I have
that we should be more concerned about these risks than we
were, uh, six months ago?
And sort of having a data-driven view of, uh,
risks across the entire life cycle.
I think that is where true value of AI is,
because ultimately what AI is able to do better is
(11:31):
take patterns, identify patterns, take these patterns
and extrapolate them
and look across the enterprise to identify
where else could this be happening?
These are not easy questions, uh, without the data,
without the technology to answer.
And those, that is where the highest value.
Yeah, I think you're exactly right.
The, like, the risk that you run into when you take this
(11:52):
very mechanistic approach is that you sort of do it in,
back in the RPA days, we used
to say don't pave the cow path, right?
And I think the same, the same risk is true if you have a
very traditional internal audit approach
and you go through step by step,
you may actually end up automating things that don't need
to be done in, in the first place in an AI driven
or datadriven world.
(12:12):
Exactly. I think like the thing is like if you are
automating something that doesn't even need to be done,
you can, you can provide a higher value by sort
of re-imagining your work.
I think automation makes it easier
to be stuck in the past sometimes.
Like I think we think of automation as the way
to the future, but sometimes
that's what's keeping you in the past
(12:33):
because you're not rethinking, okay, if
I can do something else altogether, why am I still,
you know, because now you have said these are the steps
and this is the automation you have to use,
and now you're stuck in this process
and that's all you will ever do.
That's such a great point.
Can you think of a, a process at, you know, as you've,
if you thought about Capital One,
(12:54):
you've thought about your internal audit process,
is there something where you can remember technology
fundamentally changing your approach where you, you either,
you know, to the point where making stop doing something
that you used to do
or do things in a completely different way
because of the technology that your teams have available?
I think like in general, uh,
(13:14):
looking a across my audit experience,
I'll give you a few examples for continuous monitoring,
for example, there was a time where audit, uh,
and uh, I know you have a hot take on this,
that if you're doing continuous monitoring,
you are managing somebody else's process.
But ultimately, like that was the thought process
for a while on continuous monitoring.
That you do something, you do the same thing every quarter
(13:36):
and you opine on the same thing.
But ultimately what ends up happening is your,
even your stakeholders know every quarter these are the same
questions they're gonna ask me.
And it is now no longer you've no made, no changes
to your plan scope and you're just running this process
and ultimately going from there to data-driven
(13:58):
and letting the data tell you what are the outliers,
what are the emerging risks that we've not seen?
We don't know how to opine on these or document these,
but we are gonna go and investigate
and do targeted audits where we think there is a risk
that we have never seen before
and we think that could end up, uh, materializing.
That's one example. Another is, I think going back to sort
(14:21):
of, a lot of people talk about the audit processes,
but thinking about assurance itself,
ultimately doing audits is the audit department's job,
but I don't think people think enough about
assurance itself.
So if you think about something like, uh,
going from a sample of 50 a hundred
to doing a full population test over billions
(14:41):
of transactions, what ends up happening,
even if you find something that is outlier of 1% of
that population, it helps you investigate the root causes
and get to the true deficiencies in your control environment
instead of, you know, a check the box.
We did look at it, we didn't find anything.
(15:01):
I'd love to, to double click on the continuous
monitoring idea.
So one of the things that
auditors often ask me when they start getting into the world
of continuous monitoring
or continuous auditing is when you have, I'll say anomalies,
something that requires follow up may
or may not result in a finding, you know,
(15:22):
they're disconnected from the traditional
episodic audit report.
So traditional internal audit, I have an audit,
I do the audit, I do the report,
and I may not touch that area, maybe never again,
or maybe not for a few years.
And so that report is the state of the union,
or at least our point of view.
When you have new information coming about a process every
(15:43):
month or every quarter, what's your approach
to following up on that,
but then reporting on what you find?
How do you, you know, that
that doesn't seem like an audit report,
but you also need to tell people
that you're looking at whether it's management
or the board, how do you think through that
and what's your continuous monitoring follow
(16:04):
up and reporting look like?
And I think that this is a really good question
because this is a challenge all audit,
uh, shops are going through.
And again, uh, how I did it previously, I can't today
because, you know, methodologies are different
across audit shops.
But what I've found sort of to be the best practice is
essentially you have to let the data tell you, right?
(16:25):
You cannot go in with the view that this is
how we look at continuous monitoring
and we document them, and that's it.
That is boxing yourself in.
So it could lead to one of multiple routes,
and you have to be open to those
and be also cognizant of the fact you need
to define some guardrails on which paths
(16:46):
can be taken under which circumstances
and what is the checks and controls
and reviews in place for those to happen.
So you could go down there out of, I looked at it
with confirms our view of the risk
and last audit opinion is still valid.
You document documented, uh, what has been looked at
in a way that you can leverage it if you are
(17:06):
doing an related audit
or something that it needs to be referenceable.
So the evidencing is really important of what was looked at.
How did you reach the conclusion,
even if it's not a audit report.
Very important that you always,
every time audit gives an opinion, it is grounded in, uh,
you know, rationale and documentation
that backs a third party can read
and get to the same conclusion.
(17:28):
The other routes are basically doing a targeted audit.
You're like, okay, we don't necessarily think we know enough
about this to opine.
We need to go and investigate more.
So you go down the target, it almost becomes a sort
of quasi risk assessment.
You identify something, so now we're gonna go exactly.
And, uh, or it might be you've had some conversations, uh,
your view of the risk has changed,
(17:49):
but you think you, you standard enough that you don't need
to go and deeper dive.
In that case, you would change your risk assessment
and change it from say, a medium to a high
and that would change your audit plan.
It could go one of those routes. Yeah.
So as you know, more
and more I've been seeing internal audit teams refresh their
internal audit strategy, like the global internal audit
(18:11):
standards now have prompted this rethinking of
how internal audit's gonna work.
And, and I'm seeing a lot of continuous auditing on
that, those strategies.
And I always think about the interplay between
the strategy that we wanna do
because of the nature of the business,
because of the needs of the stakeholders,
(18:33):
but that's informed
by the technological capabilities, right?
New technologies like gen ai, open strategic opportunities
that simply weren't there in the past.
And so you have this hopefully virtuous cycle of strategy
influencing decisions about process and technology,
but technology capabilities influencing our strategy.
(18:53):
And I'd I'd love to hear you kind of think about
how you have approached your internal audit strategy
and your innovation strategy, um, at cap one, given some
of the disruptions in technology lately.
And I think, uh, I always say like, innovation is
where what's impactful meets what's possible, right?
And in light of what's possible, you essentially go back
(19:14):
and reimagine sort of which are the processes?
Ultimately I go, if the way you think about it is also it
has to be rooted in what audits value is.
So you have to look at
and say, where are the areas where we are not able
to provide a higher level of assurance
because we're limited by what was possible.
So you go and explore those, right?
(19:35):
Audit departments and audits are drowning in
data these days.
Like it's the opposite problem of say 20 years ago
where people like, where I don't know how
to find data now people are like, I have just too much data.
I have no confidence that I'm opining, uh,
my opinion factors in everything that I've been given.
And, and, and by the way, too much good data, right?
(19:55):
Good high quality, curated managed data is everywhere.
And we sort of have the opposite problem of, you know,
you would, you would hear 10 years ago, well,
we've got all this data, but it's not
really reliable, it's not really good.
It's, it takes a lot of prep to work with that data
and now so many investments have been made,
we're reaping the benefits of that
(20:16):
and we're a little bit overwhelmed, you know?
Yeah. So I think ultimately
how I think about it strategy wise
is like, what is the risk?
How are we gonna opine on it?
What does the technology allow us to do?
And then you come to how do I reframe my entire process?
So maybe going back to a little bit about
audit cycles, right?
So we've traditionally had like these are, these are,
(20:38):
this is how you cycle through audits,
but now you can move to sort of a more flexible audit plan
where you pick go where the highest risks are,
you place reliance on the work you've done
unless there is a change in risk.
And instead of like doing the same audit
and coming up with the same opinion,
you can now do flexible auditing
and let it be driven by risks in the enterprise.
(21:00):
So all of this has been made possible by sort of some
of the emerging technologies,
but what we are still not getting enough of maybe,
and I think I I hope in the future years will be, is a lot
of the unstructured data can now be analyzed in the same way
that structured data could be analyzed, right?
So a lot of places where auditor says,
(21:21):
I don't have good data
that actually meant I don't have tabular data,
and now they have, you know, governance memos,
risk assessments, pages
and pages of documents that can be converted into
structured data that you can analyze.
Yeah, I think that's such a great point
because the ability to not just analyze
(21:41):
and synthesize these large unstructured data sets,
but the ability to now on the fly create knowledge
assistance that will help you deal with, you know,
regulations, with frameworks,
with internal audit methodologies, you know,
the capabilities of of, of things like, uh, co-pilot
and Gemini and these other things that allow you
to just reference your own documents really opens up a whole
(22:03):
world of how we interact
with these large unstructured documents and data sets.
I think that's a really a great point.
And I think like how I also think about it, right as you go
through it, like one of the things I would always say is the
accuracy of the model is the clarity
of the subject matter expert, right?
It's the ability of your subject matter expert, whether
(22:27):
that's a regulatory expert
or a risk expert to articulate
how do they pass information, how do they get to an opinion?
And then that allows you to teach a high
or, you know, build automation that follows the
best in class approach to evaluating risk
(22:47):
and opining on, right?
So I imagine a lot of subject matter experts would hear that
and think, oh no, like, that's the end of my world, right?
So, you know, talk to me a little bit about
the human side of this.
How are we, you know, I I I don't think either of us believe
that AI or AI agents are here to take everyone's jobs.
(23:08):
In my experience, the list of to-dos
and internal audit is way longer than we've ever gotten to.
So if we get to a few more, I think that's great,
but as an innovator
and someone who introduces technology, not just gene ai,
but you've been doing this for years, right?
So you introduce new technology, RPA was like, oh no,
the automation's coming for our jobs
and then gene AI came out coming for our jobs.
But how do you, how do you think through that
(23:29):
and how do you talk to teams
or other leaders about how this technology impacts
or influences their staffing and resourcing strategies?
And I think for, for the foreseeable future, a AI
or any of these technologies are gonna be human in the
middle for sure, right?
There is no autonomous ai, at least
(23:50):
for the foreseeable future.
I do want to go back to how do,
how should people think about it, right?
I think you are right in that people think of it
in a very sort of, will this take my job away sometimes and
or in a very narrow, how can this make my life easier?
And I think those are both narrow questions, right?
(24:12):
It is in fact an exciting time to be in order
because you can do all like the most interesting work of
when you have all this technology available to give you the
patterns, the risks, the themes and trends to investigate.
And basically it allows you to be more,
(24:34):
do more interesting work,
but also do the highest value work, which is work
that requires expertise, human judgment,
and navigating gray space
that you cannot simply train a model to do,
or you cannot have this technology do it for you.
And I think that's the way to think about it. Yeah.
(24:54):
I, I love that. I mean, the thing that drew me
to audit at the beginning of my career was
that I feel like I'm a really inherently curious person.
I always want to ask questions
and the ability to go into the business
and ask questions about virtually any business process,
any function that was really, really appealing to me.
I never got into internal audit
because I wanted to like test three-way
(25:16):
match controls, right?
Or look at change management controls or anything like that.
So I think if you get down to the root of
why do we love what we do?
Why is this a job that I want to have?
Those are not the kinds of things that, at least in, in,
in my experience, that we are targeting
technologies at, right?
We're targeting technology at how do I do, uh, you know,
(25:37):
user access testing more effectively,
or how do I make sure that, you know,
our financial controls are working the way they're supposed
to work at scale,
maybe a hundred percent population or whatever it is.
It's not about eliminating curiosity,
it's about feeding curiosity.
And I think that's the thing
that people are maybe missing in this, is
(25:57):
all this technology gives audit a better seat at the table,
and audit does have a unique seat at the table, which is
what I personally found most exciting coming into audit in
the second half of my career.
Uh, because in audit is a unique place
because you're looking across businesses, across processes
and the ability to connect the dots
(26:19):
and sort of bring these insights
that people like I would not have known,
just being in my own silo, in my own world,
that these problems are more universal than mine,
and there are others in the enterprise
or the wider industry who found solution to these.
So that ability to find sort of deep rooted problems,
common thematic problems that,
(26:41):
and help people think through the solutions, I think that
that is where internal audit can really, you know,
bring a lot of value to all enterprises
and it, this technology, uh, transformation enables order
to have a better seat at the table, have more sort
of higher impact conversation, then sort of narrow
why do 50 people have access
(27:02):
to this application when they shouldn't?
Which is really sort of not the kind of thing you want
to talk to execs about, right?
Yeah, exactly. Well, I think, I think it comes back to that,
that fear is, is rooted in not having a culture
of innovation where you expect things to change
and you expect to disrupt yourself.
And as a, as an innovation leader,
(27:23):
and you've got a pretty big team and,
and an extended team of, of co-source providers and,
and vendors and others, building that culture
of innovation is so core to innovating,
to keeping people engaged, to dealing with some
of those concerns about is it coming from my job?
Let's talk a little bit about things that leaders can do
to drive the spirit of innovation in their team so
(27:46):
that they really embrace this new technology rather than
being concerned about it or confused by it.
So I think as leaders, there's, you need to think
of innovation in multiple ways, right?
Uh, a lot of leaders just think of innovation in sort
of the technology.
Like, just tell me what is a technology,
show me what it can do.
And that's how they sort of think of innovation.
(28:08):
But that leaves a lot at, on the table.
How I think about it is like you're thinking about like
what has, like cutting edge innovation essentially
for me is what's only just become possible
meets impactful, right?
So you need to look at, okay, what could I not have done?
You know, there's that kind
of top down transformative innovation,
but what you also need to embed into your day-to-day is
(28:32):
identifying innovation where you, while you're doing it,
you're like, we struggle
with coming up with an opinion on this.
We struggle to risk assess this,
and you, you are collecting all these ideas.
So that's what we, I've done at every place I've been is
introduced sort of a process
where people doing the job while they're doing the job,
they can submit in two minutes an idea.
(28:53):
And you look across these, the power of that has been,
when you look across these ideas also you see a pattern
and that tells you a lot about where your processes,
where your value is weaker,
where your personnel are struggling to perform, execute
what you state as a policy or as a goal.
(29:14):
And connecting that like, this is where we want to go.
This is where our people are struggling, then allows you
to find technology that solves for those problems
that truly transform the day to day.
Yeah. I, I think that's, that's so great.
And I think, you know, you're in a really fortunate position
of working for a company where innovation is,
is core to your brand.
And so it's, it's sort of in the,
(29:36):
in the soup at cap one, right?
Um, and, and I think regardless of whether
that's the world you're li you're living in,
or if you're in an organization that's maybe not quite
as forward leaning when it comes to innovation in general
and tech specifically, the advice I always give
to internal audit teams is you should
keep pace with the business.
(29:56):
It's pretty rare, in my experience,
that internal audit is the first adopter
of any new technology.
It happens sometimes, but I think it,
I think it's the exception rather than the rule.
And what I think is far more common is
that internal audit lags the business by several years
in terms of their adoption of technology.
Somebody in operations will get some great new tool
(30:17):
and then internal audit gets around to it later.
I wonder, do you agree with that idea of, of keeping pace
and, and how should I think about, um,
or how should audit leaders think about really capitalizing
on the investments that have been made at an enterprise
level without putting technology first,
but understanding that you've got all these tools on the
(30:39):
workbench, you've gotta decide which ones you wanna pick up
and when you wanna pick 'em up.
So I have a sort of a nuance view on one of those things,
but I will get to sort of your first point of keeping pace.
I think as more and more controls get automated, more
and more technology gets introduced in risk sort
of managing risk and how operations work, you do need
to keep pace because you cannot challenge a model
(31:01):
with a sample testing
or if there is a model out there, you need to be able
to build a challenger model.
That challenge is sort of the efficacy of, uh,
the models used similar to that.
I think there is a need for order to be able to
process the same level of information, right?
And make their own parallel independent opinion
of those decisions in the same way that first
(31:23):
and second line functions are doing it.
But where I sort of have a now more nuanced view,
having been in this industry for a long time,
I did have the same view that coming into audit.
I was like, is audit really the place that will be for me
because I want to be at the frontier of innovation.
What I've come to realize over sort
of my career is audit can be actually the beacon
(31:45):
for innovation for the entire bank.
Because one audit has several advantages in that, right?
One, while innovation comes in ops
and revenue generating functions,
I have in my career seen innovation
and risk management functions is lagging.
And that's where I do think in fact,
audit might be ahead in some cases
of risk management functions, which then helps audit, inform
(32:09):
and educate business on how they can better manage, assess,
uh, and, you know, mitigate the risk.
The second thing is audit has a unique, uh, point of view in
that you're able to look across the enterprise.
So going to the board,
you can say these different functions, they're struggling
with the same root cause.
So we are, these are the things we need
to prioritize as an enterprise.
(32:30):
So again, you do need to keep pace,
but you should also, I think audit departments should think
of like, where is it that audit can provide
that unique value to the enterprise, be a beacon
for innovation and risk management.
Because a lot of businesses are essentially, uh,
risk management is a core part of the business.
I mean, cer certainly in financial services,
it's absolutely front and center.
(32:52):
Yeah, I love internal audit as a beacon for innovation.
We should get some bumper stickers printed up. The I a a.
That's fantastic. You know, I think in early 2025,
we can't have a conversation without looking into agents.
It is the topic right now. Everyone is talking about agents.
Maybe just to start off like gimme your definition of agents
(33:15):
and then let's talk a little bit about some of the places
that you're either using
or planning to use agents in, um, the internal audit world.
So I think, uh, lot of, uh, talk about agent tech ai.
I think autonomous ai, I do personally think some
of it is overblown, uh, artificial general intelligence.
Some of it is overblown, uh, in that, like, there,
(33:38):
there will be a niche of the chat GPTs, the, uh, anthropics
of the world who have the resources, energy, billions
of dollars to burn on creating
artificial general intelligence.
Where I focus as a sort of leader in innovation, uh,
since I don't work for Sam Altman, is sort
of application, right?
(33:59):
Where can we apply these, uh, technologies
and get real business value?
I think, again, coming to my sort of view on this
and where I see this going, I think
for the foreseeable future it'll be human in the middle.
There will be no, you know, self-driving audit, uh,
shop, so to speak.
(34:19):
And how I think about it is, like
how leaders should think about it is
how does this transform auditing?
What are the things that are core principles do you want?
And if you are thinking about building
and investing in this kind of a thing, you want your, think
of it as a, as you develop a
expert in your audit department,
(34:40):
your expert would know the core principles
of auditing, right?
Uh, know how to take a risk based approach, fundamental sort
of principles that never change.
So those are things that you are training your model, right?
You're investing money, teach them these things
and these things are fundamental.
They will never change. Then there are aspects which are
(35:00):
like, general, I don't need
to teach this person specifically, this is just
how things are done.
Uh, communication, you know, basic general ideas.
So those are things that you can take out
of the box from a lot of the, you know, open source models.
Then there are things where you say, okay, I need areas
where I want to teach somebody how
(35:23):
to look at this specific process in our enterprise
and opine in this according
to these specific mythology that we've developed.
So in those cases, you get sort of enter the space of rag
and you know, having sort of using a full
like external developed general model,
(35:44):
but building small specialized expertise
and use your own data instead
of just using general out of the box information.
So again, I give a very, I try to keep it non-technical,
but essentially there is difference between
how much resources you need to spend.
You don't want to be training your models on things
that constantly change,
(36:04):
and you are not really looking for the model
to know and remember all the time.
And there are things that are basically I needed just
to do this audit step,
and then you can use them as in the context window
and you basically start doing them out of the box instead
of investing, trying to over engineer it.
(36:26):
I love that. So one last question before we wrap it up here.
You didn't start off as an internal auditor
and now you've been an internal audit for several years.
So what is the most controversial point
of view you have about internal audit coming in from
an outside perspective?
(36:46):
So my most controversial view has been,
and I think people do, uh,
find it interesting also, is that audit.
If you think about audit as a profession, it's sort of,
it's the fundamental pillar of capitalism.
So audit has been done for like over hundreds of years,
but more
or less in the same way, there's been more data,
(37:07):
there's been more technology,
but it's more or less in the same way.
My view is we are at a point in, in the audit, sort of
where we are in audit, how audits are done,
what an auditor's job is, is at, we are at the precipice
of it can be fundamentally transformed.
And to do that, we need to reimagine
(37:29):
how audits have always been done.
And I think that is sort of something that leaders need to
start thinking about more deeply, right?
We've done audit cycles, we've done, uh, cycle audits,
we've done, you know, risk assessment memos, we've done
continuous monitoring dashboards,
but all of that doesn't exist in a world where you're able
(37:52):
to, uh, now process information, real time, provide,
connect, connect the dots across the enterprise, have more,
you know, bigger, deeper conversations instead of narrow
specific views that maybe are lower value.
This has been a great conversation.
(38:13):
Imran Mulah, thank you so much for being here today.
Thank you, Charles. Pleasure.
Small internal audit teams boost your efficiency at the
2025 audit sphere.
This one day virtual conference on June 17th offers expert
strategies and practical tools just for you.
Register now@theiia.org.
(38:35):
If you like this podcast, please subscribe and rate us.
You can subscribe wherever you get your podcasts.
You can also catch other episodes on YouTube or@theia.org.
That's THEI a.org.
The Institute of Internal Auditors presents all
things internal audit tech.
In this episode, Charles King talks
with Emron moolah about the transformative potential
of AI In internal auditing,
they discuss AI's evolution from basic automation
to advanced applications, the challenges of adoption
and the future of internal auditing.
(00:22):
The conversation also covers continuous monitoring,
risk assessment and the role
of human judgment in an AI driven world.
Iran, I'm so excited to have this conversation with you.
I feel like you always bring so many great ideas
and sort of forward thinking
and what, what you do at CAP one
is, is really, really impressive.
It's great to be here and talking to you.
(00:44):
Uh, you always bring out the best conversation.
So excited for this one. Excellent. All right.
Well, when we were back at gam, we had a session on AI
with Anton Dam from Audit board.
And you know, one of the things that was really eye-opening
for me in that session is every time I do a session on ai,
I ask the audience, how many of you are using AI at work?
(01:08):
And historically, over the past year,
let's say it's been roughly half, maybe,
maybe a slight majority, 60% of people raise their hands.
When we were having that session,
it was three quarters at least, maybe more.
I do think like a lot of more audit shops have gone ahead
and got their auditors, uh, gen AI to sort
of the white label and just out of the box gen ai.
(01:30):
So I think definitely the conversation has changed just from
few months ago when we did this.
You know, it was, uh, maybe 20%, 30% to 60, 70%. Yeah.
We are seeing more people with Gen AI in their day to day.
What they do with it is another question, right?
Well, I think that's, that's
exactly the right question to ask.
How is it being used? Because when we ask the question about
how many of you are using this
(01:52):
for assurance related activities,
the number was a lot smaller.
And that's one of the things that we see in our work
with clients as well, is that people buy AI tools,
but really widespread adoption, the kind of adoption
that's leading to really transformational outcomes,
we're not seeing that yet.
And I know you were an early adopter
(02:12):
and you know, definitely I
consider you a leader in the space.
How are you thinking about driving adoption in your team
and making sure that you're getting some
of the ROI from these AI investments?
I think like they sort
of common pitfall in adopting AI is going to auditors
and asking them what the, how are you gonna use it, right?
(02:35):
It's similar to the conversations
that we've had on analytics, right?
You have a auditor asking an analytical person, like,
how do you want to use analytics?
It often comes down to people want to,
I always say AI is the new outsourcing, right?
You've been in there where
automation was the new outsourcing.
So now AI is the new outsourcing.
(02:55):
A lot of auditors want to outsource to AI what they find
tedious, you know, hard to do.
But that is not where true transformation is.
I think where true transformation is, is reimagining audit
and thinking if AI can do all the things
that audits could do
or doing manually, what should an audit shop do?
(03:17):
What is highest value?
What are the biggest insights we can bring to our execs,
boards, stakeholders?
And how do we reimagine how we work?
So in terms of think, how do I sort of encourage teams
to think about it as start with the end and work backwards.
Don't start with, okay, I have gen ai,
what could I use it for?
(03:38):
And don't throw everything at Gen ai.
And what ends up happening, which we, I think I see it, you,
you probably see it is most people use it
to wordsmith reports, issues
and things that they spend a lot of time doing.
True. There is some value in that,
but it does not fundamentally change the value
of the audit function.
Yeah, I sometimes think of it as the,
(03:59):
the paradox of quick wins.
You can go say, oh, we did all these things
and they, you know, had some incremental value,
but quick wins, rarely transform the function
and get these kind of radical improvements and efficiency
and effectiveness that we're looking
for from these big technology investments.
A hundred percent. I think the, it's the, it's quick, easy,
(04:22):
convenient for auditors,
but in that you miss the asking the bigger questions of
how do we truly transform the value
that audit banks to the enterprise.
Yeah. So you and I both love technology.
We both think that internal audit is, you know,
right in the middle of being disrupted.
I'd love to look out, you know, let's look out 10 years
and let's talk about what does an internal audit shop look
(04:45):
like 10 years from now?
When, when all of this agentic AI and ai
and all of this is, you know, more fully embedded into
what we're doing, how has internal
audit changed in your opinion?
How I think of it is, in 10 years from now,
internal audit functions
and everything that how an internal audit function
operates will fundamentally change.
(05:08):
Let me explain a little bit, give it a bit of color.
So everybody talks about AgTech ai.
So you, in my view,
you'll probably not have one autonomous AI doing it all.
But instead of that, what you will have is a core AI
that knows your enterprise policies, knows your
processes, knows your risk.
(05:30):
So you think of it
as this generalist expert on all things
related to your enterprise.
And what that will combine with is sort
of small language models
that are specialists at certain things.
So there's a small, small language model
that is a specialist at evaluating third party contracts.
There is another small language model
(05:50):
that's an expert at evaluating BCP
business continuity programs.
And essentially methodology will become, will include,
or methodology will include a library of prompts
and sort of AI tool calling that
when you are in a certain part of the,
either the planning process, risk assessment,
(06:10):
certain agents come back
and give you sort of, here's the key outliers,
here's the key risks, here are the key themes,
here are the key trends that we want to go and evaluate.
And auditor's job then becomes, instead of going
and collecting all these data
and doing stakeholder interviews, it becomes about
investigating and opining on, based on these data points,
(06:32):
where do we think risks are?
Where do we think our controls are in terms
of mitigating those risks?
So that's a pretty big change from where we are today.
So I wonder, you know, that
that's gonna take some investment,
that's gonna take some change management.
You know, if, if I, I came to you
and said, you've got an unlimited budget
for innovation, right?
Dollars don't matter.
(06:53):
We just want really innovative next
generation internal audit.
Talk to me a little bit about
how do you prioritize those investments?
Where do you start? What should you,
what should other internal audit leaders be focused on in
terms of bringing that future to the present?
I think like the mistake most people make is they pick the
tool and then they decide what to do with it.
(07:14):
So you pick gen ai then like, what can we throw at gen?
I think that's the wrong way to think about it.
In audit, you should always start with the risks.
What are your highest risks?
Then you find the tool or technology
or AI that is, uh, appropriate for that.
So in lot of cases,
you might be better off not using gen AI today, right?
Because the technology is not mature
(07:36):
enough to give you what you need.
It might be, uh, machine learning
or NLP that gives you more predictable, reliable outcomes
that can sort of fundamentally transform your process.
So in terms of where to start, you want to think about it,
will this be better into my process?
And does this now become the default
(07:57):
process by which we do this?
We cover this risk. If it is something
that sits on the side, that is a nice to have
that you could still do it without,
and there's nobody would know.
I think you've not realized that.
You know, one of the things that we've been doing at KPMG is
really kind of breaking down the internal audit process,
start to finish, right?
(08:17):
We have a whole process decomposition,
and we're going through every one of those tasks
and we're saying, which of these can be
technology enabled, right?
Not just gen ai, but really looking at the, the full stack
of, you know, gen AI is, is a piece,
but there's, you know, I'll say traditional AI ML kind
of solutions, basic analytics that a lot
(08:39):
of internal audit teams already have.
Automation, you know, whether it's RPA or workflow.
Looking at your audit management tools that you have
and thinking about how do you really optimize the use
of those and, and get value out of them.
And then going through and saying, well,
there are some things that we can a hundred percent,
we can automate this,
or we can use gen AI to perform this task.
(09:01):
And there's other tasks where we can use some kind
of technology to partially automate it,
and there may still be a human piece.
And then there's pieces like stakeholder engagement
that we may not ever want to automate
because those are the parts of our job
that we think are really important.
And where you, we still need a lot of human interaction.
And as we think about moving through that process, we try
(09:23):
to layer in a view of, well,
what is gonna be the biggest return on that investment?
And return may be that may be dollars return, may be, um,
a little bit more intangible, like, you know, what is going
to create more job satisfaction for our people?
What is gonna take something away
that they maybe aren't as interested in doing?
(09:43):
Or make the doing of that a more interesting, more rewarding
activity or what is gonna add value
to our other stakeholders, right?
Not people on our team, but
what are other executives gonna see
as valuable from the internal audit function in terms
of the value we deliver to them, maybe in the form
of reports or insights,
but also in terms of the interactions
(10:05):
with audit stakeholders
and reducing the amount of time they need
to spend on an audit, reducing the,
the non-value add time they need to spend gathering data
for us or answering questions for us,
rather than having a really meaningful dialogue.
And that's sort of been my approach is if you really kind
of flow chart out an audit and really pinpoint those areas
(10:25):
and just move through the life cycle of an audit, I think
that can be a really valuable exercise.
I think that is the right approach.
The only thing I would add is as you go through this step,
like I think it's important to go back to
what is the true value of performing this process, right?
Instead of getting caught in the mechanics of it,
I think the mistake, like people do hear that,
(10:46):
but then they got get caught up in the mechanics
of like step one, step two, step three.
Can we, which of these steps can we automate?
But I think that can distract the conversation away, right?
And it gives you something very quick and easy to latch onto
and solve for, but doesn't truly,
you know, up the value.
So if you're doing risk assessment, what you are trying
(11:08):
to opine on is what is the highest risk?
How, what evidence do I have
that we should be more concerned about these risks than we
were, uh, six months ago?
And sort of having a data-driven view of, uh,
risks across the entire life cycle.
I think that is where true value of AI is,
because ultimately what AI is able to do better is
(11:31):
take patterns, identify patterns, take these patterns
and extrapolate them
and look across the enterprise to identify
where else could this be happening?
These are not easy questions, uh, without the data,
without the technology to answer.
And those, that is where the highest value.
Yeah, I think you're exactly right.
The, like, the risk that you run into when you take this
(11:52):
very mechanistic approach is that you sort of do it in,
back in the RPA days, we used
to say don't pave the cow path, right?
And I think the same, the same risk is true if you have a
very traditional internal audit approach
and you go through step by step,
you may actually end up automating things that don't need
to be done in, in the first place in an AI driven
or datadriven world.
(12:12):
Exactly. I think like the thing is like if you are
automating something that doesn't even need to be done,
you can, you can provide a higher value by sort
of re-imagining your work.
I think automation makes it easier
to be stuck in the past sometimes.
Like I think we think of automation as the way
to the future, but sometimes
that's what's keeping you in the past
(12:33):
because you're not rethinking, okay, if
I can do something else altogether, why am I still,
you know, because now you have said these are the steps
and this is the automation you have to use,
and now you're stuck in this process
and that's all you will ever do.
That's such a great point.
Can you think of a, a process at, you know, as you've,
if you thought about Capital One,
(12:54):
you've thought about your internal audit process,
is there something where you can remember technology
fundamentally changing your approach where you, you either,
you know, to the point where making stop doing something
that you used to do
or do things in a completely different way
because of the technology that your teams have available?
I think like in general, uh,
(13:14):
looking a across my audit experience,
I'll give you a few examples for continuous monitoring,
for example, there was a time where audit, uh,
and uh, I know you have a hot take on this,
that if you're doing continuous monitoring,
you are managing somebody else's process.
But ultimately, like that was the thought process
for a while on continuous monitoring.
That you do something, you do the same thing every quarter
(13:36):
and you opine on the same thing.
But ultimately what ends up happening is your,
even your stakeholders know every quarter these are the same
questions they're gonna ask me.
And it is now no longer you've no made, no changes
to your plan scope and you're just running this process
and ultimately going from there to data-driven
(13:58):
and letting the data tell you what are the outliers,
what are the emerging risks that we've not seen?
We don't know how to opine on these or document these,
but we are gonna go and investigate
and do targeted audits where we think there is a risk
that we have never seen before
and we think that could end up, uh, materializing.
That's one example. Another is, I think going back to sort
(14:21):
of, a lot of people talk about the audit processes,
but thinking about assurance itself,
ultimately doing audits is the audit department's job,
but I don't think people think enough about
assurance itself.
So if you think about something like, uh,
going from a sample of 50 a hundred
to doing a full population test over billions
(14:41):
of transactions, what ends up happening,
even if you find something that is outlier of 1% of
that population, it helps you investigate the root causes
and get to the true deficiencies in your control environment
instead of, you know, a check the box.
We did look at it, we didn't find anything.
(15:01):
I'd love to, to double click on the continuous
monitoring idea.
So one of the things that
auditors often ask me when they start getting into the world
of continuous monitoring
or continuous auditing is when you have, I'll say anomalies,
something that requires follow up may
or may not result in a finding, you know,
(15:22):
they're disconnected from the traditional
episodic audit report.
So traditional internal audit, I have an audit,
I do the audit, I do the report,
and I may not touch that area, maybe never again,
or maybe not for a few years.
And so that report is the state of the union,
or at least our point of view.
When you have new information coming about a process every
(15:43):
month or every quarter, what's your approach
to following up on that,
but then reporting on what you find?
How do you, you know, that
that doesn't seem like an audit report,
but you also need to tell people
that you're looking at whether it's management
or the board, how do you think through that
and what's your continuous monitoring follow
(16:04):
up and reporting look like?
And I think that this is a really good question
because this is a challenge all audit,
uh, shops are going through.
And again, uh, how I did it previously, I can't today
because, you know, methodologies are different
across audit shops.
But what I've found sort of to be the best practice is
essentially you have to let the data tell you, right?
(16:25):
You cannot go in with the view that this is
how we look at continuous monitoring
and we document them, and that's it.
That is boxing yourself in.
So it could lead to one of multiple routes,
and you have to be open to those
and be also cognizant of the fact you need
to define some guardrails on which paths
(16:46):
can be taken under which circumstances
and what is the checks and controls
and reviews in place for those to happen.
So you could go down there out of, I looked at it
with confirms our view of the risk
and last audit opinion is still valid.
You document documented, uh, what has been looked at
in a way that you can leverage it if you are
(17:06):
doing an related audit
or something that it needs to be referenceable.
So the evidencing is really important of what was looked at.
How did you reach the conclusion,
even if it's not a audit report.
Very important that you always,
every time audit gives an opinion, it is grounded in, uh,
you know, rationale and documentation
that backs a third party can read
and get to the same conclusion.
(17:28):
The other routes are basically doing a targeted audit.
You're like, okay, we don't necessarily think we know enough
about this to opine.
We need to go and investigate more.
So you go down the target, it almost becomes a sort
of quasi risk assessment.
You identify something, so now we're gonna go exactly.
And, uh, or it might be you've had some conversations, uh,
your view of the risk has changed,
(17:49):
but you think you, you standard enough that you don't need
to go and deeper dive.
In that case, you would change your risk assessment
and change it from say, a medium to a high
and that would change your audit plan.
It could go one of those routes. Yeah.
So as you know, more
and more I've been seeing internal audit teams refresh their
internal audit strategy, like the global internal audit
(18:11):
standards now have prompted this rethinking of
how internal audit's gonna work.
And, and I'm seeing a lot of continuous auditing on
that, those strategies.
And I always think about the interplay between
the strategy that we wanna do
because of the nature of the business,
because of the needs of the stakeholders,
(18:33):
but that's informed
by the technological capabilities, right?
New technologies like gen ai, open strategic opportunities
that simply weren't there in the past.
And so you have this hopefully virtuous cycle of strategy
influencing decisions about process and technology,
but technology capabilities influencing our strategy.
(18:53):
And I'd I'd love to hear you kind of think about
how you have approached your internal audit strategy
and your innovation strategy, um, at cap one, given some
of the disruptions in technology lately.
And I think, uh, I always say like, innovation is
where what's impactful meets what's possible, right?
And in light of what's possible, you essentially go back
(19:14):
and reimagine sort of which are the processes?
Ultimately I go, if the way you think about it is also it
has to be rooted in what audits value is.
So you have to look at
and say, where are the areas where we are not able
to provide a higher level of assurance
because we're limited by what was possible.
So you go and explore those, right?
(19:35):
Audit departments and audits are drowning in
data these days.
Like it's the opposite problem of say 20 years ago
where people like, where I don't know how
to find data now people are like, I have just too much data.
I have no confidence that I'm opining, uh,
my opinion factors in everything that I've been given.
And, and, and by the way, too much good data, right?
(19:55):
Good high quality, curated managed data is everywhere.
And we sort of have the opposite problem of, you know,
you would, you would hear 10 years ago, well,
we've got all this data, but it's not
really reliable, it's not really good.
It's, it takes a lot of prep to work with that data
and now so many investments have been made,
we're reaping the benefits of that
(20:16):
and we're a little bit overwhelmed, you know?
Yeah. So I think ultimately
how I think about it strategy wise
is like, what is the risk?
How are we gonna opine on it?
What does the technology allow us to do?
And then you come to how do I reframe my entire process?
So maybe going back to a little bit about
audit cycles, right?
So we've traditionally had like these are, these are,
(20:38):
this is how you cycle through audits,
but now you can move to sort of a more flexible audit plan
where you pick go where the highest risks are,
you place reliance on the work you've done
unless there is a change in risk.
And instead of like doing the same audit
and coming up with the same opinion,
you can now do flexible auditing
and let it be driven by risks in the enterprise.
(21:00):
So all of this has been made possible by sort of some
of the emerging technologies,
but what we are still not getting enough of maybe,
and I think I I hope in the future years will be, is a lot
of the unstructured data can now be analyzed in the same way
that structured data could be analyzed, right?
So a lot of places where auditor says,
(21:21):
I don't have good data
that actually meant I don't have tabular data,
and now they have, you know, governance memos,
risk assessments, pages
and pages of documents that can be converted into
structured data that you can analyze.
Yeah, I think that's such a great point
because the ability to not just analyze
(21:41):
and synthesize these large unstructured data sets,
but the ability to now on the fly create knowledge
assistance that will help you deal with, you know,
regulations, with frameworks,
with internal audit methodologies, you know,
the capabilities of of, of things like, uh, co-pilot
and Gemini and these other things that allow you
to just reference your own documents really opens up a whole
(22:03):
world of how we interact
with these large unstructured documents and data sets.
I think that's a really a great point.
And I think like how I also think about it, right as you go
through it, like one of the things I would always say is the
accuracy of the model is the clarity
of the subject matter expert, right?
It's the ability of your subject matter expert, whether
(22:27):
that's a regulatory expert
or a risk expert to articulate
how do they pass information, how do they get to an opinion?
And then that allows you to teach a high
or, you know, build automation that follows the
best in class approach to evaluating risk
(22:47):
and opining on, right?
So I imagine a lot of subject matter experts would hear that
and think, oh no, like, that's the end of my world, right?
So, you know, talk to me a little bit about
the human side of this.
How are we, you know, I I I don't think either of us believe
that AI or AI agents are here to take everyone's jobs.
(23:08):
In my experience, the list of to-dos
and internal audit is way longer than we've ever gotten to.
So if we get to a few more, I think that's great,
but as an innovator
and someone who introduces technology, not just gene ai,
but you've been doing this for years, right?
So you introduce new technology, RPA was like, oh no,
the automation's coming for our jobs
and then gene AI came out coming for our jobs.
But how do you, how do you think through that
(23:29):
and how do you talk to teams
or other leaders about how this technology impacts
or influences their staffing and resourcing strategies?
And I think for, for the foreseeable future, a AI
or any of these technologies are gonna be human in the
middle for sure, right?
There is no autonomous ai, at least
(23:50):
for the foreseeable future.
I do want to go back to how do,
how should people think about it, right?
I think you are right in that people think of it
in a very sort of, will this take my job away sometimes and
or in a very narrow, how can this make my life easier?
And I think those are both narrow questions, right?
(24:12):
It is in fact an exciting time to be in order
because you can do all like the most interesting work of
when you have all this technology available to give you the
patterns, the risks, the themes and trends to investigate.
And basically it allows you to be more,
(24:34):
do more interesting work,
but also do the highest value work, which is work
that requires expertise, human judgment,
and navigating gray space
that you cannot simply train a model to do,
or you cannot have this technology do it for you.
And I think that's the way to think about it. Yeah.
(24:54):
I, I love that. I mean, the thing that drew me
to audit at the beginning of my career was
that I feel like I'm a really inherently curious person.
I always want to ask questions
and the ability to go into the business
and ask questions about virtually any business process,
any function that was really, really appealing to me.
I never got into internal audit
because I wanted to like test three-way
(25:16):
match controls, right?
Or look at change management controls or anything like that.
So I think if you get down to the root of
why do we love what we do?
Why is this a job that I want to have?
Those are not the kinds of things that, at least in, in,
in my experience, that we are targeting
technologies at, right?
We're targeting technology at how do I do, uh, you know,
(25:37):
user access testing more effectively,
or how do I make sure that, you know,
our financial controls are working the way they're supposed
to work at scale,
maybe a hundred percent population or whatever it is.
It's not about eliminating curiosity,
it's about feeding curiosity.
And I think that's the thing
that people are maybe missing in this, is
(25:57):
all this technology gives audit a better seat at the table,
and audit does have a unique seat at the table, which is
what I personally found most exciting coming into audit in
the second half of my career.
Uh, because in audit is a unique place
because you're looking across businesses, across processes
and the ability to connect the dots
(26:19):
and sort of bring these insights
that people like I would not have known,
just being in my own silo, in my own world,
that these problems are more universal than mine,
and there are others in the enterprise
or the wider industry who found solution to these.
So that ability to find sort of deep rooted problems,
common thematic problems that,
(26:41):
and help people think through the solutions, I think that
that is where internal audit can really, you know,
bring a lot of value to all enterprises
and it, this technology, uh, transformation enables order
to have a better seat at the table, have more sort
of higher impact conversation, then sort of narrow
why do 50 people have access
(27:02):
to this application when they shouldn't?
Which is really sort of not the kind of thing you want
to talk to execs about, right?
Yeah, exactly. Well, I think, I think it comes back to that,
that fear is, is rooted in not having a culture
of innovation where you expect things to change
and you expect to disrupt yourself.
And as a, as an innovation leader,
(27:23):
and you've got a pretty big team and,
and an extended team of, of co-source providers and,
and vendors and others, building that culture
of innovation is so core to innovating,
to keeping people engaged, to dealing with some
of those concerns about is it coming from my job?
Let's talk a little bit about things that leaders can do
to drive the spirit of innovation in their team so
(27:46):
that they really embrace this new technology rather than
being concerned about it or confused by it.
So I think as leaders, there's, you need to think
of innovation in multiple ways, right?
Uh, a lot of leaders just think of innovation in sort
of the technology.
Like, just tell me what is a technology,
show me what it can do.
And that's how they sort of think of innovation.
(28:08):
But that leaves a lot at, on the table.
How I think about it is like you're thinking about like
what has, like cutting edge innovation essentially
for me is what's only just become possible
meets impactful, right?
So you need to look at, okay, what could I not have done?
You know, there's that kind
of top down transformative innovation,
but what you also need to embed into your day-to-day is
(28:32):
identifying innovation where you, while you're doing it,
you're like, we struggle
with coming up with an opinion on this.
We struggle to risk assess this,
and you, you are collecting all these ideas.
So that's what we, I've done at every place I've been is
introduced sort of a process
where people doing the job while they're doing the job,
they can submit in two minutes an idea.
(28:53):
And you look across these, the power of that has been,
when you look across these ideas also you see a pattern
and that tells you a lot about where your processes,
where your value is weaker,
where your personnel are struggling to perform, execute
what you state as a policy or as a goal.
(29:14):
And connecting that like, this is where we want to go.
This is where our people are struggling, then allows you
to find technology that solves for those problems
that truly transform the day to day.
Yeah. I, I think that's, that's so great.
And I think, you know, you're in a really fortunate position
of working for a company where innovation is,
is core to your brand.
And so it's, it's sort of in the,
(29:36):
in the soup at cap one, right?
Um, and, and I think regardless of whether
that's the world you're li you're living in,
or if you're in an organization that's maybe not quite
as forward leaning when it comes to innovation in general
and tech specifically, the advice I always give
to internal audit teams is you should
keep pace with the business.
(29:56):
It's pretty rare, in my experience,
that internal audit is the first adopter
of any new technology.
It happens sometimes, but I think it,
I think it's the exception rather than the rule.
And what I think is far more common is
that internal audit lags the business by several years
in terms of their adoption of technology.
Somebody in operations will get some great new tool
(30:17):
and then internal audit gets around to it later.
I wonder, do you agree with that idea of, of keeping pace
and, and how should I think about, um,
or how should audit leaders think about really capitalizing
on the investments that have been made at an enterprise
level without putting technology first,
but understanding that you've got all these tools on the
(30:39):
workbench, you've gotta decide which ones you wanna pick up
and when you wanna pick 'em up.
So I have a sort of a nuance view on one of those things,
but I will get to sort of your first point of keeping pace.
I think as more and more controls get automated, more
and more technology gets introduced in risk sort
of managing risk and how operations work, you do need
to keep pace because you cannot challenge a model
(31:01):
with a sample testing
or if there is a model out there, you need to be able
to build a challenger model.
That challenge is sort of the efficacy of, uh,
the models used similar to that.
I think there is a need for order to be able to
process the same level of information, right?
And make their own parallel independent opinion
of those decisions in the same way that first
(31:23):
and second line functions are doing it.
But where I sort of have a now more nuanced view,
having been in this industry for a long time,
I did have the same view that coming into audit.
I was like, is audit really the place that will be for me
because I want to be at the frontier of innovation.
What I've come to realize over sort
of my career is audit can be actually the beacon
(31:45):
for innovation for the entire bank.
Because one audit has several advantages in that, right?
One, while innovation comes in ops
and revenue generating functions,
I have in my career seen innovation
and risk management functions is lagging.
And that's where I do think in fact,
audit might be ahead in some cases
of risk management functions, which then helps audit, inform
(32:09):
and educate business on how they can better manage, assess,
uh, and, you know, mitigate the risk.
The second thing is audit has a unique, uh, point of view in
that you're able to look across the enterprise.
So going to the board,
you can say these different functions, they're struggling
with the same root cause.
So we are, these are the things we need
to prioritize as an enterprise.
(32:30):
So again, you do need to keep pace,
but you should also, I think audit departments should think
of like, where is it that audit can provide
that unique value to the enterprise, be a beacon
for innovation and risk management.
Because a lot of businesses are essentially, uh,
risk management is a core part of the business.
I mean, cer certainly in financial services,
it's absolutely front and center.
(32:52):
Yeah, I love internal audit as a beacon for innovation.
We should get some bumper stickers printed up. The I a a.
That's fantastic. You know, I think in early 2025,
we can't have a conversation without looking into agents.
It is the topic right now. Everyone is talking about agents.
Maybe just to start off like gimme your definition of agents
(33:15):
and then let's talk a little bit about some of the places
that you're either using
or planning to use agents in, um, the internal audit world.
So I think, uh, lot of, uh, talk about agent tech ai.
I think autonomous ai, I do personally think some
of it is overblown, uh, artificial general intelligence.
Some of it is overblown, uh, in that, like, there,
(33:38):
there will be a niche of the chat GPTs, the, uh, anthropics
of the world who have the resources, energy, billions
of dollars to burn on creating
artificial general intelligence.
Where I focus as a sort of leader in innovation, uh,
since I don't work for Sam Altman, is sort
of application, right?
(33:59):
Where can we apply these, uh, technologies
and get real business value?
I think, again, coming to my sort of view on this
and where I see this going, I think
for the foreseeable future it'll be human in the middle.
There will be no, you know, self-driving audit, uh,
shop, so to speak.
(34:19):
And how I think about it is, like
how leaders should think about it is
how does this transform auditing?
What are the things that are core principles do you want?
And if you are thinking about building
and investing in this kind of a thing, you want your, think
of it as a, as you develop a
expert in your audit department,
(34:40):
your expert would know the core principles
of auditing, right?
Uh, know how to take a risk based approach, fundamental sort
of principles that never change.
So those are things that you are training your model, right?
You're investing money, teach them these things
and these things are fundamental.
They will never change. Then there are aspects which are
(35:00):
like, general, I don't need
to teach this person specifically, this is just
how things are done.
Uh, communication, you know, basic general ideas.
So those are things that you can take out
of the box from a lot of the, you know, open source models.
Then there are things where you say, okay, I need areas
where I want to teach somebody how
(35:23):
to look at this specific process in our enterprise
and opine in this according
to these specific mythology that we've developed.
So in those cases, you get sort of enter the space of rag
and you know, having sort of using a full
like external developed general model,
(35:44):
but building small specialized expertise
and use your own data instead
of just using general out of the box information.
So again, I give a very, I try to keep it non-technical,
but essentially there is difference between
how much resources you need to spend.
You don't want to be training your models on things
that constantly change,
(36:04):
and you are not really looking for the model
to know and remember all the time.
And there are things that are basically I needed just
to do this audit step,
and then you can use them as in the context window
and you basically start doing them out of the box instead
of investing, trying to over engineer it.
(36:26):
I love that. So one last question before we wrap it up here.
You didn't start off as an internal auditor
and now you've been an internal audit for several years.
So what is the most controversial point
of view you have about internal audit coming in from
an outside perspective?
(36:46):
So my most controversial view has been,
and I think people do, uh,
find it interesting also, is that audit.
If you think about audit as a profession, it's sort of,
it's the fundamental pillar of capitalism.
So audit has been done for like over hundreds of years,
but more
or less in the same way, there's been more data,
(37:07):
there's been more technology,
but it's more or less in the same way.
My view is we are at a point in, in the audit, sort of
where we are in audit, how audits are done,
what an auditor's job is, is at, we are at the precipice
of it can be fundamentally transformed.
And to do that, we need to reimagine
(37:29):
how audits have always been done.
And I think that is sort of something that leaders need to
start thinking about more deeply, right?
We've done audit cycles, we've done, uh, cycle audits,
we've done, you know, risk assessment memos, we've done
continuous monitoring dashboards,
but all of that doesn't exist in a world where you're able
(37:52):
to, uh, now process information, real time, provide,
connect, connect the dots across the enterprise, have more,
you know, bigger, deeper conversations instead of narrow
specific views that maybe are lower value.
This has been a great conversation.
(38:13):
Imran Mulah, thank you so much for being here today.
Thank you, Charles. Pleasure.
Small internal audit teams boost your efficiency at the
2025 audit sphere.
This one day virtual conference on June 17th offers expert
strategies and practical tools just for you.
Register now@theiia.org.
(38:35):
If you like this podcast, please subscribe and rate us.
You can subscribe wherever you get your podcasts.
You can also catch other episodes on YouTube or@theia.org.
That's THEI a.org.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.