Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Narrator (00:07):
You're listening
to The Assurance Show.
The podcast for performanceauditors and internal auditors
that focuses on data and risk.
Your hosts are ConorMcGarrity and Yusuf Moolla.
Yusuf (00:20):
So last episode was
about forward work plans.
And we left that at thecreation of the forward work
plan . One thing that we didn'ttalk about then, that would
come before that forward workplan is finalized, is what
does the work plan comprise?
In relation to ensuring abalance between the different
types of objectives.
Conor (00:40):
To recap there, we're
talking about the four main
objectives, economy, efficiency,effectiveness, and compliance.
Yusuf (00:46):
And we'll, steer
clear, for this discussion
anyway, of limited assuranceengagements or referrals.
And focus on reasonableassurance engagements.
Because limited assuranceengagements and referrals
largely will have a veryspecific matter to deal with.
While your broader performanceaudit program, which is
reasonable assurance focused orwhere you may do some limited
(01:08):
assurance work would cover oneof those four broad objectives.
Conor (01:12):
Yeah.
So you've gone through yourassessment process to arrive
at your forward work plan.
So your three-year horizonand all those topics.
So the next thing you needto consider is the balance
of your program across thoseelements of economy, efficiency,
effectiveness and compliance.
Yusuf (01:27):
Before we talk about
how we do that, what are we
seeing, or what have we seenhistorically, around that
balance within forward workprograms or within the reports
that are produced in termsof the four key objectives.
Conor (01:39):
The topics have been
heavily skewed towards audits
that look at effectiveness andreport on effectiveness, as
opposed to the other elementswe just mentioned then.
Yusuf (01:49):
What's the
result of that?
So what's the impact of theabsence of focus on efficiency.
Conor (01:54):
Well, not
just efficiency.
But I guess if westart with efficiency.
The primary effect initiallyfor the audit office, is that
they're not fully using theirmandate, for example, as
envisaged by the parliament orthe Congress, or whoever gives
them that power to do that.
Because if they've suggestedthere are four aspects or
elements you need to look at,and you're spending 80% of
your work only looking at oneof them being effectiveness,
then you have to ask (02:18):
Is that
the use of resources, that
parliamentarians envisaged, insetting up your organization
to do performance audits?
Yusuf (02:27):
In some jurisdictions
that may lean towards
compliance as well.
So effectiveness and compliancewould be the top two focus
areas out of the four.
There's a basis for needingto do some sort of balancing,
as in you want to be able tocover off on those objectives.
What's a reasonable wayforward in getting to that
balance, given that we have soheavily been weighted towards
(02:47):
effectiveness and compliance?
Conor (02:49):
It's probably too big
of a leap to expect an audit
office that's traditionallydelivered the majority of
its audits as effectivenessaudits to automatically
jump, to doing a huge numberof efficiency, audits, or
audits focused on economy.
So that's probably overlyambitious and not realistic.
However, there needs to be somemore movement and acceptance
(03:10):
that efficiency audits needto have a greater place in
your overall mix of audits.
Yusuf (03:15):
Most people that
will be listening to this
will have an understandingof this, or, pretty good
understanding of this topic.
But when we are talkingabout the objective of
an audit, there's oftenmultiple objectives.
So we could be coveringeffectiveness and compliance,
effectiveness and economyin a particular audit.
What we're talking aboutthere is moving towards more
efficiency as an objectivewithin the audit, either
(03:38):
the sole objective or oneof the multiple objectives.
Is that fair to say.
Conor (03:42):
Yeah, it is.
And there are probably morecircumstances where you'll
see efficiency alongsideeffectiveness rather than
efficiency called outjust by itself as the
objective of the audit.
Yusuf (03:53):
And why is that?
Conor (03:54):
Quite often, you
have to look at aspects of
effectiveness when you'redetermining efficiency.
Yusuf (03:59):
Okay.
So would there be somesort of overlap in either
the criteria or the auditquestions that are asked or
the evidence that is obtainedas between effectiveness
and efficiency objectives?
Conor (04:10):
The last one you
mentioned there Yusuf is
probably the most prominent inthat the evidence you'd have to
obtain to measure effectivenesswould also go a long way, if
you're looking at efficiency.
Yusuf (04:19):
Given that you have
that overlap in the design
of the audit work programyou might be able to cover
off on both objectives whichwould be less than covering
off on each objectiveindividually within two audits.
Conor (04:31):
Absolutely.
So that in itself gives yousome efficiencies there on
how you deliver your work..
Yusuf (04:35):
Sometime last year,
we looked at the range of
performance audits thatwere conducted across
several jurisdictions,particularly in Australia,
and a few other offices.
But what we saw was thatthere was a significant skew
towards effectiveness audits,a little bit compliance,
efficiency audits accountedfor less than 10% of the
overall number of audits.
And that is where efficiencyis either the objective
(04:58):
or one of the objectives.
One of the things thatpotentially is holding us back
is the fact that, historically,we haven't been doing much of
it as performance audit teams.
What else is it that detersus from going down that line.
Conor (05:11):
Based on the people
we've spoken to from various
performance audit backgrounds,there seems to be a perception
that efficiency audits are moredifficult to actually execute
than effectiveness audits.
Yusuf (05:21):
And that perception,
is that based on historically
not doing much with it, oris it based on having done it
and found it to be difficult.
Conor (05:27):
I think it's both.
I think there's a bit ofthe fear factor there.
We haven't really donemuch of them before.
We think it's more difficult,it might be getting more into
sort of economics territory.
We may not have the skills,so there's certainly
an aspect of that.
Some entities may havetried to do some efficiency
audits in the past.
And they may not have goneas smoothly as they wanted.
So either they took a lotlonger to arrive at a report.
(05:50):
Or maybe they justcouldn't deliver it to the
standard that they wanted.
So this there'sprobably a mixture of
things going on there.
Yusuf (05:56):
about economy
focused audits, what's
the hold up there.
What's preventing usfrom doing more of those.
Conor (06:03):
I suspect the reasons
are exactly the same.
Although they economy auditsdon't seem to feature as
prominently in discussionsabout why they're not being
done as efficiency audits.
I don't know why that isthat they're not as prominent
in discussions, but mostof the discussions seem
to focus on the efficiencyaspects being difficult.
(06:24):
But one would suspectthat the reasons for more
economy audits not beingdone are probably similar
to the efficiency audits.
Yusuf (06:32):
So not withstanding
that the performance audit
mandate for most officeswould cover all four.
Is it that economy auditsjust aren't that valuable?
Conor (06:40):
That's quite
a loaded question.
Again, we'll come to perception.
They may not be perceivedas being as valuable because
there are not that manyof them we see around.
we have no real yard stick todetermine how good they've been
or what impact they have had.
In the absence ofthat, evidence, it's
difficult to answer that
Yusuf (06:56):
But the bottom line
is that we should be moving
towards a better balance?
Conor (07:00):
Before we move on that
we need to just mention the
fact that compliance auditsare still important, but it's
pretty well held, universallythat they are generally more
straightforward audits toexecute, because for example,
there may well be a presetset of regulations, or metrics
that have been well-establishedand define that we can just go
(07:21):
automatically and test against.
So they're generally consideredto be less risky audits
and more straightforward.
Yusuf (07:28):
It's almost basic
checks and balances that you
following your own sort ofbasic controls and preventing
things like fraud and ensuringthat you are compliant
with privacy expectationsand, security expectations.
Conor (07:40):
Yeah, black and
white rules generally.
Yusuf (07:41):
Which, in an ideal
world would be covered by
internal audit functionswithin the individual agencies.
And then the audit officereally should be coming
over the top of that.
And just checkingthat across the board.
However, we don't necessarilysee that happening from
within internal audit teams,which is when you need the
auditor general to step in.
Conor (07:58):
Yep.
Absolutely.
And it does actually make alot of sense, even from an
efficiency perspective, ifinternal audit is picking
up some of those compliancebased stuff, When the
external auditors or theAuditor-General comes in,
they can see what's been donealready and don't need to cover
that off in their footprint.
Yusuf (08:12):
Okay.
So one way that you can improvethat is by measuring the
effectiveness of internal auditfunctions the related boards.
And there's obviously whatwe've seen over the last year is
that there is more around that.
So there are some audit officersthat are starting to look at.
Or looking at that again.
Also because it just means thatthey can be more efficient as a
team and make the whole system.
(08:33):
Going back to efficiencythen, So let's p ut efficiency
and economy into one basketfor for purposes of this
discussion, we all know whateffectiveness audits are.
We all know whatcompliance audits are.
What would we do in order toundertake an efficiency audit?
What would the objective be?
So what are welooking to achieve?
And then if you haven't donean efficiency audit before, what
(08:54):
are the sorts of things that youneed to be thinking about, that
will be different to when you'redoing any effectiveness orders.
Conor (08:59):
So if we go back
to our definition, that's
trying to maximize the ratiobetween inputs and outputs.
A place to start would be intrying to understand whether
there's been historicalchallenges with where that
the ratio as between outputsand inputs looks extreme or
looks as if there's a heckof a lot of work on the input
(09:20):
side, that doesn't seem toflow through to the output side.
So that's trying to understandwhat that actually looks like in
terms of service delivery let'stalk about the health sphere,
and that's always a targetfor efficiency, dividends,
or efficiencies and savings.
Particularly now inthis current climate.
There's always been concerns,generally held among
(09:42):
parliaments and also membersof the public that hospitals
are very expensive to run.
So that's the one thing, butthere seems to be a lot of
costs Service inputs that don'tnecessarily deliver a good
return on patient throughput.
So how many people we're gettingmoving through the system?
So that's an example.
(10:03):
Trying to find those typesof efficiencies in the
health sphere that has beenfront of mind for a lot
of auditors general andvarious jurisdictions.
So how can we get more of ourmembers of the public through
the health system withoutnecessarily having all these
extra costs, inputs to do that?
Yusuf (10:19):
Okay.
So in that example, what we'resaying is we still want to
have patient outcomes and allsorts of governance in place.
However, there are certainthings that we are likely to
find within the inputs thatgo into running a hospital,
for example, or running ahealth service, for example,
that could be eliminatedwithout reducing those
(10:40):
outputs or could be reducedwithout reducing the actual
outputs or the outcomes thatparticular service is providing.
Conor (10:47):
One of the pitfalls
is that, because there are
so many variables, there'sso many inputs into a
massive system like that.
Where do you commence?
Trying to find efficiencies.
How do you actually identifythose activities or inputs
that are not providing anyvalue to the organization?
And in fact, maybe actuallydetracting from your
ability to do your job.
So that's the one thing.
(11:08):
So then identifying thatthat's, a significant global
problem, we just try and zonein on a particular function
type or a particular service orstream within that environment
and focus on that firstly.
Yusuf (11:22):
What if you went
back to front the one thing
that you need to be able todetermine is are the outputs
outcomes effective, right?
So you need to, possibly youwant to go first and look
at compliance within that.
system and then go and lookat effectiveness of the
outcomes within that system.
And then go and look at,okay, we know that they
are compliant or they'reworking towards compliance.
(11:42):
We know that they areeffective or working
towards effectiveness.
, now that we understand howthe system works and what
we're trying to get out, nowwe can go and see what there
is that potentially, extrasuperfluous, whatever you want
to call it within the inputs.
So working back from theoutcomes to what would
generate those outcomes.
Conor (12:00):
Yeah, and that's a
really good way to explain it.
So it's almost like ahierarchy of activity.
So you might do the compliancething first, because we want
to make sure that people aregetting healthcare, according
to legislation and appropriatehealth standards effectiveness
might be is the health servicebeing delivered properly?
Is the quality there?
So people are not leavingour hospital less well
than they entered thehospital, that sort of thing.
(12:22):
So effectiveness willtake care of that.
And then once, once we knowthat those two things are
ticked off or headed, wherethey need to go, then how can
we make sure that we're gettingthat same level of outcome
potentially with fewer inputs.
Yusuf (12:33):
So that's great because
similar thinking would apply
to internal audit teams.
If you're putting together planfor addressing internal audits.
Now, like we said, before,many internal audit teams
are going for this sort ofthree plus nine approach
where you determine what yourinitial audits need to be.
And you're reasonablysecure on that.
And then, do that for threemonths and then work out
(12:54):
what do I need to do next?
the Certain topics.
that will remain year onyear, just not because we
want to have a rolling auditplan of any kind, but because
they are so important.
So in, certain industries,customer service may be
the most important thingor revenue, maybe the most
important thing or marketingor sales, whatever it might be.
And within internal audit.
If you are going to look at whatis my work plan going to be,
(13:16):
it might make sense to say, letme look at compliance first.
And then for that particulartopic, then look at
effectiveness and then lookat efficiency so that we
slowly building up in termsof the value that we can
provide to those departmentsor functions or business units.
But also because it's moredifficult to do efficiency
traditionally, If we understand.
(13:38):
How the system works throughhaving done a compliance focused
audit then and effectivenessfocused audit that efficiency
audit becomes that much easier.
Conor (13:46):
That's common
sense approach.
If you've done those complianceaudits, effectiveness audits
you'll know that organizationpretty wellby the third time
you come round to do yourefficiency audits, which in
itself will help you plan morequickly, deliver more quickly
and report more accurately.
Because you've alreadygot that solid grounding.
So just to turn that on itshead slightly, it would almost
(14:06):
seem unwise for a performanceaudit team that probably hasn't
done a lot of efficiency auditsto go in and expect to do an
efficiency, audit of, say abrand new government program.
So not only is it brand newand we don't know anything
about it, we haven't even yetdone a bedrock compliance,
audit or effectiveness audit.
That would seem really strange.
(14:28):
And we'd suggest that teamsdon't take that approach at
trying to do an efficiencyaudit straight off the bat.
Yusuf (14:33):
And unless bringing
subject matter experts in.
Conor (14:35):
Unless bringing the
subject matter expert in.
But then again, if it is anew program, for example, or
a new type of service thenhow much subject matter
expertise is there goingto be around the place?
Yusuf (14:46):
Oh, so you mean if
it's a new type of program,
generally not a new type ofprogram for your government.
Conor (14:50):
Correct.
Yusuf (14:51):
Cool.
All right.
So that was episode two of theperformance audit mini series
and the encouragement we triedto provide within this little
discussion that we had is tofocus on or try to have a level
of focus on audits that falloutside of the traditional.
Objective focus, beingefficiency and economy audits.
(15:12):
And what we ended on therewas if you are struggling
to understand how to do it,then one easy way to do it
is to build it into your workprogram as a later audit that
you use earlier compliance andeffectiveness audits to build on
in terms of your understandingof how it all works.
And getting comfort thatit is all working before
you try to go and make itcheaper, to actually work.
Conor (15:35):
Yup.
Sounds like a good plan.
Yusuf (15:37):
Good stuff.
Thanks Conor.
Conor (15:38):
Cheers Yusuf.
Narrator (15:38):
If you enjoyed
this podcast, please share
with a friend and rateus in your podcast app.
For immediate notificationof new episodes, you can
subscribe at assuranceshow.com.
The link is in the show notes.