All Episodes

September 28, 2021 17 mins

This is the third in a series of episodes that focus on performance auditing.

In this episode, we discuss the benefits of adopting a data focused approach during audit planning.


About this podcast
The podcast for performance auditors and internal auditors that use (or want to use) data.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Mark as Played
Transcript

Episode Transcript

Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Narrator (00:07):
You're listening to the Assurance Show.
The podcast for performanceauditors and internal auditors
that focuses on data and risk.
Your hosts are ConorMcGarrity and Yusuf Moolla.

Conor (00:20):
We're continuing our little miniseries today
on performance auditing.
In the previous episodewe spoke about developing
forward work plans.
The followup episode to thatspoke about, the importance
of getting balance in thatplan, objectives of economy,
efficiency, effectivenessand compliance and what that
might look like going forward.
Okay.
So you've goteverything settled.

(00:40):
You actually need to startexecuting some of your audits.
What are some of the thingsthat you may need to consider
in using data for theplanning of those audits?

Yusuf (00:49):
Okay.
So that's the next logical step.
Now that you've actuallyselected your topics, you're
going to go into a particulartopic Most people are very
familiar with what traditionalplanning would look like so
research and understanding theentity, understanding the topic.
And what we want to talkabout in particular is
what the use of data withinplanning can help with.

Conor (01:11):
Yeah.
And so some of the benefitsof a data focused approach,
probably haven't beenarticulated that clearly.
Cause generally teams justget into planning and they
don't have time obviouslyto sit back and think of
what else can be deliveredusing data in this process.
We'll talk a littlebit about that today.

Yusuf (01:26):
And the proliferation of data, particularly open
data and obviously a range ofdifferent corporate datasets
means that you could takethis approach for pretty
much any performance audit.
So what are the benefits ofusing data within planning?

Conor (01:40):
The first one is obtaining the data early.
That allows us to explore theperformance of the organization
to try and understand what ourkey audit questions might be.
Things that are possible to beanswered in this audit, but also
equally importantly, it allowsus to zone in on things that we

(02:00):
may have to exclude only becauseeither the data doesn't allow
us to answer those questions,or we might have to go to
another source or it may betoo time consuming or complex.
So it allows us really toframe up our scope of what's
possible, and explore howwe might during the conduct
phase of the audit go aboutanswering those questions.

Yusuf (02:20):
Potentially using data to identify, like
you said, a sample, butalso scope exclusions.
What would that look like?
So how do we use datato identify a potential
scope exclusion?

Conor (02:31):
So one example springs to mind straight away is where,
we may be able to use the datathat we have to half answer a
question, but we may have to getextra data from another entity
to fully answer the question.
Now, it's very important torealize that upfront because
there may be limitationsto actually engaging that

(02:53):
other entity be able tofully answer the questions.
There may be constraintsaround time, to do that.
And there may even beprohibition or legislative
constraints aroundasking for that data.
That's a really importantconstraint that having data up
front allows us to understandso that we can potentially
exclude that from our inquiry.

Yusuf (03:12):
Does that mean that we're excluding it on the basis
that we can't get the data.
Because, surely we can thenget the information or ask
the questions regardless ofour ability to actually get
the underlying data, or isit about how deep we can go
on that particular matter?

Conor (03:28):
It's more the latter about how deep
we can actually go.
Because, to be quite frankhere, it may well be that it's
not worth the extra effortto be able to answer this
question completely, in thegrander scheme of the audit.

Yusuf (03:39):
And so what if the data tells us that there
might need to be an exclusion,but excluding it means that
we're not able to answer afundamental question that
is being asked in the audit.

Conor (03:49):
That circumstance would put in jeopardy the
entire objective of ouraudit to be able to conclude
reasonably on that objective.
So either we need to go backand revisit the objective
of the audit and recastwhat we can actually do.
And that may well mean that wecan't provide a conclusion to
the extent that we wanted to.
Or it may well mean that wehave to completely change focus.

(04:13):
If that missing piece or thatexclusion that we've identified,
if that is so substantial, thatit means we can't meet the audit
objective, then we'll have toentirely change that objective.
But if it's more about how wego about getting evidence cause
data really is just evidence.

(04:35):
Then it may not be sosignificant that we
can't obtain that evidencefrom another source.

Yusuf (04:40):
In summary, what we're saying is that using data,
upfront during planning,helps us to properly answer
the questions around whatwe can do, what we should do
and how we should do that.
So we're better informed duringthe planning stage about the
direction and the approachthat we are able to take based

(05:01):
on the information that wealready seeing, rather than
coming up with what we thinkwe're going to do, and then
finding during conduct thatwe're not able to do that.

Conor (05:10):
What's possible.
What's not possible.
The extent to which wecan answer questions.
Providing us with a defensiblebasis so that when we're
asked later on, why didyou guys exclude this from
the performance audit?
We can say we looked atsome data to help us form
the scope of the audit.
And these are the reasons whywe slightly changed tack, or
we modified the objective.

Yusuf (05:30):
And we were able to go and look at alternate sources.
We were able to look atalternative approaches and
we aren't reasonably ableto answer that question.

Conor (05:38):
Auditors General, who are attending parliament to
talk about their performanceaudit quite reasonably, they

may get asked the question (05:43):
How come you said in your forward
work plan, your objectivewas this and then it modified.
So it gives the defensiblebasis for them to say we looked
at the data and we had tochange the objective slightly.

Yusuf (05:56):
Okay.
So that's about the overallobjective of the audit
and, broadly some of theexclusions and the approach
that we're going to take.
What else does an earlyfocus on data help us
to do during planning?

Conor (06:06):
We've spoken at length, in various episodes, about the
duration of performance audits.
And some of the shorter ones arearound the six month period, and
they can go up to beyond a year.
When you think aboutthat in real terms, the
world's moving so fast.
Your audience may lose sight ofwhat's actually going on from
commencement to reporting, giventhat elapsed period of time.

(06:30):
Now, one way you can maintaintheir engagement and enthusiasm
and give them a product orsomething to see as an outcome
of what you're actually doinguntil you table your report,
is through these ancillaryor byproducts or call them
what you will, that you arecreating during the audit
based on your analysis of data.

Yusuf (06:50):
So that's providing a dashboard or some sort
of other insight early onduring the performance audit.

Conor (06:56):
Absolutely.
So let's assume that aperformance audit takes nine
months, let's assume that yourplanning takes a couple of
months and as part of that,you've grabbed a whole heap
of data and you've done whatwe just previously discussed.
You've done your explorationof the data and you've had
to put it into some sortof shape and visualize it
to see what the story is.
Now, if you can, at the endof that planning phase, even

(07:19):
before you move into conduct oreven in the middle of conduct.
If you can clean that up, thatanalysis and make it digestible,
and in a format that membersof the public can consume,
or the parliament can consumein the intervening period,
as you do your conduct work,then surely that's a benefit
to all those stakeholderswho are really interested

(07:41):
in that performance audit.

Yusuf (07:42):
Okay.
And what about the entities?
Is there potential forputting information out that
the entities can consume,understand and use to
potentially fix up some of theissues that may be identified
before the end of the audit.

Conor (07:57):
So that would be an ideal scenario.
By the time an Auditor Generalcame to table the report in
parliament, all the issuesidentified during the course of
the audit would have been fixedup quick smart by the entity.
And we always talkabout efficiencies.
And that's an efficientsystem there in and of itself
such that the prompt hasbeen the audit in itself.

(08:21):
If something is published orreleased partway through that.
And it could be, like wesaid, based on that analysis
of data that has illuminatedthe problems or the issues,
they have then been fixed upin the intervening period.
Then surely that performanceaudit, regardless of being
tabled in parliament,has created value for

(08:41):
all those stakeholdersalong that delivery chain.

Yusuf (08:44):
So an example of that might be where we take some
data early on from the entity.
And we understand what thedata quality gaps might be.
Now if it's purely justdata, quality gaps in the
work that we're doing.
That's one thing because that'sabout ensuring that the data is
clean, but where we're lookingat things like how data is used
for monitoring performance,managing performance.

(09:05):
If the data itself is notvery clean we can then
highlight those data qualityissues because that will
directly relate to the abilityto monitor performance.
And so if we can show whatthose potential dirty data
issues are and the entityis then able to fix that up.
That means that before the endof the audit, they're actually

(09:25):
able to improve the way inwhich they monitor performance
based on that underlying data.

Conor (09:30):
That's right.
And just one more point ondata quality, is that a lot
of entities with the bestwill in the world want to
capture performance data.
A lot of the times theyrely on other government
departments or agencies toprovide that performance
data to them so that they cancoalesce it and then bring
together overall performance.
Their focus through that processmay not have been on the quality

(09:52):
of what they're receiving.
So if an external organization,like the audit office can say,
these are where your specificdeficiencies in data quality
lie, it gives that agency theability to go out to those
other collection points andsay, these are the specific
matters that are deficient interms of your data quality.
We can't accurately monitorperformance of this program

(10:14):
or of this activity untilyou help us fix those up.

Yusuf (10:17):
The first one, we said there was ability
to frame up properly, ourobjectives, questions, lines
of inquiry, scope exclusions.
The second one was providingsome data to ensure
early insights in whatnormally would be a long
process and potentiallyfixing some of those up.
What else can we do withdata early on to help the
way in which we executeour audits, either internal
benefits or benefits out tothe public and parliament.

Conor (10:41):
The expectation among the general public and even
parliaments now is that therewill be data readily made
available to them to consume.
So we've seen the rise in datajournalism, for example, during
COVID where there's graphs andcharts and statistics and all
sorts of data visuals appearingin newspapers and TV screens.
So the expectation amongthe public one would suggest

(11:02):
is growing around dataand those visualizations.
So their appetite probablygrowing as well as their ability
to actually understand what'sbeen put in front of them.
So what we need to do asauditors is make sure that
we are feeding that appetite,and there is a large amount
of information that we haveavailable to us, through
our performance audits thatcan help us in that respect.

Yusuf (11:23):
Third benefit is, our ability to continue to be
relevant in relation to what ourstakeholders are looking for,
what their changing needs are,and how we can address those.
So the last question I wantto ask then is, you've got
the traditional, performanceaudit objectives that focus

(11:44):
on efficiency, effectiveness,compliance, and economy,
as everybody knows.
But what we're seeing more andmore is fake news, incorrect
information, poor data quality.
Lots of information that isbeing put out, particularly,
as you said now with thepandemic and the continued
rise of data journalism, ormaybe the way in which data

(12:05):
journalism is now potentiallyfar better understood by the
public, who have an expectationto consume some of the data
and understand what it means.
Do you think that there mayneed to be change in exactly
what an audit office does.
Open data, publicly availabledata we've seen over the years,

(12:27):
that It's not always consistent,it's not always very clean.
Is there a role for auditoffices to either separately or
as part of the audits evaluatethe quality of open data that
is being provided and used.

Conor (12:43):
So the answer is yes.
I think it goes broaderthan just evaluating the
quality of Open Data.
There's a growing patternof Auditors General being
given hard questions byparliaments and members of the
public, call them referralsor requests for audits.
They are getting more andmore of that sort of work
because they are viewed asan honest broker about what

(13:07):
is real information and whatis not real information.
There will be a growth in thattype of work coming to Auditors
General, which will changethe way in which they do their
work and their focus on otherthings like compliance, economy,
efficiency and effectiveness.

(13:28):
And they'll have to allow forthat honest broker role as
part of their overall work.

Yusuf (13:33):
So if we said add a fifth objective.
Maybe we'll call it Tfor transparency, which
everybody is asking for.
Which may fall into one ofthe other four objectives,
but may actually be aseparate objective in itself.
What's the opportunityfor something like that,
without changing an auditoffice's mandate necessarily.

Conor (13:54):
There's a fair bit in that sentiment there.
So the first thing istransparency should
always factor into thosefour main objectives.
We talked about exploring datato try and get transparency
for ourselves about whatwe should look at, we've
talked about producing datavisuals, for example, through
our audit that providestransparency as we go along.

(14:15):
However, the second and bigger,and probably more strategic
question around whether or nottransparency should be part of
our mandate as a fifth limb, Ithink that's a matter for debate
and each jurisdiction will bedifferent, but how does that
impact the way in which AuditorsGeneral think about their work?
There's probably scopethere for, when they are

(14:37):
formulating their audit workprogram and they're trying
to pick which topics aremore meritorious than others.
It may be an interestingquestion to ask, when we're
thinking about those topics,which of these topics do we
think has the least amount oftransparency attached to it?
So if it's a governmentprogram, for example, maybe

(14:57):
we add a metric or make someevaluation around Is this a
program that we don't reallyknow anything about or there's
not much public information.
Would conducting thisperformance audit really
enhance transparency.
So perhaps that's somethingthat needs to factor into
that planning discussion.

Yusuf (15:12):
Performance statements within the public sector
are being evaluated bysome Auditors General.
They're looking at theway in which service,
delivery statements orperformance statements are
crafted and whether theinformation is accurate.
So is that then a separatematter or a separate type of
topic that can then be appliedto other similar domains.

Conor (15:35):
My view would be that the transparency of
information and integrity ofinformation combined together
in the same sort of stream.
So the abundance of information,or like you said, the
proliferation that's out there,now, perhaps we have a stream
of work within our forward workprogram that is entirely based
on looking at transparencyand integrity of information.
So like you said, there'sa whole heap of information

(15:57):
being put out there, but we'rethrough our performance audit
work program going to validatewhether or not it's transparent
complete and accurate.

Yusuf (16:07):
A number of matters that we spoke about there, but
to summarize the first thingis using data as part of our
planning can help us properlyinform our objectives defend
what it is that we're goingto be looking at, not looking
at, understand our approach,understand scope exclusions, and
go deeper than where we would beif we were only to start to grab
data during conduct, if at all.

(16:27):
The second was aroundproviding transparency.
Or providing earlyindications to the public,
to parliament and also tothe entities, to potentially
fast track the remediationprocess for any findings
that we may have later on.
So the third one wasaudit offices meeting
people where they are.
Understanding what peopleare already looking for,

(16:49):
and starting to meet thatexpectation of publicly
available information thatthey can use and understand
and consume in an easy way.
So graphs and charts, et cetera.
And then lastly, we spoke abouttransparency and integrity being
something that we've been doingas audit offices all along,
through financial statementaudits and performance audits.
Potentially there might bean opportunity to increase

(17:10):
what we do in that area.
To not necessarily just holdgovernments to account, which is
something that we do obviously,but also to enhance the way in
which information is produced.
And enhance the way in whichinformation that the public
rely on can be trusted.
So improving trust by thepublic, in the information

(17:31):
that is produced bygovernment, via an audit
office stamp of approval,which is something that is
always highly sought after.

Conor (17:39):
Cool, thanks Yusuf.

Yusuf (17:40):
Thanks Conor.

Narrator (17:41):
If you enjoyed this podcast, please share
with a friend and rateus in your podcast app.
For immediate notificationof new episodes, you can
subscribe at assuranceshow.com.
The link is in the show notes.
Advertise With Us

Popular Podcasts

Stuff You Should Know
Dateline NBC

Dateline NBC

Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com

24/7 News: The Latest

24/7 News: The Latest

The latest news in 4 minutes updated every hour, every day.

Music, radio and podcasts, all free. Listen online or download the iHeart App.

Connect

© 2025 iHeartMedia, Inc.