July 18, 2021 • 14 mins
In this episode we discuss the use of risk/performance indicators.
These can be used by audit teams, risk teams and integrity agencies.
For audit teams, they are useful throughout the audit lifecycle - planning / risk assessment, fieldwork / conduct and reporting.
We explore why they are useful, what to do and briefly how to get started.
About this podcast
The podcast for performance auditors and internal auditors that use (or want to use) data.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Narrator (00:07):
You're listening to The Assurance Show.
The podcast for performanceauditors and internal auditors
that focuses on data and risk.
Your hosts are ConorMcGarrity and Yusuf Moolla.
Yusuf (00:20):
Today, we're talking about performance and risk
indicators for use withininternal audit, performance
audit, and then also extendingto first-line and second-line,
and various other individualsthat want to understand how
different indicators cancome together to provide
a view of performance, orrisk, more holistically.
Conor (00:39):
When we say risk indicators, what are we
actually talking about there?
Yusuf (00:42):
Well, depending on the subject matter that we're
looking at, it would vary.
What are the differentareas that will contribute
to our understandingof a particular domain?
Conor (00:52):
Use the data we have to build up profiles of an
entity or subject matterthat is of interest to us.
Yusuf (00:57):
Yeah, and the more data that we have from
sources that are different,the better the view would be.
Meaning not just bringingall data together from
one system or one domain.
We always talk about bringingcomplaints data in, and
complaints risk is one ofthose things that we don't
normally include as part of anaudit or a risk evaluation.
(01:18):
So that's an example.
Open data, obviously.
But then also, if you justthink about across functions
within the entity, acrossdepartments, within the
entity, bringing some of thosedifferent data points together.
Obviously, they need toactually relate to the
risks or the performancearea that we're looking at.
But they don't necessarilyhave to be directly related.
You can have a combination ofdirectly related indicators
(01:41):
that would influence strongly.
And then others thathave a minor influence.
Conor (01:45):
That all seems really intuitive and
sensible and reasonable.
There's benefits to thatapproach at the planning
stage, where you can combinerisk indicators to try and
help you select the samplefor your internal audit
or your performance audit.
Yusuf (01:58):
In the performance audit world, it would be
determining which entitiesare we going to audit as
part of a particular topic?
Within internal audit, whatis the sample that we're
going to be looking at?
So you have withinbanking, within retail,
within production andmanufacturing, which entities
are we going to look at?
Which branches arewe going to look at?
So that sort ofthing is being done.
But we can apply the conceptto a greater number of audits
(02:21):
than we're currently doing.
Conor (02:22):
There's clearly benefits, in conducting your audit, where
you can actually draw togetherindicators, whether they be from
internal data or external data,to profile the activity to try
and understand what's going on.
And then there's alsoindicators that we can use
as part of our reporting.
Yusuf (02:38):
As auditors, when reporting on these things, we
can use it to demonstrate towhoever the stakeholder is.
Whether it be the generalpublic, or internal to the
organization, or the boardor the audit committee,
or management, whoever.
Why we did what we did.
Because often people wantto know why is it that
you looked at a particularentity or particular sample?
Also we can use it toexplain, what is going on.
(02:58):
Seeing that broader pictureis enabled by bringing all of
this together, for reporting.
Conor (03:03):
Taking a risk indicator approach is really helpful
where you have, limitedcontrol over the work that
comes into your organization.
And we might see that, forexample, in a regulatory field,
if it's a regulator or anintegrity body or, somebody
that receives complaintsfrom members of the public.
Now, those types oforganizations have constraints
on resourcing, yet theyreally have no control over
(03:25):
what comes through the frontdoor in terms of their work.
So they really arereactive in nature.
And what we're seeinggenerally, is that a lot
of those organizations inthat type of arena, are now
understanding that they canactually prioritize their work
when it comes through the frontdoor by really trying to build
up indicators about what'sgoing on in their operating
environment and use that todrive their resourcing model.
Yusuf (03:48):
Yeah.
And the same could be saidfor internal audit functions.
The same can be said forsecond line risk functions
,first-line functions.
It's important then to targetyour resources at the areas that
are going to have the highestimpact and bringing this sort
of data together enables that.
Conor (04:05):
So the use of indicators is a logical response to
trying to control where youfocus your limited efforts.
Where do you start?
What's your first point of callin trying to build up a risk
indicator approach to your work?
Yusuf (04:18):
It definitely is iterative.
But you do need to startoff with something that
at least gives you areasonable starting point.
So you want to havethree or four indicators
at least to begin with.
There's no point in tryingto bring one thing together.
What's this, that's not reallygiving you a comparison.
So you need at leastthree or four to start.
How you get to thatis bottom up top down.
(04:39):
So the top down is, what isit that I'm interested in.
Bottom up is what data do I havethat I can feed into this model?
The bottom up is important, butwe do need to not throw data
in just because we have it.
So top down, what do we need?
And then bottom up, what dowe have to address the need?
And what we don't have,how do we collect it
(04:59):
for future use or can wecollect it for current use?
Conor (05:02):
And that top-down approach of trying to determine
what we need that can ofcourse change over time.
So for example, if yourorganizational priorities
change or if your strategychanges or your focus area
changes, then you shouldalways be cognizant of that
having a potential impact onthe indicators that you need.
Yusuf (05:20):
Yeah.
And sometimes you'llsee, also impact of that
through the data as well.
So you see that bringinga particular indicator.
It doesn't give you a betteranswer than you had before.
or a particular indicator,regardless of the weighting
that you have, it doesn'tactually give you a
better overall result.
Conor (05:34):
As you said, you need to be ready to iterate.
And add to thoseindicators over time.
Perhaps remove some of themthat have become redundant.
Or slightly tweak some of themthat still remain relevant.
There is possibility for alittle bit of fluctuation
in what that stack ofindicators looks like
as the business goes on.
Yusuf (05:52):
The other interesting thing that we've seen a few
times now is that you can usecertain indicators repeatedly
across multiple subjects.
And that's really usefulbecause you just need to add
to it as opposed to startingfrom scratch each time.
However, there is a howeverwith everything, however,
you need to make sure thatyou're not just using an
indicator because it exists.
Don't just grab certainindicators and pull them
across just because theymay have some sort of remote
(06:14):
relevance to the new subjectmatter you're looking at.
Conor (06:17):
So every time you look to bring in a new
indicator, you really needto ask why is this important
for our focus at the minute?
And how is this going tocontribute to the audit that
we're trying to deliver.
Narrator (06:27):
The Assurance Show is produced by Risk Insights.
We work with performanceauditors and internal
auditors, delivering audits,helping audit teams use data,
and coaching auditors toimprove their data skills.
You can find out more aboutour work at datainaudit.com.
Now, back to the conversation.
Yusuf (06:47):
I spoke about top down.
How do you determinethat top down?
Because you do a lot more thanme, of the "what indicators
are we going to be using?"What's your thinking?
How do you approach thatwhen you just have a
blank sheet of paper?
Conor (07:01):
I'm pretty old fashioned in this regard.
And I always go back to ahierarchy of needs approach.
So at the top of that mightbe, is there a legislative
imperative for what we'retrying to achieve, either for
the public or in a certaindomain that dictates that we
need to be grabbing this sortof indicator and using it.
And the next thing mightbe then what are our three
(07:23):
strategic objectives for thisorganization over the next,
usually three to five years, andtry and understand what are the
contributing factors to those.
So for example, in the corporateworld, it might be, we want to
expand our geographic footprintinto another continent.
So, I might be interested inunderstanding what are some of
the indicators of activity inthat continent that would speak
(07:46):
to my business that I need topotentially bring in to enable
that strategy to be realized.
Yusuf (07:51):
So regardless of the audit topic, because you have a
certain limited set of strategicpriorities, you want to try
to include those in everytopic that you're looking at
because there may be relevance.
Conor (08:04):
Absolutely.
Of course, there's a costbenefit there that you
need to make a judgmentcall , how much does it cost
to bring these indicators in.
Is the data complex,do we need to clean it?
Is the benefit goingto outweigh the cost?
Yusuf (08:14):
There'd be some audits that are purely compliance
focused and have a verynarrow objective that don't
relate directly to some ofthe more forward-thinking
strategic objectives.
And in that case, maynot necessarily need
this sort of approach.
But that's not themajority of audits.
Some entities are verycompliance focused, but
most audit teams will havea little bit of compliance
work that they just needto do housekeeping work.
(08:36):
But the majority of the effortwill be on helping ensure
that we achieve or ensure thatwe prevent against risks to
achieving those objectives.
Conor (08:44):
Briefly going back to the hierarchy.
So we talked about understandingwhat's happening in the
external environment,government priorities if
you're in the public sector.
What do I need to do orgrab that could help the
people in my jurisdiction?
But there's also a tertiary.
And this is to reallymature how you use data
within your organization.
That's an internalbenefit, but it's still a
really important benefit.
If you can grab some externaldata that is an indicator
(09:06):
that would also add to yourmaturity of how you use data,
then that's still a reallyhelpful thing to aim for.
Yusuf (09:12):
That feeds into the cost benefit analysis.
There may be a benefit thatfalls outside of the immediate
work that's being done.
Conor (09:17):
Yep.
We've spoken there aboutwhy it's important.
We've talked about whatyou can possibly focus on.
What about the how Yusuf?
Once we've ticked offon those two things, we
know we need to do it, weknow what we need to do.
What do we need tofocus on for the how?
What are the three top things?
Yusuf (09:34):
Okay.
Yeah.
glad you asked for threetop things because this
conversation could go for days.
Three top things that we needto think about in the how.
The first is, do we haveready access to the data?
Are we actually going to be ableto get the data within time?
If we are able to get it, thenthe next thing is how are we
going to bring it together?
And that is a reallyimportant step.
(09:55):
Ensuring that we are cleansingthe data appropriately.
Taking out any data thatwe aren't going to need.
So columns that we aren'tgoing to need or rows that
we aren't going to need.
There's a whole bunchof technical steps, like
how are we going to join?
We have five differentdata sets, how are we gonna
actually join them up together?
And then importantly, howdo you correlate them?
You have five indicators,you've cleansed it, you're
able to join it all.
(10:16):
How do you then actuallybring that together to be
able to provide that one view?
So we want one metricthat comes out of it.
And there's quite afew different ways in
which you can do this.
We obviously usevisualization tools.
But what we found is that bythrowing things directly into
a visualization tool, you cando joins, you can do cleansing.
But, first of all, you haveperformance issues because some
of the data can get quite big.
In terms of how thedata is actually coming
(10:37):
together, creating thescenarios that are needed.
All those sorts of thingsare better done in a backend
analytics tool and then broughtinto the visualization tool.
You've cleansed your data,you join your data, the
next thing you need to dois normalize that data.
So if you are, bringing aparticular indicator from a
particular data set, that couldlook very different to a second
indicator from a second dataset.
(10:59):
You need to make sureyou normalize it.
So you actually bringingall of the data into the
same range of values.
We usually use either zeroto one or zero to a hundred.
And then when you joiningit together and now you need
to say, okay, am I addingit and multiplying it, what
I'm actually going to do?
Typically add it and thendivide it by the number of
indicators that we have.
But that's a verysimplistic approach.
The better approach, which isslightly more complex, slightly
(11:20):
more sophisticated, is thatyou use scenarios for each
of the indicators so that youcan weight those indicators.
You know, 0/1 or 0/1/2.
So that when you bring yourdata into a dashboard, if you
like, you can actually decidewhich of the indicators to
switch on or switch off withthe 0/1 (zero one) approach.
With a 0 1, 2, et cetera,approach, switch off, lower
risk, higher risk, et cetera.
(11:42):
And so that's the sort of thingthat you want to do that you
do, like I said, in the backend.
And then when you bring itout to the front, you try to
structure it such that you havethat one overarching indicator.
The other thing that isgood to do is to make
it a relative score.
Remember bringingindicators together, there's
nothing real about thatnumber that's generated.
(12:02):
But really you want to beable to see it relative
to each other, that'sthe whole point of it.
And then having those scenarios.
Then in the dashboard, you havethat one overall indicator,
you have the differentscenario options, and then a
range of filters, dependingon what you're looking at.
So it may be year, it may bemonth, it may be particular
entities that you want toselect or groupings of entities
that you want to select.
And then also having thetable of details below that,
(12:25):
that you can actually drillinto to understand when I'm
looking at a particular number,what is it comprised of?
How was that generated so thatwhen you're exploring it or
you're explaining it, you canget to that level of detail.
Conor (12:37):
That ability to visually situate those entities
against each other, that'sparticularly helpful, I think.
And like you said, thatgives you the ability to
further drill into what'sdriving the performance
in that particular entity.
Yusuf (12:51):
Three things about how there.
One is, can I get the data?
Secondly, what am I doingin terms of normalizing
the data when I get it in?
And then thirdly, howdo I visualize it?
So getting the data, calculatingall of the fields and cleansing
it and getting it ready forvisualization, and then making
sure you have enough in thefront when you visualize it.
Conor (13:09):
And try not to piece together too many
indicators and be tooambitious from the outset.
So don't start with20 indicators across
various data sets.
Like you said.
Probably start with threeor four based on reliable
data and just build upthe profile from there.
Yusuf (13:22):
The number one thing though is to start doing it.
The more we understand thedata that we have available
to us, the better we canunderstand the businesses
that the data underpins.
And also the more we use thisdata, the better we're able
to drill into the details tounderstand some of the nuances.
Conor (13:38):
Okay.
So wrapping up.
First thing, if you'renot already doing it you
got to start doing it.
Take a sensible approach.
Maybe start off with threeindicators, stack them together.
Yusuf (13:46):
Second thing.
You can use it for planning.
You can use it for fieldwork.
You can use it for reporting.
There's various ways inwhich that will work.
And this is withininternal audit and
within performance audit.
Obviously if you're in 1stline risk or 2nd line risk,
this is something that canbecome part of your continuous
monitoring or something youcan use ongoing to be able
to understand those risks.
If you work for a regulator,it can help you identify
(14:07):
particular entities.
The application is broad, butparticularly within the audit
sphere it can be used throughoutthe lifecycle of your audit.
Conor (14:15):
And lastly, It will be a process of iteration.
Don't expect to use allthe right indicators,
to get exactly the rightoutcome, from the start.
You'll need to keep buildingon those indicators,
and some will fall away,new ones will come in.
But just know that,you're getting closer and
closer to having accuratemeasures of performance.
Yusuf (14:33):
Good stuff.
Thanks Conor.
Conor (14:34):
Thanks Yusuf.
Narrator (14:35):
If you enjoyed this podcast, please share
with a friend and rateus in your podcast app.
For immediate notificationof new episodes, you can
subscribe at assuranceshow.com.
The link is in the show notes.
You're listening to The Assurance Show.
The podcast for performanceauditors and internal auditors
that focuses on data and risk.
Your hosts are ConorMcGarrity and Yusuf Moolla.
Yusuf (00:20):
Today, we're talking about performance and risk
indicators for use withininternal audit, performance
audit, and then also extendingto first-line and second-line,
and various other individualsthat want to understand how
different indicators cancome together to provide
a view of performance, orrisk, more holistically.
Conor (00:39):
When we say risk indicators, what are we
actually talking about there?
Yusuf (00:42):
Well, depending on the subject matter that we're
looking at, it would vary.
What are the differentareas that will contribute
to our understandingof a particular domain?
Conor (00:52):
Use the data we have to build up profiles of an
entity or subject matterthat is of interest to us.
Yusuf (00:57):
Yeah, and the more data that we have from
sources that are different,the better the view would be.
Meaning not just bringingall data together from
one system or one domain.
We always talk about bringingcomplaints data in, and
complaints risk is one ofthose things that we don't
normally include as part of anaudit or a risk evaluation.
(01:18):
So that's an example.
Open data, obviously.
But then also, if you justthink about across functions
within the entity, acrossdepartments, within the
entity, bringing some of thosedifferent data points together.
Obviously, they need toactually relate to the
risks or the performancearea that we're looking at.
But they don't necessarilyhave to be directly related.
You can have a combination ofdirectly related indicators
(01:41):
that would influence strongly.
And then others thathave a minor influence.
Conor (01:45):
That all seems really intuitive and
sensible and reasonable.
There's benefits to thatapproach at the planning
stage, where you can combinerisk indicators to try and
help you select the samplefor your internal audit
or your performance audit.
Yusuf (01:58):
In the performance audit world, it would be
determining which entitiesare we going to audit as
part of a particular topic?
Within internal audit, whatis the sample that we're
going to be looking at?
So you have withinbanking, within retail,
within production andmanufacturing, which entities
are we going to look at?
Which branches arewe going to look at?
So that sort ofthing is being done.
But we can apply the conceptto a greater number of audits
(02:21):
than we're currently doing.
Conor (02:22):
There's clearly benefits, in conducting your audit, where
you can actually draw togetherindicators, whether they be from
internal data or external data,to profile the activity to try
and understand what's going on.
And then there's alsoindicators that we can use
as part of our reporting.
Yusuf (02:38):
As auditors, when reporting on these things, we
can use it to demonstrate towhoever the stakeholder is.
Whether it be the generalpublic, or internal to the
organization, or the boardor the audit committee,
or management, whoever.
Why we did what we did.
Because often people wantto know why is it that
you looked at a particularentity or particular sample?
Also we can use it toexplain, what is going on.
(02:58):
Seeing that broader pictureis enabled by bringing all of
this together, for reporting.
Conor (03:03):
Taking a risk indicator approach is really helpful
where you have, limitedcontrol over the work that
comes into your organization.
And we might see that, forexample, in a regulatory field,
if it's a regulator or anintegrity body or, somebody
that receives complaintsfrom members of the public.
Now, those types oforganizations have constraints
on resourcing, yet theyreally have no control over
(03:25):
what comes through the frontdoor in terms of their work.
So they really arereactive in nature.
And what we're seeinggenerally, is that a lot
of those organizations inthat type of arena, are now
understanding that they canactually prioritize their work
when it comes through the frontdoor by really trying to build
up indicators about what'sgoing on in their operating
environment and use that todrive their resourcing model.
Yusuf (03:48):
Yeah.
And the same could be saidfor internal audit functions.
The same can be said forsecond line risk functions
,first-line functions.
It's important then to targetyour resources at the areas that
are going to have the highestimpact and bringing this sort
of data together enables that.
Conor (04:05):
So the use of indicators is a logical response to
trying to control where youfocus your limited efforts.
Where do you start?
What's your first point of callin trying to build up a risk
indicator approach to your work?
Yusuf (04:18):
It definitely is iterative.
But you do need to startoff with something that
at least gives you areasonable starting point.
So you want to havethree or four indicators
at least to begin with.
There's no point in tryingto bring one thing together.
What's this, that's not reallygiving you a comparison.
So you need at leastthree or four to start.
How you get to thatis bottom up top down.
(04:39):
So the top down is, what isit that I'm interested in.
Bottom up is what data do I havethat I can feed into this model?
The bottom up is important, butwe do need to not throw data
in just because we have it.
So top down, what do we need?
And then bottom up, what dowe have to address the need?
And what we don't have,how do we collect it
(04:59):
for future use or can wecollect it for current use?
Conor (05:02):
And that top-down approach of trying to determine
what we need that can ofcourse change over time.
So for example, if yourorganizational priorities
change or if your strategychanges or your focus area
changes, then you shouldalways be cognizant of that
having a potential impact onthe indicators that you need.
Yusuf (05:20):
Yeah.
And sometimes you'llsee, also impact of that
through the data as well.
So you see that bringinga particular indicator.
It doesn't give you a betteranswer than you had before.
or a particular indicator,regardless of the weighting
that you have, it doesn'tactually give you a
better overall result.
Conor (05:34):
As you said, you need to be ready to iterate.
And add to thoseindicators over time.
Perhaps remove some of themthat have become redundant.
Or slightly tweak some of themthat still remain relevant.
There is possibility for alittle bit of fluctuation
in what that stack ofindicators looks like
as the business goes on.
Yusuf (05:52):
The other interesting thing that we've seen a few
times now is that you can usecertain indicators repeatedly
across multiple subjects.
And that's really usefulbecause you just need to add
to it as opposed to startingfrom scratch each time.
However, there is a howeverwith everything, however,
you need to make sure thatyou're not just using an
indicator because it exists.
Don't just grab certainindicators and pull them
across just because theymay have some sort of remote
(06:14):
relevance to the new subjectmatter you're looking at.
Conor (06:17):
So every time you look to bring in a new
indicator, you really needto ask why is this important
for our focus at the minute?
And how is this going tocontribute to the audit that
we're trying to deliver.
Narrator (06:27):
The Assurance Show is produced by Risk Insights.
We work with performanceauditors and internal
auditors, delivering audits,helping audit teams use data,
and coaching auditors toimprove their data skills.
You can find out more aboutour work at datainaudit.com.
Now, back to the conversation.
Yusuf (06:47):
I spoke about top down.
How do you determinethat top down?
Because you do a lot more thanme, of the "what indicators
are we going to be using?"What's your thinking?
How do you approach thatwhen you just have a
blank sheet of paper?
Conor (07:01):
I'm pretty old fashioned in this regard.
And I always go back to ahierarchy of needs approach.
So at the top of that mightbe, is there a legislative
imperative for what we'retrying to achieve, either for
the public or in a certaindomain that dictates that we
need to be grabbing this sortof indicator and using it.
And the next thing mightbe then what are our three
(07:23):
strategic objectives for thisorganization over the next,
usually three to five years, andtry and understand what are the
contributing factors to those.
So for example, in the corporateworld, it might be, we want to
expand our geographic footprintinto another continent.
So, I might be interested inunderstanding what are some of
the indicators of activity inthat continent that would speak
(07:46):
to my business that I need topotentially bring in to enable
that strategy to be realized.
Yusuf (07:51):
So regardless of the audit topic, because you have a
certain limited set of strategicpriorities, you want to try
to include those in everytopic that you're looking at
because there may be relevance.
Conor (08:04):
Absolutely.
Of course, there's a costbenefit there that you
need to make a judgmentcall , how much does it cost
to bring these indicators in.
Is the data complex,do we need to clean it?
Is the benefit goingto outweigh the cost?
Yusuf (08:14):
There'd be some audits that are purely compliance
focused and have a verynarrow objective that don't
relate directly to some ofthe more forward-thinking
strategic objectives.
And in that case, maynot necessarily need
this sort of approach.
But that's not themajority of audits.
Some entities are verycompliance focused, but
most audit teams will havea little bit of compliance
work that they just needto do housekeeping work.
(08:36):
But the majority of the effortwill be on helping ensure
that we achieve or ensure thatwe prevent against risks to
achieving those objectives.
Conor (08:44):
Briefly going back to the hierarchy.
So we talked about understandingwhat's happening in the
external environment,government priorities if
you're in the public sector.
What do I need to do orgrab that could help the
people in my jurisdiction?
But there's also a tertiary.
And this is to reallymature how you use data
within your organization.
That's an internalbenefit, but it's still a
really important benefit.
If you can grab some externaldata that is an indicator
(09:06):
that would also add to yourmaturity of how you use data,
then that's still a reallyhelpful thing to aim for.
Yusuf (09:12):
That feeds into the cost benefit analysis.
There may be a benefit thatfalls outside of the immediate
work that's being done.
Conor (09:17):
Yep.
We've spoken there aboutwhy it's important.
We've talked about whatyou can possibly focus on.
What about the how Yusuf?
Once we've ticked offon those two things, we
know we need to do it, weknow what we need to do.
What do we need tofocus on for the how?
What are the three top things?
Yusuf (09:34):
Okay.
Yeah.
glad you asked for threetop things because this
conversation could go for days.
Three top things that we needto think about in the how.
The first is, do we haveready access to the data?
Are we actually going to be ableto get the data within time?
If we are able to get it, thenthe next thing is how are we
going to bring it together?
And that is a reallyimportant step.
(09:55):
Ensuring that we are cleansingthe data appropriately.
Taking out any data thatwe aren't going to need.
So columns that we aren'tgoing to need or rows that
we aren't going to need.
There's a whole bunchof technical steps, like
how are we going to join?
We have five differentdata sets, how are we gonna
actually join them up together?
And then importantly, howdo you correlate them?
You have five indicators,you've cleansed it, you're
able to join it all.
(10:16):
How do you then actuallybring that together to be
able to provide that one view?
So we want one metricthat comes out of it.
And there's quite afew different ways in
which you can do this.
We obviously usevisualization tools.
But what we found is that bythrowing things directly into
a visualization tool, you cando joins, you can do cleansing.
But, first of all, you haveperformance issues because some
of the data can get quite big.
In terms of how thedata is actually coming
(10:37):
together, creating thescenarios that are needed.
All those sorts of thingsare better done in a backend
analytics tool and then broughtinto the visualization tool.
You've cleansed your data,you join your data, the
next thing you need to dois normalize that data.
So if you are, bringing aparticular indicator from a
particular data set, that couldlook very different to a second
indicator from a second dataset.
(10:59):
You need to make sureyou normalize it.
So you actually bringingall of the data into the
same range of values.
We usually use either zeroto one or zero to a hundred.
And then when you joiningit together and now you need
to say, okay, am I addingit and multiplying it, what
I'm actually going to do?
Typically add it and thendivide it by the number of
indicators that we have.
But that's a verysimplistic approach.
The better approach, which isslightly more complex, slightly
(11:20):
more sophisticated, is thatyou use scenarios for each
of the indicators so that youcan weight those indicators.
You know, 0/1 or 0/1/2.
So that when you bring yourdata into a dashboard, if you
like, you can actually decidewhich of the indicators to
switch on or switch off withthe 0/1 (zero one) approach.
With a 0 1, 2, et cetera,approach, switch off, lower
risk, higher risk, et cetera.
(11:42):
And so that's the sort of thingthat you want to do that you
do, like I said, in the backend.
And then when you bring itout to the front, you try to
structure it such that you havethat one overarching indicator.
The other thing that isgood to do is to make
it a relative score.
Remember bringingindicators together, there's
nothing real about thatnumber that's generated.
(12:02):
But really you want to beable to see it relative
to each other, that'sthe whole point of it.
And then having those scenarios.
Then in the dashboard, you havethat one overall indicator,
you have the differentscenario options, and then a
range of filters, dependingon what you're looking at.
So it may be year, it may bemonth, it may be particular
entities that you want toselect or groupings of entities
that you want to select.
And then also having thetable of details below that,
(12:25):
that you can actually drillinto to understand when I'm
looking at a particular number,what is it comprised of?
How was that generated so thatwhen you're exploring it or
you're explaining it, you canget to that level of detail.
Conor (12:37):
That ability to visually situate those entities
against each other, that'sparticularly helpful, I think.
And like you said, thatgives you the ability to
further drill into what'sdriving the performance
in that particular entity.
Yusuf (12:51):
Three things about how there.
One is, can I get the data?
Secondly, what am I doingin terms of normalizing
the data when I get it in?
And then thirdly, howdo I visualize it?
So getting the data, calculatingall of the fields and cleansing
it and getting it ready forvisualization, and then making
sure you have enough in thefront when you visualize it.
Conor (13:09):
And try not to piece together too many
indicators and be tooambitious from the outset.
So don't start with20 indicators across
various data sets.
Like you said.
Probably start with threeor four based on reliable
data and just build upthe profile from there.
Yusuf (13:22):
The number one thing though is to start doing it.
The more we understand thedata that we have available
to us, the better we canunderstand the businesses
that the data underpins.
And also the more we use thisdata, the better we're able
to drill into the details tounderstand some of the nuances.
Conor (13:38):
Okay.
So wrapping up.
First thing, if you'renot already doing it you
got to start doing it.
Take a sensible approach.
Maybe start off with threeindicators, stack them together.
Yusuf (13:46):
Second thing.
You can use it for planning.
You can use it for fieldwork.
You can use it for reporting.
There's various ways inwhich that will work.
And this is withininternal audit and
within performance audit.
Obviously if you're in 1stline risk or 2nd line risk,
this is something that canbecome part of your continuous
monitoring or something youcan use ongoing to be able
to understand those risks.
If you work for a regulator,it can help you identify
(14:07):
particular entities.
The application is broad, butparticularly within the audit
sphere it can be used throughoutthe lifecycle of your audit.
Conor (14:15):
And lastly, It will be a process of iteration.
Don't expect to use allthe right indicators,
to get exactly the rightoutcome, from the start.
You'll need to keep buildingon those indicators,
and some will fall away,new ones will come in.
But just know that,you're getting closer and
closer to having accuratemeasures of performance.
Yusuf (14:33):
Good stuff.
Thanks Conor.
Conor (14:34):
Thanks Yusuf.
Narrator (14:35):
If you enjoyed this podcast, please share
with a friend and rateus in your podcast app.
For immediate notificationof new episodes, you can
subscribe at assuranceshow.com.
The link is in the show notes.
Popular Podcasts
Fudd Around And Find Out
UConn basketball star Azzi Fudd brings her championship swag to iHeart Women’s Sports with Fudd Around and Find Out, a weekly podcast that takes fans along for the ride as Azzi spends her final year of college trying to reclaim the National Championship and prepare to be a first round WNBA draft pick. Ever wonder what it’s like to be a world-class athlete in the public spotlight while still managing schoolwork, friendships and family time? It’s time to Fudd Around and Find Out!
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy, Jess Hilarious, And Charlamagne Tha God!