July 15, 2026 • 41 mins
Today's guest is Andy Wheeler, a renowned criminologist, data scientist, and founder of Crime Decoder. Andy is a terrific voice for understanding the role of technology in analyzing and decreasing crime.
We talk through Andy’s work with police departments, including crime trend monitoring, predictive analytics, focused deterrence, and the challenge of turning data into useful operational decisions. We also discuss the practical realities of artificial intelligence in criminal justice. Rather than focusing on the hype around AI replacing workers or transforming society overnight, Andy walks through how these tools are actually being used today: helping analysts write code, automate repetitive work, build dashboards, and make better use of messy public safety data.
This is a great conversation about how the future of criminal justice analysis is already here.
Andrew P. Wheeler, PhD and founder of CRIME De-Coder, collaborates with police departments across the United States on custom software and data analytics. His work focuses on predictive analytics, operations research, and policy analysis. See his firm and other resources at https://crimede-coder.com/.
Andy also has a recent book, *Large Language Models for Mortals: A Practical Guide for Analysts with Python*, https://crimede-coder.com/blogposts/2026/LLMsForMortals
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
SPEAKER_00 (00:00):
I'm Jeff Asher, and this is the Jeffalytics Podcast.
Artificial intelligence issurrounded by big promises and
even bigger questions.
Depending on who you ask, it'seither about to transform
society or destroy it.
My guest today spends a lot lesstime talking about the hype and
a lot more time talking aboutwhat these tools can actually
do, especially in the world ofcriminal justice.
Andy Wheeler is a criminologist,data scientist, and founder of
(00:23):
Crime Decoder.
His work sits at theintersection of crime analysis,
technology, and public safety.
In this episode, we talk abouthow AI is already being used in
public safety, where it'shelping, where it's falling
short, and why the future islikely to be more incremental
than revolutionary.
We also get into how technologyand AI fit into focus
deterrence, predictiveanalytics, and the challenge of
making better decisions in anincreasingly data-rich world.
(00:47):
Technology can provide moreinformation than ever before.
Knowing what to do with it is adifferent challenge entirely.
Let's get started.
My guest today is Andy Wheeler.
Andy, thanks so much for joiningme.
SPEAKER_01 (00:58):
Yeah.
Thank you for having me, Jeff.
SPEAKER_00 (01:01):
Question I start everybody off.
And you a little, I think, moremysterious than most.
Like, what is your backgroundand what is it that you say that
you do?
What is it that you do?
SPEAKER_01 (01:11):
Yeah, so to start, I got my PhD in criminal justice
at SUNY Albany.
I most of my work was doingnumber crunching with police
departments.
So basic research, policyanalysis, operations research,
and predictive analytics for themost part.
(01:31):
I was then a professor ofcriminology at the University of
Texas at Dallas for a few years.
And then in 2019, I went over tobe a data scientist in the
private sector.
So day gig, I work uh for ahealthcare company that uh
examines fraud, waste and abusefor Medicaid claims.
But in addition to that, I stilldo consulting with police
(01:53):
departments.
And so I have a consulting firmcalled Crime Decoder, and that's
kind of where we probablyoverlap the most then.
SPEAKER_00 (02:01):
I would imagine so.
Although the fraud stuff soundsfascinating as well.
So what sort of motivates you todo all of this work?
SPEAKER_01 (02:08):
Yeah, I mean, a lot of it's just based on my
interactions with policedepartments and my interest in
really technical problems.
I view if somebody comes to meand they're like, hey, I have
this problem, whether it'smonitoring crime patterns or
doing this type of interventionor building software, I enjoy
(02:31):
working on those types ofthings, probably the same way
that some people enjoy doingcrossword puzzles or different
types of puzzles like that.
I just enjoy doing technicalwork with police departments.
SPEAKER_00 (02:42):
And so what kind of problems are you solving with
police departments?
Is there a typical type or iseach one different?
SPEAKER_01 (02:48):
Yeah, it's a lot of times it may be technical
algorithm development.
So I worked while I was gettingmy PhD, I worked as a crime
analyst.
And so some of those projectswere just born out of different
projects that I was working onat on the time while I was a
crime analyst.
(03:08):
So like one of the examples wasjust simply to tell if I'm
monitoring crimes week overweek.
So if I'm in a, if I was inTroy, New York, so we may have
10 car break-ins in last weekand then 20 car break-ins this
week, is that a significantincrease?
Is that at the point where Ishould go and dig in to those
(03:29):
crimes and try to identify ifthere's like an itinerant group
that's committing a bunch ofoffenses, or if that's just like
normal variation over time.
So that's one of the projects Iworked on is just monitoring how
do you how do you developmetrics to monitor crime over
time and identify those spikes.
Other examples are identifyingthe best individuals to give
(03:54):
focus deterrence messages to.
So one of the interventions thatpolice departments do is
basically uh gang networkinterventions, where the focus
deterrence model is basicallyyou give a deterrence message to
the whole gang.
If one gang member messes up,we're gonna go after the rest of
the gang.
Uh, so I developed an algorithmabout who to best prioritize to
(04:16):
give those particular messagesto.
And those are just a couple ofexamples in my career.
So, like I said, a lot of timesit's just born out of a
particular problem a policedepartment comes to me with, and
then I just focus on the bestway to solve that problem.
SPEAKER_00 (04:30):
I want to dig in into the folks' deterrents thing
because it's that's kind ofreally interesting.
Did you have any sort of broadconclusions?
Is there a general solution tothat problem, or is it sort of a
very much a case-by-case eachagency and each individual is
different?
SPEAKER_01 (04:46):
Yeah.
So to back up a little bit onthat, folks may, if we're
talking about gangs or orgroups, a lot of times they're
not like uh they're not likebloods and crips, it's just like
loosely affiliated groups ofmostly younger individuals for a
lot of different cities.
And so folks may think thatgroups are entirely connected.
(05:09):
So everyone in that gang or thatgroup knows everyone else.
In reality, they're not.
And so you may have two peopleaffiliated with that group or
gang, but do but don't actuallylike know each other, don't
necessarily hang out.
And so in the focus deterrenceinitiative, basically they call
in individuals from that gangand deliver that deterrence
(05:32):
message with the expectationthat you're going to disseminate
it to the rest of the group.
When I looked at the actual dataon who they called in for a few
of the different jurisdictionsthat I was working with at the
time in upstate New York, Icould tell that they called in
people that were very suboptimalto basically spread the message.
(05:52):
So if you draw, if you look at anetwork graph, it would
basically be they only called inindividuals that were on the
periphery of the graph and notlike embedded in the graph very
well.
And so I basically justdeveloped an algorithm about who
to prioritize.
So if you can only call in fivepeople, who are the best five
(06:12):
people to call in?
And so it really it probablyapplies to any department doing
that focused deterrenceintervention.
But honestly, if folks just drewthe graphs and then said, like,
uh, we should probably call inthese people who are more in the
middle of the graph as opposedto the outside of the graph,
that probably would improve thatintervention by for a lot of
(06:34):
different departments.
SPEAKER_00 (06:37):
Do you have any impression as to why they would
operate suboptimally from anorganizational standpoint?
Is it just because these are theeasiest people to call in?
Or was it throwing darts at adart bard?
Was there any sort ofconsistency there?
SPEAKER_01 (06:50):
Yeah, it was mostly just because people didn't look
at the information.
It's definitely the case thatyou can't force people to come
into those call-ins.
And so uh my experience with,and this would have been in
Albany or Syracuse, New York, atthat particular time.
There's basically groups, it mayjust be a detective who's
(07:11):
working with a crime analyst andis saying, Hey, let's call in A
or let's call in B, or invitethose two individuals to the
call-in, I should say.
And so it was very ad hoc.
So there wasn't just muchthought put behind about who to
call in at all.
And so it was basically justtaking the idea about like we
(07:32):
should look at the actualnetwork and then prioritize who
to do the call-ins for based onactual data, as opposed to, like
I said, it was just somebodysitting at a desk somewhere and
is like, uh, let's call in AndyWheeler, as opposed to actually
looking at data and making aninformed decision.
SPEAKER_00 (07:47):
And what is the response when you sort of
provide this analysis ofbasically you're doing this
wrong?
Are people receptive?
Or is there sort of pushback onthat?
SPEAKER_01 (07:56):
It's definitely I can definitely see scenarios
where there's there's going tobe pushback.
Honestly, if departments arealready working with someone
like a researcher like me,they're basically already pretty
receptive to it.
And so most of the departmentsthat they're self-selected in
that they actually want feedbacklike that.
(08:19):
It is a little bit moredifficult if, say, you're a
crime analyst in a departmentwith sort of an old school chief
that, or old school detectivesthat don't really want that
level of feedback.
But a lot of times to it improvereceptivity, you just need to
actually provide useful advice.
A lot of crime analysts willprovide a bunch of numbers, but
(08:40):
not really give useful advicefor folks.
So if it's very specific, youshould call in B instead of A, a
lot of folks won't necessarilygive pushback to that,
especially if it's like you canjust give plain reasoning about
why that person is better orthis or doing this intervention
is better.
A lot of people, a lot ofanalysts struggle with that
though.
SPEAKER_00 (09:00):
We got very very into the weeds very suddenly.
So I want to take a step backand just talk about sort of when
you kind of look at the bigpicture, what are the overall
misperceptions that you see andmisconceptions at an agency
level, at the public level,around all of the work that you
do?
SPEAKER_01 (09:15):
Yeah, definitely.
And so I think probably in termsof public perception, in terms
of what crime analysts do, a lotof people don't even realize
that police departments arereally independent.
So I'm in the research trianglearea of North Carolina now.
People don't realize that theDurham Police Department and the
(09:35):
Raleigh Police Department aretwo totally isolated systems.
They basically don't, ifsomething happens in Durham, a
crime report, there isn'tanybody in Raleigh that can
perceive that particular crimeincident report.
And so I think a lot of people,especially if you're you have
concerns about like thesurveillance state, a lot of
(09:57):
people think police departmentsare just all interconnected.
The FBI can go in and seewhatever information that they
want.
In reality, it's just we haveall these little isolated
departments doing their ownthing.
SPEAKER_00 (10:10):
And so now I want to talk about AI, because I know
that you're a heavy user, and Ithink uh you've written a ton
about this, you do a lot ofthis, and you talk about this
extensively in blog posts and onLinkedIn.
So can you walk me through sortof how do you use AI just to
start?
SPEAKER_01 (10:26):
Yeah.
And so, like I said, I my daygig, I work as a software
engineer.
And so that's really the biggestarea that I've seen AI use is
basically just using AI to helpyou write computer code to write
software.
And so with the recent, so youhad Chat GPT come out in late
(10:49):
2022, I believe.
And so a lot of differentconsumers that have seen, I like
have experience of using thatchat application.
You can just go into Chat GPTand say, hey, help me write this
Python function.
And even when it first came out,it did that, it did that very
well.
And part of the reason is thatthere's just a lot of computer
(11:11):
code on the web and they trainthe models basically on those
historical legacy code.
And it's really only gottenbetter over time.
And so if you're a softwareengineer and you write computer
code, just using the tools in avery simple way, hey, I have
this function or hey, I havethis idea for an app, they
worked really well even back in2023, and they're just getting
(11:34):
better over time to do that.
And so that's the mainapplication that I have been
using AI day to day, both in myjob and in my consulting work.
SPEAKER_00 (11:45):
And so how can this be applied sort of to the crime,
criminal justice space?
SPEAKER_01 (11:51):
Yeah.
And so the way that I view it isI expect there to be sort of two
different broader use cases ofit.
One of them is in terms ofsoftware applications, different
groups will basically bake insome of these different AI
applications into their productsdirectly.
(12:13):
And so, like there's a littlebutton in Microsoft Word
documents to use Copilot.
Different software tools,whether it's uh companies like
Peregrine that do data analyticsor whether it's records
management software, they'llbasically start having an easy
button of, hey, can I use AI todo XYZ?
(12:35):
And if we're talking about crimeanalysis applications, it may be
things like, hey, show me thecrime trends of burglaries from
motor vehicles, or hey, pull outdifferent burglaries that use
this particular type of modusoperandi.
And behind the scenes, the AIjust writes the computer code to
(12:57):
go do that particular analysis.
The other application is justcities themselves.
The AI can be used to help,because it can be used to help
write software, it really makesit easier for cities themselves,
whether it's police departmentsor other city organizations, to
really write software themselvesto do different applications.
(13:20):
So instead of paying an externalfirm to have software to do data
analysis or to do a dashboard,now it's basically the case that
a like a pretty well-motivatedcrime analyst can go in and ask
the AI, hey, help me build thisdashboard and write out a decent
application on their own withouthaving to pay an external firm
(13:40):
to do that.
SPEAKER_00 (13:42):
Is that something that's happening?
Do you have sort of examples ofthe kinds of things that those
like well-motivated analysts areproducing?
SPEAKER_01 (13:50):
Yeah, it's not happening currently all that
much now.
So I don't have any real greatexamples besides like little
idiosyncratic things analystsare building on the sides of
their desk now, which are reallynot any different than things
that they've been doing.
So, like, say automating aweekly report, now they're using
some of the AI tools to helpthem write the computer code to
(14:12):
do that.
So not so nothing real sexy totalk about there.
But I do suspect as the toolsget more widely integrated and
accepted that we'll be seeingmore of that type of work.
Um Yeah.
SPEAKER_00 (14:25):
Why do you think that is?
Is it just a people are slow toadopt new technology things or
bureaucratic?
SPEAKER_01 (14:32):
It's partly due to speed, but it's also partly due
to the technical capabilities ofa lot of analysts or city
workers for particulardepartments.
So uh one of the things that Ido is training for crime
analysts.
So I'll go in and help analystsget up to speed with Python, for
example.
(14:53):
And so the biggest struggle withmost of the departments that I
work with is you'll have a mixof younger folks who are really
interested in learning newtechnology, but then you have a
lot of folks who have beenworking there for 10, 15, 20
years who are just notinterested at all and really
just want to work at Excel or dowhatever sort of work that
(15:14):
they've done for the past 20years.
And so part of it is thatthere's just resistance to
change, as well as liketechnical capabilities of folks.
But I do think that it willimprove over time, just
naturally, but simultaneouslyit's gonna happen over time just
due to like fiscal demands forpeople to be more productive.
SPEAKER_00 (15:34):
And thinking about AI from a specifically strictly
policing standpoint, I had IanAdams on a couple of months ago
and he talked about his researchlooking at the report writing
technology.
Have you thought about the wayslike in that these technologies
are implemented not from ananalyst standpoint, but from a
wider policing standpoint andthe degree to which they're
(15:56):
either effective, or to Ian's umresearch that shows that it's
not inherently saving time rightnow?
SPEAKER_01 (16:04):
It's definitely super hard to implement these
different AI tools and savetime.
And so at my day job, one of thethings that we're doing is we're
building similar tools tobasically help nurses do audits
of Medicaid claims.
And so it's it's really hard.
And so a lot of times we'respending significant amounts of
(16:27):
engineering to save nurses likefive minutes here, six minutes
there to do those particularaudits.
And so they're not a lot of theapplications that I see, there's
a there's definitely a lot ofbuzz around AI.
And hey, saving a bunch of folksfive minutes on something is
definitely worth doing.
(16:48):
But for a lot of theapplications, I don't foresee
being able to cut the human outof the loop entirely.
And so we're talking more aboutsome of these marginal
incremental improvements asopposed to like magically it's
gonna cut report times to fiveminutes for things that took an
hour before.
And in terms of whether we'retalking about the police
(17:09):
departments or different typesof government agencies, a lot of
agencies just don't have theinfrastructure in place to be
able to really effectivelyenforce a like even know how
long that they spend onparticular things.
So I I think like a goodcounterexample are call centers.
So if folks are familiar withcall centers, those folks are
(17:32):
like metrics out the wazoo.
So folks know how long folks ittake to answer calls, how long
they spend on calls, how much oftheir downtime that they have.
And so most State Departments,whether it's we're talking about
what police officers do day today or analysts do day to day,
don't have any internal capacityto like determine if an officer
(17:56):
saves five minutes on writing areport now.
So they don't even have theinfrastructure set up to know if
it's saving people time.
The last part of that is it'sreally hard to generate savings
in terms of labor cost savingswith AI, because the only way to
actually realize savings is ifyou save a police officer five
(18:17):
minutes of their time, youaren't actually realizing any
labor cost savings.
You're already basically you'rehiring the police officer full
time around those hourly wages,essentially, to only if you're
actually if your goal is to savemoney, you actually have to
reduce headcount.
And so most places, they're notinterested in reducing
headcount.
(18:38):
They're really just interestedin doing the job better and
being able to do additionalwork.
And so I suspect it in terms ofthe AI writing, given the
advancements of the tools andget like knowledge of the
current quality of policeofficers writing reports, I
think probably the betterbenefit of those tools will be
(18:58):
it'll improve report writingover time.
It's not there yet, but I thinkthe future state, there's a good
chance that using those types oftools can improve report writing
for officers, but I don't knowif it'll be like massive time
savings in that.
SPEAKER_00 (19:14):
I foresee always needing a human.
Can you uh I heard you do thison the Jason Elders crime
analyst podcast, whose name I'msuddenly blanking on?
I'm sorry.
Uh I've been a guest on and it'skilling me.
But you just sort of walkthrough the different tools that
are available.
And can you do that here?
Talk through the different AIinfrastructure, what tools exist
from a like a Claude and andCodex and sort of the
(19:37):
differences between them?
SPEAKER_01 (19:39):
Yeah, definitely.
So Jason's podcast, which I'llgive a little bit of plugs, was
since it was mentioned, LawEnforcement Analyst Podcast
Leap.
He does a really great job.
He's done it for a long time.
And absolutely.
So yeah, so I recently wrote abook, Large Language Models for
Mortals, a practical guide foranalysts with Python.
(20:00):
And one of the reasons that Iwrote that book is like I said,
I'm I work in softwareengineering now.
It really, when ChatGPT cameout, it massively shifted the
software, the focus to not onlyusing those coding, those tools
to help you write computer code,but also in applications to use
those basically those samemodels under the hood to do a
(20:23):
lot of the stuff I work with.
It's people think AI is cool,it's so boring.
Like I'm mostly re working toreplace fax machines and like
document processing and thingslike that.
And so in that book, I basicallygive an overview of the
different types of commonapplications for those large
language models.
(20:44):
So one of the most basic ones,like I said, it's super boring,
is just what's called likestructured output extraction.
So imagine you get a PDF or ascan or an image of a check or a
police report or basically anytype of input document, these
tools are very good in ways thatwere basically a step change
(21:06):
over prior over prior tools tobe able to just take those PDFs
and extract out information fromthose.
So if we're going with a crimeanalysis example, you can take
an officer's just sort of plaintext description of what happens
at a particular incident andthen extract out things like
names and dates and addressesand even more technical things
(21:29):
like the modus operandi for howsomebody broke into an apartment
or things like that.
So it's basically like insteadof writing a specialized tool to
pull out addresses, you havelike this general tool now.
You can just tell it what toextract and it will extract that
information.
And so it's not a realinteresting application, but a
(21:51):
lot of different thingsbasically fall into that
structured output extractionsort of camp.
And so behind the scenes, It'slikely the case that the AI
report writing tools are doingthat at a particular step in the
transaction.
So Axon has all the body worncamera footage.
(22:11):
They basically turn the footageinto a transcription and then
extract out the information inthe transcript to basically fill
in the police report.
And so a lot of different,pretty boring applications, but
things that that help save timesort of look like that.
One of the other applicationsfor LLMs are basically they're
(22:35):
very good at summarizingdocuments.
And so imagine you're a policeofficer in the field, and I like
I enjoy watching the body worncam or cops or things like that.
And so one of the ones recentlywas they pulled over somebody in
an auto drive Tesla.
It was a few years ago, and theofficer wasn't, didn't know
(22:56):
which citation to give theindividual who was sleeping,
even though they were onautopilot.
So they're like, is thisreckless?
Is this reckless driving?
Like, what is this?
One of the example use cases ofAI is basically what's called
RAG, retrieval augmentedgeneration.
And that's just you have yourdocuments and you have a system
(23:16):
set up in place whereessentially you query the doc,
you ask plain text questionslike, hey, here's my incident.
What's the I have somebodyasleep at the wheel using Tesla
Autopilot.
What's the most likely criminalcharge or or traffic citation I
should use for this scenario?
(23:37):
And so the way that thoseapplications work now is you can
go in Chat GPT and ask that andit'll give an answer, but it
won't necessarily be an answerbased on your department's
documents.
And so the way to change that tobe based on your specific state
codes or your particulardepartment uh documents would be
(24:00):
you build a system where youessentially scan over your
internal documents to find themost relevant ones, and then
only feed those relevantdocuments into the chatbot and
then have the chatbot summarizebased on your local documents.
And so that's what's called RAG.
The last application that I talkabout in the book is what's
(24:21):
called, it's kind of complicatedbehind the scenes.
It's called tool calling.
But ultimately, when ChatGPTfirst came out and you ask a
question, so say um it came outin like late November 2022, I
think sometime around then.
And it was only trained on datalike earlier in the year.
So if you asked it a question,say, hey, tell me about crime
(24:46):
trends in December 2022.
So after the data that it wastrained on, it wouldn't be able
to answer that question.
It doesn't internally in theweights and in the way that the
model was trained, have thatinformation.
The way to solve that though isto do what's called tool
calling.
(25:06):
So if you do that same questionnow, what ChatGPT will do is
basically under the hood and belike, I know that I don't know
this information.
I need to actually go toexternal sources.
And it may go to like one ofyour websites, Jeff, or one of
your news articles in the Timesor things like that, and go look
up those particular crime trendsthat are more up to date and
(25:29):
then return back theinformation.
And it's so it's the same ideaas rag before, but it's
basically the model can go intomultiple steps and get the
results of those tool calls tofeed back in to further
downstream processes.
And that's pretty powerfulbecause it can not only be like
searching web documents, but itcan basically be tool calls to
(25:52):
do any intermediate output thatyou want, like whether it's
querying a local table orpulling in different documents,
like I talked about in the priorexample, or even coders, a lot
of times what we're doing iswe're having the Claude or
Codecs or whatever tool writePython code to do that work.
(26:12):
So basically we say, Hey, canyou answer this question?
And it'll write a little Pythoncode to try to answer that
question and then return theresponse.
SPEAKER_00 (26:21):
How do you account for hallucinations in the
machine that we know thatthere's imperfections?
Uh when ChatGPT first came out,I would like to be like, you
know, what has Jeff Asherwritten?
And it would come up with all ofthese articles.
I'd be like, those would begreat.
You know, those sound exactlylike things I would have
written.
I haven't written them, but andobviously the models have
improved significantly sincethen.
But are there techniques orresponses to basically just
(26:45):
hallucinations and the fact thatthese things aren't perfect?
SPEAKER_01 (26:47):
Yeah.
It it really depends on theparticular application.
And so the example I gave withtool calling.
So one of the popular ways thatpeople try to critique these
models is you can ask it howmany R's are in strawberry, and
it would give not the correctanswers.
Or you can ask it to do somesimple math questions and it
(27:09):
would get the particular mathquestions wrong.
And so I'm not sure thereprobably is a real technical
definition of hallucination, butI'm just broadly saying it's
like anytime it gives an answerwhen the answer is the answer is
wrong.
Now, if you have a particularsituation that there's a known
(27:31):
definite output, you canbasically use tool calling like
I gave the example before.
So instead of just having theLLM basically take the text in
and give its guest answer forhow many R's are in Strawberry,
it actually writes Python codeto go and check the text, check
(27:53):
the number of Rs that are inthat word.
I know that that's a trivialexample, but it basically works
that type of instead of the LLMdirectly answering the question,
it writes computer code, andthen you can audit the computer
code and make sure that thecomputer code is right to answer
the question.
That's one of the ways, andthat's really the most popular
way that people solve that forcurrent software engineering
(28:17):
applications now.
It's the system, and it's one ofthe reasons why it's so popular
in software engineering, isbecause we have constrained
systems and like we know thecorrect inputs and outputs.
And so you can basically justwrite what are called tests.
You can rewrite tests that arelike, I know if I get this
input, I should get this inoutput.
And so the agent writes computercode, you have your tests, and
(28:40):
if you know you failed yourtest, like something's wrong in
your computer code.
Now, for the case of the moregeneral, like summarized
information, I kind of liken itto an on-demand Wikipedia
currently.
Like it'll do the summaries,it'll often give you citations
(29:00):
to like external, externalsources, and it's pretty good,
but it's definitely notinfallible.
SPEAKER_00 (29:07):
And sort of what advice do you have for kind of
the non-software engineer userthat for everything from sort of
the non-technical crime analystto the policymaker to the just,
you know, your relative thatwants to use AI and chatbots?
SPEAKER_01 (29:21):
Yeah, it's definitely the even just the
free tools now.
I definitely encourage folks togo use them.
And like I gave the example, Ithink it's pretty equivalent to
an on-demand Wikipedia with sortof similar quality currently.
And so any folks who have beenin school recently knows your
(29:41):
professors do not like you tocite a Wikipedia page if you're
trying to use a point.
But Wikipedia overall is prettyhigh quality.
And so it's definitelyworthwhile for you to go in.
And if you have a generalquestion, to go in and just ask
these tools, whether it's thedifferent tools are pretty
similar in quality.
(30:02):
So it doesn't really matterwhether it's ChatGPT or Claude's
tool or Google's Gemini.
They're all to me, they're veryexchangeable, honestly.
And so you can go in and askyour question, but you basically
need to pay attention to thoseexternal resources that they've
provided.
And in addition to that, one ofthe biggest problems with the
(30:23):
models now is they're what'scalled psychophantic.
And so they'll tend to respondpositively if you ask a
question.
So if you ask, so say you wereasking about a medical condition
and you're like, hey, I think Ihave high cholesterol.
Here are my symptoms.
It's more likely to respondaffirmative, yes, I think you
have high cholesterol, asopposed to if you just ask the
(30:45):
questions and say, here are mysymptoms.
They tend to basically confirmyour suspicions.
So you definitely want to be,when you're asking these
questions, you want to becritical of the sources and you
want to avoid basically tellingthe machine what you think the
answer is to begin with.
You kind of want to come into anopen mind with those
conversations.
SPEAKER_00 (31:04):
So I keep asking it, I think I'm smart.
Do you agree?
And it keeps telling me yes.
SPEAKER_01 (31:08):
So that's you're telling me that that's not I
think you're smart, Jeff.
So I wouldn't worry about thattoo much.
But basically know that themachines are definitely
fallible.
And so there are a lot of peoplethat just take it as ground
truth.
And if you're doing that, ifyou're at a workplace and you
just have the LLM write youremail and it's like 10 pages,
(31:30):
nobody likes that.
So like take a little bit oftime to like read the output,
sort of understand it.
They're really great as teachingtools as well.
That's probably one of my mostcommon use cases of them, is
like, I don't really understandthis.
Can you give me like a more amore general explanation and
then go deep dive into that,into whatever you're learning,
(31:54):
whether it's I'm mostly focusedon tech stuff, but obviously
most people aren't going to goand ask Python questions or
database questions to Chat GPTlike I do?
But they're really excellenttools to go and help you teach
yourself those applications.
SPEAKER_00 (32:09):
Just know it's like it's fallible, just like any
human would be if they're Isthere a prediction being made
today about AI or crime data orany of this type of work that
you think will sort of be seenas ridiculous in a few years?
Kind of like if you'd askedyourself four years ago about
where ChatGPT would be now orwhere the this landscape would
be now, you never would haveguessed it's here.
SPEAKER_01 (32:31):
I haven't seen it specifically for crime analysts
in particular, but to me, I'mreally in the camp of AI is
likely to be a complement tocurrent analysts or software
engineers or basically anybodywho does stuff on the computer.
So there's a large contingencynow that really thinks that AI
(32:52):
can really disrupt the industryand and basically take
everyone's jobs.
And so this the the one of thefounders of anthropic, Ariel
Amadai, like he's big into that,basically going online and
saying, like, everybody's we'regonna take everybody's jobs,
essentially.
Which doesn't endear thetechnology to Yeah, and so I'm
(33:14):
really being like a professionalsoftware engineer, working with
the types of things thatanalysts do, I really don't,
it's really hard to cut humansout of the loop.
And so I'm really not, I'mreally bearish on that.
I think that there's a mucheasier path, though, to
basically have folks who do workon the computer have them use
(33:37):
these AI tools to really becomplement to their work now.
And so I do suspect it will belike into one of the common
things that analysts sort of getstuck doing are open records
requests.
So I know a bunch of analystswho say the majority of things I
do with my job are open recordsrequests.
And so that job is it shouldalmost be automated.
(34:00):
It should almost be you build atool that basically takes in the
records requests, builds thebasically the query language,
the SQL, to generate it, andthen just give the results and
the human do a quick pass tomake sure if it makes sense or
not.
And so that analyst, even ifthey did that for 90% of their
(34:21):
job before, there's gonna beother work for them to do now.
They're gonna be able to go domore interesting work,
basically, with their jobs.
So definitely bearish on AItaking a bunch of jobs.
But that said, I think it can bea real boon for basically
everybody who works on acomputer now.
So it will be transformative,but probably more on the margins
(34:42):
and less on the it's gonna takeeverybody's job.
SPEAKER_00 (34:44):
Aaron Powell And how do you try to account for some
of the the clear downsides ofAI, talking about you know the
environmental downsides, thesort of injecting computers into
art and things like that reallyprovoke strong reactions sort of
against the technology?
SPEAKER_01 (35:00):
Aaron Powell Yeah, there's definitely one of the
areas I've thought about it themost is actually in writing.
So there's a lot of folks usingusing these different tools to
the term of art now is slop.
So to produce AI slop,essentially.
And so there will always bethere's a lot of things like art
(35:20):
or writing that have a lot ofobjective or or not objective,
but subjective, like what isgood or what is bad.
That said, a lot of folks canlook at particular art or look
at particular writing andbasically use the Potter Stewart
approach and saying, like, thisis good or this is bad.
(35:42):
And so the that subjectiveperceptions of the work are not
going to go away.
And people will have to learnhow to use these tools to put
quality work.
And a lot of things in terms oflike subjective, you can ask
ChatGPT to generate a blog postand like first sort of this
(36:07):
superficial pass.
A lot of times it seems likereally good, but then when you
pay a little bit more attentionto it, it's pretty vapid, it's
like filled with a bunch ofplatitudes, doesn't really say
specific information.
And so there will still beindividuals who basically can
tell, okay, this is bad, but howdo I use these tools to help me
(36:29):
do my job better or faster orsmarter?
And so I think in it's thecomplement perspective.
So instead of like taking ahuman entirely out of it, how
can we use these tools to dowhat we're doing now, but do
them better?
I think that's the path forwardthat the most successful
individuals will end up being inreally the near future, not even
(36:52):
that long-term of future.
SPEAKER_00 (36:54):
And so I guess that takes me to my last question is
sort of what does the futurelook like?
The short term, the medium term,the long term with this
technology.
SPEAKER_01 (37:01):
Yeah.
I think uh for the short term,there's really just going to
continue to be incrementalimprovements into these models
that come out.
And so for the day-to-day folks,there's not going to be a new
anthropic or open AI model thatcomes out that like
fundamentally that like takesyour job tomorrow.
(37:22):
That's just not going to happen.
They're going to be theseincremental improvements to the
models, which even at thispoint, it's hard for me to tell
like the massive them theimprovements to the models in my
particular day-to-day work.
And so they've been veryhelpful, but the difference
between the Sonnet 4.1 model andthe Sonnet 4.6 model is not all
(37:47):
that, not that big of a changein my day-to-day work for
helping me write software,essentially.
And so the biggest sort ofmedium term that I'm not quite
sure how it's going to shape outis these models now are getting
more and more expensive.
(38:08):
And so there's this idea ofinduced demand.
And so a lot of times they useit for building highways on the
road.
So, like if you build morehighways, it tends to people who
took surface roads before willtake the highways now.
So it's basically a free good.
And so a lot of people whoweren't using that before are
(38:28):
using it now.
The AI models aren't necessarilya free good, like they cost
money.
And so those particular in themedium term, these different
vendors, I'm not quite sure whatthe pricing models are going to
look like.
So now you pay 20 bucks to beable to use some of these tools
on a regular basis.
(38:49):
I think that they're going toget more expensive, but there's
going to be some more,especially for companies or
enterprises rolling these out tolike 5,000 individuals at once,
there's going to be a lot ofcompetition among OpenAI and
Anthropic and Google aboutpricing.
So I do expect that these modelsto get cheaper over time.
(39:10):
And in terms of like really farout, I mean, maybe not super far
out, but maybe like say in thenext 10 years, there's really
most of the work currently rightnow has really been focused on
text in and text out.
And so there's definitely someof the tools now, you can also,
you can also include images andsay, like, hey, describe what's
(39:32):
in this image.
A lot of the work for modelimprovements, I see in terms of
uh there there needs tobasically be those same types of
level of innovations.
And it's starting to happen now,but it's really like it's really
years behind where it is fortext for video and audio and
(39:54):
images, and that's where a lotof the innovations are probably
going to come over in thecriminal justice side.
So being able to basically sayto Google's model, hey, here's
my body worn camera footage,like pull out, pull out the the
important points that I want toknow.
Or monitoring a CCTV camera,hey, give me an alert when
(40:19):
somebody fires a gun and we seea muzzle flash.
The models right now, the likethe big those are mostly being
developed by sort of these likesmall idiosyncratic teams.
I suspect that the models fromGoogle or OpenAI anthropic will
eventually be able to competewith those.
(40:40):
And that really opens it up forlike just a local department to
build that type of softwarethemselves, as opposed to
needing like several milliondollars to be able to develop
those applications.
SPEAKER_00 (40:51):
Crazy frontier.
SPEAKER_01 (40:55):
Yeah, so go to my website crimede-coder.com is is
probably the best place to lookme up.
I'm also on LinkedIn, and so Ihave a crime decoder page on
LinkedIn where I where I postthe most frequently as well.
SPEAKER_00 (41:15):
Well, thank you so much for coming on.
I appreciate it.
I know I learned a lot, andhopefully others did as well.
SPEAKER_01 (41:21):
Yeah, thank you very much, Jeff.
SPEAKER_00 (41:23):
Thanks for listening to the Jeffalytics Podcast.
Be sure to subscribe and tolearn more, head on over to
ahdatalytics.com for moreinformation and previous
episodes.
If you like what you heard,please leave a glowing review,
which will help others todiscover the show.
Until next time, I'm Jeff Asher.
I'm Jeff Asher, and this is the Jeffalytics Podcast.
Artificial intelligence issurrounded by big promises and
even bigger questions.
Depending on who you ask, it'seither about to transform
society or destroy it.
My guest today spends a lot lesstime talking about the hype and
a lot more time talking aboutwhat these tools can actually
do, especially in the world ofcriminal justice.
Andy Wheeler is a criminologist,data scientist, and founder of
(00:23):
Crime Decoder.
His work sits at theintersection of crime analysis,
technology, and public safety.
In this episode, we talk abouthow AI is already being used in
public safety, where it'shelping, where it's falling
short, and why the future islikely to be more incremental
than revolutionary.
We also get into how technologyand AI fit into focus
deterrence, predictiveanalytics, and the challenge of
making better decisions in anincreasingly data-rich world.
(00:47):
Technology can provide moreinformation than ever before.
Knowing what to do with it is adifferent challenge entirely.
Let's get started.
My guest today is Andy Wheeler.
Andy, thanks so much for joiningme.
SPEAKER_01 (00:58):
Yeah.
Thank you for having me, Jeff.
SPEAKER_00 (01:01):
Question I start everybody off.
And you a little, I think, moremysterious than most.
Like, what is your backgroundand what is it that you say that
you do?
What is it that you do?
SPEAKER_01 (01:11):
Yeah, so to start, I got my PhD in criminal justice
at SUNY Albany.
I most of my work was doingnumber crunching with police
departments.
So basic research, policyanalysis, operations research,
and predictive analytics for themost part.
(01:31):
I was then a professor ofcriminology at the University of
Texas at Dallas for a few years.
And then in 2019, I went over tobe a data scientist in the
private sector.
So day gig, I work uh for ahealthcare company that uh
examines fraud, waste and abusefor Medicaid claims.
But in addition to that, I stilldo consulting with police
(01:53):
departments.
And so I have a consulting firmcalled Crime Decoder, and that's
kind of where we probablyoverlap the most then.
SPEAKER_00 (02:01):
I would imagine so.
Although the fraud stuff soundsfascinating as well.
So what sort of motivates you todo all of this work?
SPEAKER_01 (02:08):
Yeah, I mean, a lot of it's just based on my
interactions with policedepartments and my interest in
really technical problems.
I view if somebody comes to meand they're like, hey, I have
this problem, whether it'smonitoring crime patterns or
doing this type of interventionor building software, I enjoy
(02:31):
working on those types ofthings, probably the same way
that some people enjoy doingcrossword puzzles or different
types of puzzles like that.
I just enjoy doing technicalwork with police departments.
SPEAKER_00 (02:42):
And so what kind of problems are you solving with
police departments?
Is there a typical type or iseach one different?
SPEAKER_01 (02:48):
Yeah, it's a lot of times it may be technical
algorithm development.
So I worked while I was gettingmy PhD, I worked as a crime
analyst.
And so some of those projectswere just born out of different
projects that I was working onat on the time while I was a
crime analyst.
(03:08):
So like one of the examples wasjust simply to tell if I'm
monitoring crimes week overweek.
So if I'm in a, if I was inTroy, New York, so we may have
10 car break-ins in last weekand then 20 car break-ins this
week, is that a significantincrease?
Is that at the point where Ishould go and dig in to those
(03:29):
crimes and try to identify ifthere's like an itinerant group
that's committing a bunch ofoffenses, or if that's just like
normal variation over time.
So that's one of the projects Iworked on is just monitoring how
do you how do you developmetrics to monitor crime over
time and identify those spikes.
Other examples are identifyingthe best individuals to give
(03:54):
focus deterrence messages to.
So one of the interventions thatpolice departments do is
basically uh gang networkinterventions, where the focus
deterrence model is basicallyyou give a deterrence message to
the whole gang.
If one gang member messes up,we're gonna go after the rest of
the gang.
Uh, so I developed an algorithmabout who to best prioritize to
(04:16):
give those particular messagesto.
And those are just a couple ofexamples in my career.
So, like I said, a lot of timesit's just born out of a
particular problem a policedepartment comes to me with, and
then I just focus on the bestway to solve that problem.
SPEAKER_00 (04:30):
I want to dig in into the folks' deterrents thing
because it's that's kind ofreally interesting.
Did you have any sort of broadconclusions?
Is there a general solution tothat problem, or is it sort of a
very much a case-by-case eachagency and each individual is
different?
SPEAKER_01 (04:46):
Yeah.
So to back up a little bit onthat, folks may, if we're
talking about gangs or orgroups, a lot of times they're
not like uh they're not likebloods and crips, it's just like
loosely affiliated groups ofmostly younger individuals for a
lot of different cities.
And so folks may think thatgroups are entirely connected.
(05:09):
So everyone in that gang or thatgroup knows everyone else.
In reality, they're not.
And so you may have two peopleaffiliated with that group or
gang, but do but don't actuallylike know each other, don't
necessarily hang out.
And so in the focus deterrenceinitiative, basically they call
in individuals from that gangand deliver that deterrence
(05:32):
message with the expectationthat you're going to disseminate
it to the rest of the group.
When I looked at the actual dataon who they called in for a few
of the different jurisdictionsthat I was working with at the
time in upstate New York, Icould tell that they called in
people that were very suboptimalto basically spread the message.
(05:52):
So if you draw, if you look at anetwork graph, it would
basically be they only called inindividuals that were on the
periphery of the graph and notlike embedded in the graph very
well.
And so I basically justdeveloped an algorithm about who
to prioritize.
So if you can only call in fivepeople, who are the best five
(06:12):
people to call in?
And so it really it probablyapplies to any department doing
that focused deterrenceintervention.
But honestly, if folks just drewthe graphs and then said, like,
uh, we should probably call inthese people who are more in the
middle of the graph as opposedto the outside of the graph,
that probably would improve thatintervention by for a lot of
(06:34):
different departments.
SPEAKER_00 (06:37):
Do you have any impression as to why they would
operate suboptimally from anorganizational standpoint?
Is it just because these are theeasiest people to call in?
Or was it throwing darts at adart bard?
Was there any sort ofconsistency there?
SPEAKER_01 (06:50):
Yeah, it was mostly just because people didn't look
at the information.
It's definitely the case thatyou can't force people to come
into those call-ins.
And so uh my experience with,and this would have been in
Albany or Syracuse, New York, atthat particular time.
There's basically groups, it mayjust be a detective who's
(07:11):
working with a crime analyst andis saying, Hey, let's call in A
or let's call in B, or invitethose two individuals to the
call-in, I should say.
And so it was very ad hoc.
So there wasn't just muchthought put behind about who to
call in at all.
And so it was basically justtaking the idea about like we
(07:32):
should look at the actualnetwork and then prioritize who
to do the call-ins for based onactual data, as opposed to, like
I said, it was just somebodysitting at a desk somewhere and
is like, uh, let's call in AndyWheeler, as opposed to actually
looking at data and making aninformed decision.
SPEAKER_00 (07:47):
And what is the response when you sort of
provide this analysis ofbasically you're doing this
wrong?
Are people receptive?
Or is there sort of pushback onthat?
SPEAKER_01 (07:56):
It's definitely I can definitely see scenarios
where there's there's going tobe pushback.
Honestly, if departments arealready working with someone
like a researcher like me,they're basically already pretty
receptive to it.
And so most of the departmentsthat they're self-selected in
that they actually want feedbacklike that.
(08:19):
It is a little bit moredifficult if, say, you're a
crime analyst in a departmentwith sort of an old school chief
that, or old school detectivesthat don't really want that
level of feedback.
But a lot of times to it improvereceptivity, you just need to
actually provide useful advice.
A lot of crime analysts willprovide a bunch of numbers, but
(08:40):
not really give useful advicefor folks.
So if it's very specific, youshould call in B instead of A, a
lot of folks won't necessarilygive pushback to that,
especially if it's like you canjust give plain reasoning about
why that person is better orthis or doing this intervention
is better.
A lot of people, a lot ofanalysts struggle with that
though.
SPEAKER_00 (09:00):
We got very very into the weeds very suddenly.
So I want to take a step backand just talk about sort of when
you kind of look at the bigpicture, what are the overall
misperceptions that you see andmisconceptions at an agency
level, at the public level,around all of the work that you
do?
SPEAKER_01 (09:15):
Yeah, definitely.
And so I think probably in termsof public perception, in terms
of what crime analysts do, a lotof people don't even realize
that police departments arereally independent.
So I'm in the research trianglearea of North Carolina now.
People don't realize that theDurham Police Department and the
(09:35):
Raleigh Police Department aretwo totally isolated systems.
They basically don't, ifsomething happens in Durham, a
crime report, there isn'tanybody in Raleigh that can
perceive that particular crimeincident report.
And so I think a lot of people,especially if you're you have
concerns about like thesurveillance state, a lot of
(09:57):
people think police departmentsare just all interconnected.
The FBI can go in and seewhatever information that they
want.
In reality, it's just we haveall these little isolated
departments doing their ownthing.
SPEAKER_00 (10:10):
And so now I want to talk about AI, because I know
that you're a heavy user, and Ithink uh you've written a ton
about this, you do a lot ofthis, and you talk about this
extensively in blog posts and onLinkedIn.
So can you walk me through sortof how do you use AI just to
start?
SPEAKER_01 (10:26):
Yeah.
And so, like I said, I my daygig, I work as a software
engineer.
And so that's really the biggestarea that I've seen AI use is
basically just using AI to helpyou write computer code to write
software.
And so with the recent, so youhad Chat GPT come out in late
(10:49):
2022, I believe.
And so a lot of differentconsumers that have seen, I like
have experience of using thatchat application.
You can just go into Chat GPTand say, hey, help me write this
Python function.
And even when it first came out,it did that, it did that very
well.
And part of the reason is thatthere's just a lot of computer
(11:11):
code on the web and they trainthe models basically on those
historical legacy code.
And it's really only gottenbetter over time.
And so if you're a softwareengineer and you write computer
code, just using the tools in avery simple way, hey, I have
this function or hey, I havethis idea for an app, they
worked really well even back in2023, and they're just getting
(11:34):
better over time to do that.
And so that's the mainapplication that I have been
using AI day to day, both in myjob and in my consulting work.
SPEAKER_00 (11:45):
And so how can this be applied sort of to the crime,
criminal justice space?
SPEAKER_01 (11:51):
Yeah.
And so the way that I view it isI expect there to be sort of two
different broader use cases ofit.
One of them is in terms ofsoftware applications, different
groups will basically bake insome of these different AI
applications into their productsdirectly.
(12:13):
And so, like there's a littlebutton in Microsoft Word
documents to use Copilot.
Different software tools,whether it's uh companies like
Peregrine that do data analyticsor whether it's records
management software, they'llbasically start having an easy
button of, hey, can I use AI todo XYZ?
(12:35):
And if we're talking about crimeanalysis applications, it may be
things like, hey, show me thecrime trends of burglaries from
motor vehicles, or hey, pull outdifferent burglaries that use
this particular type of modusoperandi.
And behind the scenes, the AIjust writes the computer code to
(12:57):
go do that particular analysis.
The other application is justcities themselves.
The AI can be used to help,because it can be used to help
write software, it really makesit easier for cities themselves,
whether it's police departmentsor other city organizations, to
really write software themselvesto do different applications.
(13:20):
So instead of paying an externalfirm to have software to do data
analysis or to do a dashboard,now it's basically the case that
a like a pretty well-motivatedcrime analyst can go in and ask
the AI, hey, help me build thisdashboard and write out a decent
application on their own withouthaving to pay an external firm
(13:40):
to do that.
SPEAKER_00 (13:42):
Is that something that's happening?
Do you have sort of examples ofthe kinds of things that those
like well-motivated analysts areproducing?
SPEAKER_01 (13:50):
Yeah, it's not happening currently all that
much now.
So I don't have any real greatexamples besides like little
idiosyncratic things analystsare building on the sides of
their desk now, which are reallynot any different than things
that they've been doing.
So, like, say automating aweekly report, now they're using
some of the AI tools to helpthem write the computer code to
(14:12):
do that.
So not so nothing real sexy totalk about there.
But I do suspect as the toolsget more widely integrated and
accepted that we'll be seeingmore of that type of work.
Um Yeah.
SPEAKER_00 (14:25):
Why do you think that is?
Is it just a people are slow toadopt new technology things or
bureaucratic?
SPEAKER_01 (14:32):
It's partly due to speed, but it's also partly due
to the technical capabilities ofa lot of analysts or city
workers for particulardepartments.
So uh one of the things that Ido is training for crime
analysts.
So I'll go in and help analystsget up to speed with Python, for
example.
(14:53):
And so the biggest struggle withmost of the departments that I
work with is you'll have a mixof younger folks who are really
interested in learning newtechnology, but then you have a
lot of folks who have beenworking there for 10, 15, 20
years who are just notinterested at all and really
just want to work at Excel or dowhatever sort of work that
(15:14):
they've done for the past 20years.
And so part of it is thatthere's just resistance to
change, as well as liketechnical capabilities of folks.
But I do think that it willimprove over time, just
naturally, but simultaneouslyit's gonna happen over time just
due to like fiscal demands forpeople to be more productive.
SPEAKER_00 (15:34):
And thinking about AI from a specifically strictly
policing standpoint, I had IanAdams on a couple of months ago
and he talked about his researchlooking at the report writing
technology.
Have you thought about the wayslike in that these technologies
are implemented not from ananalyst standpoint, but from a
wider policing standpoint andthe degree to which they're
(15:56):
either effective, or to Ian's umresearch that shows that it's
not inherently saving time rightnow?
SPEAKER_01 (16:04):
It's definitely super hard to implement these
different AI tools and savetime.
And so at my day job, one of thethings that we're doing is we're
building similar tools tobasically help nurses do audits
of Medicaid claims.
And so it's it's really hard.
And so a lot of times we'respending significant amounts of
(16:27):
engineering to save nurses likefive minutes here, six minutes
there to do those particularaudits.
And so they're not a lot of theapplications that I see, there's
a there's definitely a lot ofbuzz around AI.
And hey, saving a bunch of folksfive minutes on something is
definitely worth doing.
(16:48):
But for a lot of theapplications, I don't foresee
being able to cut the human outof the loop entirely.
And so we're talking more aboutsome of these marginal
incremental improvements asopposed to like magically it's
gonna cut report times to fiveminutes for things that took an
hour before.
And in terms of whether we'retalking about the police
(17:09):
departments or different typesof government agencies, a lot of
agencies just don't have theinfrastructure in place to be
able to really effectivelyenforce a like even know how
long that they spend onparticular things.
So I I think like a goodcounterexample are call centers.
So if folks are familiar withcall centers, those folks are
(17:32):
like metrics out the wazoo.
So folks know how long folks ittake to answer calls, how long
they spend on calls, how much oftheir downtime that they have.
And so most State Departments,whether it's we're talking about
what police officers do day today or analysts do day to day,
don't have any internal capacityto like determine if an officer
(17:56):
saves five minutes on writing areport now.
So they don't even have theinfrastructure set up to know if
it's saving people time.
The last part of that is it'sreally hard to generate savings
in terms of labor cost savingswith AI, because the only way to
actually realize savings is ifyou save a police officer five
(18:17):
minutes of their time, youaren't actually realizing any
labor cost savings.
You're already basically you'rehiring the police officer full
time around those hourly wages,essentially, to only if you're
actually if your goal is to savemoney, you actually have to
reduce headcount.
And so most places, they're notinterested in reducing
headcount.
(18:38):
They're really just interestedin doing the job better and
being able to do additionalwork.
And so I suspect it in terms ofthe AI writing, given the
advancements of the tools andget like knowledge of the
current quality of policeofficers writing reports, I
think probably the betterbenefit of those tools will be
(18:58):
it'll improve report writingover time.
It's not there yet, but I thinkthe future state, there's a good
chance that using those types oftools can improve report writing
for officers, but I don't knowif it'll be like massive time
savings in that.
SPEAKER_00 (19:14):
I foresee always needing a human.
Can you uh I heard you do thison the Jason Elders crime
analyst podcast, whose name I'msuddenly blanking on?
I'm sorry.
Uh I've been a guest on and it'skilling me.
But you just sort of walkthrough the different tools that
are available.
And can you do that here?
Talk through the different AIinfrastructure, what tools exist
from a like a Claude and andCodex and sort of the
(19:37):
differences between them?
SPEAKER_01 (19:39):
Yeah, definitely.
So Jason's podcast, which I'llgive a little bit of plugs, was
since it was mentioned, LawEnforcement Analyst Podcast
Leap.
He does a really great job.
He's done it for a long time.
And absolutely.
So yeah, so I recently wrote abook, Large Language Models for
Mortals, a practical guide foranalysts with Python.
(20:00):
And one of the reasons that Iwrote that book is like I said,
I'm I work in softwareengineering now.
It really, when ChatGPT cameout, it massively shifted the
software, the focus to not onlyusing those coding, those tools
to help you write computer code,but also in applications to use
those basically those samemodels under the hood to do a
(20:23):
lot of the stuff I work with.
It's people think AI is cool,it's so boring.
Like I'm mostly re working toreplace fax machines and like
document processing and thingslike that.
And so in that book, I basicallygive an overview of the
different types of commonapplications for those large
language models.
(20:44):
So one of the most basic ones,like I said, it's super boring,
is just what's called likestructured output extraction.
So imagine you get a PDF or ascan or an image of a check or a
police report or basically anytype of input document, these
tools are very good in ways thatwere basically a step change
(21:06):
over prior over prior tools tobe able to just take those PDFs
and extract out information fromthose.
So if we're going with a crimeanalysis example, you can take
an officer's just sort of plaintext description of what happens
at a particular incident andthen extract out things like
names and dates and addressesand even more technical things
(21:29):
like the modus operandi for howsomebody broke into an apartment
or things like that.
So it's basically like insteadof writing a specialized tool to
pull out addresses, you havelike this general tool now.
You can just tell it what toextract and it will extract that
information.
And so it's not a realinteresting application, but a
(21:51):
lot of different thingsbasically fall into that
structured output extractionsort of camp.
And so behind the scenes, It'slikely the case that the AI
report writing tools are doingthat at a particular step in the
transaction.
So Axon has all the body worncamera footage.
(22:11):
They basically turn the footageinto a transcription and then
extract out the information inthe transcript to basically fill
in the police report.
And so a lot of different,pretty boring applications, but
things that that help save timesort of look like that.
One of the other applicationsfor LLMs are basically they're
(22:35):
very good at summarizingdocuments.
And so imagine you're a policeofficer in the field, and I like
I enjoy watching the body worncam or cops or things like that.
And so one of the ones recentlywas they pulled over somebody in
an auto drive Tesla.
It was a few years ago, and theofficer wasn't, didn't know
(22:56):
which citation to give theindividual who was sleeping,
even though they were onautopilot.
So they're like, is thisreckless?
Is this reckless driving?
Like, what is this?
One of the example use cases ofAI is basically what's called
RAG, retrieval augmentedgeneration.
And that's just you have yourdocuments and you have a system
(23:16):
set up in place whereessentially you query the doc,
you ask plain text questionslike, hey, here's my incident.
What's the I have somebodyasleep at the wheel using Tesla
Autopilot.
What's the most likely criminalcharge or or traffic citation I
should use for this scenario?
(23:37):
And so the way that thoseapplications work now is you can
go in Chat GPT and ask that andit'll give an answer, but it
won't necessarily be an answerbased on your department's
documents.
And so the way to change that tobe based on your specific state
codes or your particulardepartment uh documents would be
(24:00):
you build a system where youessentially scan over your
internal documents to find themost relevant ones, and then
only feed those relevantdocuments into the chatbot and
then have the chatbot summarizebased on your local documents.
And so that's what's called RAG.
The last application that I talkabout in the book is what's
(24:21):
called, it's kind of complicatedbehind the scenes.
It's called tool calling.
But ultimately, when ChatGPTfirst came out and you ask a
question, so say um it came outin like late November 2022, I
think sometime around then.
And it was only trained on datalike earlier in the year.
So if you asked it a question,say, hey, tell me about crime
(24:46):
trends in December 2022.
So after the data that it wastrained on, it wouldn't be able
to answer that question.
It doesn't internally in theweights and in the way that the
model was trained, have thatinformation.
The way to solve that though isto do what's called tool
calling.
(25:06):
So if you do that same questionnow, what ChatGPT will do is
basically under the hood and belike, I know that I don't know
this information.
I need to actually go toexternal sources.
And it may go to like one ofyour websites, Jeff, or one of
your news articles in the Timesor things like that, and go look
up those particular crime trendsthat are more up to date and
(25:29):
then return back theinformation.
And it's so it's the same ideaas rag before, but it's
basically the model can go intomultiple steps and get the
results of those tool calls tofeed back in to further
downstream processes.
And that's pretty powerfulbecause it can not only be like
searching web documents, but itcan basically be tool calls to
(25:52):
do any intermediate output thatyou want, like whether it's
querying a local table orpulling in different documents,
like I talked about in the priorexample, or even coders, a lot
of times what we're doing iswe're having the Claude or
Codecs or whatever tool writePython code to do that work.
(26:12):
So basically we say, Hey, canyou answer this question?
And it'll write a little Pythoncode to try to answer that
question and then return theresponse.
SPEAKER_00 (26:21):
How do you account for hallucinations in the
machine that we know thatthere's imperfections?
Uh when ChatGPT first came out,I would like to be like, you
know, what has Jeff Asherwritten?
And it would come up with all ofthese articles.
I'd be like, those would begreat.
You know, those sound exactlylike things I would have
written.
I haven't written them, but andobviously the models have
improved significantly sincethen.
But are there techniques orresponses to basically just
(26:45):
hallucinations and the fact thatthese things aren't perfect?
SPEAKER_01 (26:47):
Yeah.
It it really depends on theparticular application.
And so the example I gave withtool calling.
So one of the popular ways thatpeople try to critique these
models is you can ask it howmany R's are in strawberry, and
it would give not the correctanswers.
Or you can ask it to do somesimple math questions and it
(27:09):
would get the particular mathquestions wrong.
And so I'm not sure thereprobably is a real technical
definition of hallucination, butI'm just broadly saying it's
like anytime it gives an answerwhen the answer is the answer is
wrong.
Now, if you have a particularsituation that there's a known
(27:31):
definite output, you canbasically use tool calling like
I gave the example before.
So instead of just having theLLM basically take the text in
and give its guest answer forhow many R's are in Strawberry,
it actually writes Python codeto go and check the text, check
(27:53):
the number of Rs that are inthat word.
I know that that's a trivialexample, but it basically works
that type of instead of the LLMdirectly answering the question,
it writes computer code, andthen you can audit the computer
code and make sure that thecomputer code is right to answer
the question.
That's one of the ways, andthat's really the most popular
way that people solve that forcurrent software engineering
(28:17):
applications now.
It's the system, and it's one ofthe reasons why it's so popular
in software engineering, isbecause we have constrained
systems and like we know thecorrect inputs and outputs.
And so you can basically justwrite what are called tests.
You can rewrite tests that arelike, I know if I get this
input, I should get this inoutput.
And so the agent writes computercode, you have your tests, and
(28:40):
if you know you failed yourtest, like something's wrong in
your computer code.
Now, for the case of the moregeneral, like summarized
information, I kind of liken itto an on-demand Wikipedia
currently.
Like it'll do the summaries,it'll often give you citations
(29:00):
to like external, externalsources, and it's pretty good,
but it's definitely notinfallible.
SPEAKER_00 (29:07):
And sort of what advice do you have for kind of
the non-software engineer userthat for everything from sort of
the non-technical crime analystto the policymaker to the just,
you know, your relative thatwants to use AI and chatbots?
SPEAKER_01 (29:21):
Yeah, it's definitely the even just the
free tools now.
I definitely encourage folks togo use them.
And like I gave the example, Ithink it's pretty equivalent to
an on-demand Wikipedia with sortof similar quality currently.
And so any folks who have beenin school recently knows your
(29:41):
professors do not like you tocite a Wikipedia page if you're
trying to use a point.
But Wikipedia overall is prettyhigh quality.
And so it's definitelyworthwhile for you to go in.
And if you have a generalquestion, to go in and just ask
these tools, whether it's thedifferent tools are pretty
similar in quality.
(30:02):
So it doesn't really matterwhether it's ChatGPT or Claude's
tool or Google's Gemini.
They're all to me, they're veryexchangeable, honestly.
And so you can go in and askyour question, but you basically
need to pay attention to thoseexternal resources that they've
provided.
And in addition to that, one ofthe biggest problems with the
(30:23):
models now is they're what'scalled psychophantic.
And so they'll tend to respondpositively if you ask a
question.
So if you ask, so say you wereasking about a medical condition
and you're like, hey, I think Ihave high cholesterol.
Here are my symptoms.
It's more likely to respondaffirmative, yes, I think you
have high cholesterol, asopposed to if you just ask the
(30:45):
questions and say, here are mysymptoms.
They tend to basically confirmyour suspicions.
So you definitely want to be,when you're asking these
questions, you want to becritical of the sources and you
want to avoid basically tellingthe machine what you think the
answer is to begin with.
You kind of want to come into anopen mind with those
conversations.
SPEAKER_00 (31:04):
So I keep asking it, I think I'm smart.
Do you agree?
And it keeps telling me yes.
SPEAKER_01 (31:08):
So that's you're telling me that that's not I
think you're smart, Jeff.
So I wouldn't worry about thattoo much.
But basically know that themachines are definitely
fallible.
And so there are a lot of peoplethat just take it as ground
truth.
And if you're doing that, ifyou're at a workplace and you
just have the LLM write youremail and it's like 10 pages,
(31:30):
nobody likes that.
So like take a little bit oftime to like read the output,
sort of understand it.
They're really great as teachingtools as well.
That's probably one of my mostcommon use cases of them, is
like, I don't really understandthis.
Can you give me like a more amore general explanation and
then go deep dive into that,into whatever you're learning,
(31:54):
whether it's I'm mostly focusedon tech stuff, but obviously
most people aren't going to goand ask Python questions or
database questions to Chat GPTlike I do?
But they're really excellenttools to go and help you teach
yourself those applications.
SPEAKER_00 (32:09):
Just know it's like it's fallible, just like any
human would be if they're Isthere a prediction being made
today about AI or crime data orany of this type of work that
you think will sort of be seenas ridiculous in a few years?
Kind of like if you'd askedyourself four years ago about
where ChatGPT would be now orwhere the this landscape would
be now, you never would haveguessed it's here.
SPEAKER_01 (32:31):
I haven't seen it specifically for crime analysts
in particular, but to me, I'mreally in the camp of AI is
likely to be a complement tocurrent analysts or software
engineers or basically anybodywho does stuff on the computer.
So there's a large contingencynow that really thinks that AI
(32:52):
can really disrupt the industryand and basically take
everyone's jobs.
And so this the the one of thefounders of anthropic, Ariel
Amadai, like he's big into that,basically going online and
saying, like, everybody's we'regonna take everybody's jobs,
essentially.
Which doesn't endear thetechnology to Yeah, and so I'm
(33:14):
really being like a professionalsoftware engineer, working with
the types of things thatanalysts do, I really don't,
it's really hard to cut humansout of the loop.
And so I'm really not, I'mreally bearish on that.
I think that there's a mucheasier path, though, to
basically have folks who do workon the computer have them use
(33:37):
these AI tools to really becomplement to their work now.
And so I do suspect it will belike into one of the common
things that analysts sort of getstuck doing are open records
requests.
So I know a bunch of analystswho say the majority of things I
do with my job are open recordsrequests.
And so that job is it shouldalmost be automated.
(34:00):
It should almost be you build atool that basically takes in the
records requests, builds thebasically the query language,
the SQL, to generate it, andthen just give the results and
the human do a quick pass tomake sure if it makes sense or
not.
And so that analyst, even ifthey did that for 90% of their
(34:21):
job before, there's gonna beother work for them to do now.
They're gonna be able to go domore interesting work,
basically, with their jobs.
So definitely bearish on AItaking a bunch of jobs.
But that said, I think it can bea real boon for basically
everybody who works on acomputer now.
So it will be transformative,but probably more on the margins
(34:42):
and less on the it's gonna takeeverybody's job.
SPEAKER_00 (34:44):
Aaron Powell And how do you try to account for some
of the the clear downsides ofAI, talking about you know the
environmental downsides, thesort of injecting computers into
art and things like that reallyprovoke strong reactions sort of
against the technology?
SPEAKER_01 (35:00):
Aaron Powell Yeah, there's definitely one of the
areas I've thought about it themost is actually in writing.
So there's a lot of folks usingusing these different tools to
the term of art now is slop.
So to produce AI slop,essentially.
And so there will always bethere's a lot of things like art
(35:20):
or writing that have a lot ofobjective or or not objective,
but subjective, like what isgood or what is bad.
That said, a lot of folks canlook at particular art or look
at particular writing andbasically use the Potter Stewart
approach and saying, like, thisis good or this is bad.
(35:42):
And so the that subjectiveperceptions of the work are not
going to go away.
And people will have to learnhow to use these tools to put
quality work.
And a lot of things in terms oflike subjective, you can ask
ChatGPT to generate a blog postand like first sort of this
(36:07):
superficial pass.
A lot of times it seems likereally good, but then when you
pay a little bit more attentionto it, it's pretty vapid, it's
like filled with a bunch ofplatitudes, doesn't really say
specific information.
And so there will still beindividuals who basically can
tell, okay, this is bad, but howdo I use these tools to help me
(36:29):
do my job better or faster orsmarter?
And so I think in it's thecomplement perspective.
So instead of like taking ahuman entirely out of it, how
can we use these tools to dowhat we're doing now, but do
them better?
I think that's the path forwardthat the most successful
individuals will end up being inreally the near future, not even
(36:52):
that long-term of future.
SPEAKER_00 (36:54):
And so I guess that takes me to my last question is
sort of what does the futurelook like?
The short term, the medium term,the long term with this
technology.
SPEAKER_01 (37:01):
Yeah.
I think uh for the short term,there's really just going to
continue to be incrementalimprovements into these models
that come out.
And so for the day-to-day folks,there's not going to be a new
anthropic or open AI model thatcomes out that like
fundamentally that like takesyour job tomorrow.
(37:22):
That's just not going to happen.
They're going to be theseincremental improvements to the
models, which even at thispoint, it's hard for me to tell
like the massive them theimprovements to the models in my
particular day-to-day work.
And so they've been veryhelpful, but the difference
between the Sonnet 4.1 model andthe Sonnet 4.6 model is not all
(37:47):
that, not that big of a changein my day-to-day work for
helping me write software,essentially.
And so the biggest sort ofmedium term that I'm not quite
sure how it's going to shape outis these models now are getting
more and more expensive.
(38:08):
And so there's this idea ofinduced demand.
And so a lot of times they useit for building highways on the
road.
So, like if you build morehighways, it tends to people who
took surface roads before willtake the highways now.
So it's basically a free good.
And so a lot of people whoweren't using that before are
(38:28):
using it now.
The AI models aren't necessarilya free good, like they cost
money.
And so those particular in themedium term, these different
vendors, I'm not quite sure whatthe pricing models are going to
look like.
So now you pay 20 bucks to beable to use some of these tools
on a regular basis.
(38:49):
I think that they're going toget more expensive, but there's
going to be some more,especially for companies or
enterprises rolling these out tolike 5,000 individuals at once,
there's going to be a lot ofcompetition among OpenAI and
Anthropic and Google aboutpricing.
So I do expect that these modelsto get cheaper over time.
(39:10):
And in terms of like really farout, I mean, maybe not super far
out, but maybe like say in thenext 10 years, there's really
most of the work currently rightnow has really been focused on
text in and text out.
And so there's definitely someof the tools now, you can also,
you can also include images andsay, like, hey, describe what's
(39:32):
in this image.
A lot of the work for modelimprovements, I see in terms of
uh there there needs tobasically be those same types of
level of innovations.
And it's starting to happen now,but it's really like it's really
years behind where it is fortext for video and audio and
(39:54):
images, and that's where a lotof the innovations are probably
going to come over in thecriminal justice side.
So being able to basically sayto Google's model, hey, here's
my body worn camera footage,like pull out, pull out the the
important points that I want toknow.
Or monitoring a CCTV camera,hey, give me an alert when
(40:19):
somebody fires a gun and we seea muzzle flash.
The models right now, the likethe big those are mostly being
developed by sort of these likesmall idiosyncratic teams.
I suspect that the models fromGoogle or OpenAI anthropic will
eventually be able to competewith those.
(40:40):
And that really opens it up forlike just a local department to
build that type of softwarethemselves, as opposed to
needing like several milliondollars to be able to develop
those applications.
SPEAKER_00 (40:51):
Crazy frontier.
SPEAKER_01 (40:55):
Yeah, so go to my website crimede-coder.com is is
probably the best place to lookme up.
I'm also on LinkedIn, and so Ihave a crime decoder page on
LinkedIn where I where I postthe most frequently as well.
SPEAKER_00 (41:15):
Well, thank you so much for coming on.
I appreciate it.
I know I learned a lot, andhopefully others did as well.
SPEAKER_01 (41:21):
Yeah, thank you very much, Jeff.
SPEAKER_00 (41:23):
Thanks for listening to the Jeffalytics Podcast.
Be sure to subscribe and tolearn more, head on over to
ahdatalytics.com for moreinformation and previous
episodes.
If you like what you heard,please leave a glowing review,
which will help others todiscover the show.
Until next time, I'm Jeff Asher.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Betrayal Weekly
Betrayal Weekly is back for a new season. Every Thursday, Betrayal Weekly shares first-hand accounts of broken trust, shocking deceptions, and the trail of destruction they leave behind. Hosted by Andrea Gunning, this weekly ongoing series digs into real-life stories of betrayal and the aftermath. From stories of double lives to dark discoveries, these are cautionary tales and accounts of resilience against all odds. From the producers of the critically acclaimed Betrayal series, Betrayal Weekly drops new episodes every Thursday. If you would like to share your story, you can reach out to the Betrayal Team by emailing them at betrayalpod@gmail.com and follow us on Instagram at @betrayalpod and @glasspodcasts. Please join our Substack for additional exclusive content, curated book recommendations, and community discussions. Sign up FREE by clicking this link Beyond Betrayal Substack. Join our community dedicated to truth, resilience, and healing. Your voice matters! Be a part of our Betrayal journey on Substack.