All Episodes

March 11, 2026 44 mins

Have you heard about the police department in Utah where report drafting AI interpreted footage from an officer’s body camera of The Princess and the Frog playing in the background of an incident to mean the officer had morphed into a frog? 

AI has come a long way in the last few years but it still isn’t perfect. Within AI is the potential for revolutionary disruption of traditional processes, but there is also the danger of relying too heavily on a tool that is only right most of the time for efforts that require perfection or near perfection. 

For this conversation, I turned to Ian Adams. Ian is an assistant professor of criminology and criminal justice at the University of South Carolina. Before taking his PhD in political science at the University of Utah, he was a police officer and police labor executive. His research is focused on policing, broadly construed, with a focus on behavior and technology.

Ian has also researched and written extensively about AI, and today’s conversation is all about the uses of AI in policing, the potential/actual pitfalls, and where this technology might be heading in the world of criminal justice. 

If you’re interested in some extra credit work, two papers related to this topic you should check out are:

  1. Adams, I. T., Barter, M., McLean, K., Boehme, H. M., & Geary, I. A. (2024). No man’s hand: Artificial intelligence does not improve police report writing speed. Journal of Experimental Criminology. https://doi.org/10.1007/s11292-024-09644-7
  2. Adams, I. T., McLean, K., & Alpert, G. P. (2026). Improving police behavior through artificial intelligence: Pre-registered experimental results in two large US agencies. Criminology, 0(0), 1–15. https://doi.org/10.1111/1745-9125.70028

To get in touch or peruse peruse different papers/projects/dashboards, Ian’s website is ianadamsresearch.com

And while you’re here, be sure to check out these other recent great episodes:

Politics podcaster Galen Druke

Arnold Ventures Executive Vice President Jennifer Doleac

FBI Assistant Director Timothy Ferguson

Orleans Parish District Attorney Jason Williams

Resources:

Follow the Jeff-alytics Podcast:

Website: The Jeff-alytics Podcast

Listen
Watch
Mark as Played
Transcript

Episode Transcript

Available transcripts are automatically generated. Complete accuracy is not guaranteed.
SPEAKER_01 (00:00):
I'm Jeff Asher, and this is the Jeffalytics Podcast.

(00:05):
Policing has always been shapedby technology.
What's different now is howquickly it's happening.
Tools that sounded experimentala couple years ago or even a
couple of months ago are alreadysitting inside real departments,
affecting how officers actuallywrite reports, how supervisors
review body cam footage, and howagencies think about
accountability.
In this episode, I'm joined byIan Adams, who's a former police

(00:25):
officer turned researcher andprofessor at the University of
South Carolina.
Ian studies how emergingtechnologies and especially AI
are actually being used inpolicing, and where the promises
don't necessarily match reality.
We talk about what AI andpolicing looks like on the
ground, from using body worncamera footage to generate
feedback on officercommunication to the idea that
AI can dramatically speed upreport writing.

(00:47):
Ian breaks down one project thatmeasurably improved officer
behavior and another that failedto deliver the efficiency gains
everyone expected, despiteofficers thinking ahead.
This episode examines howtechnology is being tested in
real departments and what thefuture of these technologies
might look like.
My guest today is Ian Adams.
Ian, thank you so much forjoining the program.

SPEAKER_00 (01:06):
Oh, my pleasure.
I'm very happy to be here.

SPEAKER_01 (01:08):
First question to all the guests.
Walk me through your background.
What what brought you heretoday?

SPEAKER_00 (01:12):
Yeah, a series of misfortunate events that ended
well, I suppose.
Uh I was a police officer, so Iwas I was in law enforcement for
close to 13 years.
Um, then did some uh laborexecutive sort of management
stuff uh while I was doing myPhD there at the University of
Utah.
I I come from a politicalscience sort of policy

(01:33):
evaluation background, but allmy work in grad school and and
since has been in uh policing.
And so I ended up in a uhcriminology and criminal
criminology and criminal justicedepartment at the University of
South Carolina, and that's whereI still hang out now.

SPEAKER_01 (01:50):
And so a lot of your work has centered on, and what
I'd love, I'm I'm excited totalk about this.
I um I guess I I only hear aboutit when it's in the news, which
probably makes it not as uh asuseful of a knowledge base.
But uh you study AI andpolicing.
Why, why look at that?

SPEAKER_00 (02:11):
Yeah, I mean, it kind of grew out of just a
general fascination uh withtechnology.
I've always been a bit of ahobbyist in technology and what
we used to just call machinelearning, but now has gets
called artificial intelligenceuh in some ways.
Um, you know, where it startedfor me was in I don't know,
early 90s.
I was one of those uh kids wholiked to uh build PCs uh back in

(02:35):
the day.
And I don't know if youremember, but back then uh the
audio card was not embedded intothe motherboard, right?
It wasn't it wasn't uh built in.
You it was kind of like videocards are today.
You would slot it in, you wouldpurchase it separate.
Anyway, there's um a an oldmanufacturer of sound cards was
called Sound Blaster.
And when you ordered a SoundBlaster card, you got this

(02:56):
little floppy disk that camewith a little program called Dr.
Sabetsu.
And what it was was really justlike a text puzzle, right?
Like it sort of mimicked beingable to um have a conversation,
but I was pretty fascinated withit.
Um, and that just over thedecades kind of stayed
consistent for me as a somethingI was interested in.

(03:19):
So when ChatGP or when earlyversions of GPT started to come
out in like 2017, um, and it wasall an open source project back
then, I was I was um in gradschool at a good point to sort
of take advantage of that andcontinue to uh monitor it.
And then I naturally startstarted seeing a lot of my um
research was around body worncameras at the time.

(03:40):
That was the big technology thatwas um being studied.
And uh I saw an opportunity andan interest in the artificial
intelligence piece, startedseeing agencies and officers
begin to use these products umon their own or sort of
off-the-shelf commercialproducts as they became
available.
And so I've been extremelyfortunate to have the good luck

(04:02):
to be in that space just as itwas sorting sort of starting to
emerge.

SPEAKER_01 (04:06):
So I'll I'll put you on the spot.
Um massive question here.
If you were to summarize, howare police departments using AI?

SPEAKER_00 (04:14):
Great, uh, poorly, a lot, and not at all.
Um it's it's a wild world,right?
Um You know, police agencies areextremely varied in the US.
Uh the the number we like tokind of throw around is there's
about 18,000 local or stateagencies.
Um the meat, the the mean or theaverage size of that agency is

(04:36):
somewhere between 12 and 15officers, which means, you know,
half of the agencies in thecountry, less than 20 officers
for sure.
Um the agencies we normallythink about, like the bigger
ones, let's say over 100, that'sjust about 6% of the agencies or
so.
Um, and so what you you'veyou're you've started to see
like a concentration oftechnology in some of the

(04:57):
largest agencies.
So you have large agencies thatare that are spending hundreds
of millions of dollars in umaxon packages for for body
cameras, uh uh uh real-timetranslation, evidence storage,
drones, report writing.
And then you have some agenciesthat don't have any of that

(05:18):
technology at all.
Um the degree to which they areable to use them wisely, I
think, is still an open questionas the technology develops, and
certainly an area where we'retrying to contribute to that uh
the that research base.

SPEAKER_01 (05:33):
And does the public have really any awareness of how
these tools are being used?
Is this just is this just a wayfor the police to use it, or is
this something that you thinkthat there's at least a
conversation understanding ofthe trade-offs that come with
using more advancedtechnologies, especially as you
get into, you know, civilliberties and surveillance and

(05:55):
things like that?

SPEAKER_00 (05:58):
Certainly there's some public awareness of this.
Um, you know, my one of my teamsin a paper that was led by
Kaylin Schiff at uh Purdue umabout a year ago, we put out
what what we found was that thepublic is actually pretty
trusting of at least localpolice, their local sheriffs,
police departments, that sort ofthing, as compared to federal

(06:19):
agencies.
They're pretty trustworthyworthy, or they they are pretty
trusting of those local agenciesto use this technology and
pretty demanding that they do.
So the public is is expectingagencies to sort of keep up to
date, uh, regardless of the sortof ability of the agency to
afford and use that technologywell.

(06:41):
Um, there are some differencesin, as there always are in the
US around partisanship.
So um, the more left-leaningrespondents really wanted to see
agencies using this kind oftechnology for internal
controls, right?
To to bring more accountabilityand transparency to the police
departments.
Whereas respondents who mightlean more to the right, uh what

(07:03):
we saw was a pattern where theyreally wanted agencies to be
using these tools for externalcrime fighting uh uh reasons.
And that gets to one of thedifficulties of these
conversations is what do we meanwhen we say police are using AI?
Do our do we mean like policeare using AI to help write
reports, sort of an internaloperational piece?
Or do we mean we want police ordon't want police using AI in

(07:27):
surveillant capabilities on theoutside of the agency?
Sort of um think Flock and uhother technologies companies
that are more aimed atcollecting information and
feeding it into theinvestigative pipeline.
The same person might have verydifferent views about those just
those two uses, let alone sortof the panoply of um uses that

(07:48):
we might imagine agencies coulduse.

SPEAKER_01 (07:51):
So I guess let's start at kind of the middle
then.
Body worn cameras is somethingthat I think that AI has been
used for and that you'vestudied.
What have you found about howare police departments using AI
to supplement their body worncameras?
And is it impactful making moremaking their departments more
efficient, or is there justnothing happening?

SPEAKER_00 (08:12):
Yeah, let's underline that word efficiency
because it's it'll come upagain, I think, in the rest of
the conversation.
But just to um answer yourquestions, so body cameras come
around, we begin rapidlyadopting in the United States
around 2015.
They immediately start producinga tsunami of data storage
problems and data reviewproblems.
The basic problem is that forevery hour of body-worn camera

(08:35):
footage, to give that a goodaudit to really review it takes
at least one human hour, right?
And so we just don't have thecapacity within law enforcement
or really probably within theUnited States to adequately
review all that body worn camerafootage.
So all the sort of hoped-forbenefits of body cameras, right?

(08:56):
Maybe they maybe they producebetter training, maybe they can
produce better transparency, allthose problems never really get
solved because we don't have away of actually looking at the
body camera footage to determineany of that.
This was my insight in gradschool, and basically what I
rolled my dissertation on waswell, what if we could use um
advances in machine learning andartificial intelligence to audit

(09:20):
100% of that footage, extractinformation out of it, and use
that as sort of a database thatwe can then get information out
back out of that huge datastorage.
Um agencies that that technologysoon became available in the
early 2020s.
There's a company called Truliothat is sort of the um original

(09:42):
uh company doing that and stilldoing it today.
And so beginning in 2022, myteam set out and started up an
experiment, an RCT in twoplaces.
One was Aurora, Colorado, onewas Richland County, South
Carolina.
And what we wanted to do therewas see if the the see if

(10:02):
Trulio's approach was successfulin changing officer behavior for
the better.
And this is what they do theytake the audio from every one of
those body cameras, get atranscript, and then use machine
learning and natural languageprogramming to extract features
of that conversation, theofficer's language, the

(10:23):
civilian's language, and then inreal time feed that back to the
officers in near real time, Ishould say.

So think about it this way: you're an officer, you're on (10:29):
undefined
patrol, you've had um, you know,eight hours of work the previous
day.
When you come in today, you havea dashboard that can tell you
basically a grade of how you didthe previous day.
Did you use a lot of explanatorylanguage?
That would be considered highlyprofessional.
Did you use maybe threatening oruh profanity or insults, right?

(10:54):
That would be kind of a lowgrade.
Or did you just kind of like,you know, do standard
professional work?
Well, what we found at the endof a one year of experimenting
across these two agencies wasthat yes, officers actually do
use that sort of self-learningmode to improve their
professionalism.
In other words, in Aurora, wefound um a very high decrease in

(11:19):
the amount of unprofessionallanguage.
So the insults, the threats, theum sort of the language we don't
want to see officers using withmembers of the public.
And in Richland, we saw um theopposite.
Um, instead of uh raising thefloor, it raised the ceiling,
meaning like officers uh beganusing up to, I think, about 80%
more explanatory language withwith the public about why they

(11:44):
were there, what and why theywere doing what they were doing.
Um and that so what we saw wasbasically a consistent picture
across these two large agenciesof um the AI feedback loop
improving policeprofessionalism.

SPEAKER_01 (11:58):
Is there any any look at sort of the efficiency
landscape from a supervisorperspective?
Does it make it easier formanagement to basically go
through all of this footage andand find either reasons to
commend or reasons to uhdiscipline, or is is the AI not
being used for that?

SPEAKER_00 (12:18):
I think we can think about that as an efficiency
issue, right?
I mean, the the problem is is inbusiness as usual in in
agencies, um often supervisorsare required or requested at
least to audit some small numberof these videos per month.
But I want you to put youyourself in a really busy
sergeant's shoes for a minute.

(12:39):
And you're supposed to review,let's say, five of your squad's
videos per week.
Which videos do you think youselect?
Probably not the ones that aresix hours long on a standoff,
right?
You probably pick ones that area little bit easier to deal
with.
Um, now some agencies have triedto like impose a randomization
schedule on that to um greateror lesser success, but the

(13:04):
ultimate um problem is that youdon't have enough time to
actually review one, even oneofficer's 40-hour week, let
alone five or six officers'40-hour weeks.
I think of it less as efficiencyand more as a step change in the
gain on actually auditing allthe videos.

(13:25):
It makes it easier for thesergeant or supervisor to
surface that information withoutactually doing a
second-by-second audit yourself.
But I want to point outsomething important.
The channel that I talked abouton improving professionalism
operates differently when thatinformation is provided directly
back to the officer versusthrough a supervisory channel.

(13:46):
So I didn't really get into allthe details, but in this
experiment, we randomly assignedofficers to either business as
usual control or a self-learningsort of mode where that in
trulio-based information iscoming directly back to the
officer, or a supervisory mode,where the information goes to
their supervisor and then getstranslated to the officer.
We saw lesser effects on thesupervisory channel when we're

(14:09):
talking about increasing thehighly professional behavior
using that explanatory language.
But we saw greater effects inthe um sort of uh improving low
professionalism when thatinformation was coming from the
supervisor.
So it's a complex uh study, it'sa complex area.
But the general lesson comingout of it is that when we want

(14:30):
to think about an agencyimproving the highly
professional behavior of itsofficers, it looks like we
probably get our best bang forthe buck by providing that
information directly back to theofficers themselves.
But when we want to sort ofreduce unwanted behavior, sort
of deter unwanted behavior,having that supervisor in the
loop seemed to um get the besteffects.

SPEAKER_01 (14:54):
So uh kind of switching gears.
Obviously, this is like veryminutiae switching gears, but um
looking at report writing,there's a very famous uh story,
I guess, that made the roundsearlier this year of a police
department that was usingsoftware, AI reporting software
to um basically capture what washappening and it didn't go so

(15:17):
well.
Can you walk me through thatincident and then also what that
says about um kind of the otherside of the coin of using AI to
improve efficiency, very much onthe back end in terms of
improving your efficiency withreport writing?

SPEAKER_00 (15:32):
Yeah, I think you're talking about the princess and
the frog out of Hebrew, Utah.
Yeah.
Um that's my old stompinggrounds.
I I come from Utah, so I'm I'mquite familiar with the area.
Uh, this is a smaller agency,and they were testing out Axon's
draft one product.
So Axon is probably the largesttechnology provider in the
space, right?
Um, they're the manufacturer ofthe taser, for example, but they

(15:55):
have their hands in a lot ofdifferent technology, advanced
technology sectors, um,including report writing.
They were the first to launchthis draft one product.
And what happens in the draftone product is similar to the
Trulio approach, right?
We're gonna take a body camera,we're gonna separate out the
audio and generate a transcript.
And then what the draft one doesis it sends that transcript

(16:16):
along with a set of custominstructions that the officer
doesn't really see.
But if you've used ChatGPTbefore, think about it as giving
some directions to the um toChat GPT on how to create a
police report from thistranscript.
So that goes out, it hits theOpenAI API, and it returns the
first draft, right?

(16:37):
Thus the name draft one to theofficer within the software
program.
The officer has to make somechanges to it and then submits
and then is able to like sort ofcopy and paste that over into
their agency's report managementsystem and submit the report.
But what happened in Heber,unfortunately, is a breakdown in
a couple places.
First, the officer wasresponding to apparently a

(16:59):
domestic call.
And in the background of thatdomestic call was um a Disney
movie, Princess and the Frog.
So the dialogue from Princessand the Frog was recorded on the
transcript.
This transcript then got sentout to the API.
It returned a police report inwhich the officer was apparently
turned into a frog, and then theofficer apparently didn't edit

(17:23):
that uh report at all beforesubmission.
And so that's a problem.
That's a big problem.
But uh importantly, it's is it atechnology problem or is it a
people problem?
And we run into that quite a bitin this area.
That one seems to me to be apeople problem, right?
The technology actually operatedkind of how we asked it to.
It it's taking a transcript andit's creating a report.

(17:44):
What why it's creating a reportwhere the officer turns into a
frog, who knows?
But the safeguard in that systemis supposed to be the officer.
The officer is supposed to be uhreviewing that report for
accuracy and completeness beforesubmitting.
And it sounds like maybe thatdidn't happen.
So um that did not accomplishits efficiency goals, obviously,

(18:05):
uh, and probably a bunch ofother goals uh as well.

SPEAKER_01 (18:09):
There's probably a good movie script coming out of
that, though.
I feel like you're like 90% ofthe way there.

SPEAKER_00 (18:15):
I'm not a big fan of the police procedurals, but I'd
watch that one, yeah.

SPEAKER_01 (18:20):
So is this the solution here better technology,
or is this one where come on,guys, like the software tells
you what to do when you didn'tdo it?

SPEAKER_00 (18:33):
Yeah, we're always gonna have human failure, right?
And so like we probablyshouldn't judge uh an entire
class of products off of thatstory as amusing as it is.
But, you know, my team didproduce the first and only
experimental evidence on thistype of technology about a year
ago.
So we teamed up with an agencyuh that was testing out that
same draft one product.
A very simple approach, right?

(18:54):
What we were interested in istesting whether the uh software
was actually able to improveefficiencies, because the number
one claim, like the marketingclaim that Axon was using and
still uses, is that this willmassively speed up officer
report writing time and save asave a lot of time for the
officers so they can get backout there onto the road and do
more productive tasks.

(19:15):
In fact, Axon claimed that itwould save a very specific
number, 82% of an officer'stime, which is a weirdly
specific number, right?
My team set out, and what we didwas we assigned half of patrol
to business as usual.
They were just going to writereports as usual.
Half of the officers had accessto the draft one tool and were
able to create reports withindraft one.

(19:35):
And at the end of that, we weremeasuring how long did it take
those officers from the timethey began a report to the time
they submitted it.
And what we found was not an 82%savings.
In fact, we found a null.
It didn't help save any time atall.
Certainly a surprise to me.
My, my, admittedly, my priorswere sort of going into this
study were that, sure, ofcourse, AI, you know, I'd use

(19:56):
Chat GPT.
Of course, it's gonna like sortof help on the efficiency front.
But not only did we find itdidn't work there, that's not a
policing only story.
There's been a couple otherstudies in the that have come
out since in both um sort ofmedical record keeping and in
computer programming that havefound similar, if even negative,

(20:16):
results.
So in the computer programmingstudy, for example, set up very
similar to ours, but whatthey're asking is can that same
sort of approach, can AI helpcomputer programs become more
efficient?
And what it found was actually a19% loss in time.
So, and then in the medicalrecords attempt, they found it
basically a null as well.
So there's sort of an emergingconsensus in these studies that

(20:38):
the hoped-for efficiency gainsare running into context,
professional contexts, that sortof hinder or even prevent those
same uh hoped-for efficiencygains.

SPEAKER_01 (20:48):
What are some of the things that would prevent those
gains from being realized?

SPEAKER_00 (20:52):
Yeah, policing has a very specific um issue.
One, and that is the technologystack.
So if you've ever hung out in apolice department, you're gonna
find out that they haveincredibly complex idiosyncratic
technology stacks.
What I mean by that is it might,if you were naive and new to
policing, you might imagine thattwo systems, um, that is the

(21:13):
report management system, whereofficers write reports, store
digital evidence, linkeverything together, and the um
CAD, which is the computer-aideddispatch uh system, that's how
officers get dispatched from thedispatch center.
It's sort of a mapping and umincident tracking uh type of
software.

(21:33):
You might imagine, like naively,that those two things probably
work really well together, maybeeven from the same uh technology
manufacturer or vendor.
But what you'd find out is like,no, these things were adopted at
different points in history,developed by very different
people.
Like, you know, it might havebeen your RMS may have been
developed by somebody's like 14year old nephew in 1982 on Coban

(21:53):
Hall.
And like it's never been updatedsince.
And the CAD is like superadvanced and um very up to date.
So they don't sit in the samepart of the technology stack.
They don't even talk to eachother in many places.
And then a second feature isthat maybe we're thinking about
the police, quote unquote,police report wrong.
The police report is more thanjust a narrative.

(22:14):
The police report is a verycomplex data entry process.
In the agency we were studying,for example, there were up to, I
think, more than 200 separatetext entries that an officer
could make, right?
They're not going to be makingthem for every incident, but
could make outside of justtyping up the narrative, uh
recounting of sort of what theydid and what they saw and what

(22:37):
they learned as part of theirinvestigation.
And so the sort of AI generatednarrative might save you, even
if it did save you a little bitof time, it's not going to save
you any uh any time on sort ofsome of the main uh activities
that an officer has to do inorder to complete the report.
So we should probably expectless marginal gains than if it's

(22:59):
just a if it's just one piece ofthat whole puzzle.
Um, those are just two of two ofthe the basic problems.
The third is that not every callactually generates a transcript,
right?
So, like a a lot of calls thatofficers take that still need to
have a report are done over thephone and won't have that sort

(23:19):
of um transcript of aconversation going on between
them and somebody in the public.
So we shouldn't expect that thisis uh a sort of silver bullet
that can solve thehundred-year-old problem of how
do we get officers to writebetter, faster reports.

SPEAKER_01 (23:33):
So, what do officers think about sort of the
effectiveness of these tools?
Are they annoyed by it or dothey think that this can change
everything that they do?

SPEAKER_00 (23:43):
Yeah, good question because we went back.
So, our second study in thatsame department, what we wanted
to find out is how did thosesame officers think about the
technology that we had just umallowed them to use?
And we found some fascinatingstuff.
First, officers are prettypositive about this wave of
technology.
They they're pretty excitedabout it.

(24:04):
They're somewhere betweenneutral and positive.
And that is a little bit of asurprise just uh if you think
about it in the context of bodycameras, right?
When body cameras were firstsort of introduced into the
policing workspace, they werereceived quite negatively.
They were seen as maybe a toolof where uh management was going
to use them as a as a fishingexpedition to find small policy

(24:25):
violations and jam, jam officersup.
But they they sort of quicklybecame very, very popular within
policing.
So if you ask officers todayabout body cameras, they're very
positive.
On the this AI report writingfront, what we found was
somewhere between neutral tovery positive, uh basically
across the entire um uhdepartment from the officers to
their sergeants.

(24:46):
But then a really fascinatingthing showed up in our analysis.
About half of the officers toldus this technology definitely
helped them write fasterreports, right?
They they they were in factendorsing that marketing message
that Axon was using.
And so we went back and welooked because, of course, we
have records of those officersin black and white data.

(25:09):
Like, I can actually find out.
And lo and behold, what we foundout was there was this huge gap
between perception and reality.
The officers were telling usit's saving them time.
But when we go look at theiractual records, we we see, like,
no, it's not saving them anytime at all.
And so that's one of the thingsI like to warn uh police
executives about, especially, isthat if you don't do a careful

(25:30):
evaluation here, you you mightbe misled.
If you just do a traditionalsort of check-in with the
troops, how's it going?
Did you like the technology?
Did it work?
That might give you one answerthat's very, very different than
the a sort of more rigorousapproach where we're looking at
their at the underlying data.

SPEAKER_01 (25:46):
Do we have any idea why that is?
Is it like I'm thinking that,you know, I'm I can either sit
at this traffic stop and it'sgonna take 10 minutes to get
through all of thisconstruction, or I can go the
other way around and it's gonnatake 10 minutes to go the other
way around, but at least I'll bedriving and I'll feel like I'm
doing something.
Is that the same thing, but forpolice officers?

SPEAKER_00 (26:04):
I think you're on to something there.
Um the the the honest academicanswer is like we don't know the
pure mechanism that's causingthis discrepancy, right?
To be clear.
But it's probably something likethat, or probably something
like, hey, if if if I come inand I, you're an officer, you're
doing a difficult job.
It is a it is a challenging workenvironment.

(26:27):
There's a lot of time pressure,there's a lot of task pressure.
Um, every single year, callvolume goes up, right?
Even as over the last fiveyears, there's been pretty
significant labor pressureswithin policing, meaning we
don't have as many officers aswe need to answer all those
calls.
So if you're in that environmentand somebody comes in and says,
hey, I've got a magic toolthat's gonna save you all kinds

(26:48):
of time, I think it's just humannature to kind of feel like,
yeah, that saved me some time.
There's a bit of confirmationbias.
But again, who knows about theexact mechanisms?
I know there's um Brandon May isa psychologist down in Florida
who's very interested in thistechnology.
And that's that's more hisinterest, is like what exactly
is occurring um in the sort ofperception uh uh boundary of

(27:09):
this of this problem.
Um, but what I can tell you isthat right now, as far as the
most rigorous evidence in theworld is concerned, there was no
time savings.
And yet about half of theofficers using the technology
said it they perceived that itwas.
And so we should be aware ofthat when we're trying to sort
of on balance figure out is thistechnology worth the huge

(27:32):
dollars that agencies are beingasked to put towards it.
So, just as one example, in thatagency that we studied, the cost
for just the draft one productwas equal to about 11% of this
agency's after labor budget.

SPEAKER_01 (27:45):
Jeez, right?

SPEAKER_00 (27:47):
So, like this agency has about a$32 million budget a
year.
But in policing, somewherearound 95% in this agency, I
think it was 96%.
96% of its budget goes directlyto sort of paying the troops and
and and making sure that theycan answer 911 calls and
investigate crimes.
So you have about 2.2 millionleft over.
And the cost for just the draftone piece was around 11% of that

(28:10):
per year.
And it's, you know, chiefs andsheriffs are under a lot of
budget pressure all the time.
They always need more to domore.
And so asking them to spend 11%of their budget on a product
that isn't having the primaryeffect that it was promised is
probably too much to ask.
And we don't really knowfunctionally how much agencies

(28:32):
are actually spending in the US.
I mean, we can we can observe uhquarterly earning reports from
Axon and know that um in some oftheir latest big dollar
contracts, they're getting toabout$600 per officer per month
in annual recurring revenue.
So that can give you some ideaof the dollars at stake, but it

(28:53):
doesn't tell us in aggregate howmuch is being spent because of
course that's just one vendor,albeit the biggest one.

SPEAKER_01 (28:58):
Is there a happy medium between how AI can best
serve policing and how policedepartments can responsibly and
effectively and efficientlyintegrate AI?

SPEAKER_00 (29:10):
Well, yes, at least in theory, right?
I mean, that that's part of theprocess that's occurring right
now across the United States indifferent agencies and different
um jurisdictions.
This is a good time to give ashout out to the um Council on
Criminal Justice.
I sit on the AI task force atthe Council on Criminal Justice,
so I quite like thatorganization, especially it has
a it has a pretty even-handedapproach to this sort of thing.

(29:32):
And what's coming out of thosesort of meetings is one, it's
really hard to come up with theanswer to the question on
everybody's mind, which is sortof like, is it worth it?
Right.
It's some sort of balancingquestion.
Is it worth, you know, whatbenefits can we get?
What risks are we trading offon?
That's ultimately a big publicpolicy question.
To the degree that it's usefulto have an empirical grounding

(29:57):
for that question, I would saywe are nowhere near knowing the
answer to either the benefits orthe costs, right?
Like I've covered two umevaluations of AI and policing.
One of them uh we saw reallyimportant effects, right?
It's important that officers uhbehave professionally when
they're dealing with members ofthe public.
In the other, we got a big null,right?

(30:18):
We didn't that that sametechnology didn't save officers
time when they're generatingreports.
That's not much of an empiricalbasis to proceed on in like a
policy uh question.
Uh, but I would also argue thatwe we are confronting the exact
same, if not worse, picture whenit comes to the risks.
We don't even know really how todefine the risks.

(30:40):
I think they sort of get bakedinto questions about the
surveillance and constitutionalquestions and due process, all
of which are super interestingand important.
But we are absent in empiricalrecord of showing like that that
stuff actually shows upsomewhere in the data.
And so I'm a data guy.
That's where I live.
It's uh it doesn't mean thatthose more normative questions

(31:02):
aren't important.
But when we want to ultimatelyget to the sort of um end
question of those normativediscussions, I think that it's
helpful to have good data.
We just need more of it.
We need more researchers, moreresearch funding, more time, and
more agencies willing to do thehard evaluations up front to

(31:22):
sort of get ahead of that uhthat bad outcome where agencies
are spending 10, 11% of theirbudget on a technology that
feels good but isn't actuallydoing anything.
Do you get a sense that this issomething?

SPEAKER_01 (31:34):
I mean, we before we went on, we were talking about
Cloud Code and just howincredible the advances have
been over what, six months?

SPEAKER_00 (31:41):
Is that, you know, I would even say like a month,
right?
Like I would say in the lastmonth that thing is taken off.

SPEAKER_01 (31:48):
Is this something where just the conversation
we're having now today in early2026 will be just completely
null and void, and thetechnology will have advanced to
the point where it'll becompletely new in six months, a
year from now?

SPEAKER_00 (32:03):
There's a big part of me that always has that in
the back of my mind when I'mtalking about these things,
right?
Just like the academic's uhworst nightmare of being wrong
at some point.
But um the the answer is like,of course, things are gonna
change.
There, there is a counterweightto that, and that is that
policing is an inherentlyconservative, like small C
conservative institution.
It doesn't move quick.

(32:24):
Um, and so we have time.
I don't want, I am a technologyoptimist.
Sometimes people hear about thethe report writing study and
they think I'm a technologyhater.
It's like, no, that's just whatthe experiment showed.
I actually think that we are umtoday is the worst that any of
these technologies will ever be.
Next month we're gonna be in amuch better spot and let alone

(32:44):
in 10 months.
So I yes, the world will look alot different.
I sometimes, when I'm talking topolice executives, like to use
the uh the pager metaphor.
So I ask them like, who hereremembers getting a pager for
the first time?
And usually there's somebodyaround who started in maybe
their late 90s and they remembergetting the pager and how cool
it was.
And then they remember gettingthe first time they got a mobile

(33:05):
phone and a laptop in their car.
And and I ask them, like, well,if I had asked you back in '97
to predict what technology lookslike today from that pager,
could you have done it?
And the answer is always no.
And like, and yet you livedthrough 30 years of a profession
relatively rapidly adoptingmodern technology into police

(33:26):
practices.
We will find a way.
There will be a new normal atsome point.
We're living through probablythe fastest technology adoption
cycle that we've ever seen,right?
I would argue that AI is beingadopted much faster than body
worn cameras uh were, let alonelaptops and that and all that
technology.
But um eventually we'll getthere.

(33:48):
Uh the the safer we can do itwith good research, the better.

SPEAKER_01 (33:53):
Can I ask you about for science?

SPEAKER_00 (33:55):
Yeah.
I didn't expect you to, but youcertainly can.
Yeah.

SPEAKER_01 (33:58):
Good.
Well, I I had seen some of it,and I just I watched the full
upload of the video for theperf.
So I I I want to talk about thisjust because you're you're here,
and I think this is fascinatingand such a good example of just
sort of ethics and responsiblyapproaching research and and
data.
What is force science and sortof what is the controversy, the
the issue that you've beenlooking at?

SPEAKER_00 (34:19):
Yeah, okay.
So force science is a privatecompany.
It's a vendor, it's a trainingvendor mostly, in that is very,
very widespread in policing.
Um, they position themselves inthe market as a scientific voice
that can give scientificinformation about what happens

(34:39):
in sort of the most uh uhcritical incidents in policing,
right?
Use of force, shootings, thatsort of thing.
That's their bread and butter.
They hold um dozens and dozensand dozens of trainings across
the United States every year,training officers and
investigators in this umpurportedly scientific approach.

(35:01):
The problem is, of course, whenyou have a a scientific
background and you begin to lookat their scientific corpus, the
the collection of studies thatthey say they've put together,
anybody that's taken even a verybasic research design course
will look at one of theirstudies and uh be horrified,

(35:21):
right?
Extremely small sample sizes,samples that were uh not just
convenient, but superconveniently sampled, um, bad
reporting, incomplete reporting,inferring to uh generalized
populations that weren't eventested in the original study.
It's pretty bad.
It's really, really bad.

(35:41):
So um my team published a studylast year that reviewed all of
their studies, right?
They put them all together in abook and they said these are our
most critical studies, our 24most critical studies.
And we just sent each studythrough very basic scientific
review processes, right?
You may have heard of like theMaryland scale, which is um
widely used in criminology tosort of judge a study or a set

(36:04):
of studies on its ability tosafely inform policy, for
example.
Um, but we used and we used twoothers.
That's the details may notmatter as much, but if you're
interested, uh the paper'savailable at police quarterly.
And at the end of it, webasically found that across this
corpus, there was um very, verylow scientific reliability,

(36:25):
meaning they just didn't followbasic, normal scientific
processes in either the design,the execution, or the reporting
of those studies, and thusshould not form the a reliable
basis for police training orevaluation.
Um you called it controversial.
It certainly is.

(36:45):
That study was raised eyebrowsbecause this is a company that
is well received withinpolicing, right?
I they they've they're wellloved.
Um, they give pseudoscientificanswers that officers often want
to hear.
I was one of those officersonce, by the way.
I've been through the fourscience training.
Um, and there's something quiteseductive when you're an officer

(37:07):
and you're sitting in a room andthey're parading a bunch of
people who insist on beingcalled doctor in front of you,
and they tell you the answersyou want to hear, and they say
not only that, that's whatscience says.
That's if you're not, if youdon't have a scientific
background and don't know how toevaluate that information, um,
that could be a pretty seductivemessage.
It it it all of us like to betold what we were hoping to

(37:30):
hear, but science doesn't giveus the answers we were hoping to
hear, right?
That's not how science works.
Science doesn't promise someanswer, it promises answers to
well-constructed questions thatwe can um sort of know how that
scientific answer came to be andevaluate it.
The unfortunate reality is thatwhen Force Science and I met for

(37:51):
a debate at the Perth Town Halluh this last October, they
didn't send a scientist todefend their results and they
haven't ever really even triedto rehabilitate those results.
Um, they sent a lawyer, right?
That's who they sent to debate.
I see myself as one of thepeople who has the unique
perspective of a street cop,somebody who's been through

(38:13):
force science, somebody who hasa rigorous scientific education
and I think contributes usefullyto scientific conversations in
policing.
And so um hopefully we can sortof keep pushing because I think
it's really, really importantthat agencies do have an a
scientific evidence base whenit's available to help guide
them in their operations.

(38:34):
Like that is the best approach.
We know that's the best approachfrom all kinds of professions,
right?
Um, policing's no different.
And I don't, what I don't wantto see is sort of a universal
distaste for scientificinformation that comes out of
that, right?
That's not my goal.
My goal is to improve thescientific evidence base and
scientific knowledge andeducation and make police

(38:56):
agencies better consumers ofthat information, ultimately.

SPEAKER_01 (38:59):
That's great.
And if anybody's interested, umthe video is available on
YouTube uh through the policeexecutive research forum.
So um I I think it's a greatlesson and even if you have no
background in it, I think it itwas very interesting.
And and um many of the nerdsthat have, I think, been on this
podcast or will eventually be onthis podcast were made an

(39:21):
appearance there.
So it was it was fascinating towatch.

SPEAKER_00 (39:23):
Ian, what what's next for you?
Man, uh I am super excited aboutClaude Code and other agentic
coding.
Um, I'm super excited about theability to maybe use some of
these tools to improveinvestigative processes.
Um in policing, we have a realreal terrible problem with
solving specific types ofcrimes.

(39:46):
Just as an example, right?
You you're you and yourlisteners will be well aware
that we don't solve every murderin the United States.
But even worse are the ones thatare near murders.
So um this oftentimes thedifference between a shooting
that results in a death and theshooting that results in a
life-altering injury is nothingmore than stochatic chance.

(40:06):
It's just randomness.
Uh, where that exactly thatbullet lands in your body, a
millimeter this this way or thatway.
But our solve rate on murdershovers around nationally around
60%.
You you'd be better on thatnumber.
Yeah, that's about right.
It gets as low, it gets under20%, sometimes close to 10% on
um what these attempted murdersuh got by gunfire.

(40:30):
And like we know that those twotypes of events are very, very
alike one another.
And so we shouldn't see that bigof a gap in solve rate.
So why are we seeing that solverate?
Well, there's something aboutinvestigative effort that's
going on, right?
I know cops, they care aboutthose cases too.
But if you are a murder cop inLA or uh Albuquerque or Houston,

(40:56):
you've got a pretty bigcaseload.
And a lot of those are uh goingto be both homicide and near
homicide.
So your investigative effort,there's only so many hours in a
day.
So, how can we use technology tobetter improve those processes?
Is something that I see a lot ofcompanies attacking right now.
To be clear, I don't develop.
I'm not a developer, I'm anevaluator.

(41:16):
But I see these um agenciesbeginning to experiment with
that technology, and I'm superexcited about like maybe we can
actually get a handle on thatproblem uh sometime in the next
year or two.

SPEAKER_01 (41:27):
That's great.
And I I know that that is afascinating question of we've
had all of these advances in thelast 25 years, and yet the
murder clearance rate isbasically slightly lower than it
was 25 years ago.

SPEAKER_00 (41:40):
Right.
And the other technology that uhI'm very excited about, and and
it is being used more often thanpeople know, is drones in
policing.
And there's a couple differentways to use drones.
Like in the last 10 years,you've seen a real upswelling of
drones in the police car, right?
Uh drones as operationalresponse, meaning there's a cop

(42:00):
in a car and he can get thedrone out if he wants and maybe
use it um sporadically.
We haven't seen good evaluationsof that, to be clear, and and
adoption has been rather uhstumbly, let's say.
But over the last year or two,we've started to see DFR drones,
drones as first responders.
And um, I've had the chance tosit in at agencies watching this

(42:24):
technology work.
And in fact, my um uh one of myteams has uh an active dual-site
RCT in two large American citiesright now using this technology.
This is different.
These sit on pods on the top ofbuildings and are able to um, in
one of our cities, get on top ofa reported incident within 72

(42:45):
seconds of the call, not thedispatched call.
So um, just as a reminder of howthis process works, somebody
calls 911.
You are talking to a call taker,usually takes a few minutes to
get information before thatinformation is dispatched out to
the police agents, policeofficer for actual response.
So there's already like thistwo-minute gap, and that's

(43:08):
before it takes, you know, theofficer a few minutes to a long
time to get to that actual call.
Well, these uh these agenciesare able to um have sergeants
and corporal pilots who arealready experienced officers who
are hearing the 911 call come inin real time and make a quick
snap decision about whether toget a drone up and going towards

(43:32):
that location right now and beover that target on average
within 72 seconds of that call.
That's a huge change.
Um, has tons of opportunity, notonly for like the public service
piece of this, people like fastpolice response, to be clear.
So, like that's an important umsort of primary goal.
But but unknown and stillunproven is like, can we get

(43:54):
better operational outcomes outof this as well?
Um, can we get better suspectIDs?
In violent crimes.
Can we, um, instead of sending abunch of officers into a foot
pursuit, can we use the dronetechnology to sort of make for a
safer environment, not only forthe officers, but for the
subject themselves?
So there's all kinds ofquestions to come.
I think we'll get some of thoseanswers here in the next six
months.

SPEAKER_01 (44:15):
That's awesome.
Oh, excited to hear about that.
We'll have to have you back onlater this year then.
Anytime.
Anytime.
All right.
Ian, thanks so much for coming.
Thanks.
Thanks for listening to theJeffalytics Podcast.
Be sure to subscribe and tolearn more, head on over to
ahdatalytics.com for moreinformation and previous
episodes.
If you like what you heard,please leave a glowing review,

(44:35):
which will help others todiscover the show.
Until next time, I'm Jeff Asher.
Advertise With Us

Popular Podcasts

Stuff You Should Know
Dateline NBC

Dateline NBC

Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com

Crime Junkie

Crime Junkie

Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by Audiochuck Media Company.

Music, radio and podcasts, all free. Listen online or download the iHeart App.

Connect

© 2026 iHeartMedia, Inc.

  • Help
  • Privacy Policy
  • Terms of Use
  • AdChoicesAd Choices