Avoiding the Pitfalls of Using AI in a Legal Practice

I spoke with Binit Agrawal, the Chief of Staff at Lucio, an India-based AI-native workspace for lawyers. We discussed the pitfalls of using AI in a legal practice, the mindset and skill set that junior lawyers should adopt, and how AI in the legal profession is evolving.

Episode Transcript

(00:01):
Welcome to Reinventing Professionals,a podcast hosted by industry analyst
Ari Kaplan, which shares ideas,guidance, and perspectives from market
leaders shaping the next generationof legal and professional services.
This is Ari Kaplan and I'm speakingtoday with Bennett Agrawal, the chief

(00:24):
of staff at Lucio, an India basedAI native workspace for lawyers.
Hi Bennett.
How are you?
Hi, Ari.
I'm doing well.
Great to meet you.
Tell us about your backgroundand your role at Lucio.
I happened to be a lawyer practiced asa competition lawyer for a bit of years.
Studied at national Law School of India.

(00:44):
Went to University of Viennaworked at a few forms around the
world doing some internships.
And what I ended up discovering inthis entire process, even though I was
loving the good stuff that lawyers.
Great colleagues, lot of interestingreading at the intersection of law,
society, economics, all of that high Octaconversations and obviously great pay.

(01:05):
What I started to discover wasthat law was very slow and at
some point my firm, where I wasdoing competition law in India.
Started using this legal AIplatform called Lucio for all
sorts of different things.
And there was a realization thatLucio is not just great for me, but
it could be great on all fronts.
It could be great ifclients started using it.

(01:26):
It could be great if the regulatorsthat we engaged with started using it.
And when that realization startedto dawn upon me, I called up these
guys at Lucio who happened to befriends from college, and I was like.
Can I join you guys And Yes.
Ended up there.
I'm the chief of staff overseeinga team of about 50 people, lawyers,
AI engineers, all sorts of people.
So that's how I ended up at Lucio.

(01:48):
What is an AI native workspace?
We started off and still arepartly a legal AI platform.
What I mean by platform is somethinglike chat, GPT, it's a website.
Lawyers who are our users endup on this website whenever they
want to do some kind of resource.
They want to get some drafting done.
Want to ask queries of a document?

(02:09):
They come here, they uploadstuff, they get something done.
They would copy it, paste it onto theirword file, or whatever file they're
working on, make edits there, and that'sthe process which is followed when
you are talking about an AI platform.
We thought that's not efficient.
That has a lot of friction points.
Firstly, a lawyer is usuallyspending time on a PDF on a word.

(02:31):
They are having to think of something.
They're having to think, okay,now I need to call upon AI and use
ai, and they end up on a website.
They have to upload stuff, download stuff.
So we thought, let's try toreimagine this and what we have been
working on and what we have been.
Trying to pitch to the worldis more of an AI workspace.
It's a complete suit of tools forlawyers trying to use ai, but not always.

(02:54):
The idea is to make AI availableto them when they need it and
when they want to call upon it,instead of just selling to them.
Ai ai, which may not necessarilyhave the great adoption rate,
which has been talked about, orwhich has been sold to the world.
You mentioned using Lucio at yourprior firm and others using it.

(03:15):
What kinds of organizations are takingadvantage of Lucio and in what roles?
It's very similar to what otherlegal AI platforms are seeing.
You have your big law firms, the tophundreds, top two hundreds from around
the world taking up legal ai becausethey're really the only ones who can, per
se, afford the cost of legal AI Today.

(03:35):
Legal AI happens to be pretty expensive.
So we do have.
Big firms in India.
India's largest law firm.
They use us but we do also have alot of in-house teams about 30, 40
in-house teams mostly in India, in uain London using us we have started to
see a lot of pickup from boutique formsor 10 member forms, 15 member forms.

(03:56):
Who are trying to match thespeed of these big firms.
Some of these firms havealso started using us.
I would say there are about 200organizations, small and big around
the world, spread across eightjurisdictions using CIO today.
How does CIO improve theway lawyers practice?
There are three views on this really.

(04:17):
One view is that Lucio actuallyimproves the lives of lawyers in
terms of reducing the grant work.
In terms of saving themas there's a set of users.
I would say about 20% of our totaluser base is using Lucio in a way
that saves them a lot of time.
Then there's a set of user base, Iwould say about 30% who use Lucio.

(04:37):
To some advantage at different points,but it's not life changing for them.
And then there's a last set of user base.
Who log into Lucio oncea month, twice a month.
But it's not lifechanging at all for them.
They wouldn't pay for it on their own.
They would pay.
They're using it because theirfirm has made it available to them.
And that's across the board.
We are noticing this for alllegal AI platforms, small, big

(04:59):
in London, or in Delhi or Mumbai.
In India the first set of users reallyare power users or people who have
tracked how to make use of legal ai ina way that really changes life for them.
They are using it to get drafting done.
Often lawyers would have templatized work.
They're looking at some old drafts firmhas some institutional knowledge base.
They're taking things outof those, modifying it.

(05:21):
They automate that process today.
They put it on Lucio, they put ina new term sheet, new information,
and ask Lucio to modify it for them.
So you have a new first draftready and then you review it.
A lot of lawyers are actually usingLucio to be a bit more creative and
enjoy the practice of law a bit more.
To take my own example, how Ifell in love with Lucio was.

(05:43):
As a competition lawyer I would havejudgments, orders from different
jurisdictions us, eu, uk, India.
I would put all of them into onefolder on Lucio, let's say 1000
orders from different jurisdictions.
I would ask it random questions onjurisprudence, on history of competition
law, history of antitrust criminalization.
All sorts of interesting questions, whichI always wanted to know the answers to,

(06:06):
but I never really dared to read so muchand find so many things interesting stuff.
I ended up doing that on Lucio.
So that's another waypeople are using Lucio.
The third way they're using Lucio reallyis to cut down time on research, cut
down time on reviewing documents sodifferent sorts of usage in and around
reviewing documents, synthesizinginformation, analyzing information.

(06:30):
What are the pitfalls of using AI inpractice and how is Lucio addressing them?
The pitfalls are actually quite stuck,I would say, as you would have already
heard the same things from many others.
The biggest pitfall that has, that wehave come across is lawyers, at least
the senior, mid to senior lawyers.

(06:51):
Worrying about their junior colleagues.
When junior colleagues enter theprofession earlier, it was thought
that they would undergo this threeyears of training where you push them
to read a lot, you push them to writea lot, you push them to think a lot.
And the worry today is that.
As they start using AI a lot theirability to think is there, there's
an obstacle in their ability to thinksimilarly, then there's an obstacle

(07:15):
when it comes to writing good drafts.
There's an obstacle when it comes towriting polished grammatically correct
drafts because they're increasinglyrelying on AI to do some of these things.
So there's a fear that this younggeneration may not get the right.
Training set as they progressahead and there may be a skill gap.
That's one set of worries.
The other set of worriesis really around data.

(07:37):
That's, I would say huge becauselawyers are worried that they might
lose jobs to AI at some point.
And today there's a feeling thatas they upload client documents,
as they upload their own.
They're essentially training AI to getbetter and better at jobs that lawyers do.
So in a way, you aretraining your own downfall.
So that's the fear that a lotof lawyers are having today.

(07:58):
And the third fear is aroundconfidentiality of information.
Particularly if it's clientinformation, it's information
coming in proprietary informationcoming in from big corporations
who are very possessive about it.
And when you put it on ai sometimes youdon't really know if it's secure, even.
When the AI companies claimit's secure that sense of
con comfort is not yet there.

(08:20):
And at times you would see things likechat, g pt, confidentiality issue, which
has been going on now which is that ifyou share a chat screen with someone,
it automatically becomes a public link.
It's a script on it's scriptable onGoogle, in fact, on Google search.
So that's a real fear thathas been moving around.
For junior lawyers enteringthe field, what mindset and

(08:42):
skillset should they demonstrate?
What we have been increasinglyhearing, and when we meet lawyers,
especially managing partners of lawfirms everyone's talking about two
essential skill sets for junior lawyers.
One is the skillset in and aroundtechnology understanding and knowing
tech well being am able to using techbecause there's a set of young lawyers.

(09:08):
Who aren't really used to using aiin the workflows or there's a set of
young lawyers who aren't really able tounderstand the backend of technology.
And when you don't understand thebackend of technology, something as
simple as the fact that AI is notdeterministic, it's output might differ.
Same query three different times,same platform, different results.

(09:29):
Now that recognition, that understandingof tech, that it might give you wrong
results, that understanding that.
It may give you outputs which arenot really on point, and you have
to think through it again and again.
You have to review it, you have toedit it, you have to map it against
information that you otherwise have.
That understanding is something thatyoung lawyers need to develop and that is

(09:50):
an expectation that we are increasinglyseeing at the managing partner level.
They want to hire peoplewho are great lawyers.
Who are great at writing, who have donetheir academic work, but who are also
great at understanding and learning andpicking up technology and understanding
the backside of technology, understandingthat black box which is there behind
the outputs that you're getting.
That's one.
The second skillset, which a lot oflaw firms have now started looking is.

(10:14):
The ability of junior lawyersto really build relationships,
maintain relationships, andleverage those relationships.
The expectation really is now thatby the time you are in your second
or third year, you should alreadybe able to engage with your clients.
You should be able to speak to them well.
You should be able to.
Understand their pain points, you shouldbe able to understand what their real
needs are and satisfy those through thelegal services that you're providing.

(10:38):
That ability to build thoserelationships on client side.
I think that is increasinglyimportant as we move ahead,
how do you see AI and legal evolving?
AI and legal isn't doinggreat at the moment.
The adoption rates are not crazy.
It's not at the level where onecan justify that you pay me two

(10:59):
$50 a month for access to my legalAI platform or legal AI tool.
Because you have, let's saythere's a law firm of 1000 lawyers
and they take 1000 licenses.
You just have about 10 or 15%of the people making actual
use of it every single day.
And that's not used.
That is multiple hours a day.
It's maybe a few times aday, few minutes a day.

(11:20):
That's just 10 to 15% ofthe overall deployment size.
You have about 50 to 60% peoplewho are going to AI just once or
twice a month, and that's not great.
In terms of ROI from a firm'sperspective, what we have also seen
is that over the last two years.
There was a lot ofutilitarian hype around ai.
It'll reduce work hours by 70%.

(11:41):
It'll make junior lawyers obsolete.
It would cut down deliverabletimes significantly.
That created expectationon the client side.
Clients were like, I'msending this to you now.
You have that great Harveyai or this ai, or that ai.
Where is my deliverable?
Why is it taking so long?
But law firms are not reallyserving that demand at the moment
because AI is not there yet.

(12:02):
AI is not magical as it was expectedto be from a law firm's perspective.
Because of that.
We are seeing a trend wherepeople are being put off by ai.
Lawyers are being put off by ai,they're turning away from it.
But our hope is that we move them awayfrom this entire legal AI platform model
where they have to go to a website, uploadstuff, download stuff, do this, do that.

(12:25):
There's a lot of frictionall around that today.
Unusual law firm personis looking at Google Docs.
They're looking at Word, they'relooking at document management systems.
They have their Outlook, Gmail.
They have their WhatsApp where client istexting they are dealing on a daily basis
with about 10 to 15 different tech tools.
You add more they get more confused.
And it's a lot of cognitive loadjust in terms of learning it,

(12:47):
making use of it on a daily basis.
And what we are seeing is that hasled to an attention deficit where
their focus has gone away from theircore work to just managing all of
these different tools and systems.
So our hope is to get them to move away.
From all these fragmented techproducts and take them towards more
of a workspace where they have afew things at least blended in.

(13:10):
They work on the workspace.
They call upon AI when they need it.
If they don't want to use ai,they still work within the
workspace, get their things done.
And we do feel that is the directionwe are going to move towards where
lawyers will stop using all ofthis 10, 15, 20 different tools.
This is Ari Kaplan speaking withBennett Agrawal, the chief of

(13:33):
staff at Lucio in India basedai, native workspace for lawyers.
Bennett, really nice to talk to you.
Thanks so much.
Same here, Ari.
Great talking to you,and have a great day.
Thank you for listening to theReinventing Professionals Podcast.
Visit reinventing professionals.com orari kaplan advisors.com to learn more.