51 - Bridging the Gap Between AI Promises and Legal Practice Realities with Jack Shepherd

In this episode, Ted sits down with Jack Shepherd to dissect the practical realities of implementing AI and knowledge management tools in the legal industry. Together, they challenge conventional assumptions, explore the pitfalls of over-hyped technologies, and highlight the understated value of focusing on basic, everyday challenges. With candid insights into AI adoption, enterprise search challenges, and the importance of clean data, this conversation is a must-listen for anyone seeking actionable strategies to tackle inefficiencies in legal workflows and knowledge management.

In this episode, Jack shares insights on how to:

• Evaluate AI tools effectively in the legal industry
• Address data hygiene challenges in law firm document management systems
• Integrate generative AI into law firm workflows without over-promising results
• Balance risk and reward when adopting enterprise search solutions in legal contexts
• Encourage lawyer engagement with knowledge-sharing initiatives

Key takeaways: 

• Generative AI tools like LLMs are limited by hallucinations and a lack of comprehension, often failing in critical legal tasks like accurate citations or respecting legal hierarchies, underscoring the need for cautious adoption in high-risk contexts.
• Many law firms invest heavily in enterprise search projects only to see them fail due to unclear ROI and poor integration, highlighting the importance of focusing on smaller, targeted use cases with tangible benefits like better document organization or locating key client information.
• Effective use of AI in law requires clean, structured data, as experiments show poor data hygiene leads to irrelevant or misleading outputs, undermining both efficiency and trust.
• The legal industry's focus on flashy, complex AI use cases often overlooks basic workflow inefficiencies—like time tracking, document versioning, or knowledge-sharing—which would yield quicker, more impactful results for lawyers.

About the guest, Jack Shepherd:

Jack Shepherd is a former lawyer and legal technology expert, with a particular interest in cutting through hype and confusion in technology to deliver value to practising lawyers and clients. He works at iManage, where he leads consulting initiatives on knowledge management projects.

“[The] accuracy thing is such a big deal around LLMs because the lawyer using a tool—they'll ask it a question, and I can guarantee you the first question they'll ask will be to test it. Then, if it gets even slightly wrong: 'No, I don't trust this. Never using it again.' You've got to be really careful about that.” – Jack Shepherd

Connect with Jack Shepherd:

Website: https://imanage.com/

LinkedIn: https://www.linkedin.com/in/jackwshepherd/

Connect with Ted Theodoropoulos:

.css-j9qmi7{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;font-weight:700;margin-bottom:1rem;margin-top:2.8rem;width:100%;-webkit-box-pack:start;-ms-flex-pack:start;-webkit-justify-content:start;justify-content:start;padding-left:5rem;}@media only screen and (max-width: 599px){.css-j9qmi7{padding-left:0;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;}}.css-j9qmi7 svg{fill:#27292D;}.css-j9qmi7 .eagfbvw0{-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;color:#27292D;}