Absence of a Bug Doesn't Equal Safety: Phil Koopman & Steve Wilson on Health Tech Safety & Security

For three decades, both Phil Koopman and Steve Wilson have been working in AI. They joined us on Making AI Work for You for a conversation about health tech safety and security. 

 

With exceptional backgrounds and unparalleled experience, they share their perspectives on:

• Why cybersecurity is an “arms race”
• How safety standards differ from machine learning testing
• The considerations about the level of agency you give an AI system
• Why there needs to be a human “captain of the ship”
• Struggles with the software supply chain
• Why it’s critical to have rigorous engineering processes

 

Phil is an associate professor at Carnegie Mellon University, is the originator of UL 4600, and has been working on self-driving cars for 30 years. 

 

Steve is CPO of Exabeam, is an author, a member of the TTIC, and is an expert in machine learning, LLMs, and cybersecurity. He’s been working in AI since 1992. 

 

Topical time codes:

1:47 - Phil - The importance of humans in autonomous technology 

5:10 - Phil - You need a captain of the ship for safety

7:04 - Steve - Agentic systems 

11:09 - Steve - Software supply chains

15:39 - Phil - The challenge of making sure software is safe 

17:54 - Phil - Testing doesn’t prove safety

20:27 - Phil - The purpose & value of UL 4600

23:06 - Phil - You can’t wait until something is on fire to put out the fire

24:16 - Steve - Culture of the Product environment 

25:27 - Steve - Need for focus around safety and security for good decisions

26:07 - Phil - In embedded AI, there are real people on the other side 

27:36 - Steve - Cybersecurity is an arms race

29:38 - Phil - People short-cut safety when they’re in a rush 

 

Connect with Phil Koopman: https://www.linkedin.com/in/philip-koopman-0631a4116/ 

Learn more about UL 4600: https://ulse.org/ul-standards-engagement/autonomous-vehicle-technology

Connect with Steve Wilson: https://www.linkedin.com/in/wilsonsd/ 

Learn more about Exabeam: https://www.linkedin.com/company/exabeam/ 

Connect with Sherri Douville: https://www.linkedin.com/in/sdouville/  

Learn more about TTIC: https://medium.com/@news_52674/about