Ep 264 Why Memory is the Unsung Hero of AI, with Thomas Coughlin

Guest: Dr. Tom Coughlin, President, Coughlin Associates, IEEE Past President (2025)

Website: https://tomcoughlin.com

FMS Conference: https://futurememorystorage.com/

Episode Summary: Join us for an enlightening conversation with Dr. Tom Coughlin, a seasoned digital storage analyst and consultant with over 40 years in the industry. Tom, the President of Coughlin Associates and former IEEE President, shares unparalleled insights into the foundational technologies shaping our digital world. We delve into the crucial role of memory in AI's development, the surprising realities of storage demand, and the fascinating world of breakthrough memory technologies. Discover why memory often gets overlooked in AI discussions, critical considerations for data privacy, and the global impact of the IEEE. Tom also previews the upcoming Future of Memory and Storage (FMS) conference and offers invaluable career advice for tech entrepreneurs.

Key Discussion Points:

• Behind-the-Scenes of Storage Innovation: Tom shares a surprising story about the 25-year research journey behind HAMR technology now rolling out in HDDs.
• Evolving Storage Demands:
  • Learn how SSDs have become primary data center storage and replaced HDDs in personal computers and consumer applications.
  • Understand HDDs' shift to colder storage in data centers—this is their growth market, and much of the world’s data lives on HDDs.
  • Discover magnetic tape's vital role in archiving and backing up cloud data.
  • Explore new archive storage technologies being developed, such as optical recording and DNA storage.
• Memory's Critical Role in AI:
  • Memory, particularly DRAM, is playing a big role in training AI models.
  • Approaches are emerging that reduce the need for expensive DRAM (especially in HBM) for inference applications, using storage technologies like SSDs (e.g., Kioxia's AiSAQ for tuning LLMs).
• er optical storage or DNA for long-term data storage and preservation.
• Why Memory is Overlooked in AI: Insights into why people tend to focus more on processing (GPUs) than on the data itself, despite memory and storage advances being as impressive as those in GPUs.
• Data Privacy & Security in Storage: Essential considerations include having copies of data on immutable storage for ransomware recovery, using AI for anomaly detection on networked systems to prevent malware, and proper encryption use in storage systems for data security.
• The Global Impact of IEEE: Learn about IEEE as the world’s largest technical professional organization with nearly half a million members in over 190 countries. IEEE puts on over 2,000 conferences and events each year and publishes a good percentage of the world’s technical literature. IEEE standards enable interoperability and industries, with a recent focus on sustainability and ethical AI practices to solve global problems and benefit humanity.
• Future of Memory and Storage (FMS) Conference: Dr. Coughlin, the general chair, provides details on the 2025 FMS (August 4-7, 2025, at the Santa Clara Convention Center). The conference will feature keynotes by major players in the digital storage and memory industry and sessions covering all major technologies and applications. FMS is the largest independent event focused on digital storage and memory.
• Highlight Speakers at FMS: Keynote talks include representatives from Kioxia, Fadu, Micron, Silicon Motion, SK hynix, Samsung, Neo, Sandisk, Max Linear, VergeIO, and Kove. There will also be a special session on AI, memory, and storage organized by NVIDIA, and Dr. Coughlin will give a talk on his experiences as IEEE President in 2024. Many parallel sessions will feature speakers from important industry players.
• Major Disruption in Digital Storage: Dr. Coughlin predicts that just managing the massive amounts of data generated by AI and IoT will be a huge challenge. He also foresees a growing need for technology to ensure data provenance, to identify false information and curate data for AI training.
• Career Advice for Tech Professionals: Dr. Coughlin advises aspiring tech professionals to be part of their industry and join technical professional organizations like the IEEE. This provides opportunities to develop professional networks and learn important skills like working with others and communicating