The idea of model cards, which was introduced as a measure to increase transparency and understanding of LLMs, has been perverted into the marketing gimmick characterized by OpenAI's o1 system card. To demonstrate the adversarial stance we believe is necessary to draw meaning from these press-releases-in-disguise, we conduct a close read of the system card. Be warned, there's a lot of muck in this one.
Note: All figures/tables discussed in the podcast can be found on the podcast website at https://kairos.fm/muckraikers/e009/
- (00:00) - Recorded 2024.12.08
- (00:54) - Actual intro
- (03:00) - System cards vs. academic papers
- (05:36) - Starting off sus
- (08:28) - o1.continued
- (12:23) - Rant #1: figure 1
- (18:27) - A diamond in the rough
- (19:41) - Hiding copyright violations
- (21:29) - Rant #2: Jacob on "hallucinations"
- (25:55) - More ranting and "hallucination" rate comparison
- (31:54) - Fairness, bias, and bad science comms
- (35:41) - System, dev, and user prompt jailbreaking
- (39:28) - Chain-of-thought and Rao-Blackwellization
- (44:43) - "Red-teaming"
- (49:00) - Apollo's bit
- (51:28) - METR's bit
- (59:51) - Pass@???
- (01:04:45) - SWE Verified
- (01:05:44) - Appendix bias metrics
- (01:10:17) - The muck and the meaning
Links
- o1 system card
- OpenAI press release collection - 12 Days of OpenAI
Additional o1 Coverage
- NIST + AISI [report] - US AISI and UK AISI Joint Pre-Deployment Test
- Apollo Research's paper - Frontier Models are Capable of In-context Scheming
- VentureBeat article - OpenAI launches full o1 model with image uploads and analysis, debuts ChatGPT Pro
- The Atlantic article - The GPT Era Is Already Ending
On Data Labelers
- 60 Minutes article + video - Labelers training AI say they're overworked, underpaid and exploited by big American tech companies
- Reflections article - The hidden health dangers of data labeling in AI development
- Privacy International article = Humans in the AI loop: the data labelers behind some of the most powerful LLMs' training datasets
Chain-of-Thought Papers Cited
- Paper - Measuring Faithfulness in Chain-of-Thought Reasoning
- Paper - Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
- Paper - On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
- Paper - Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models
Other Mentioned/Relevant Sources
- Andy Jones blogpost - Rao-Blackwellization
- Paper - Training on the Test Task Confounds Evaluation and Emergence
- Paper - Best-of-N Jailbreaking
- Research landing page - SWE Bench
- Code Competition - Konwinski Prize
- Lakera game = Gandalf
- Kate Crawford's Atlas of AI
- BlueDot Impact's course - Intro to Transformative AI
Unrelated Developments
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Are You A Charlotte?
In 1997, actress Kristin Davis’ life was forever changed when she took on the role of Charlotte York in Sex and the City. As we watched Carrie, Samantha, Miranda and Charlotte navigate relationships in NYC, the show helped push once unacceptable conversation topics out of the shadows and altered the narrative around women and sex. We all saw ourselves in them as they searched for fulfillment in life, sex and friendships. Now, Kristin Davis wants to connect with you, the fans, and share untold stories and all the behind the scenes. Together, with Kristin and special guests, what will begin with Sex and the City will evolve into talks about themes that are still so relevant today. "Are you a Charlotte?" is much more than just rewatching this beloved show, it brings the past and the present together as we talk with heart, humor and of course some optimism.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.