The MAD Podcast: How AI Gets Built — with Matt Turck

Afleveringen

1. State of AI 2026: The Biggest Stories of the Year (and AGI in 2027?) | Nathan Benaich

   29 oktober 2026 · 1 uur 8 min

   Afspelen
   AGI in 2027. That’s the last line of this year’s State of AI Report, and its author Nathan Benaich (Air Street Capital) is back for the third year to defend it. AI is now building AI: Claude leads 26% of Anthropic’s own model R&D, up from under 1% in February, but Nathan thinks “research taste” is the wall. OpenAI’s evaluation agents ended up inside Hugging Face’s systems, only a Chinese open-weight model could help investigate, and Nathan now argues cyber risk matters more than existential risk. Catching up to OpenAI or Anthropic from scratch costs about a trillion dollars, which is why “you either die trying to get to the frontier, or live long enough to serve inference.” Americans would rather live next to a nuclear plant than a data center, $50B of projects are stalled, and all 27 EU countries together have fewer GPUs than one top US cluster. Sovereign AI is “like a national airline: it’s still someone else’s airplane.” Also: the three-lab race and the fate of the neo labs, models vs. harnesses, why he won’t give a personal AI agent his credit card, the Codex adoption gap and why OpenAI should open a store, enterprises owning their own AI instead of renting it, and the morning a solo GP woke up to twelve expert interviews his agent ran overnight. Lees meer

   (00:00) Cold open and intro

   (00:56) Why Nathan Benaich predicts AGI in 2027

   (02:28) AI building AI: Claude leads 26% of Anthropic’s research

   (04:00) Research taste: can AI pick the right problem?

   (08:34) Doomerism is back, but cyber risk beats extinction

   (10:13) Prediction: a frontier model’s weights get stolen

   (13:05) How OpenAI’s agents ended up attacking Hugging Face

   (15:29) Why only a Chinese open-weight model could investigate

   (17:33) What frontier-lab cyber defense could look like

   (19:08) Self-replicating agents and the risks of AI swarms

   (21:16) Can frontier labs agree to slow down?

   (24:03) Why communities are blocking $50B of data centers

   (24:44) Nuclear plant vs. data center: what Americans prefer…

2. Mistral Is SO Back: Inside Le Chonk, the West’s Biggest Open Model (Pierre Stock, VP Science, Mistral)

   22 oktober 2026 · 1 uur 19 min

   Afspelen
   Mistral AI is back in the frontier AI race. Mistral Large 4, nicknamed “Le Chonk,” is a 1-trillion-parameter open-weight model, trained in two months on 3,800 GPUs in Mistral’s own European datacenters — and independently rated the best open model outside China alongside Meta’s Muse Spark Open. After a year when open-weight AI meant DeepSeek, Kimi, Qwen and GLM, October brought a wave of Western open models, with Mistral at the front of it. Pierre Stock, Mistral’s VP of Science and first employee, walks through how the model was built: mixture-of-experts explained, what Mistral borrowed from DeepSeek, reinforcement learning at scale on coding, cyber, finance and chip design, why the RL run is still going, the model that pinged its own trainer from inside the sandbox, open weights vs cyber refusals, why 10-trillion-parameter models are coming, and Mistral Large 5, already in pretraining. Lees meer

   (00:00) Cold open

   (00:59) The Western open-weight comeback: Beam, Le Chonk, Muse, Nemotron

   (04:09) The US shutdown of Anthropic’s models — “AI is like electricity”

   (06:07) Why “Le Chonk”

   (07:56) “Mistral fell behind”: the comeback from the inside

   (10:29) Training on 3,800 GPUs in Mistral’s own datacenters

   (13:24) Meta keeps its best model closed, Mistral opens Chonk

   (15:40) Chonk vs Beam, Muse Spark Open and the Chinese models

   (18:44) Why Chinese labs are still at the frontier

   (21:08) Pierre’s story: FAIR, Yann LeCun, the “Mistral mafia”

   (25:14) From compressing networks to a trillion parameters

   (27:30) Mixture of experts explained: the hospital analogy

   (32:25) 10-trillion-parameter models are coming

   (35:29) What Mistral borrowed from DeepSeek

   (36:46) One model, several reasoning modes

   (39:15) Native multimodality: satellites, manuals, computer use

   (41:56) Data, synthetic data and model collapse

   (46:32) Reinforcement learning as an exam

   (49:59) Where the efficiency comes from

   (52:18) Le Chonk is still training right now

   (53:39) What RL trains on: coding, cyber, …

3. Jev: The AI That Makes Decisions, Not Text - and Why It Changes Everything (Diogo Almeida)

   15 oktober 2026 · 1 uur 8 min

   Afspelen
   What is Jev? TypeSafe’s Jev is a new kind of AI model, a “decision model,” that doesn’t write text. It makes fast, cheap, calibrated decisions inside software, and it has taken the AI world by storm: within weeks of launch it became the most copied model in AI, with OpenAI, Cloudflare, Amazon and Databricks all shipping their own versions, and TypeSafe just raised an $870 million Series A at a $7.5 billion valuation. In this episode, Jev’s creator Diogo Almeida, co-inventor of RLHF at OpenAI and now CEO of TypeSafe, explains Jev in plain English: what a “System One” or decision model actually is, why ChatGPT-style language models can’t be trusted inside software (“95% Einstein, 5% Mr. Bean”), how RLHF and RLVR created the problem, what RLCD (reinforcement learning for calibrated decisions) does differently, where the training data comes from, which models Jev is built on, why “just a classifier” is a compliment, and why he calls pre-training “a giant dumpster fire of wasting resources.” Lees meer

   We also cover the copycats and why he thinks they missed the hard part, why benchmarks are useless, whether Jev will go open source, the Jevons paradox business model behind the name, why he doesn’t believe AI is on the path to recursive self-improvement, and why he thinks the AI winter he feared has been averted.

   (00:00) Cold open & Intro: “95% Einstein, 5% Mr. Bean”

   (01:09) Jev’s breakout launch: why intelligence per dollar matters

   (03:25) Why TypeSafe built in stealth for two years

   (05:50) “We could have had AGI last year”

   (06:48) Why the product carries the marketing

   (09:33) What is Jev? Machine-native AI, built for software

   (10:53) System One vs. System Two: the limits of reasoning

   (12:26) From simple decisions to composable AI

   (15:21) Automating email: giant prompts vs. smart if-statements

   (20:03) Probabilities, thresholds, and developer control

   (23:43) Why Jev is faster and cheaper than chat models

   (25:39) “Classification is AI-complete”

   (27:16) RLHF: how OpenAI…

4. What Happens When Billions of AI Agents Hit Your Database? (Andy Pavlo)

   8 oktober 2026 · 1 uur 17 min

   Afspelen
   AI agents are becoming the biggest users of databases: creating them, querying them, and sometimes deleting them. Andy Pavlo, Carnegie Mellon's database professor and now VP of Database Research at ClickHouse, explains what changes when billions of agents hit the data layer, and why most of the AI world's mental model of databases stopped at vector databases and RAG. Lees meer

   Andy's CMU courses taught a generation of engineers (and, it turns out, the AI models); he writes the year-in-review on databases the whole industry reads; and this summer he joined ClickHouse to found ClickHouse Labs. We cover why vector databases were "just an index," what it means for an agent to create a database, why agents could mean 10 to 100x more queries, whether files or databases should hold an agent's memory, how text-to-SQL went from 60% to 99.5% accuracy, why 60% of open-source databases now have commits from coding agents, and what ten years of self-driving database research taught him. Plus ClickHouse, Postgres, the vector/graph/GPU database verdicts, Larry Ellison, and the Wu-Tang Clan.

   Disclosure: FirstMark, where Matt is a General Partner, is an investor in ClickHouse.

   (00:00) Cold open & Intro

   (01:21) From vector databases and RAG to the age of agents

   (04:09) Neon's stat: agents create 80% of databases?

   (07:25) Why agents keep deleting production databases

   (08:19) Guardrails: the toddler-and-stairs rule

   (10:51) The four eras of database volume: 10–100x more queries

   (15:10) Agent memory: files vs. databases ("everything is a database")

   (18:24) Which database do AI models recommend? The new SEO

   (19:25) "It cites me back to myself"

   (23:14) MCP for databases, and text-to-SQL from 60% to 99.5%

   (27:09) Trust an agent the way you'd trust a junior developer

   (27:47) Can AI build an entire database? Opus 4 and the CMU projects

   (29:57) 60% of open-source databases now have AI commits

   (30:43) Ten years of self-driving databases, Peloton to today

   (33:52) What LLMs changed: 85% o…

5. AI Models Are Now Hiding Their Cheating | Eric Ho (Goodfire)

   1 oktober 2026 · 1 uur 10 min

   Afspelen
   This summer, AI agents broke out of their test sandboxes at every major lab and hacked real companies, and the safety tests didn't see it coming. Eric Ho is the co-founder and CEO of Goodfire, the Anthropic-backed mechanistic interpretability lab whose new paper, "Models Know When They're Reward Hacking," found that top open-source AI models cheat on agent tasks up to 96% of the time, that a clear "cheating" signal exists inside the model, and that cheap activation probes can catch it, including hacks that chain-of-thought monitors miss. In this conversation, Eric explains reward hacking in plain terms (AI agents as "amoral students with a mostly absent teacher"), why reinforcement learning turned a known problem into a crisis, why chain-of-thought monitoring is fading as models start to think in "neuralese," and why nobody, including the people who build them, understands how these models work. Lees meer

   We then go inside the science: what interpretability actually is, what probes and steering do, how Goodfire found the cheating signal and turned it up and down, and what can be done about it, from real-time activation monitoring to "intentional design," where you give gradient descent a choice about what the model learns. Along the way: a model caught planning its hack to evade its own monitor, a new Alzheimer's biomarker discovered inside a diagnostic model, why only a few hundred people in the world work on this, and Eric's bet that we'll decode neural networks by 2028.

   (00:00) Cold open & intro

   (01:06) Amoral students with an absent teacher

   (02:49) What is reward hacking?

   (04:13) Models cheat up to 96% of the time

   (06:08) Do models know they're cheating?

   (08:10) The Hugging Face hack: why tests missed it

   (11:35) "The dumbest models we'll ever deal with"

   (11:58) Should AI slow down?

   (13:56) Mechanistic interpretability 101

   (16:06) Models are grown, not built

   (19:10) How labs do alignment today

   (21:30) Is this an alignment crisis?

   (23:26) Why agents changed e…

6. Who Feeds the GPUs? Inside AI's Hidden $30B Layer | Renen Hallak, VAST Data

   24 september 2026 · 1 uur 10 min

   Afspelen
   Everyone talks about GPUs. Almost nobody talks about the layer that feeds them. Renen Hallak is the founder & CEO of VAST Data — the $30 billion company powering xAI and some of the world's biggest AI clouds — and he sits in the hidden layer of the AI stack. Lees meer

   In this episode, we cover what an AI factory actually is, why every company will eventually own its own AI, the architecture bet behind VAST (DASE, explained simply), KV caches and agent memory, and DataEnclave — VAST's brand-new confidential AI announcement with NVIDIA that lets leading models run on the world's most sensitive data.

   Plus: the demand signal that scares even him (a customer went from 500 petabytes to 2 exabytes), circular financing, which neoclouds survive, sovereign AI, working with NVIDIA and Elon Musk's xAI — and why the next 10 years will bring more change than the last 1,000.

   (00:00) Intro

   (00:51) The hidden software layer in NVIDIA's AI stack

   (02:28) What actually makes an "AI factory"?

   (05:13) Should Walmart and Goldman Sachs build their own AI?

   (06:20) "We infer during the day, fine-tune at night"

   (13:16) The announcement: models become a resource to manage

   (15:32) From P vs. NP to founding VAST Data

   (17:32) OpenAI, Navier–Stokes and 10,000 collaborating agents

   (20:25) The pre-transformer insight behind VAST

   (21:55) DASE: VAST's "shared everything" architecture explained

   (25:18) "Storage was where startups go to die"

   (27:39) Trillions of vectors: why old databases break

   (29:01) Are S3, Snowflake and Databricks ready for AI?

   (31:29) Data gravity, vendor lock-in and zero churn

   (33:13) Training vs. inference: why the infrastructure changes

   (34:46) Model routing, KV caches, RAG and agent memory

   (36:59) Identity, permissions and security for AI agents

   (40:27) Can multi-agent systems unlock scientific discovery?

   (41:55) DataEnclave: how confidential AI protects data and weights

   (45:16) Who should be AI's trust layer?

   (46:37) "Sometimes it scares me": 500 petabytes to 2 e…

7. When AI Improves Itself | Richard Socher (Recursive)

   10 september 2026 · 1 uur 14 min

   Afspelen
   What happens when AI begins improving itself, and then turns that intelligence toward science? Richard Socher, pioneering AI researcher and CEO and co-founder of Recursive, joins Matt Turck to explore the vision behind his new book, The Eureka Machine. They discuss why scientific progress may be slowing, how large language models can learn the hidden languages of proteins and biology, and why simulations, verifiers and autonomous experiments could unlock superhuman AI capabilities. The conversation covers recursive self-improvement, AI drug discovery and cancer research, hallucination as creativity, virtual cells, self-driving laboratories, agent swarms, the AI Economist, Recursive’s plans, and the compute and data needed to build an AI scientist that never stops learning—and may eventually discover what humans cannot. Lees meer

   (00:00) Intro: AI That Improves Itself

   (00:55) Why Scientific Progress Is Slowing

   (03:08) The Labyrinth of Human Knowledge

   (05:59) Can AI Put Science Back Together?

   (07:57) How LLMs Learn Biology and Proteins

   (10:56) Next-Token Prediction as a World Model

   (16:44) Can AI Generate Truly Original Ideas?

   (17:32) Simulations, Verifiers and Superhuman AI

   (22:18) The Path to Recursive Self-Improvement

   (24:49) Why AI Hallucinations Can Drive Discovery

   (27:42) From Reading Biology to Writing It

   (31:31) Can AI Accelerate Drug Discovery?

   (33:31) Will AI Help Cure Cancer?

   (38:03) AI Breakthroughs in Biology, Energy and Materials

   (40:19) Will Some Societies Reject AI?

   (45:07) Building the AI Economist

   (52:22) The Scientific Data Bottleneck

   (53:41) The Four Pillars of the Eureka Machine

   (55:01) Teaching AI the Rules of Reality

   (57:44) Simulations and Virtual Cells

   (1:00:40) Self-Driving Robotic Laboratories

   (1:02:51) Agent Swarms and Open-Ended Discovery

   (1:04:30) The Compute Bottleneck

   (1:05:44) Inside Recursive

   (1:07:33) What Recursive Will Build First

   (1:10:10) How Do We Define Intelligence?

   (1:11:32) How Far Can Intelligence Go?

8. AI Could Take Over in 2029. Is It Already Too Late? | Ryan Greenblatt

   27 augustus 2026 · 1 uur 19 min

   Afspelen
   Could AI take over as soon as 2029? Ryan Greenblatt, Chief Scientist at Redwood Research and the researcher who first caught an AI faking its own alignment, says the scenario he actually expects ends with AI systems "competently scheming" against their creators. In this episode, he explains why he recommends planning for fully automated AI research by 2029, why today's models are already more misaligned than the one that made him famous, and what happens in the year-by-year path from AI coding assistants to superintelligence. Then we walk through the alternative he helped design: AI 2040 Plan A, the most detailed blueprint anyone has written for how the US and China could avoid a reckless race to superintelligence, built on radical research transparency, chip tracking, and a deterrence regime he calls mutually assured compute destruction. Lees meer

   We also cover the recent letter signed by 1,200 AI insiders, including Anthropic CEO Dario Amodei, asking the government for the tools to slow AI down; OpenAI pausing its Astra model after it hit the first-ever critical cybersecurity threshold; the 30-day government review that frontier AI models now go through before release; Mark Zuckerberg's open superintelligence manifesto and why Ryan thinks it ignores the real problems; what Plan A would do to NVIDIA, OpenAI, and Anthropic valuations; the state of AI control and alignment research; and whether it is already too late to change course. Stay for the last ten minutes, where Ryan lays out, step by step, how he believes the transition to superintelligence actually unfolds.

   AI 2040 - https://ai-2040.com/

   Alignment faking paper: https://blog.redwoodresearch.org/p/alignment-faking-in-large-language

   Ryan Greenblatt

   LinkedIn - https://www.linkedin.com/in/ryan-greenblatt-4b9907134

   Blog - https://substack.com/@ryangreenblatt

   Redwood Research

   Website - https://www.redwoodresearch.org

   X/Twitter - https://x.com/redwood_ai

   Matt Turck (General Partner)

   Blog - https://mattturck.com
   …

9. “OpenAI’s Model Hacked Us” - Hugging Face’s Thomas Wolf

   7 augustus 2026 · 58 min

   Afspelen
   An OpenAI-powered agent penetrated Hugging Face during cyber testing - even though it was never tasked with attacking Hugging Face. It did it as a side quest. Lees meer

   Thomas Wolf, co-founder and Chief Science Officer of Hugging Face, joins Matt Turck to unpack what actually happened, why closed AI models refused to help during the live incident, how an open-source model helped the team fight back, and why the old equation of “closed equals safe, open equals dangerous” no longer holds.

   They also discuss model deception and social engineering, the limits of sandboxes and guardrails, the state of open-source AI in 2026, AI sovereignty, the economics of open models, recursive self-improvement, and whether the frontier should deliberately slow down.

   (00:00) An AI Agent Hacked Hugging Face

   (00:30) Introduction

   (01:00) 17,000 Attacker Events—and a Strange Target

   (04:28) The Attack Was a “Side Quest”

   (06:13) AI Training Runs Left Notes for Each Other

   (07:09) Closed AI Refused to Help

   (09:47) Fighting Back With an Open-Source Model

   (13:15) Open vs. Closed Is the Wrong Safety Debate

   (15:46) AI Agents Start Social-Engineering Humans

   (22:24) The Three Walls: Sandboxes, Guardrails, Alignment

   (24:34) “Neuralese”: Can Humans Still Read AI Reasoning?

   (25:28) Why Monitoring AI Agents Gets So Hard

   (28:10) Reward Hacking and the “Paperclip Problem”

   (32:02) The State of Open-Source AI in 2026

   (33:47) Router Models and the Enterprise Shift to Open

   (37:01) The Real Economics of Open Models

   (39:41) Can Chinese AI Models Be Trusted?

   (41:37) AI Sovereignty: Who Controls the Switch?

   (43:16) Why Western Open-Source AI Matters

   (48:16) Is AI Heading Toward an Oligopoly?

   (49:41) The Race Toward Recursive Self-Improvement

   (51:54) Why Thomas Signed the AI Slowdown Letter

   (55:14) AI Slowdown—or Regulatory Capture?

10. How to Build Long-Horizon AI Agents — Mitch Troyanovsky, Basis

    6 augustus 2026 · 1 uur 23 min

    Afspelen
    AI agents can write code for hours, but ask them to do real work in the real economy, and they break. Mitch Troyanovsky is co-founder of Basis, a unicorn AI company whose agents run autonomously for hours — sometimes days — completing complex tax returns end to end. His answer to the reliability problem: stop grading outcomes, and start supervising the process. Lees meer

    This is a definitive, reference-style conversation on building long-horizon AI agents. Mitch walks through the full history — from ReAct and the AutoGPT crash to reasoning models and RLVR — and explains why the industry abandoned process supervision in 2023, and why it's now coming back at a completely different scale. We go deep on behavior specs, the open standard Basis just released with Braintrust for defining and evaluating how agents behave across entire trajectories, with no ground truth required.

    Along the way: why context is really runtime training data, why your documentation must be treated like a codebase, ontologies as "worlds for agents to live in," the judge-as-agent architecture, why Basis hires philosophy majors as Language Architects, deploying agents as "onboarding 300 brilliant alien employees," and Mitch's prediction for when the bitter lesson swallows the harness.

    (01:09) Why Basis Engineers Whisper to Their Agents
    (04:12) Accounting as Compression: an Intelligence Layer Over the Economy
    (06:11) Defining Long-Horizon: When You Exceed the Context Window
    (08:24) Anatomy of a Multi-Day Autonomous Trajectory
    (10:19) Handoff Design: Optimizing Output for the Reviewer
    (11:17) ReAct and Why Reasoning Must Regulate Its Own State
    (12:33) Large Working Memory, No Long-Term Memory
    (14:13) Compounding Errors: Why AutoGPT and BabyAGI Broke
    (15:51) Opus 3, o1, o3: the Three Real Paradigm Shifts
    (17:07) Titrating Inference Compute Across Easy and Hard Steps
    (18:23) Process Reward vs. Outcome Reward: "Let's Verify Step by Step"
    (20:32) RLVR and Why the METR Curve Overstates Reliability
    (22:09) Verif…

Alle 136 afleveringen →