Chain of Thought | AI Agents, Infrastructure & Engineering
Alle afleveringen
82 afleveringen, van nieuw naar oud
-
Agents Are Overtaking Humans in Search: Why Search Must Be Rebuilt
1 november 2026 · 57 min
Saahil Jain joined You.com as a founding engineer in 2020, when it was a consumer search engine competing with Google. Now its CTO, he builds search for AI agents, and he argues that however smart models get, they will still need tools to find out what is happening in the world. Lees meer
His analogy: we built the self-driving cars before the roads. He walks through which work he expects to move inside the model, why better models made harnesses matter less, and why cheaper open models and routing are giving harnesses a new job. He says he changed his mind on that last point in the past few months.
Then we get into how agents search differently than people do: thousands of queries at once, recall over ranking, and a much wider range of latency needs. Saahil traces a query from fan-out to the index to re-ranking, and explains why You.com judges search by whether the agent answered the question, and why he trusts customer evals and benchmarks that change over time more than static public ones.
We cover:
- Core intelligence versus augmented intelligence, and the self-driving cars that arrived before the roads
- Which tasks Saahil expects models to absorb, and where unverifiable rewards slow them down
- Why frontier labs keep going vertical while open-model labs give weights away
- How stronger models shrank the gap between good and bad harnesses, and how routing across open models can revive them
- How You.com post-trains models with its search tool while trying not to overfit to any one model
- How agent search differs from human search: query volume, context, ranking and latency
- The life of a query: subqueries, lexical and semantic indexes, re-ranking and enrichment
- Why static benchmarks can be memorized, and what dynamic benchmarks and customer evals measure insteadMentioned in this episode:
- I Paid an AI Agent $8 to Write About its ‘Life’: https://newsletter.chainofthought.show/p/i-paid-an-ai-agent-8-to-write-about
- MotionBench: https://motionbench.shConnect w…
-
How Cisco Secures Personal AI Assistants for 90,000 Employees | DJ Sampath
29 oktober 2026 · 44 min
Cisco is giving each of its 90,000 employees a personal AI agent. Some of those same employees are already connecting agents from outside the security boundary without asking IT. DJ Sampath, SVP and GM of AI Software and Platform at Cisco, walks through what had to be in place before the rollout: scanning every prompt, sandboxing each agent, keeping agents multi-tenant, scanning the skills and tools people bring in, and settling which identities an agent may act as. Lees meer
New episodes, the ideas behind them and Conor's essays land in the Chain of Thought newsletter first. Subscribe: https://newsletter.chainofthought.show/
We recorded at Cisco's San Francisco office, a block or two from OpenAI, where Cisco was an early Codex design partner. DJ explains how engineers rework their skills files with each model generation, how per-engineer PR bots review agent-written code, and why defenders now have to move at agent speed when an exploit can show up within minutes of a CVE going public.
We cover:
What Cisco's IT team required before rolling out MyAgent to 90,000+ employees
Prompt injection by email, and why the first integration everyone makes is the riskiest
Cisco's AI security taxonomy, DefenseClaw, and Project Quarry, a new open source language for describing agent behavior and policy
How the agents are actually used: daily briefs, meeting actions, live dashboards, 2 trillion tokens in a single week
Why skills files need refactoring every model generation
Discovering and governing shadow agents with zero trust access
Multi-model code review, spec drift, and evals built from pull request traces
How Cisco Cloud Control was built in about eight months with agentic coding
Chapters:
(0:00) Guardrails tested inside Fortune 100 companies
(0:21) Securing 90,000 employee AI agents
(8:04) Cisco's open source agent security stack
(12:01) Project Quarry: a policy language for agents
(17:07) Personal agent use cases at Cisco
(19:08) Early design partner on Codex
(23:15) Re… -
For AI Agent Fleets, the Model Is Rarely the Bottleneck | Theory Ventures' Andy Triedman
25 oktober 2026 · 49 min
Andy Triedman is a partner at Theory Ventures, and he is building his own software factory, while assessing how startups are leveraging theirs. His read after running agent fleets: the models are frankly good enough for most of the work. The vast majority of things you'd ask an agent to do are not intelligence limited. They are context limited. We have infrastructure work to do - but we can be more ambitious! Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/79-for-ai-agent-fleets-the-model-is-rarely-the-bottleneck-theory-ventures-andy-triedman/
So the hard part moves upstream. Deciding what the agents should build, what they need to know, and when they should stop and ask a person is now most of the job.
We cover:
Moving from solving one problem at a time to building systems that solve whole classes of problems, like generating and testing integrations
Which tasks still need human approval, and which can be fully handed off
Why a short feature spec leaves too much unsaid, and how better questions up front cut review interruptions
Keeping shared context and company decisions current, because an agent takes everything it reads as true
Inference ROI, and when well-scoped tasks can go to cheaper models
Sandboxes and simulated customer environments for agents
First steps for teams: shared skills, cloud agents and approval gates
Protecting customer data, and organizing agent memory from session traces to company priorities
New episodes, the ideas behind them and Conor's essays land in the Chain of Thought newsletter first. Subscribe: https://newsletter.chainofthought.show/
Chapters:
(0:00) Agents are context limited, not intelligence limited
(0:34) From coding agents to software factories
(5:53) What still needs human approval
(10:16) Agent loops that improve the product on their own
(14:52) Where agent automation breaks down
(17:59) The bitter lesson and engineering as experimentation
(20:08) From writing code to directing agent fleets
(24:… -
What Breaks When Agents Become the Customer | Svix CEO Tom Hacohen
15 oktober 2026 · 53 min
What breaks when agents become the customer? Most of the integration layer, APIs, SDKs and webhooks alike, was built for human developers and for servers that are always listening. Agents are neither. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/78-what-breaks-when-agents-become-the-customer-svix-ceo-tom-hacohen/
When an agent runs on a laptop, closing the lid takes down the tunnel its webhooks come through. When the lid opens, it gets a storm of retries from 12 hours earlier. Svix founder and CEO Tom Hacohen's answer is to let the agent poll for what it missed.
Svix delivers webhooks for companies like Clerk and Resend, and it created Standard Webhooks, the spec OpenAI, Anthropic and Google Gemini have adopted. In this episode of Chain of Thought, Tom joins Conor Bronsdon to talk through designing APIs for agents that adapt, whether agents still need SDKs, and why he wants less code even when agents write it. He also introduces Poke, Svix's new spec for making event-driven APIs work for agents.
This episode is sponsored by Svix.
We cover:
Why events are harder for agents than API calls: the sender controls them, and they arrive outside the agent's session
How polling endpoints and Svix's plugins for Hermes and OpenClaw let an agent pick up where it left off
Why Tom calls JSON an abomination and webhooks duct tape, and why worse is better anyway
Why Svix created Standard Webhooks when everyone implemented webhooks differently, and how the spec let Kong and ngrok add webhook verification
Whether agents still need SDKs, and why Svix's skills work like a solutions engineer while its docs stay written for humans
Why Tom wants less code even when agents write it, and what piling patches on patches costs
Poke, Svix's early spec for letting agents subscribe to and trigger events across services, released so feedback can shape it while it's still malleable
Chapters:
(0:00) Duct tape, retry storms and the case for simplicity
(1:19) Why inte… -
Switching Models Shouldn't Mean Starting Over | Walrus Protocol's Kimberly Logan
7 oktober 2026 · 38 min
If your agent's memory lives inside one model provider, switching models or harnesses means starting over. You need portable memory - yet when switching agent memory between models, you can swing accuracy by thirteen points or more, depending on which model reads them back. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/77-switching-models-shouldnt-mean-starting-over-walrus-protocols-kimberly-logan/
Kimberly Logan, Head of Product at the Walrus Foundation, where she builds Walrus Memory, joins Chain of Thought to discuss how to build portable and verifiable memory for AI agents - and why Walrus's decentralized storage network has proved an apt primitive. She and host Conor Bronsdon get into why model lock-in is a memory problem, what people use agent memory for beyond coding, how you prove what happened to your data, and why Walrus doesn't publish memory quality benchmarks, plus more. This episode is sponsored by Walrus.
We cover:
Why longer context windows don't help once you switch models or harnesses, and why an import function only fixes one point in time
Why Kimberly doesn't benchmark Walrus Memory against memory startups, and why she sees the opportunity as bigger than memory alone
What people build with Walrus Memory besides coding agents, which Walrus puts at maybe 30% of usage: long-running study aids, trading agents, and security teams comparing production snapshots with code changes
Walrus's figure that almost 85% of the memories written are read back more than 30 days later
How attestations on the blockchain and a delegate key you control let you show what happened to your data, and why unchanged data still isn't proof that it's accurate
Why Kimberly treats recall quality and latency as table stakes, and how a knowledge graph keeps recall on the freshest memories
Why she left almost 20 years in traditional software, including seven at Google, for a blockchain company, and where shared memory across companies and trust bounda…
-
Your Best AI Engineer Might Have the Worst Metrics | Sonar CTO Andrea Malagodi
5 oktober 2026 · 57 min
Andrea Malagodi is CTO at Sonar, which builds AI code verification and governance tools. When finance asked about the size of his team's AI bills, he calculated cost per PR. At one extreme was an engineer with more than 500 PRs in a short period and a low cost per PR. At the other was an engineer with very high spend and very few PRs. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/76-your-best-ai-engineer-might-have-the-worst-metrics-sonar-cto-andrea-malagodi/
Judged on cost alone, the first engineer wins and the second looks like a problem. Malagodi looked closer. The first had built a personal agent factory, with specification, design review, documentation records, coding, verification and functional testing, and about two-thirds of the output was tests and validations. The second was working on a hard problem that needed the AI to reason through long, multi-turn sessions and didn't reduce to a line count.
Numbers alone tell you something, but not which engineer was doing the more valuable work.
We cover:
Why cost per PR is becoming the industry's standard AI spend metric, and how Malagodi's 500-PR engineer and high-spend engineer broke it
The history of bad proxies, from lines of code to story points to token leaderboards, and why Malagodi says the people who removed code were often the most important
Sonar's guide, verify, solve loop: Vortex for codebase context, algorithmic plus LLM-based PR analysis, a remediation agent for legacy debt, and the Hunter agent for security vulnerabilities
Why Malagodi breaks agent work into small pieces, and why a five-day session that produces 400,000 lines of code leaves you with no idea what is in your codebase
Setting up disagreement between agents with an orchestrator and personas for engineering, product management and quality, plus Conor Bronsdon's cross-model-family review setup
Rolling AI tools out at enterprise scale: one cost view for finance, up to 30% savings from cutting repeated contex…
-
Why Context Alone Isn't Enough for Enterprise AI Agents | WisdomAI CPO Kapil Chhabra
24 september 2026 · 1 uur 23 min
An AI agent answering questions about your business needs to understand how that business works. How do you calculate churn? When does your fiscal year end? Who has the authority to change those definitions? Connecting a model to company data leaves those questions unresolved. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/75-why-context-alone-isnt-enough-for-enterprise-ai-agents-wisdomai-cpo-kapil-chhabra/
That’s the work of context engineering: giving agents the business definitions, instructions and examples they need to interpret your data. But that context changes as the business evolves, and someone has to keep it accurate.
In this episode of Chain of Thought, WisdomAI co-founder and CPO Kapil Chhabra joins Conor Bronsdon to explain how his team approaches that challenge. We explore how companies maintain shared context, why data teams are taking on the role of AI context engineers, and how a specialized harness plans queries, checks results and repairs errors before returning an answer.
Recorded at WisdomAI’s San Mateo office, this episode is sponsored by WisdomAI.
We cover:
Why the context layer is broader than a semantic layer or catalog, and how it differs from memory
How context drifts, and why a subject-matter expert has to approve changes to shared definitions
Why Kapil sees an AI context engineer role emerging, and why data teams are moving from providing insights to providing context
How WisdomAI's harness decomposes a question, federates queries across data sources and repairs errors, including a customer that replaced a $5M-a-year analytics pipeline
The four ingredients of a trustworthy AI answer: accuracy, consistency, governance and explainability
Live Apps: governed analytics apps built from a single prompt, and what keeps them live
Why your context is your IP and should stay portable
Chapters:
(0:00) Do your agents have the right context?
(1:56) The four ingredients of trust
(5:13) The criticality and impact 2x2
… -
AI Codes: Product Engineers Decide What to Build | Laurie Voss, Arize
22 september 2026 · 56 min
Laurie Voss co-founded npm - now head of developer relations at Arize, he argues that engineers will increasingly earn their keep as 'product engineers': understanding what users need and directing AI agents to build it. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/74-ai-codes-product-engineers-decide-what-to-build-laurie-voss/
One example: a bakery owner who knows how to make a croissant but has no interest in building software. Someone still has to turn that owner's needs into requirements. Laurie sees that work becoming central to product engineering, with cheaper code making software for narrower industries more viable.
We discuss where he still sees a need for human code review and operational knowledge, what he would look for in a computer science course if he were starting out today (and what he wouldn't do), and why he compares AI today to the web in 1997. He is optimistic about the technology and skeptical of the valuations, while leaving one question unresolved: how do junior engineers learn the judgment this work demands?
We cover:
Why the "aha moment" of solving a problem survives even when agents type the code
Where Laurie still sees a need for human code review, operations, and tacit knowledge
Whether a $15,000 coding bootcamp or a theory-heavy CS degree is still worth it
Why AI in 2026 looks like the web in 1997, and what that says about the bubble
How AI-generated pull requests burden open source maintainers, and why Laurie expects cheaper code to pressure closed-source business models
Why the systems analyst returns as the product engineer, and why that means niche software for bakeries and auto parts
Why Laurie predicts open-model competition and diminishing returns could compress frontier-model margins
How apprenticeships could help junior engineers develop product judgment
Chapters:
(0:00) The shift in software jobs
(0:44) Why Laurie is optimistic about the code generation explosion
(3:06) The aha moment moves … -
Genspark's Bet: AI Agents Become the Users of Software | Wen Sang
17 september 2026 · 1 uur 2 min
Genspark went from launch to $250 million in ARR in about a year. Along the way it shipped a card-thin meeting recorder, open sourced an office suite that Wen Sang says one engineer prototyped in a week, and started running product triage with agents instead of product managers. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/73-gensparks-bet-ai-agents-become-the-users-of-software/
Wen's bet is that agents, not people, become the next users of software.
Wen Sang is co-founder and COO of Genspark. In this episode he walks through the company's three-layer architecture (models, tools and premium data as the execution layer, a memory layer he calls the second brain, and a collaboration layer called Gen Team), why a meeting note should be the start of work rather than the end of it, the engineering behind the SecondBrain Note, and where he thinks knowledge work goes once agents absorb the busy work.
Disclosure: Genspark provided the SecondBrain Note recorder discussed in this episode at no cost. Genspark is not a sponsor of this episode.
We cover:
Why Genspark builds the self-driving car around the frontier labs' engines, and what that means for people who cannot code
How Genspark's mixture-of-agents architecture routes work across 70+ models, 150+ in-house tools and paid data sets
Evals that grade whether the sales proposal answered the RFP, not whether the model can solve a differential equation
What a meeting turns into a week later when an agent needs it: proposals, pricing models, research, follow-ups
The SecondBrain Note's microphone array, battery decisions, and consent in a two-party state
Why GenOffice went open source, and the one-week prototype story behind it
How Genspark runs product feedback triage with agents and no dedicated PMs
Chapters:
(0:00) Cold open
(0:32) Geniuses with goldfish memories
(3:47) Engines and vehicles: Genspark builds the self-driving car
(6:25) Mixture of agents: models, in-house tools and premium dat… -
Tuning GPU Performance with AI Agents | AMD’s Anush Elangovan on ROCm 10
9 september 2026 · 53 min
Watch the full conversation on YouTube Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/72-tuning-gpu-performance-with-ai-agents-amds-anush-elangovan-on-rocm-10/
AI agents can keep tuning GPU workloads after you step away from the keyboard. Anush Elangovan, Corporate VP of AI Software at AMD, returns to Chain of Thought to explain how that works with Hyperloom and ROCm 10.
Anush and Conor Bronsdon trace the process from installing ROCm through Claude Code or Codex to profiling workloads, finding slow kernels, and testing optimizations while preserving numerical accuracy. Anush shares a Hyperloom run spanning 14,000 models and explains why clear goals and feedback matter when agents are doing the tuning.
They also explore what comes next for engineers: keeping skills and frameworks reliable, managing the security and accountability of autonomous agents, and applying AI to the last mile of useful software.
We cover:
How agents help install ROCm and serve models through natural language
How Hyperloom profiles workloads and uses LLMs to explore optimizations
The role of GEAK in tuning kernels while preserving numerical accuracy
Anush’s account of optimizing 14,000 models in one pass
How software improvements get more performance from existing GPUs
Keeping agent skills current and testing across AI frameworks
Security, accountability, and the next bottlenecks in agent-driven development
Chapters:
(0:25) A decade of ROCm, now agent native
(3:03) What agentic ROCm looks like in practice
(6:17) Installing ROCm then versus now
(9:14) An order of magnitude more CI across every framework
(10:44) Anush’s workflow: agents and deployment
(12:21) Speed is the moat
(15:00) Success is a stranger who cannot spell ROCm serving an LLM
(17:09) Keeping agent skills from going stale
(21:38) Co-designing kernels with the frontier labs
(24:06) Hyperloom, GEAK, and 14,000 models in one pass
(26:45) Managing autonomous agents: control and liability
(32:21) Secu… -
Slack Wants to Be the Context Harness for Code | CPO Jaime DeLanghe
2 september 2026 · 55 min
Jaime DeLanghe has spent nine years at Slack turning search, machine learning, and now agents into product. Her team just shipped Slack Code: tag a coding agent like Claude Code, Devin, Codex, or the GitHub agent in a conversation, and it spins up a code channel where everyone in that conversation gets a live development environment, diffs post as artifacts, and the channel winds down when the task is done. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/71-slack-wants-to-be-the-context-harness-for-code-cpo-jaime-delanghe/
Slack's bet is that AI at work is multiplayer. Agents belong in the channels where teams already work, not in a private chat with one person. Jaime explains why Anthropic pushes so much of its code through Slack, how the channel permission model became the agent context model, and what has to change in engineering culture when the branch is public and the whole team is steering the same agent.
The bigger question is whether Slack becomes the context harness where enterprise agents actually run.
In this conversation:
What happens mechanically when an agent creates a code channel, from authentication to diffs as artifacts
Why engineering at Slack now looks like delegating discrete tasks to agents instead of copy-pasting from a chat
How Slack's channel permission model doubles as the context and access model for agents
Why Anthropic ships code through Slack: the conversation is where the issue emerges
How culture decides whether a multiplayer coding session converges or splits
Why solo-terminal coding with an "army of Claudes" reinforces bias, and what social spaces fix
Slack as an accidental knowledge management system that ranks recency and engagement over correctness
(0:00) Slack as an IDE and a GitHub for your team
(0:29) Who is Jaime DeLanghe
(1:21) The reaction to the Slack Code launch
(5:30) Why coding agents belong in a context-rich environment
(6:08) Engineers now manage agents, not copy-paste code
(7:24) The p… -
11 Cameras, Dozens of Mics: The AI That Reads the Room | Tormod Ree, Neat
27 augustus 2026 · 55 min
Tormod Ree puts 11 or more cameras and dozens of microphones into a single meeting room, then runs computer vision on all of it to figure out who is present, who is talking, and who is looking at whom. He is the chief product and engineering officer at Neat, the Zoom-backed video hardware company. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/70-11-cameras-dozens-of-mics-the-ai-that-reads-the-room/
Before Neat, Tormod co-founded AVA, a computer vision security company Motorola acquired, and spent close to eight years at Cisco running the Spark Board. He explains how Neat turns a room into a system that directs the meeting instead of just framing whoever talks, why almost all of the AI has to run at the edge, and how the company builds computer vision models without ever collecting a customer's audio or video.
In this conversation:
Why the real advantage is the harness that combines signals from different detectors, not the models themselves
How Neat draws a hard line: no customer audio or video ever trains its models
The "captain" device that orchestrates a room full of cameras and mics, directing from metadata rather than re-running inference on every stream
Why the meeting "director" is still deterministic today, and what changes when it becomes a trained model
Building AI under a five-year device lifetime and phone-class compute
Agentic fleet management, an MCP server, and what Tormod calls "agentic healing"
How Neat gets its own non-technical teams building with AI
(0:00) Reading the room: 11 cameras, dozens of mics
(0:27) Who is Tormod Ree
(2:07) Turning a meeting room into a system that directs itself
(3:39) What it takes to actually read a room
(5:13) Why the media path has to run at the edge
(6:27) Open models, in-house models, and the harness that matters
(7:51) The data problem when you can't touch customer meetings
(11:09) The captain device: distributing compute across the room
(15:38) Two users: the people in the room and… -
Thomson 1: The New $40M Legal AI Model | Thomson Reuters Joel Hron
24 augustus 2026 · 54 min
Behind Thomson, the new legal AI model from Thomson Reuters, is a $40 million investment in people, compute, and evaluation methods. The final training run cost just $450,000. CTO Joel Hron, whose teams build Westlaw, Practical Law, and CoCounsel for millions of professionals in more than 100 countries, joined us for the launch to break down why the 175-year-old company chose to own its model layer instead of solely renting frontier intelligence. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/69-thomson-1-the-new-40m-legal-ai-model-thomson-reuters-joel-hron/
We cover:
Why Thomson Reuters trained its own model instead of relying only on Claude, GPT, or Gemini
The compute, data, and expertise flywheel behind the Thomson model
How rebuilding CoCounsel around agent-native tools took one-shot accuracy from 25% to over 70%
The rent-versus-buy case for owning model weights and compounding expert feedback over time
The dangers of AI inaccuracies in legal work
Citation ledgers, deep research, and verifying legal work with no ground-truth oracle
Joel's advice to CTOs weighing open models and training on their own data
Chapters:
(0:00) Cold open: a $40M model and 25% to 70%
(0:27) Why Thomson Reuters built the Thomson model
(2:46) From information services to an AI company
(5:14) The flywheel: compute, data, and expertise
(8:21) The oldest company to ship a model?
(9:47) Training for users without catastrophic forgetting
(14:37) Continuous pre-training on Westlaw and Checkpoint
(15:29) Fine-tuning, DPO, and agentic reinforcement learning
(17:37) Rebuilding CoCounsel: 25% to 70% overnight
(21:48) Capturing expert judgment: own versus rent the model
(28:59) Managing lawyer time and protecting customer IP
(31:45) Eval results and avoiding catastrophic forgetting
(34:34) Tabular analysis and legal deep research
(36:26) Benchmarks, Harvey, and frontier comparisons
(39:26) Verifying legal work with no ground-truth oracle
(41:54) Citation ledgers and th… -
Time to Exploit is Negative: AI Broke the Patch Cycle | Dan Lorenc, Chainguard
19 augustus 2026 · 47 min
Attackers used to take months, sometimes 270 days, to weaponize a disclosed vulnerability. Now it happens in weeks, minutes if the incentive is there, and independent reports from Mandiant and CrowdStrike show the average time to exploit has gone negative. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/68-time-to-exploit-is-negative-ai-broke-the-patch-cycle-dan-lorenc/
Dan Lorenc's conclusion: finding flaws is no longer the hard part. Fixing them first is.
Dan is the co-founder and CEO of Chainguard. Before that he spent years at Google building the backbone of software supply chain security and created Sigstore. In June his team launched Athena, a coalition of more than two dozen companies including JPMorgan, Cloudflare, Cisco, and Kyndryl, built for the era where AI finds vulnerabilities faster than maintainers can patch them. Last month alone it processed more than 40,000 AI-discovered findings.
In this conversation:
Why the average time to exploit went negative, and what collapsed the fat tail of never-exploited bugs
How models chain "low severity" flaws into working exploits, like Project Zero's zero-click iPhone takeover
Inside Athena: what happens between a member submitting a finding and a fix landing upstream
Why 40,000 findings is not 40,000 CVEs: validation, dedup, and vulnerability archaeology
Fuzzing outpaced patching for a decade, and AI is the first tool that speeds up the fixing side
The Log4j thought exercise for solo maintainers and enterprise CISOs alike
Fork economics, "state your intentions," and defense in depth for agent infrastructure
Chapters:
(0:00) Cold open: how time to exploit goes negative
(0:31) The 20-year assumption that just died
(3:21) What a negative time to exploit actually means
(6:04) Two new realities: attacks democratized, more bugs than anyone knew
(8:55) Chaining tiny flaws: the Project Zero iPhone story
(11:26) Why fixing AI-found vulnerabilities takes a coalition
(13:27) 40,000 findings in… -
Parenting Your AI Agents for Best Results | Cisco's Jeetu Patel
12 augustus 2026 · 46 min
Agents are like teenagers: profoundly intelligent, extremely resourceful, no fear of consequence, and missing the judgment to know right from wrong at all times. That's how Cisco President and Chief Product Officer Jeetu Patel thinks about securing AI agents, and it's why he says static allow/block rules are already obsolete. Agents are smart enough to route around them. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/67-parenting-your-ai-agents-for-best-results-ciscos-jeetu-patel/
Jeetu returns to Chain of Thought to map cyber's third phase: an agent trust platform where security and observability fuse into one discipline. He explains how Cisco Cloud Control spins up a digital twin to test every agent-recommended fix before it touches production, why Cisco moved from unlimited tokens to rationing them like headcount, and why the gap between people who are fluent with AI and people who aren't is now a 10x differential, not 10%. He also makes the contrarian case that AI will create more jobs than it destroys - and of course, we talk infrastructure for this new era.
We cover:
Why agents need dynamic runtime guardrails instead of static allow/block rules
The agent trust platform: how security and observability are fusing into one discipline
Agentic ops in Cisco Cloud Control: ambient agents, digital twins, and human-in-the-loop
How Cisco rations AI tokens the way it rations headcount
The intelligence, cost, and control trade-offs behind open vs closed models
Action control vs access control: what rights you actually grant an agent
Why every automation step creates a new human bottleneck, and more jobs
Chapters:
(0:00) Agents are like teenagers: cold open
(0:25) Welcome back Jeetu Patel
(1:18) Open weight vs closed models
(2:26) Intelligence, cost, and control: the model trade-off triangle
(10:13) Shrinking model half-life and the economics of frontier training
(11:38) Why token costs still outrun token value
(14:15) Build your own evals a… -
Data Federation, Not Centralization, Is What Enterprise AI Needs | Jitender Aswani, Starburst
16 juli 2026 · 53 min
Jitender Aswani was customer zero for Presto at Meta, where a billion daily active users generated queries that took hours to return. He watched that drop to minutes, scaled the same technology at Netflix across 300 million subscribers, and now runs engineering and security at Starburst, the $3.35 billion platform built on Trino. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/66-data-federation-not-centralization-is-what-enterprise-ai-needs/
His argument: every enterprise AI project that stalls is fighting the same hidden battle. The agents can query the model fine. They just can't reach the data. The average enterprise runs 52 to 200 data sources, and a decade of moving all of it into one lake produced ETL debt, governance problems, and pipelines that break whenever a SaaS vendor adds a column.
Federation is the only model that scales with entropy.
We cover:
Why Presto changed what Meta could experiment on, and how that compounded product velocity
What broke when Jitender took the same technology to enterprises running 52 to 200 data sources
Why centralization stopped working once data grew faster than the ability to move it
What happened to Starburst's query volume the day they shipped an MCP server
The FinOps agent that fired queries for 30 minutes against data it never had
How AIDA turns ad hoc analysis into workflows using skills and MCP servers
Why a context graph is different from a knowledge graph, and why ontology decides agent accuracy
(0:00) Enterprises run on 52 to 200 data sources
(0:25) Intro
(2:18) Customer zero for Presto at Meta
(9:50) Scaling to trillions of events at Netflix
(15:11) Taking Trino from Silicon Valley to 10,000 enterprises
(20:24) The 2011 research that predicted conversational analytics
(28:54) Why centralization can't scale with entropy
(32:26) The agent query explosion and what MCP did to volume
(41:44) Inside AIDA, Starburst's conversational analytics product
(46:56) Context graphs versus knowledge… -
You Can't Secure an AI Agent with Software | Charles Guillemet, Ledger
1 juli 2026 · 53 min
Charles Guillemet is CTO of Ledger and the founder of the Donjon, Ledger's internal offensive security lab whose job is to break the company's own products before attackers do. He spent a decade in cryptography and hardware security before Ledger, including designing secure integrated circuits. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/65-you-cant-secure-an-ai-agent-with-software/
His argument is blunt: you cannot secure an AI agent with software alone. As agents start moving real money, API keys and trust scopes leave no physical verification layer, and Charles makes the case that hardware has to sit in the loop.
This one turned into a wide-ranging thought piece (and some debate) on what the agentic economy actually looks like, and how to stay safe inside it.
We cover:
Why Charles thinks "securing an AI agent" with software permissions and API keys is a false promise
The economic asymmetry between attackers and defenders, and how AI is collapsing it
How a policy engine plus a hardware-enforced signature can delegate rights to an agent safely
Why Charles thinks the agentic economy settles on blockchain rails over Visa and Mastercard
Secure elements, HSMs, and zero-knowledge proofs as execution-integrity guarantees
How Ledger uses hardware authorization internally for passkeys, signed releases, and multisig
A practical way to classify assets by threat model and match security to value
(0:00) Why securing an AI agent in software alone is impossible
(0:30) Delegating execution power inside your security perimeter
(2:28) The attack-defense asymmetry AI is erasing
(6:00) The alignment problem and delegating rights to agents
(9:24) Policy engines, intents, and hardware-enforced signatures
(13:19) From developer experience to agent experience
(15:12) Secure elements, HSMs, and execution integrity
(20:00) Zero-knowledge proofs, proving without revealing
(27:24) Convincing the skeptics on agent-driven payments
(34:49) Why Ledger bet on de… -
Stop Token Maxxing: Find Where AI Actually Pays Off | Jiaona Zhang, Laurel
25 juni 2026 · 58 min
Jiaona Zhang(JZ) is the Chief Product Officer at Laurel, where the team runs its own product on itself to see exactly where AI helps and where it doesn't. Before Laurel, JZ built products at Airbnb, Dropbox, Webflow, and Linktree, and she has taught product management at Stanford for nearly a decade. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/64-stop-token-maxxing-find-where-ai-actually-pays-off-jiaona-zhang/
Companies are spending billions on AI tooling, but most still can't say where it returns time or revenue. Jiaona breaks down how to get that visibility, why blanket AI mandates backfire, and what it takes to re-architect a team so anyone can ship.
Her argument is simple: stop token maxing and start measuring time back.
We cover:
Why most organizations can't see where AI is actually working, and how Laurel uses time data to fix it
The token max trap that "use AI everywhere" mandates create, and how to drive efficient use instead
Why former managers make the best operators of agent fleets
How Laurel lets PMs, designers, and customer success ship features end to end
The bottom-up plus top-down playbook for re-architecting a team around AI
Why technology moats are falling away while brand and data moats endure
Laurel's bet on returning time to people instead of replacing them
(0:00) The token max trap
(1:47) Why companies can't see where AI is working
(5:03) What Laurel does: turning time into data
(8:53) Agents as an extension of the workforce
(13:43) Why former managers make the best AI users
(18:23) Lean teams and shipping end to end
(22:29) Enabling non-engineers to ship features
(28:30) Re-architecting teams: bottom-up and top-down
(32:09) Keeping your professional identity as AI shifts work
(38:53) The context layer is the new race
(42:06) Fundamentals plus tinkering: how to learn
(48:45) Brand and data moats when tech moats fall away
(54:31) Laurel's movement: returning time to peopleConnect with Jiaona Zhang(JZ):
Linke…
-
Most of the Web Will Never Get APIs for AI Agents | Dhruv Batra, Yutori
18 juni 2026 · 55 min
Most of the web will never get APIs for AI agents. School district sites, small business pages, government offices, and the long tail of e-commerce were built for humans, and they will keep working that way for years. So how do agents actually get things done across the web? Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/63-most-of-the-web-will-never-get-apis-for-ai-agents-dhruv-batra/
Dhruv Batra is co-founder and chief scientist of Yutori, the company building specialized browser and computer-use agents. He previously led embodied AI at Meta's FAIR lab, training robots in simulation and shipping the image question-answering model on Ray-Ban Meta glasses. His bet: the web is a shared roadway, much like roads split between human drivers and self-driving cars, and agents will be built to use it the way people already do.
Pixels in, clicks out. That is the API.
In this conversation:
Why the long tail of the web won't re-architect itself for agents
How Yutori's Navigator perceives pixels and writes JavaScript on the fly to shorten task trajectories
Why Navigator runs 2-3x faster and 4-5x cheaper than Opus 4.7 and GPT-5.5 on browser tasks
Learning from live websites, and using URL query parameters as privileged verifiers instead of cloning sites
What the shift from American to Chinese open-weight models means for startups
How smart glasses and robots share the same perception-action loop
Why demand for inference compute is pushing models smaller and onto devices
Chapters:
(00:00) Pixels in, clicks out
(01:37) Why most of the web will never get APIs
(08:47) Aggregation, specialization, and human friction
(11:39) Digital niches and specialized models
(16:41) The web's heavy tail and where browser agents win
(20:40) Inside Yutori's Navigator and Scouts
(24:08) N1.5: writing JavaScript to cut trajectory length
(27:45) Training on live websites
(33:29) Open source: FAIR's legacy and the Chinese frontier
(37:22) Agent frameworks: OpenClaw, Her… -
The First Fully Autonomous AI Attack is Coming | Kris Lovejoy, Kyndryl
11 juni 2026 · 46 min
Kristin "Kris" Lovejoy has spent her career inside the systems the global economy runs on: banks, hospitals, energy grids, governments. Today she is Global Head of Strategy at Kyndryl, the world's largest IT infrastructure services provider, working with mission-critical enterprises across more than 60 countries. Before that she ran security businesses at EY and IBM, founded the AI security company BluVector (acquired by Comcast), and now sits on the board of Dominion Energy. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/62-the-first-fully-autonomous-ai-attack-is-coming-kristin-lovejoy/
Her prediction: the first fully autonomous AI attack, where an AI takes down an enterprise network with no human driving it, lands within 18 months.
Conor and Kris dig into why 62% of enterprise AI initiatives are still stuck in pilots even as spend climbs 33% year over year, why attackers chaining low-risk vulnerabilities changes the patching math, and why she has a fraught relationship with policy as code.
We cover:
The electricity analogy: we can build the models, but the transmission lines for industrial AI don't exist yet
Productivity AI vs mission-critical AI, and why banks and healthcare systems aren't running agentic AI at production scale
Why deterministic policy as code clashes with autonomous systems, and "human on top" vs human in the loop
The 18-month prediction: chaining low-risk vulnerabilities, outcome-oriented agents that take systems down by accident, and insiders armed with AI attack tools
The data center build-out from a Dominion Energy board member: PJM load forecasts that miss by double digits every year, water use, density, and rack optimization
Privacy as a double-edged sword: data combinations that suddenly become PII and the shift to continuous compliance
What's next: open source everywhere, sovereignty as control, autonomous robotics, and quantum
Chapters:
(00:00) Meet Kris Lovejoy: Kyndryl, EY, IBM, and Dominion Energy
(02… -
The AI Framework Era Is Over: Why Context Is the Moat | Jerry Liu of LlamaIndex
3 juni 2026 · 53 min
Jerry Liu built one of the most installed pieces of AI plumbing of the last three years. LlamaIndex became the indexing and retrieval layer a whole generation of RAG apps were stitched together with. Then he started arguing that the framework era he helped create is over. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/61-the-ai-framework-era-is-over-why-context-is-the-moat-jerry-liu-of-llamaindex/
Jerry is co-founder and CEO of LlamaIndex. In this conversation he walks through the company's pivot from open-source framework to managed document infrastructure with LlamaCloud and LlamaParse, and why he is betting that context quality is the one moat that compounds as agent loops get good enough to absorb the scaffolding.
If you are a founder worried a frontier lab or a coding agent is about to eat your product, this is the playbook for reinventing your ICP without losing the thread.
In this conversation:
Why Jerry says the AI framework era is over, and what actually survives
How agent harnesses like Claude Code collapsed the old framework patterns into the model
Why context quality is the durable moat, not the agent loop
How LlamaParse beats legacy OCR and frontier models on document accuracy and cost
Why 95%+ accuracy is the real bar for legal, insurance, and financial document work
How LlamaIndex disrupted its own product and reinvented its ICP to stay alive
Jerry's take on agent memory, model personalities, and why LLMs are still bad writers
(0:00) Is the AI framework era over?
(1:56) What died and what survived
(6:31) Why context quality is the moat
(8:12) Defining the context layer
(13:18) Coding and vision as the abstraction layer
(18:13) The bet that context compounds
(23:59) Which verticals are adopting
(25:14) Why 95%+ accuracy is the real bar
(29:49) The file system as an agent primitive
(34:33) Surviving your own pivot
(37:15) Reinventing strategy and hiring
(42:00) Agent memory as persistent context
(44:41) Model personalitie… -
We Built Agents, Nobody Built HR | Tyler Akidau, Redpanda
27 mei 2026 · 51 min
Tyler Akidau spent 12 years on streaming systems at Google and five years at Snowflake before joining Redpanda as CTO. He wrote the O'Reilly Streaming Systems book most of the field has on its shelf. His new piece on O'Reilly Radar (Post-Human: We All Built Agents, Nobody Built HR) argues that enterprises are stuck in the prototype-to-production gap because they're applying human-era identity, auth, and observability tools to a workforce that's unpredictable in structurally novel ways, runs at machine speed, and follows bad instructions to a fault. Inline guardrails like CLAUDE.md work until they don't. Governance has to be enforced through channels the agent can't see, modify, or override. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/60-we-built-agents-nobody-built-hr-tyler-akidau-redpanda/
We cover:
Why AI agents are a new kind of co-worker (unpredictable, machine-speed, directable to a fault) and what that means for enterprise infrastructure
The four pillars of agent governance: identity, authorization, observability and explainability, accountability and control
Why task-scoped, short-lived identity is the foundation everything else builds on
Authorization that's deny-capable and intersection-aware (Tyler's "guest badge" model)
Why OpenTelemetry is the right starting point for recording every prompt, tool call, and response
How Redpanda's Agentic Data Plane combines streaming topics, Oxla SQL, and Postgres under the hood
Tyler's academic paper with a psychologist on the neurobiological systems humans have that AI agents are missing
Chapters:
(00:00) Why nobody built HR for AI agents
(02:12) Three ways agents differ from human employees
(07:53) The four pillars of out-of-band governance
(10:29) Identity: task-scoped, short-lived, chained to humans
(14:40) Authorization: deny-capable and intersection-aware
(18:57) Observability: record everything via OpenTelemetry
(24:24) Redpanda's agents and the $1,000 trade limit example
(30:10)… -
How Superhuman Built AI Into a 100ms Product | Loïc Houssier
22 mei 2026 · 50 min
Loïc Houssier leads engineering at Superhuman, the email client Grammarly acquired for ~$825 million in July 2025. Before Superhuman he was CTO of OpenTrust (acquired by DocuSign), ran engineering at ProductBoard, and started his career in applied cryptography for France's defense industry, including work on nuclear submarine systems. Loïc joined Superhuman in early 2024 and within 30 days was leading a six-week sprint to ship AI Inbox. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/59-how-superhuman-built-ai-into-a-100ms-product-lo-c-houssier/
Superhuman's brand is built on speed: every interaction under 100 milliseconds. LLMs do not run in 100 milliseconds. So Loïc walks Conor through how his team retrofitted AI into a product that was already winning without it: pre-caching context for the mobile voice feature, starting every feature on the smartest available model and only then fine-tuning down to cheap dedicated infrastructure, treating "look foolish" as a P0 bug class, and refusing to auto-send any email even when their agents could.
This is a practitioner's tour of what it actually takes to put AI on top of a product that has to stay fast, stay quiet, and never embarrass the user.
We cover:
The model-routing strategy: Opus and frontier models to prove a feature, then fine-tuned BERT classifiers on dedicated inference
Pre-caching voice and tone context separately from dictation to keep the mobile voice feature feeling fast
Why eval engineering at Superhuman is owned by PMs, and how a single "how much time did I spend in Waymo last month" query exposes the eigenvectors a feature has to cover
Why "look foolish" is a P0 bug class, and where the boundary between agent agency and agent laziness actually sits
How Superhuman's pod structure (PM, tech lead, designer) and a central AI platform team support aligned autonomy
Hiring for AI fluency: how interview questions are changing and what self-augmenting engineers look like
Pattern dete…
-
The AI Hiring Doom Loop: Applications Up 239%, Hires Down 75% | Daniel Chait, Greenhouse
6 mei 2026 · 57 min
Job applications are up 239% since ChatGPT launched, tech layoffs show no signs of slowing down, and the market for technical talent is a topsy turvy mess. Lees meer
Greenhouse has a unique vantage point to understand all of this: they process 22 million job applications a month across 7,500+ companies including HubSpot, Anthropic, Coinbase, and the NFL. CEO Daniel Chait has had a front-row seat to the strangest hiring market in decades, and he's here to advise us all on how to navigate it.
Full transcript and chapters: https://chainofthought.show/podcast/58-the-ai-hiring-doom-loop-applications-up-239-hires-down-75/
Daniel coined the term "AI doom loop" for what's happening: applications up 239% since ChatGPT launched, resume hacks like white-fonting and prompt injection up 500%, and 75% fewer applications reaching the hire stage. 91% of recruiters have spotted candidate deception. 38% of job seekers walk away from processes that include an AI interview.
It's the worst job market for candidates and the hardest hiring market for recruiters.
Daniel explains how technical talent can break the loop.
We cover:
Why software engineers, according to Greenhouse data, are the worst auto-appliers and what to do instead
The North Korean infiltration problem: deepfakes, laptop farms, and why companies are flying candidates in for in-person interviews again
How AI screener interviews open up the funnel when companies are transparent about using them, and break it when they aren't
Greenhouse Dream Jobs: how a single high-signal application a month converts at 5x the rate
Why take-home assignments don't survive contact with AI and what Greenhouse uses instead
What a coding interview looks like when leetcode is dead and engineers run 10+ Claude Code sessions in parallel
The case for killing the resume entirely and rebuilding hiring around AI conversations
Chapters:
(00:00) Cold open: 239% more applications, 75% fewer hires
(02:14) Galileo
(03:05) The AI doom loop, defined
(04:0… -
Every AI Agent Has an Evaluation Gap | Alex Ratner, Snorkel AI
29 april 2026 · 43 min
Alex Ratner co-founded Snorkel AI out of Chris Ré's Stanford lab and helped establish data-centric AI as a field. Today, Snorkel is a $1.3B company shipping thousands of data sets and environments a week to frontier labs and vertical AI teams like Harvey. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/57-every-ai-agent-has-an-evaluation-gap-alex-ratner-snorkel-ai/
In this conversation, he argues our ability to build AI agents has outpaced our ability to measure them. That gap is what's keeping most enterprise agents stuck in demo purgatory.
If you can't measure it, you can't improve it. And you can't deploy it.
In this conversation:
The three axes of the evaluation gap: input complexity, autonomy horizon, and output complexity
Big Law Bench: how Snorkel and Harvey benchmarked legal agents on deep-research tasks that take lawyers 10-15 hours
What Snorkel's $3M Open Benchmarks Grant is funding, and why "benchmaxxing" critiques don't kill the case for public benchmarks
Why 40-50% of Snorkel's data work is still review and labeling, even with the best models in the loop
The "expert-agentic" era, where domain expertise (law, finance, coding, even woodworking) is the new bottleneck
Why self-supervision is a dead end outside narrow cases like distillation
The false dichotomy between data and environments, and why pure-environment vendors miss how AI actually works
Chapters
(00:00) Intro: Alex Ratner and Snorkel AI
(02:50) What the evaluation gap actually is
(06:05) Moravec's paradox and the jagged frontier
(08:46) Where AI agents fall down in enterprise work
(10:40) Big Law Bench: benchmarking Harvey's legal agents
(12:00) The three axes: input, autonomy horizon, output
(18:31) Snorkel's $3M Open Benchmarks Grant
(22:33) From "janitorial" to epicenter: 15 years of data-centric AI
(29:26) The expert-agentic data era
(34:54) The false dichotomy between data and environments
(40:05) DoorDash Tasks and expert data at scaleConnect with Alex Ra…
-
250,000 Lines of Code/Week: Inside an AMD VP's Agent-First Workflow | Anush Elangovan
22 april 2026 · 51 min
What happens when a VP of AI Software at a major chip company goes all-in on AI coding agents for his own team's work? Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/56-250-000-lines-of-code-week-inside-an-amd-vps-agent-first-workflow-anush-elangovan/
Anush Elangovan runs 10–12 Claude Code agents across three machines, burns 6.5 billion tokens a week, and rewrote a 25-year-old project (Slurm → Spur in Rust) in a single night.
He does it all on dangerously-skip-permissions.
About Anush
Anush Elangovan is Corporate VP of AI Software at AMD. He founded Nod.ai, where his team built SHARK and was a primary contributor to Torch-MLIR and IREE. AMD acquired Nod.ai in 2023, and Anush now leads AI software strategy across AMD's full silicon portfolio. Before Nod.ai, he shipped the graphics stack on the first ARM Chromebook and led Chrome OS's migration to Gentoo.We cover:
How Anush runs 10–12 parallel agents with a geo-distributed AMD hardware rig
Why the test harness is the new code review (and why agents are "sneaky and dumb")
Rewriting a 25-year-old project in Rust overnight, without opening the editor
Why every new project is in Rust specifically because he refuses to learn it
The "HR partner fixing engineering bugs" moment and what it says about upskilling
Why normal SDLC is dead and speed is the only durable moat
AMD's fully open-source software stack and how community contributions are accelerating ROCm
"Software is just tokens" and what that means for AMD's bet against CUDA lock-in
Connect with Anush
LinkedIn: linkedin.com/in/anushelangovan
Twitter/X: @AnushElangovan
AMD AI blog: amd.com
AMD AI Developer Program: amd.com/developerConnect with Chain of Thought host Conor Bronsdon:
Newsletter: newsletter.chainofthought.show
Twitter/X: @ConorBronsdon
LinkedIn: linkedin.com/in/conorbronsdon
YouTube: @ConorBronsdonMore episodes: chainofthought.show
Chapters
0:00 Cold open
0:21 Welcome + guest intro
3:43 250K lines a week, 10–12 paral… -
Hallucinations Are a Data Architecture Problem | Sudhir Hasbe, Neo4j
16 april 2026 · 52 min
Sudhir Hasbe is President and Chief Product Officer at Neo4j, the graph database company powering 84 of the Fortune 100 (Walmart, Uber, Airbus) at $200M+ ARR and a $2B+ valuation. Before Neo4j, he ran product for all of Google Cloud's data analytics services: BigQuery, Looker, Dataflow, and led the Looker acquisition. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/55-hallucinations-are-a-data-architecture-problem-sudhir-hasbe-neo4j/
His thesis: the hallucinations we blame on AI models are really a data architecture problem. LLMs weren't trained on your enterprise knowledge, so handing them a data lake with 10,000 disconnected tables and asking them to reason is the wrong design. The fix is knowledge graphs: feeding the model a structured map of relationships, entities, and context so it can reason over meaning, not just vector similarity.
Sudhir breaks down the five capabilities knowledge graphs unlock for enterprise AI: GraphRAG (moving accuracy from 60% to 97%), semantic mapping across siloed systems, context graphs, agent memory, and multi-hop reasoning. He explains three architecture patterns customers are actually shipping, why giving an LLM hundreds of tools makes it worse, and what Uber, EA Sports, Klarna, and Novo Nordisk are doing differently.
This is the case for treating knowledge as infrastructure.
We cover:
Why enterprise AI needs a different playbook than consumer AI
The five data asset types every agentic system needs: system of record, historical, memory, context, and reference
How GraphRAG combines vector search and graph traversal to move from 60% accuracy to 95%+
Three architecture patterns: semantic layer only, semantic map plus domain data, full consolidation (the Klarna/Kiki model)
What context graphs capture that Salesforce doesn't: the Slack and email negotiation behind every deal
Why giving an LLM hundreds of tools drops accuracy, and how Uber uses knowledge graphs as a business validation layer
What Neo4j's A…
-
Why LLMs Are Plausibility Engines, Not Truth Engines | Dan Klein, Scaled Cognition
8 april 2026 · 1 uur 18 min
Every few weeks at Microsoft, someone would build an AI prototype that blew everyone's minds. Three months later? Dead. "We can never ship that." Dan Klein watched this happen for five years before he decided to do something about it. Lees meer
Dan is co-founder and CTO of Scaled Cognition, a professor of computer science at UC Berkeley, and winner of the ACM Grace Murray Hopper Award. His previous startups include adap.tv (acquired by AOL for $405M) and Semantic Machines (acquired by Microsoft in 2018), where he spent five years integrating conversational AI. His PhD students now run AI teams at Google, Stanford, MIT, and OpenAI.
At Scaled Cognition, Dan's team built APT1 (the Agentic Pre-trained Transformer) for under $11 million. It's a model designed for actions, not tokens, with structural guarantees that go beyond prompt-and-pray.
Dan makes the case that current LLMs are plausibility engines, not truth engines, and that the gap between demo and production is where most AI projects die.
Full transcript and chapters: https://chainofthought.show/podcast/54-why-llms-are-plausibility-engines-not-truth-engines-dan-klein/
Why prompting is a fundamentally unreliable control surface for production AI
How APT1's architecture gives actions and information first-class status instead of treating everything as tokens
The specific failure modes that kill enterprise AI prototypes within three months
Why stacking multiple models to check each other produces correlated errors, not reliability
How Scaled Cognition applied RL to conversational AI when there's no zero-sum winner
Why every S-curve in AI gets mistaken for an exponential — and what comes after the current plateau
The societal risk of systems that produce output indistinguishable from truth
Chapters
(0:00) Cold open: RL is about doubling down on what works
(0:28) Introducing Dan Klein and Scaled Cognition
(2:53) The demo-to-production gap: why AI prototypes die
(5:40) Why prompting is not a real control surface
(8:… -
Agent Memory: The Last Battleground in the AI Stack | Richmond Alake, Oracle
2 april 2026 · 59 min
Richmond Alake is Director of AI Developer Experience at Oracle and one of the most concrete voices on agent memory right now. His AI Engineer World's Fair talk on architecting agent memory crossed 100,000 views, he built the open-source MemoRIS library, and he co-created a course with Andrew Ng. Lees meer
In this conversation, Richmond walks through memory engineering as a distinct discipline from prompt engineering and context engineering, demos a memory-aware financial services agent that runs vector, graph, spatial, and relational search in a single query, and explains the principle that separates production-grade memory systems from prototypes: don't delete, forget. If you're building agents that need to remember anything across sessions, this is the episode.
We cover:
- Why memory engineering deserves its own name, separate from prompt and context engineering
- The two failure modes Richmond sees most: wrong mental model and deleting instead of forgetting
- Four human memory types mapped to agent architecture: working, episodic, semantic, and procedural
- Demo: AFSA, a memory-aware financial services agent with converged search across data types
- How the Generative Agents paper's decay formula (relevance + recency + importance) enables controlled forgetting
- Where context engineering ends and memory engineering begins
- Why files work for prototypes but databases win in productionChapters:
(0:00) Memory is the last battleground in AI
(0:28) Meet Richmond Alake, Oracle's AI DevEx lead
(2:23) Why memory engineering is its own discipline
(7:57) The failure modes nobody talks about
(12:49) Demo: a memory-aware financial services agent
(18:30) Segmenting context windows by memory type
(19:22) Four human memory types mapped to agent architecture
(23:51) Procedural memory in production systems
(27:11) Don't delete, forget: implementing controlled decay (33:32) Sponsor: Galileo
(35:46) Where context engineering ends and memory engineering begins
(38:50) Is agent memory f… -
Context Poisoning is Killing Your AI Agents: How to Stop it | Michel Tricot, Airbyte
25 maart 2026 · 44 min
Michel Tricot co-founded Airbyte, the open source data integration platform with 600+ free connectors that hit a $1.5 billion valuation. Now he's building the company's next product: an agent engine, currently in public beta. His thesis is that agents don't fail because models are bad. They fail because the data feeding them is wrong: context poisoning is killing them. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/52-context-poisoning-is-killing-your-ai-agents-how-to-stop-it/
Michel demos this live. A simple Gong query through raw API calls burned 30,000 extra tokens and took three minutes. The same query through Airbyte's context store ran in one minute and used a fraction of the context window. Conor and Michel dig into why RAG alone won't cut it, what a "context engineer" actually does, how Airbyte tracks entities across Salesforce, Zendesk, and Gong without embeddings, and whether the SaaS apocalypse playing out in public markets is overblown.
Chapters:
0:00 Intro
0:20 Meet Michel Tricot, CEO of Airbyte
2:27 Data Got Us to the Information Age. Context Gets Us to Intelligence.
4:48 How Context Poisoning Breaks Agents
7:49 Why Airbyte Customers Stopped Loading Into Warehouses
10:12 Live Demo: Context Store vs Raw API Calls
10:38 What Does a Context Engineer Actually Do?
14:14 RAG Isn't Dead, But How We Build It Will Die
16:41 30K Wasted Tokens Without Proper Context
22:22 Cross-System Joins: Zendesk, Gong, and Salesforce
26:12 The Open Source Agent Connector SDK
29:45 The SaaS Apocalypse Is Overblown
36:09 From Data Pipes to Agent Infrastructure
38:51 What Agents Need to Get Right by Summer
40:48 Memory Is Just Another Form of Context
43:07 OutroAbout the Guest:
Michel Tricot is the CEO and co-founder of Airbyte, the open source data integration platform used by thousands of companies to move data between systems. Before Airbyte, he led data ingestion and distribution engineering at LiveRamp. Airbyte raised at a $1.5 billion valuation a…
-
I Started r/AI_Agents and Now I'm Launching a VC Fund | Yujian Tang
10 maart 2026 · 44 min
Yujian Tang started the r/AI_Agents subreddit in April 2023. For the first year, it barely moved. Then it hit 9,000 members, he went on vacation, came back to 36,000, and now it's approaching 300,000. In this episode, Yujian talks about how that community grew alongside his event business (Seattle Startup Summit, 900+ attendees last year), his two failed startups, and why he just filed paperwork to launch his own venture fund. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/51-i-started-r-ai-agents-and-now-im-launching-a-vc-fund/
Conor and Yujian dig into the mechanics of starting a fund from scratch (Delaware PO boxes, EIN numbers, lawyers), why AI startup valuations have doubled in the last two years, whether a one-person unicorn is realistic, and what failed founders learn that successful ones sometimes miss.
Chapters:
(0:00) Cold Open: The Subreddit Growth Explosion
(0:21) Intro and Meet Yujian Tang
(1:06) From AI Research to Community Building
(7:26) Where AI Applications Are Headed
(10:03) The AI Bubble and a Valuation Reset
(10:39) Getting Deal Flow Through Community Events
(14:02) Filing the Fund: The Boring Side of VC
(16:04) How r/AI_Agents Went from Crickets to 300K
(18:39) Building an Accidental Empire
(26:37) What Two Failed Startups Taught Him
(29:52) Why Pre-Seed Valuations Are Out of Control
(37:37) The One-Person Unicorn Debate
(39:50) Seattle Startup Summit 2026
(42:17) What Chain of Thought Should Cover Next
(43:25) Outro
About the Guest:
Yujian Tang is the founder of Seattle Startup Summit, the largest startup event in the Pacific Northwest. He created the r/AI_Agents subreddit (now nearly 300K members), runs hackathons and developer events across Seattle and the Bay Area, and is launching an early-stage AI venture fund.
Guest Links:
Seattle Startup Summit: seattlestartupsummit.com
Reddit: reddit.com/r/AI_Agents
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://newsletter.chainoftho…
-
I Built an AI Coworker That Runs 90% of My Day | Sterling Chin, Postman
4 maart 2026 · 1 uur 2 min
Sterling Chin stopped thinking of AI as a tool and started treating it like a junior employee. Onboarded it with context, corrected its mistakes, and gave it writing rules. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/50-i-built-an-ai-coworker-that-runs-90-of-my-day/
Forty days later, MARVIN was handling 90% of his workday.
In this episode of Chain of Thought, Sterling (Applied AI Engineer and Senior Developer Advocate at Postman) walks through live demos of MARVIN, his personal AI assistant built on Claude Code. From pulling meeting transcripts and updating Jira tickets to drafting blog posts and managing his calendar, MARVIN runs as a full-time AI chief of staff.
We cover:
How MARVIN bookends Sterling's workday from first login to the end of the day
Personality, sub-agents, and writing rules that make MARVIN an effective co-worker
Automating meeting notes to Jira tickets
Why DIY assistants outperform big tech alternatives
How Sterling onboarded 12+ colleagues at Postman, including non-technical knowledge workers
What the compute crunch means for open source AI
Connect with Sterling:
LinkedIn: https://www.linkedin.com/in/sterlingchin/
Twitter/X: https://x.com/SilverJaw82
MARVIN Template: https://github.com/SterlingChin/marvin-template
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://conorbronsdon.substack.com/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon
YouTube: https://www.youtube.com/@ConorBronsdon
🔗 More episodes: https://chainofthought.show
Timestamps:
(0:00) Intro
(0:28) Meet Sterling Chin and the MARVIN AI Assistant
(9:10) Live Demo: How MARVIN Bookends Your Workday
(16:04) Personality, Sub-Agents, and Writing Rules
(22:00) Automating Meeting Notes to Jira Tickets
(29:30) Why DIY AI Assistants Outperform Big Tech
(40:55) Treat Your AI Like a Junior Employee
(46:41) How to Get Started with MARVIN
(55:36) The Compute Crunch and Open Source Fu…
-
How Intercom Cut $250K/Month by Ditching GPT for Open Models | Fergal Reid
26 februari 2026 · 54 min
Intercom was spending $250K/month on a single summarization task using GPT. Then they replaced it with a fine-tuned 14B parameter Qwen model and saved almost all of it. In this episode, Intercom's Chief AI Officer, Fergal Reid, walks through exactly how they made that call, where their approach has changed over time, and how all of their efforts built their Fin customer service agent. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/49-how-intercom-cut-250k-month-by-ditching-gpt-for-qwen/
Fergal breaks down how Fin went from 30% to nearly 70% resolution rate and why most of those gains came from surrounding systems (custom re-rankers, retrieval models, query canonicalization), not the core frontier LLM. He explains why higher latency counterintuitively increases resolution rates, how they built a custom re-ranker that outperformed Cohere using ModernBERT, and why he believes vertically integrated AI products will win in the long term.
If you're deciding between fine-tuning open-weight models and using frontier APIs in production, you won't find a more detailed decision process walkthrough.
🔗 Connect with Fergal:
Twitter/X: https://x.com/fergal_reid
LinkedIn: https://www.linkedin.com/in/fergalreid/
Fin: https://fin.ai/
🔗 Connect with Chain of Thought host Conor Bronsdon:
YouTube: https://www.youtube.com/@ConorBronsdon
Newsletter: https://conorbronsdon.substack.com/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
🔗 More episodes: https://chainofthought.showCHAPTERS
0:00 Intro
0:46 Why Intercom Completely Reversed Their Fine-Tuning Position
8:00 The $250K/Month Summarization Task (Query Canonicalization)
11:25 Training Infrastructure: H200s, LoRA to Full SFT, and GRPO
14:09 Why Qwen Models Specifically Work for Production
18:03 Goodhart's Law: When Benchmarks Lie
19:47 A/B Testing AI in Production: Soft vs. Hard Resolutions
25:09 The Latency Paradox: Why Slower Responses Get More Resolut…
-
How Block Deployed AI Agents to 12,000 Employees in 8 Weeks w/ MCP | Angie Jones
21 januari 2026 · 50 min
How do you deploy AI agents to 12,000 employees in just 8 weeks? How do you do it safely? Angie Jones, VP of Engineering for AI Tools and Enablement at Block, joins the show to share exactly how her team pulled it off. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/48-how-block-deployed-ai-agents-to-12-000-employees-in-8-weeks-w-mcp-angie-jones/
Block (the company behind Square and Cash App) became an early adopter of Model Context Protocol (MCP) and built Goose, their open-source AI agent that's now a reference implementation for the Agentic AI Foundation. Angie shares the challenges they faced, the security guardrails they built, and why letting employees choose their own models was critical to adoption.
We also dive into vibe coding (including Angie's experience watching Jack Dorsey vibe code a feature in 2 hours), how non-engineers are building their own tools, and what MCP unlocks when you connect multiple systems together.
Chapters:
00:00 Introduction
02:02 How Block deployed AI agents to 12,000 employees
05:04 Challenges with MCP adoption and security at scale
07:10 Why Block supports multiple AI models (Claude, GPT, Gemini)
08:40 Open source models and local LLM usage
09:58 Measuring velocity gains across the organization
10:49 Vibe coding: Benefits, risks & Jack Dorsey's 2-hour feature build
13:46 Block's contributions to the MCP protocol
14:38 MCP in action: Incident management + GitHub workflow demo
15:52 Addressing MCP criticism and security concerns
18:41 The Agentic AI Foundation announcement (Block, Anthropic, OpenAI, Google, Microsoft)
21:46 AI democratization: Non-engineers building MCP servers
24:11 How to get started with MCP and prompting tips
25:42 Security guardrails for enterprise AI deployment
29:25 Tool annotations and human-in-the-loop controls
30:22 OAuth and authentication in Goose
32:11 Use cases: Engineering, data analysis, fraud detection
35:22 Goose in Slack: Bug detection and PR creation in 5…
-
Gemini 3 & Robot Dogs: Inside Google DeepMind's AI Experiments | Paige Bailey
14 januari 2026 · 51 min
Google DeepMind is reshaping the AI landscape with an unprecedented wave of releases—from Gemini 3 to robotics and even data centers in space. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/47-gemini-3-and-robot-dogs-inside-google-deepminds-ai-experiments-paige-bailey/
Paige Bailey, AI Developer Relations Lead at Google DeepMind, joins us to break down the full Google AI ecosystem. From her unique journey as a geophysicist-turned-AI-leader who helped ship GitHub Copilot, to now running developer experience for DeepMind's entire platform, Paige offers an insider's view of how Google is thinking about the future of AI.
The conversation covers the practical differences between Gemini 3 Pro and Flash, when to use the open-source Gemma models, and how tools like Anti-Gravity IDE, Jules, and Gemini CLI fit into developer workflows. Paige also demonstrates Space Math Academy—a gamified NASA curriculum she built using AI Studio, Colab, and Anti-Gravity—showing how modern AI tools enable rapid prototyping.
The discussion then ventures into AI's physical frontier: robotics powered by Gemini on Raspberry Pi, Google's robotics trusted tester program, and the ambitious Project Suncatcher exploring data centers in space.
00:00 Introduction
01:30 Paige's Background & Connection to Modular
02:29 Gemini Integration Across Google Products
03:04 Jules, Gemini CLI & Anti-Gravity IDE Overview
03:48 Gemini 3 Flash vs Pro: Live Demo & Pricing
06:10 Choosing the Right Gemini Model
09:42 Google's Hardware Advantage: TPUs & JAX
10:16 TensorFlow History & Evolution to JAX
11:45 NeurIPS 2025 & Google's Research Culture
14:40 Google Brain to DeepMind: The Merger Story
15:24 Palm II to Gemini: Scaling from 40 People
18:42 Gemma Open Source Models
20:46 Anti-Gravity IDE Deep Dive
23:53 MCP Protocol & Chrome DevTools Integration
26:57 Gemini CLI in Google Colab
28:00 Image Generation & AI Studio Traffic Spikes
28:46 Space Math Academy: Gamified NASA Curricu…
-
Explaining Eval Engineering | Galileo's Vikram Chatterji
19 december 2025 · 37 min
You've heard of evaluations—but eval engineering is the difference between AI that ships and AI that's stuck in prototype. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/46-explaining-eval-engineering-galileos-vikram-chatterji/
Most teams still treat evals like unit tests: write them once, check a box, move on. But when you're deploying agents that make real decisions, touch real customers, and cost real money, those one-time tests don't cut it. The companies actually shipping production AI at scale have figured out something different—they've turned evaluations into infrastructure, into IP, into the layer where domain expertise becomes executable governance.
Vikram Chatterji, CEO and Co-founder of Galileo, returns to Chain of Thought to break down eval engineering: what it is, why it's becoming a dedicated discipline, and what it takes to actually make it work. Vikram shares why generic evals are plateauing, how continuous learning loops drive accuracy, and why he predicts "eval engineer" will become as common a role as "prompt engineer" once was.
In this conversation, Conor and Vikram explore:
Why treating evals as infrastructure—not checkboxes—separates production AI from prototypes
The plateau problem: why generic LLM-as-a-judge metrics can't break 90% accuracy
How continuous human feedback loops improve eval precision over time
The emerging "eval engineer" role and what the job actually looks like
Why 60-70% of AI engineers' time is already spent on evals
What multi-agent systems mean for the future of evaluation
Vikram's framework for baking trust AND control into agentic applications
Plus: Conor shares news about his move to Modular and what it means for Chain of Thought going forward.
Chapters:00:00 – Introduction: Why Evals Are Becoming IP01:37 – What Is Eval Engineering?04:24 – The Eval Engineering Course for Developers05:24 – Generic Evals Are Plateauing08:21 – Continuous Learning and Human Feedback11:01 – Human Feedback L…
-
Debunking AI's Environmental Panic | Andy Masley
26 november 2025 · 59 min
AI is destroying the planet—or so we've been told. This week on Chain of Thought, we tackle one of the most persistent and misleading narratives in the AI conversation. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/45-debunking-ais-environmental-panic-andy-masley/
Andy Masley, Director of Effective Altruism DC, joins host Conor Bronsdon to fact-check the absurd AI environmental claims you've heard at parties, in articles, and even in bestselling books. Andy recently went viral for discovering what he calls "the single most egregious math mistake" he's ever seen in a book—a data center water usage calculation in Karen Hao's NYT Bestseller, Empire of AI, that was off by a factor of 4,500.
In this conversation, Andy and Conor break down the myths around AI’s water and energy usage and explore:
The viral Empire of AI error and what it reveals about the broader debate
Why most AI water usage statistics are misleading or flat-out wrong
How one ChatGPT prompt represents just 1/150,000th of your daily emissions
Trade-offs around data center cooling + decision making
Why "tribal thinking" about AI is distorting environmental activism
Where AI might actually help the climate through deep learning optimization
If you've ever felt guilty about using AI tools, been cornered at a party about AI's environmental impact, or simply want to understand what the data actually says, this episode, and Andy’s deep dive articles, arm you with the facts.
Chapters:
00:00 – Introduction: The Party Guilt Problem
01:54 – Andy's Background and What Sparked This Work
03:50 – The 4,500x Error in Empire of AI
06:39 – Breaking Down the Math: Liters vs. Cubic Meters
10:39 – The Unintended Consequence: Air Cooling vs. Water Cooling
12:51 – Karen Hao's Response and What's Still Missing
19:08 – Why Environmentalists Should Focus Elsewhere
21:41 – The Danger of Tribal Thinking About AI
25:49 – What Is Effective Altruism (And Why People Attack It)
29:15 – EA, AI Ri…
-
The Critical Infrastructure Behind the AI Boom | Cisco CPO Jeetu Patel
19 november 2025 · 1 uur 18 min
AI is accelerating at a breakneck pace, but model quality isn’t the only constraint we face.. There are major infrastructure requirements, energy needs, security, and data pipelines to run AI at scale. This week on Chain of Thought, Cisco’s President and Chief Product Officer Jeetu Patel joins host Conor Bronsdon to reveal what it actually takes to build the critical foundation for the AI era. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/44-the-critical-infrastructure-behind-the-ai-boom-cisco-cpo-jeetu-patel/
Jeetu breaks down the three bottlenecks he sees holding AI back today:
• Infrastructure limits: not enough power, compute, or data center capacity
• A trust deficit: non-deterministic models powering systems that must be predictable
• A widening data gap: human-generated data plateauing while machine data explodes
Jeetu then shares how Cisco is tackling these challenges through secure AI factories, edge inference, open multi-model architectures, and global partnerships with Nvidia, G42, and sovereign cloud providers. Jeetu also explains why he thinks enterprises will soon rely on thousands of specialized models — not just one — and how routing, latency, cost, and security shape this new landscape.
Conor and Jeetu also explore high-performance leadership and team culture, discussing building high-trust teams, embracing constructive tension, staying vigilant in moments of success, and the personal experiences that shaped Jeetu’s approach to innovation and resilience.
If you want a clearer picture of the global AI infrastructure race, how high-level leaders are thinking about the future, and what it all means for enterprises, developers, and the future of work, this conversation is essential.
Chapters:
00:00 – Welcome to Chain of Thought
0:48 - AI and Jobs: Beyond the Hype
6:15 - The Real AI Opportunity: Original Insights
10:00 - Three Critical AI Constraints: Infrastructure, Trust, and Data
16:27 - Cisco's AI Strategy and Platf…
-
Beyond Transformers: How Liquid AI Is Rethinking LLM Architecture | Maxime Labonne
12 november 2025 · 53 min
The transformer architecture has dominated AI since 2017, but it’s not the only approach to building LLMs - and new architectures are bringing LLMs to edge devices Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/43-beyond-transformers-how-liquid-ai-is-rethinking-llm-architecture-maxime-labonne/
Maxime Labonne, Head of Post-Training at Liquid AI and creator of the 67,000+ star LLM Course, joins Conor Bronsdon to challenge the AI architecture status quo. Liquid AI’s hybrid architecture, combining transformers with convolutional layers, delivers faster inference, lower latency, and dramatically smaller footprints without sacrificing capability.
This alternative architectural philosophy creates models that run effectively on phones and laptops without compromise.
But reimagined architecture is only half the story. Maxime unpacks the post-training reality most teams struggle with: challenges and opportunities of synthetic data, how to balance helpfulness against safety, Liquid AI’s approach to evals, RAG architectural approaches, how he sees AI on edge devices evolving, hard won lessons from shipping LFM1 through 2, and much more.
If you're tired of surface-level AI takes and want to understand the architectural and engineering decisions behind production LLMs from someone building them in the trenches, this is your episode.
Connect with Maxime Labonne :
LinkedIn – https://www.linkedin.com/in/maxime-labonne/
X (Twitter) – @maximelabonne
About Maxime – https://mlabonne.github.io/blog/about.html
HuggingFace – https://huggingface.co/mlabonne
The LLM Course – https://github.com/mlabonne/llm-course
Liquid AI – https://liquid.ai
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon
00:00 Intro — Welcome to Chain of Thought
00:27 Guest Intro — Maxime La…
-
Architecting AI Agents: The Shift from Models to Systems | Aishwarya Srinivasan
8 oktober 2025 · 53 min
Most AI agents are built backwards, starting with models instead of system architecture. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/42-architecting-ai-agents-the-shift-from-models-to-systems-aishwarya-srinivasan/
Aishwarya Srinivasan, Head of AI Developer Relations at Fireworks AI, joins host Conor Bronsdon to explain the shift required to build reliable agents: stop treating them as model problems and start architecting them as complete software systems. Benchmarks alone won't save you.
Aish breaks down the evolution from prompt engineering to context engineering, revealing how production agents demand careful orchestration of multiple models, memory systems, and tool calls. She shares battle-tested insights on evaluation-driven development, the rise of open source models like DeepSeek v3, and practical strategies for managing autonomy with human-in-the-loop systems. The conversation addresses critical production challenges, ranging from LLM-as-judge techniques to navigating compliance in regulated environments.
Connect with Aishwarya Srinivasan:
LinkedIn: https://www.linkedin.com/in/aishwarya-srinivasan/
Instagram: https://www.instagram.com/the.datascience.gal/
Connect with Chain of Thought host Conor Bronsdon: https://www.linkedin.com/in/conorbronsdon/
00:00 Intro — Welcome to Chain of Thought
00:22 Guest Intro — Ash Srinivasan of Fireworks AI
02:37 The Challenge of Responsible AI
05:44 The Hidden Risks of Reward Hacking
07:22 From Prompt to Context Engineering
10:14 Data Quality and Human Feedback
14:43 Quantifying Trust and Observability
20:27 Evaluation-Driven Development
30:10 Open Source Models vs. Proprietary Systems
34:56 Gaps in the Open-Source AI Stack
38:45 When to Use Different Models
45:36 Governance and Compliance in AI Systems
50:11 The Future of AI Builders
56:00 Closing Thoughts & Follow Ash Online
-
The Accidental Algorithm | Humans of AI Crossover with Writer's Melisa Russak
1 oktober 2025 · 21 min
This week, we're doing something special and sharing an episode from another podcast we love: The Humans of AI by our friends at Writer. We're huge fans of their work, and you might remember Writer's CEO, May Habib, from the inaugural episode of our own show. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/41-the-accidental-algorithm-humans-of-ai-crossover-with-writers-melisa-russak/
From The Humans of AI:
Learn how Melisa Russak, lead research scientist at WRITER, stumbled upon fundamental machine learning algorithms, completely unaware of existing research — twice. Her story reveals the power of approaching problems with fresh eyes and the innovative breakthroughs that can occur when constraints become catalysts for creativity.
Melisa explores the intersection of curiosity-driven research, accidental discovery, and systematic innovation, offering valuable insights into how WRITER is pushing the boundaries of enterprise AI. Tune in to learn how her journey from a math teacher in China to a pioneer in AI research illuminates the future of technological advancement.
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon
Follow Today's Guest(s)
Check out Writer’s YouTube channel to watch the full interviews.
Learn more about WRITER at writer.com.
Follow Melisa on LinkedIn
Follow May on LinkedIn
Check out Galileo
Try Galileo
Agent Leaderboard
-
After Code Gen: What Graphite Is Building for the Post-AI Dev Stack | Greg Foster
24 september 2025 · 55 min
The incredible velocity of AI coding tools has shifted the critical bottleneck in software development from code generation to code reviews. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/40-after-code-gen-what-graphite-is-building-for-the-post-ai-dev-stack-greg-foster/
Greg Foster, Co-Founder & CTO of Graphite, joins the conversation to explore this new reality, outlining the three waves of AI that are leading to autonomous agents spawning pull requests in the background. He argues that as AI automates the "inner loop" of writing code, the human-centric "outer loop"—reviewing, merging, and deploying—is now under immense pressure, demanding a complete rethinking of our tools and processes.
The conversation then gets tactical, with Greg detailing how a technique called "stacking" can break down large code changes into manageable units for both humans and AI. He also identifies an emerging hiring gap where experienced engineers with strong architectural context are becoming "lethal" with AI tools. This episode is an essential guide to navigating the new bottlenecks in software development and understanding the skills that will define the next generation of high-impact engineers.
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon
Follow Today's Guest(s)
Connect with Greg on LinkedIn
Follow Greg on X
Graphite Website: graphite.dev
Check out Galileo
Try Galileo
Agent Leaderboard
-
Vercel's Playbook for AI Agents: From Vibe Check to Production | Malte Ubl
10 september 2025 · 54 min
What’s the first step to building an enterprise-grade AI tool? Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/39-vercels-playbook-for-ai-agents-from-vibe-check-to-production-malte-ubl/
Malte Ubl, CTO of Vercel, joins us this week to share Vercel’s playbook for agents, explaining how agents are a new type of software for solving flexible tasks. He shares how Vercel's developer-first ecosystem, including tools like the AI SDK and AI Gateway, is designed to help teams move from a quick proof-of-concept to a trusted, production-ready application.
Malte explores the practicalities of production AI, from the importance of eval-driven development to debugging chaotic agents with robust tracing. He offers a critical lesson on security, explaining why prompt injection requires a totally different solution - tool constraint - than traditional threats like SQL injection. This episode is a deep dive into the infrastructure and mindset, from sandboxes to specialized SLMs, required to build the next generation of AI tools.
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon
Follow Today's Guest(s)
Connect with Malte on LinkedIn
Follow Malte on X (formerly Twitter)
Learn more about Vercel
Check out Galileo
Try Galileo
Agent Leaderboard
-
From Demo to Defensibility: How to Build an AI Business that Lasts | Aurimas Griciūnas
27 augustus 2025 · 52 min
The technological moat is eroding in the AI era, what new factors separate a successful startup from the rest? Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/38-from-demo-to-defensibility-how-to-build-an-ai-business-that-lasts-aurimas-grici-nas/
Aurimas Griciūnas, CEO of SwirlAI, joins the show to break down the realities of building in this new landscape. Startup success now hinges on speed, strong financial backing, or immediate distribution. Aurimas warns against the critical mistake of prioritizing shiny tools over fundamental engineering and the market gaps this creates.
Discover the new moats for AI companies, built on a culture of relentless execution, tight feedback loops, and the surprising skills that define today's most valuable engineers.The episode also looks to the future, with bold predictions about a slowdown in LLM leaps and the coming impact of coding agents and self-improving systems.
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon
Follow Today's Guest(s)
Connect with Aurimas on LinkedIn
Aurimas' Course: End-to-End AI Engineering Bootcamp
Check out Galileo
Try Galileo
Agent Leaderboard
-
Mindset Over Metrics: How to Approach AI Engineering | Hamel Husain
20 augustus 2025 · 42 min
As we enter the era of the AI engineer, the biggest challenge isn't technical - it's a shift in mindset. Hamel Husain, a leading AI consultant and luminary in the eval space, joins the podcast to explore the skills and processes needed to build reliable AI. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/37-mindset-over-metrics-how-to-approach-ai-engineering-hamel-husain/
Hamel explains why many teams relying on vanity dashboards and a "buffet of metrics" experience a false sense of security, which is no substitute for customized evals tailored to domain-specific risks. The solution? A disciplined process of error analysis, grounded in manually looking at data to identify real-world failures
This discussion is an essential guide to building the continuous learning loops and "experimentation mindset" required to take AI products from prototype to production with confidence. Listen to learn the playbook for building AI reliability, and derive qualitative insights from log data to build customized quantitative guardrails.
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon
Follow Today's Guest(s)
Connect with Hamel on LinkedIn
Follow Hamel on X/Twitter
Check out his blog: hamel.dev
Check out Galileo
Try Galileo
Agent Leaderboard
-
How AI Velocity is Rewriting the Rules for Engineering Leaders | ChatPRD's Claire Vo
13 augustus 2025 · 43 min
What if your next competitor is not a startup, but a solo builder on a side project shipping features faster than your entire team? Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/36-how-ai-velocity-is-rewriting-the-rules-for-engineering-leaders-chatprds-claire-vo/
For Claire Vo, that's not a hypothetical. As the founder of ChatPRD, formerly the Chief Product and Technology Officer at LaunchDarkly, and host of the How I AI podcast, she has a unique vantage point on the driving forces behind a new blueprint for success.
She argues that AI accountability must be driven from the top by an "AI czar" and reveals how a culture of experimentation is the key to overcoming organizational hesitancy. Drawing from her experience as a solo founder, she warns that for incumbents, the cost of moving slowly is the biggest threat and details how AI can finally be used to tackle legacy codebases. The conversation closes with bold predictions on the rise of the "super IC" - who can achieve top-tier impact and salary without managing a team - and the death of product management.
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon
Follow Today's Guest(s)
Connect with Claire on LinkedIn
Follow Claire on X/Twitter
Claire’s podcast How I AI
Check out Galileo
Try Galileo
Agent Leaderboard
-
Building an AI-Native Startup | GrowthX's Marcel Santilli
6 augustus 2025 · 24 min
How do you build an AI-native company to a $7M run rate in just six months? Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/35-building-an-ai-native-startup-growthxs-marcel-santilli/
According to Marcel Santilli, Founder and CEO of GrowthX, the secret isn't chasing the next frontier model, it's mastering the "messy middle." Drawing on his deep experience at Scale AI and Deepgram, Marcel joins host Conor Bronsdon to share his framework for building durable, customer-obsessed businesses.
Marcel argues that the most critical skills for the AI era aren't technical but philosophical: first-principles thinking and the art of delegation.
Tune in to learn why GrowthX first focused on services to codify expert work, how AI can augment human talent instead of replacing it, and why speed and brand are a startup's greatest competitive advantages. This conversation offers a clear playbook for building a resilient company by prioritizing culture and relentless shipping.
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon
Follow Today's Guest(s)
Connect with Marcel on LinkedIn
Follow Marcel on X (formerly Twitter)
Learn more about GrowthX
Check out Galileo
Try Galileo
Agent Leaderboard
-
AI's Trillion-Dollar Healthcare Bet | Corti's Andreas Cleve
30 juli 2025 · 47 min
AI isn't just changing healthcare; it's providing the essential help needed to unlock a trillion-dollar opportunity for better care. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/34-ais-trillion-dollar-healthcare-bet-cortis-andreas-cleve/
Andreas Cleve, CEO & Co-founder of Corti, steps in to shed light on AI's immense, yet often misunderstood, transformative potential in this high-stakes environment. Andreas refutes the narrative of healthcare being slow adopters, emphasizing its high bar for trustworthy technology and its constant embrace of new tools. He reveals how purpose-built AI models are already alleviating the "pajama time" burden of documentation for clinicians, enabling faster and more accurate assessments in various specializations. This quiet, impactful adoption is seeing companies grow "like weeds" beyond common expectations.
The conversation addresses how AI can tackle the looming global shortage of 10 million healthcare professionals by 2030, reallocating a trillion dollars worth of administrative work back into care. Andreas details Corti’s approach to building invisible, reliable AI through rigorous, compliance-first evaluation, ensuring accuracy and efficiency in real-time. He emphasizes that AI's true role is not replacement, but augmentation, empowering professionals to deliver more care, attract talent, and drive organizational growth.
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon
Follow Today's Guest(s)
LinkedIn: linkedin.com/in/andreascleve
X (formerly Twitter): andreascleve
Corti Website: corti.ai
Check out Galileo
Try Galileo
Agent Leaderboard
-
Mastering Multi-Agent Systems | MongoDB’s Mikiko Chandrasekhar
23 juli 2025 · 40 min
AI agents offer unprecedented power, but mastering agent reliability is the ultimate challenge for agentic systems to actually work in production. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/33-mastering-multi-agent-systems-mongodbs-mikiko-chandrasekhar/
Mikiko Chandrashekar, Staff Developer Advocate at MongoDB, whose background spans the entire data-to-AI pipeline, unveils MongoDB's vision as the memory store for agents, supporting complex multi-agent systems from data storage and vector search to debugging chat logs. She highlights how MongoDB, reinforced by the acquisition of Voyage, empowers developers to build production-scale agents across various industries, from solo projects to major enterprises. This robust data layer is foundational to ensure agent performance and improve the end user experience.
Mikiko advocates for treating agents as software products, applying rigorous engineering best practices to ensure reliability, even for non-deterministic systems. She details MongoDB's unique position to balance GPU/CPU loads and manage data for performance and observability, including Galileo's integrations.
The conversation emphasizes the profound need to rethink observability, evaluations, and guardrails in the era of agents, showcasing Galileo's family of small language models for real-time guardrailing, Luna-2, and Insights Engine for automated failure analysis. Discover how building trustworthiness through systematic evaluation, beyond just "vibe checks," is essential for AI agents to scale and deliver value in high-stakes use cases.
Connect with Chain of Thought host Conor Bronsdon:
Newsletter: https://newsletter.chainofthought.show/
Twitter/X: https://x.com/ConorBronsdon
LinkedIn: https://www.linkedin.com/in/conorbronsdon/
YouTube: https://www.youtube.com/@ConorBronsdon
Follow Today's Guest(s)
Connect with Mikiko on LinkedIn
Follow Mikiko on X/Twitter
Explore Mikiko's YouTube channel
Check out Mikiko's Substack
Connec…
-
The AI Agent Trust Gap: Bridging Risk to Reliability | Elastic’s Philipp Krenn
16 juli 2025 · 44 min
The age of ubiquitous AI agents is here, bringing immense potential - and unprecedented risk. Lees meer
Full transcript and chapters: https://chainofthought.show/podcast/32-the-ai-agent-trust-gap-bridging-risk-to-reliability-elastics-philipp-krenn/
Hosts Conor Bronsdon and Vikram Chatterji open the episode by discussing the urgent need for building trust and reliability into next-generation AI agents. Vikram unveils Galileo's free AI reliability platform for agents, featuring Luna 2 SLMs for real-time guardrails and its Insights Engine for automatic failure mode analysis. This platform enables cost-effective, low-latency production evaluations, significantly transforming debugging. Achieving trustworthy AI agents demands rigorous testing, continuous feedback, and robust guardrailing—complex challenges requiring powerful solutions from partners like Elastic.
Conor welcomes Philipp Krenn, Director of Developer Relations at Elastic, to discuss their collaboration in ensuring AI agent reliability, including how Elastic leverages Galileo's platform for evaluation. Philipp details Elastic's evolution from a search powerhouse to a key AI enabler, transforming data access with Retrieval-Augmented Generation (RAG) and new interaction modes. He discusses Elastic's investment in SLMs for efficient re-ranking and embeddings, emphasizing robust evaluation and observability for production. This collaborative effort aims to equip developers to build reliable, high-performing AI systems for every enterprise.
Chapters:
00:00 Introduction
01:09 Galileo's AI Reliability Platform
01:43 Challenges in AI Agent Reliability
06:17 Insights Engine and Its Importance
11:00 Luna 2: Small Language Models
14:42 Custom Metrics and Agent Leaderboard
19:16 Galileo's Integrations and Partnerships
21:04 Philipp Krenn from Elastic
24:47 Optimizing LLM Responses
25:41 Galileo and Elastic: A Powerful Partnership
28:20 Challenges in AI Production and Trust
30:02 Guardrails and Reliability in AI Sy…