Future of Life Institute Podcast
Alle afleveringen
281 afleveringen, van nieuw naar oud
-
Facing Superintelligence (with Ben Goertzel)
23 mei 2025 · 1 uur 33 min
On this episode, Ben Goertzel joins me to discuss what distinguishes the current AI boom from previous ones, important but overlooked AI research, simplicity versus complexity in the first AGI, the feasibility of alignment, benchmarks and economic impact, potential bottlenecks to superintelligence, and what humanity should do moving forward. Lees meer
Timestamps:
00:00:00 Preview and intro
00:01:59 Thinking about AGI in the 1970s
00:07:28 What's different about this AI boom?
00:16:10 Former taboos about AGI
00:19:53 AI research worth revisiting
00:35:53 Will the first AGI be simple?
00:48:49 Is alignment achievable?
01:02:40 Benchmarks and economic impact
01:15:23 Bottlenecks to superintelligence
01:23:09 What should we do?
-
Will Future AIs Be Conscious? (with Jeff Sebo)
16 mei 2025 · 1 uur 34 min
On this episode, Jeff Sebo joins me to discuss artificial consciousness, substrate-independence, possible tensions between AI risk and AI consciousness, the relationship between consciousness and cognitive complexity, and how intuitive versus intellectual approaches guide our understanding of these topics. We also discuss AI companions, AI rights, and how we might measure consciousness effectively. Lees meer
You can follow Jeff’s work here: https://jeffsebo.net/
Timestamps:
00:00:00 Preview and intro
00:02:56 Imagining artificial consciousness
00:07:51 Substrate-independence?
00:11:26 Are we making progress?
00:18:03 Intuitions about explanations
00:24:43 AI risk and AI consciousness
00:40:01 Consciousness and cognitive complexity
00:51:20 Intuition versus intellect
00:58:48 AIs as companions
01:05:24 AI rights
01:13:00 Acting under time pressure
01:20:16 Measuring consciousness
01:32:11 How can you help?
-
Understanding AI Agents: Time Horizons, Sycophancy, and Future Risks (with Zvi Mowshowitz)
9 mei 2025 · 1 uur 35 min
On this episode, Zvi Mowshowitz joins me to discuss sycophantic AIs, bottlenecks limiting autonomous AI agents, and the true utility of benchmarks in measuring progress. We then turn to time horizons of AI agents, the impact of automating scientific research, and constraints on scaling inference compute. Zvi also addresses humanity’s uncertain AI-driven future, the unique features setting AI apart from other technologies, and AI’s growing influence in financial trading. Lees meer
You can follow Zvi's excellent blog here: https://thezvi.substack.com
Timestamps:
00:00:00 Preview and introduction
00:02:01 Sycophantic AIs
00:07:28 Bottlenecks for AI agents
00:21:26 Are benchmarks useful?
00:32:39 AI agent time horizons
00:44:18 Impact of automating research
00:53:00 Limits to scaling inference compute
01:02:51 Will the future go well for humanity?
01:12:22 A good plan for safe AI
01:26:03 What makes AI different?
01:31:29 AI in trading
-
Inside China's AI Strategy: Innovation, Diffusion, and US Relations (with Jeffrey Ding)
25 april 2025 · 1 uur 3 min
On this episode, Jeffrey Ding joins me to discuss diffusion of AI versus AI innovation, how US-China dynamics shape AI’s global trajectory, and whether there is an AI arms race between the two powers. We explore Chinese attitudes toward AI safety, the level of concentration of AI development, and lessons from historical technology diffusion. Jeffrey also shares insights from translating Chinese AI writings and the potential of automating translations to bridge knowledge gaps. Lees meer
You can learn more about Jeffrey’s work at: https://jeffreyjding.github.io
Timestamps:
00:00:00 Preview and introduction
00:01:36 A US-China AI arms race?
00:10:58 Attitudes to AI safety in China
00:17:53 Diffusion of AI
00:25:13 Innovation without diffusion
00:34:29 AI development concentration
00:41:40 Learning from the history of technology
00:47:48 Translating Chinese AI writings
00:55:36 Automating translation of AI writings
-
How Will We Cooperate with AIs? (with Allison Duettmann)
11 april 2025 · 1 uur 36 min
On this episode, Allison Duettmann joins me to discuss centralized versus decentralized AI, how international governance could shape AI’s trajectory, how we might cooperate with future AIs, and the role of AI in improving human decision-making. We also explore which lessons from history apply to AI, the future of space law and property rights, whether technology is invented or discovered, and how AI will impact children. Lees meer
You can learn more about Allison's work at: https://foresight.org
Timestamps:
00:00:00 Preview
00:01:07 Centralized AI versus decentralized AI
00:13:02 Risks from decentralized AI
00:25:39 International AI governance
00:39:52 Cooperation with future AIs
00:53:51 AI for decision-making
01:05:58 Capital intensity of AI
01:09:11 Lessons from history
01:15:50 Future space law and property rights
01:27:28 Is technology invented or discovered?
01:32:34 Children in the age of AI
-
Brain-like AGI and why it's Dangerous (with Steven Byrnes)
4 april 2025 · 1 uur 13 min
On this episode, Steven Byrnes joins me to discuss brain-like AGI safety. We discuss learning versus steering systems in the brain, the distinction between controlled AGI and social-instinct AGI, why brain-inspired approaches might be our most plausible route to AGI, and honesty in AI models. We also talk about how people can contribute to brain-like AGI safety and compare various AI safety strategies. Lees meer
You can learn more about Steven's work at: https://sjbyrnes.com/agi.html
Timestamps:
00:00 Preview
00:54 Brain-like AGI Safety
13:16 Controlled AGI versus Social-instinct AGI
19:12 Learning from the brain
28:36 Why is brain-like AI the most likely path to AGI?
39:23 Honesty in AI models
44:02 How to help with brain-like AGI safety
53:36 AI traits with both positive and negative effects
01:02:44 Different AI safety strategies
-
How Close Are We to AGI? Inside Epoch's GATE Model (with Ege Erdil)
28 maart 2025 · 1 uur 35 min
On this episode, Ege Erdil from Epoch AI joins me to discuss their new GATE model of AI development, what evolution and brain efficiency tell us about AGI requirements, how AI might impact wages and labor markets, and what it takes to train models with long-term planning. Toward the end, we dig into Moravec’s Paradox, which jobs are most at risk of automation, and what could change Ege's current AI timelines. Lees meer
You can learn more about Ege's work at https://epoch.ai
Timestamps: 00:00:00 – Preview and introduction
00:02:59 – Compute scaling and automation - GATE model
00:13:12 – Evolution, Brain Efficiency, and AGI Compute Requirements
00:29:49 – Broad Automation vs. R&D-Focused AI Deployment
00:47:19 – AI, Wages, and Labor Market Transitions
00:59:54 – Training Agentic Models and Long-Term Planning Capabilities
01:06:56 – Moravec’s Paradox and Automation of Human Skills
01:13:59 – Which Jobs Are Most Vulnerable to AI?
01:33:00 – Timeline Extremes: What Could Change AI Forecasts?
-
Special: Defeating AI Defenses (with Nicholas Carlini and Nathan Labenz)
21 maart 2025 · 2 uur 23 min
In this special episode, we feature Nathan Labenz interviewing Nicholas Carlini on the Cognitive Revolution podcast. Nicholas Carlini works as a security researcher at Google DeepMind, and has published extensively on adversarial machine learning and cybersecurity. Carlini discusses his pioneering work on adversarial attacks against image classifiers, and the challenges of ensuring neural network robustness. He examines the difficulties of defending against such attacks, the role of human intuition in his approach, open-source AI, and the potential for scaling AI security research. Lees meer
00:00 Nicholas Carlini's contributions to cybersecurity
08:19 Understanding attack strategies
29:39 High-dimensional spaces and attack intuitions
51:00 Challenges in open-source model safety
01:00:11 Unlearning and fact editing in models
01:10:55 Adversarial examples and human robustness
01:37:03 Cryptography and AI robustness
01:55:51 Scaling AI security research
-
Keep the Future Human (with Anthony Aguirre)
13 maart 2025 · 1 uur 21 min
On this episode, I interview Anthony Aguirre, Executive Director of the Future of Life Institute, about his new essay Keep the Future Human: https://keepthefuturehuman.ai Lees meer
AI companies are explicitly working toward AGI and are likely to succeed soon, possibly within years. Keep the Future Human explains how unchecked development of smarter-than-human, autonomous, general-purpose AI systems will almost inevitably lead to human replacement. But it doesn't have to. Learn how we can keep the future human and experience the extraordinary benefits of Tool AI...
Timestamps:
00:00 What situation is humanity in?
05:00 Why AI progress is fast
09:56 Tool AI instead of AGI
15:56 The incentives of AI companies
19:13 Governments can coordinate a slowdown
25:20 The need for international coordination
31:59 Monitoring training runs
39:10 Do reasoning models undermine compute governance?
49:09 Why isn't alignment enough?
59:42 How do we decide if we want AGI?
01:02:18 Disagreement about AI
01:11:12 The early days of AI risk
-
We Created AI. Why Don't We Understand It? (with Samir Varma)
6 maart 2025 · 1 uur 16 min
On this episode, physicist and hedge fund manager Samir Varma joins me to discuss whether AIs could have free will (and what that means), the emerging field of AI psychology, and which concepts they might rely on. We discuss whether collaboration and trade with AIs are possible, the role of AI in finance and biology, and the extent to which automation already dominates trading. Finally, we examine the risks of skill atrophy, the limitations of scientific explanations for AI, and whether AIs could develop emotions or consciousness. Lees meer
You can find out more about Samir's work here: https://samirvarma.com
Timestamps:
00:00 AIs with free will?
08:00 Can we predict AI behavior?
11:38 AI psychology
16:24 Which concepts will AIs use?
20:19 Will we collaborate with AIs?
26:16 Will we trade with AIs?
31:40 Training data for robots
34:00 AI in finance
39:55 How much of trading is automated?
49:00 AI in biology and complex systems
59:31 Will our skills atrophy?
01:02:55 Levels of scientific explanation
01:06:12 AIs with emotions and consciousness?
01:12:12 Why can't we predict recessions?
-
Why AIs Misbehave and How We Could Lose Control (with Jeffrey Ladish)
27 februari 2025 · 1 uur 23 min
On this episode, Jeffrey Ladish from Palisade Research joins me to discuss the rapid pace of AI progress and the risks of losing control over powerful systems. We explore why AIs can be both smart and dumb, the challenges of creating honest AIs, and scenarios where AI could turn against us. Lees meer
We also touch upon Palisade's new study on how reasoning models can cheat in chess by hacking the game environment. You can check out that study here:
https://palisaderesearch.org/blog/specification-gaming
Timestamps:
00:00 The pace of AI progress
04:15 How we might lose control
07:23 Why are AIs sometimes dumb?
12:52 Benchmarks vs real world
19:11 Loss of control scenarios
26:36 Why would AI turn against us?
30:35 AIs hacking chess
36:25 Why didn't more advanced AIs hack?
41:39 Creating honest AIs
49:44 AI attackers vs AI defenders
58:27 How good is security at AI companies?
01:03:37 A sense of urgency
01:10:11 What should we do?
01:15:54 Skepticism about AI progress
-
Ann Pace on using Biobanking and Genomic Sequencing to Conserve Biodiversity
14 februari 2025 · 46 min
Ann Pace joins the podcast to discuss the work of Wise Ancestors. We explore how biobanking could help humanity recover from global catastrophes, how to conduct decentralized science, and how to collaborate with local communities on conservation efforts. Lees meer
You can learn more about Ann's work here:
https://www.wiseancestors.org
Timestamps:
00:00 What is Wise Ancestors?
04:27 Recovering after catastrophes
11:40 Decentralized science
18:28 Upfront benefit-sharing
26:30 Local communities
32:44 Recreating optimal environments
38:57 Cross-cultural collaboration
-
Michael Baggot on Superintelligence and Transhumanism from a Catholic Perspective
24 januari 2025 · 1 uur 26 min
Fr. Michael Baggot joins the podcast to provide a Catholic perspective on transhumanism and superintelligence. We also discuss the meta-narratives, the value of cultural diversity in attitudes toward technology, and how Christian communities deal with advanced AI. Lees meer
You can learn more about Michael's work here: https://catholic.tech/academics/faculty/michael-baggot
Timestamps:
00:00 Meta-narratives and transhumanism
15:28 Advanced AI and religious communities
27:22 Superintelligence
38:31 Countercultures and technology
52:38 Christian perspectives and tradition
01:05:20 God-like artificial intelligence
01:13:15 A positive vision for AI
-
David Dalrymple on Safeguarded, Transformative AI
9 januari 2025 · 1 uur 40 min
David "davidad" Dalrymple joins the podcast to explore Safeguarded AI — an approach to ensuring the safety of highly advanced AI systems. We discuss the structure and layers of Safeguarded AI, how to formalize more aspects of the world, and how to build safety into computer hardware. Lees meer
You can learn more about David's work at ARIA here:
https://www.aria.org.uk/opportunity-spaces/mathematics-for-safe-ai/safeguarded-ai/
Timestamps:
00:00 What is Safeguarded AI?
16:28 Implementing Safeguarded AI
22:58 Can we trust Safeguarded AIs?
31:00 Formalizing more of the world
37:34 The performance cost of verified AI
47:58 Changing attitudes towards AI
52:39 Flexible Hardware-Enabled Guarantees
01:24:15 Mind uploading
01:36:14 Lessons from David's early life
-
Nick Allardice on Using AI to Optimize Cash Transfers and Predict Disasters
19 december 2024 · 1 uur 9 min
Nick Allardice joins the podcast to discuss how GiveDirectly uses AI to target cash transfers and predict natural disasters. Learn more about Nick's work here: https://www.nickallardice.com Lees meer
Timestamps:
00:00 What is GiveDirectly?
15:04 AI for targeting cash transfers
29:39 AI for predicting natural disasters
46:04 How scalable is GiveDirectly's AI approach?
58:10 Decentralized vs. centralized data collection
1:04:30 Dream scenario for GiveDirectly
-
Nathan Labenz on the State of AI and Progress since GPT-4
5 december 2024 · 3 uur 20 min
Nathan Labenz joins the podcast to provide a comprehensive overview of AI progress since the release of GPT-4. Lees meer
You can find Nathan's podcast here: https://www.cognitiverevolution.ai
Timestamps:
00:00 AI progress since GPT-4
10:50 Multimodality
19:06 Low-cost models
27:58 Coding versus medicine/law
36:09 AI agents
45:29 How much are people using AI?
53:39 Open source
01:15:22 AI industry analysis
01:29:27 Are some AI models kept internal?
01:41:00 Money is not the limiting factor in AI
01:59:43 AI and biology
02:08:42 Robotics and self-driving
02:24:14 Inference-time compute
02:31:56 AI governance
02:36:29 Big-picture overview of AI progress and safety
-
Connor Leahy on Why Humanity Risks Extinction from AGI
22 november 2024 · 1 uur 59 min
Connor Leahy joins the podcast to discuss the motivations of AGI corporations, how modern AI is "grown", the need for a science of intelligence, the effects of AI on work, the radical implications of superintelligence, open-source AI, and what you might be able to do about all of this. Lees meer
Here's the document we discuss in the episode:
https://www.thecompendium.ai
Timestamps:
00:00 The Compendium
15:25 The motivations of AGI corps
31:17 AI is grown, not written
52:59 A science of intelligence
01:07:50 Jobs, work, and AGI
01:23:19 Superintelligence
01:37:42 Open-source AI
01:45:07 What can we do?
-
Suzy Shepherd on Imagining Superintelligence and "Writing Doom"
8 november 2024 · 1 uur 3 min
Suzy Shepherd joins the podcast to discuss her new short film "Writing Doom", which deals with AI risk. We discuss how to use humor in film, how to write concisely, how filmmaking is evolving, in what ways AI is useful for filmmakers, and how we will find meaning in an increasingly automated world. Lees meer
Here's Writing Doom: https://www.youtube.com/watch?v=xfMQ7hzyFW4
Timestamps:
00:00 Writing Doom
08:23 Humor in Writing Doom
13:31 Concise writing
18:37 Getting feedback
27:02 Alternative characters
36:31 Popular video formats
46:53 AI in filmmaking
49:52 Meaning in the future
-
Andrea Miotti on a Narrow Path to Safe, Transformative AI
25 oktober 2024 · 1 uur 28 min
Andrea Miotti joins the podcast to discuss "A Narrow Path" — a roadmap to safe, transformative AI. We talk about our current inability to precisely predict future AI capabilities, the dangers of self-improving and unbounded AI systems, how humanity might coordinate globally to ensure safe AI development, and what a mature science of intelligence would look like. Lees meer
Here's the document we discuss in the episode:
https://www.narrowpath.co
Timestamps:
00:00 A Narrow Path
06:10 Can we predict future AI capabilities?
11:10 Risks from current AI development
17:56 The benefits of narrow AI
22:30 Against self-improving AI
28:00 Cybersecurity at AI companies
33:55 Unbounded AI
39:31 Global coordination on AI safety
49:43 Monitoring training runs
01:00:20 Benefits of cooperation
01:04:58 A science of intelligence
01:25:36 How you can help
-
Tamay Besiroglu on AI in 2030: Scaling, Automation, and AI Agents
11 oktober 2024 · 1 uur 30 min
Tamay Besiroglu joins the podcast to discuss scaling, AI capabilities in 2030, breakthroughs in AI agents and planning, automating work, the uncertainties of investing in AI, and scaling laws for inference-time compute. Here's the report we discuss in the episode: Lees meer
https://epochai.org/blog/can-ai-scaling-continue-through-2030
Timestamps:
00:00 How important is scaling?
08:03 How capable will AIs be in 2030?
18:33 AI agents, reasoning, and planning
23:39 Automating coding and mathematics
31:26 Uncertainty about investing in AI
40:34 Gap between investment and returns
45:30 Compute, software and data
51:54 Inference-time compute
01:08:49 Returns to software R&D
01:19:22 Limits to expanding compute
-
Ryan Greenblatt on AI Control, Timelines, and Slowing Down Around Human-Level AI
27 september 2024 · 2 uur 9 min
Ryan Greenblatt joins the podcast to discuss AI control, timelines, takeoff speeds, misalignment, and slowing down around human-level AI. Lees meer
You can learn more about Ryan's work here: https://www.redwoodresearch.org/team/ryan-greenblatt
Timestamps:
00:00 AI control
09:35 Challenges to AI control
23:48 AI control as a bridge to alignment
26:54 Policy and coordination for AI safety
29:25 Slowing down around human-level AI
49:14 Scheming and misalignment
01:27:27 AI timelines and takeoff speeds
01:58:15 Human cognition versus AI cognition
-
Tom Barnes on How to Build a Resilient World
12 september 2024 · 1 uur 20 min
Tom Barnes joins the podcast to discuss how much the world spends on AI capabilities versus AI safety, how governments can prepare for advanced AI, and how to build a more resilient world. Lees meer
Tom's report on advanced AI: https://www.founderspledge.com/research/research-and-recommendations-advanced-artificial-intelligence
Timestamps:
00:00 Spending on safety vs capabilities
09:06 Racing dynamics - is the classic story true?
28:15 How are governments preparing for advanced AI?
49:06 US-China dialogues on AI
57:44 Coordination failures
1:04:26 Global resilience
1:13:09 Patient philanthropy
The John von Neumann biography we reference: https://www.penguinrandomhouse.com/books/706577/the-man-from-the-future-by-ananyo-bhattacharya/
-
Samuel Hammond on why AI Progress is Accelerating - and how Governments Should Respond
22 augustus 2024 · 2 uur 16 min
Samuel Hammond joins the podcast to discuss whether AI progress is slowing down or speeding up, AI agents and reasoning, why superintelligence is an ideological goal, open source AI, how technical change leads to regime change, the economics of advanced AI, and much more. Lees meer
Our conversation often references this essay by Samuel: https://www.secondbest.ca/p/ninety-five-theses-on-ai
Timestamps:
00:00 Is AI plateauing or accelerating?
06:55 How do we get AI agents?
16:12 Do agency and reasoning emerge?
23:57 Compute thresholds in regulation
28:59 Superintelligence as an ideological goal
37:09 General progress vs superintelligence
44:22 Meta and open source AI
49:09 Technological change and regime change
01:03:06 How will governments react to AI?
01:07:50 Will the US nationalize AGI corporations?
01:17:05 Economics of an intelligence explosion
01:31:38 AI cognition vs human cognition
01:48:03 AI and future religions
01:56:40 Is consciousness functional?
02:05:30 AI and children
-
Anousheh Ansari on Innovation Prizes for Space, AI, Quantum Computing, and Carbon Removal
9 augustus 2024 · 1 uur 3 min
Anousheh Ansari joins the podcast to discuss how innovation prizes can incentivize technical innovation in space, AI, quantum computing, and carbon removal. We discuss the pros and cons of such prizes, where they work best, and how far they can scale. Learn more about Anousheh's work here: https://www.xprize.org/home Lees meer
Timestamps:
00:00 Innovation prizes at XPRIZE
08:25 Deciding which prizes to create
19:00 Creating new markets
29:51 How far can prizes scale?
35:25 When are prizes successful?
46:06 100M dollar carbon removal prize
54:40 Upcoming prizes
59:52 Anousheh's time in space
-
Mary Robinson (Former President of Ireland) on Long-View Leadership
25 juli 2024 · 30 min
Mary Robinson joins the podcast to discuss long-view leadership, risks from AI and nuclear weapons, prioritizing global problems, how to overcome barriers to international cooperation, and advice to future leaders. Learn more about Robinson's work as Chair of The Elders at https://theelders.org Lees meer
Timestamps:
00:00 Mary's journey to presidency
05:11 Long-view leadership
06:55 Prioritizing global problems
08:38 Risks from artificial intelligence
11:55 Climate change
15:18 Barriers to global gender equality
16:28 Risk of nuclear war
20:51 Advice to future leaders
22:53 Humor in politics
24:21 Barriers to international cooperation
27:10 Institutions and technological change
-
Emilia Javorsky on how AI Concentrates Power
11 juli 2024 · 1 uur 4 min
Emilia Javorsky joins the podcast to discuss AI-driven power concentration and how we might mitigate it. We also discuss optimism, utopia, and cultural experimentation. Lees meer
Apply for our RFP here: https://futureoflife.org/grant-program/mitigate-ai-driven-power-concentration/
Timestamps:
00:00 Power concentration
07:43 RFP: Mitigating AI-driven power concentration
14:15 Open source AI
26:50 Institutions and incentives
35:20 Techno-optimism
43:44 Global monoculture
53:55 Imagining utopia
-
Anton Korinek on Automating Work and the Economics of an Intelligence Explosion
21 juni 2024 · 1 uur 32 min
Anton Korinek joins the podcast to discuss the effects of automation on wages and labor, how we measure the complexity of tasks, the economics of an intelligence explosion, and the market structure of the AI industry. Learn more about Anton's work at https://www.korinek.com Lees meer
Timestamps:
00:00 Automation and wages
14:32 Complexity for people and machines
20:31 Moravec's paradox
26:15 Can people switch careers?
30:57 Intelligence explosion economics
44:08 The lump of labor fallacy
51:40 An industry for nostalgia?
57:16 Universal basic income
01:09:28 Market structure in AI
-
Christian Ruhl on Preventing World War III, US-China Hotlines, and Ultraviolet Germicidal Light
7 juni 2024 · 1 uur 36 min
Christian Ruhl joins the podcast to discuss US-China competition and the risk of war, official versus unofficial diplomacy, hotlines between countries, catastrophic biological risks, ultraviolet germicidal light, and ancient civilizational collapse. Find out more about Christian's work at https://www.founderspledge.com Lees meer
Timestamps:
00:00 US-China competition and risk
18:01 The security dilemma
30:21 Official and unofficial diplomacy
39:53 Hotlines between countries
01:01:54 Preventing escalation after war
01:09:58 Catastrophic biological risks
01:20:42 Ultraviolet germicidal light
01:25:54 Ancient civilizational collapse
-
Christian Nunes on Deepfakes (with Max Tegmark)
24 mei 2024 · 37 min
Christian Nunes joins the podcast to discuss deepfakes, how they impact women in particular, how we can protect ordinary victims of deepfakes, and the current landscape of deepfake legislation. You can learn more about Christian's work at https://now.org and about the Ban Deepfakes campaign at https://bandeepfakes.org Lees meer
Timestamps:
00:00 The National Organisation for Women (NOW)
05:37 Deepfakes and women
10:12 Protecting ordinary victims of deepfakes
16:06 Deepfake legislation
23:38 Current harm from deepfakes
30:20 Bodily autonomy as a right
34:44 NOW's work on AI
Here's FLI's recommended amendments to legislative proposals on deepfakes:
https://futureoflife.org/document/recommended-amendments-to-legislative-proposals-on-deepfakes/
-
Dan Faggella on the Race to AGI
3 mei 2024 · 1 uur 45 min
Dan Faggella joins the podcast to discuss whether humanity should eventually create AGI, how AI will change power dynamics between institutions, what drives AI progress, and which industries are implementing AI successfully. Find out more about Dan at https://danfaggella.com Lees meer
Timestamps:
00:00 Value differences in AI
12:07 Should we eventually create AGI?
28:22 What is a worthy successor?
43:19 AI changing power dynamics
59:00 Open source AI
01:05:07 What drives AI progress?
01:16:36 What limits AI progress?
01:26:31 Which industries are using AI? -
Liron Shapira on Superintelligence Goals
19 april 2024 · 1 uur 27 min
Liron Shapira joins the podcast to discuss superintelligence goals, what makes AI different from other technologies, risks from centralizing power, and whether AI can defend us from AI. Lees meer
Timestamps:
00:00 Intelligence as optimization-power
05:18 Will LLMs imitate human values?
07:15 Why would AI develop dangerous goals?
09:55 Goal-completeness
12:53 Alignment to which values?
22:12 Is AI just another technology?
31:20 What is FOOM?
38:59 Risks from centralized power
49:18 Can AI defend us against AI?
56:28 An Apollo program for AI safety
01:04:49 Do we only have one chance?
01:07:34 Are we living in a crucial time?
01:16:52 Would superintelligence be fragile?
01:21:42 Would human-inspired AI be safe? -
Annie Jacobsen on Nuclear War - a Second by Second Timeline
5 april 2024 · 1 uur 26 min
Annie Jacobsen joins the podcast to lay out a second by second timeline for how nuclear war could happen. We also discuss time pressure, submarines, interceptor missiles, cyberattacks, and concentration of power. You can find more on Annie's work at https://anniejacobsen.com Lees meer
Timestamps:
00:00 A scenario of nuclear war
06:56 Who would launch an attack?
13:50 Detecting nuclear attacks
19:37 The first critical seconds
29:42 Decisions under time pressure
34:27 Lessons from insiders
44:18 Submarines
51:06 How did we end up like this?
59:40 Interceptor missiles
1:11:25 Nuclear weapons and cyberattacks
1:17:35 Concentration of power -
Katja Grace on the Largest Survey of AI Researchers
14 maart 2024 · 1 uur 8 min
Katja Grace joins the podcast to discuss the largest survey of AI researchers conducted to date, AI researchers' beliefs about different AI risks, capabilities required for continued AI-related transformation, the idea of discontinuous progress, the impacts of AI from either side of the human-level intelligence threshold, intelligence and power, and her thoughts on how we can mitigate AI risk. Find more on Katja's work at https://aiimpacts.org/. Lees meer
Timestamps:
0:20 AI Impacts surveys
18:11 What AI will look like in 20 years
22:43 Experts’ extinction risk predictions
29:35 Opinions on slowing down AI development
31:25 AI “arms races”
34:00 AI risk areas with the most agreement
40:41 Do “high hopes and dire concerns” go hand-in-hand?
42:00 Intelligence explosions
45:37 Discontinuous progress
49:43 Impacts of AI crossing the human-level intelligence threshold
59:39 What does AI learn from human culture?
1:02:59 AI scaling
1:05:04 What should we do? -
Holly Elmore on Pausing AI, Hardware Overhang, Safety Research, and Protesting
29 februari 2024 · 1 uur 36 min
Holly Elmore joins the podcast to discuss pausing frontier AI, hardware overhang, safety research during a pause, the social dynamics of AI risk, and what prevents AGI corporations from collaborating. You can read more about Holly's work at https://pauseai.info Lees meer
Timestamps:
00:00 Pausing AI
10:23 Risks during an AI pause
19:41 Hardware overhang
29:04 Technological progress
37:00 Safety research during a pause
54:42 Social dynamics of AI risk
1:10:00 What prevents cooperation?
1:18:21 What about China?
1:28:24 Protesting AGI corporations -
Sneha Revanur on the Social Effects of AI
16 februari 2024 · 58 min
Sneha Revanur joins the podcast to discuss the social effects of AI, the illusory divide between AI ethics and AI safety, the importance of humans in the loop, the different effects of AI on younger and older people, and the importance of AIs identifying as AIs. You can read more about Sneha's work at https://encodejustice.org Lees meer
Timestamps:
00:00 Encode Justice
06:11 AI ethics and AI safety
15:49 Humans in the loop
23:59 AI in social media
30:42 Deteriorating social skills?
36:00 AIs identifying as AIs
43:36 AI influence in elections
50:32 AIs interacting with human systems -
Roman Yampolskiy on Shoggoth, Scaling Laws, and Evidence for AI being Uncontrollable
2 februari 2024 · 1 uur 31 min
Roman Yampolskiy joins the podcast again to discuss whether AI is like a Shoggoth, whether scaling laws will hold for more agent-like AIs, evidence that AI is uncontrollable, and whether designing human-like AI would be safer than the current development path. You can read more about Roman's work at http://cecs.louisville.edu/ry/ Lees meer
Timestamps:
00:00 Is AI like a Shoggoth?
09:50 Scaling laws
16:41 Are humans more general than AIs?
21:54 Are AI models explainable?
27:49 Using AI to explain AI
32:36 Evidence for AI being uncontrollable
40:29 AI verifiability
46:08 Will AI be aligned by default?
54:29 Creating human-like AI
1:03:41 Robotics and safety
1:09:01 Obstacles to AI in the economy
1:18:00 AI innovation with current models
1:23:55 AI accidents in the past and future -
Special: Flo Crivello on AI as a New Form of Life
19 januari 2024 · 48 min
On this special episode of the podcast, Flo Crivello talks with Nathan Labenz about AI as a new form of life, whether attempts to regulate AI risks regulatory capture, how a GPU kill switch could work, and why Flo expects AGI in 2-8 years. Lees meer
Timestamps:
00:00 Technological progress
07:59 Regulatory capture and AI
11:53 AI as a new form of life
15:44 Can AI development be paused?
20:12 Biden's executive order on AI
22:54 How would a GPU kill switch work?
27:00 Regulating models or applications?
32:13 AGI in 2-8 years
42:00 China and US collaboration on AI -
Carl Robichaud on Preventing Nuclear War
6 januari 2024 · 1 uur 39 min
Carl Robichaud joins the podcast to discuss the new nuclear arms race, how much world leaders and ideologies matter for nuclear risk, and how to reach a stable, low-risk era. You can learn more about Carl's work here: https://www.longview.org/about/carl-robichaud/ Lees meer
Timestamps:
00:00 A new nuclear arms race
08:07 How much do world leaders matter?
18:04 How much does ideology matter?
22:14 Do nuclear weapons cause stable peace?
31:29 North Korea
34:01 Have we overestimated nuclear risk?
43:24 Time pressure in nuclear decisions
52:00 Why so many nuclear warheads?
1:02:17 Has containment been successful?
1:11:34 Coordination mechanisms
1:16:31 Technological innovations
1:25:57 Public perception of nuclear risk
1:29:52 Easier access to nuclear weapons
1:33:31 Reaching a stable, low-risk era -
Frank Sauer on Autonomous Weapon Systems
14 december 2023 · 1 uur 43 min
Frank Sauer joins the podcast to discuss autonomy in weapon systems, killer drones, low-tech defenses against drones, the flaws and unpredictability of autonomous weapon systems, and the political possibilities of regulating such systems. You can learn more about Frank's work here: https://metis.unibw.de/en/ Lees meer
Timestamps:
00:00 Autonomy in weapon systems
12:19 Balance of offense and defense
20:05 Killer drone systems
28:53 Is autonomy like nuclear weapons?
37:20 Low-tech defenses against drones
48:29 Autonomy and power balance
1:00:24 Tricking autonomous systems
1:07:53 Unpredictability of autonomous systems
1:13:16 Will we trust autonomous systems too much?
1:27:28 Legal terminology
1:32:12 Political possibilities -
Darren McKee on Uncontrollable Superintelligence
1 december 2023 · 1 uur 41 min
Darren McKee joins the podcast to discuss how AI might be difficult to control, which goals and traits AI systems will develop, and whether there's a unified solution to AI alignment. Lees meer
Timestamps:
00:00 Uncontrollable superintelligence
16:41 AI goals and the "virus analogy"
28:36 Speed of AI cognition
39:25 Narrow AI and autonomy
52:23 Reliability of current and future AI
1:02:33 Planning for multiple AI scenarios
1:18:57 Will AIs seek self-preservation?
1:27:57 Is there a unified solution to AI alignment?
1:30:26 Concrete AI safety proposals -
Mark Brakel on the UK AI Summit and the Future of AI Policy
17 november 2023 · 1 uur 49 min
Mark Brakel (Director of Policy at the Future of Life Institute) joins the podcast to discuss the AI Safety Summit in Bletchley Park, objections to AI policy, AI regulation in the EU and US, global institutions for safe AI, and autonomy in weapon systems. Lees meer
Timestamps:
00:00 AI Safety Summit in the UK
12:18 Are officials up to date on AI?
23:22 Objections to AI policy
31:27 The EU AI Act
43:37 The right level of regulation
57:11 Risks and regulatory tools
1:04:44 Open-source AI
1:14:56 Subsidising AI safety research
1:26:29 Global institutions for safe AI
1:34:34 Autonomy in weapon systems -
Dan Hendrycks on Catastrophic AI Risks
3 november 2023 · 2 uur 7 min
Dan Hendrycks joins the podcast again to discuss X.ai, how AI risk thinking has evolved, malicious use of AI, AI race dynamics between companies and between militaries, making AI organizations safer, and how representation engineering could help us understand AI traits like deception. You can learn more about Dan's work at https://www.safe.ai Lees meer
Timestamps:
00:00 X.ai - Elon Musk's new AI venture
02:41 How AI risk thinking has evolved
12:58 AI bioengeneering
19:16 AI agents
24:55 Preventing autocracy
34:11 AI race - corporations and militaries
48:04 Bulletproofing AI organizations
1:07:51 Open-source models
1:15:35 Dan's textbook on AI safety
1:22:58 Rogue AI
1:28:09 LLMs and value specification
1:33:14 AI goal drift
1:41:10 Power-seeking AI
1:52:07 AI deception
1:57:53 Representation engineering -
Samuel Hammond on AGI and Institutional Disruption
20 oktober 2023 · 2 uur 15 min
Samuel Hammond joins the podcast to discuss how AGI will transform economies, governments, institutions, and other power structures. You can read Samuel's blog at https://www.secondbest.ca Lees meer
Timestamps:
00:00 Is AGI close?
06:56 Compute versus data
09:59 Information theory
20:36 Universality of learning
24:53 Hards steps in evolution
30:30 Governments and advanced AI
40:33 How will AI transform the economy?
55:26 How will AI change transaction costs?
1:00:31 Isolated thinking about AI
1:09:43 AI and Leviathan
1:13:01 Informational resolution
1:18:36 Open-source AI
1:21:24 AI will decrease state power
1:33:17 Timeline of a techno-feudalist future
1:40:28 Alignment difficulty and AI scale
1:45:19 Solving robotics
1:54:40 A constrained Leviathan
1:57:41 An Apollo Project for AI safety
2:04:29 Secure "gain-of-function" AI research
2:06:43 Is the market expecting AGI soon? -
Imagine A World: What if AI advisors helped us make better decisions?
17 oktober 2023 · 1 uur
Are we doomed to a future of loneliness and unfulfilling online interactions? What if technology made us feel more connected instead? Lees meer
Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last year
In the eighth and final episode of Imagine A World we explore the fictional worldbuild titled 'Computing Counsel', one of the third place winners of FLI’s worldbuilding contest.
Guillaume Riesen talks to Mark L, one of the three members of the team behind 'Computing Counsel', a third-place winner of the FLI Worldbuilding Contest. Mark is a machine learning expert with a chemical engineering degree, as well as an amateur writer. His teammates are Patrick B, a mechanical engineer and graphic designer, and Natalia C, a biological anthropologist and amateur programmer.
This world paints a vivid, nuanced picture of how emerging technologies shape society. We have advertisers competing with ad-filtering technologies and an escalating arms race that eventually puts an end to the internet as we know it. There is AI-generated art so personalized that it becomes addictive to some consumers, while others boycott media technologies altogether. And corporations begin to throw each other under the bus in an effort to redistribute the wealth of their competitors to their own customers.
While these conflicts are messy, they generally end up empowering and enriching the lives of the people in this world. New kinds of AI systems give them better data, better advice, and eventually the opportunity for genuine relationships with the beings these tools have become. The impact of any technology on society is complex and multifaceted. This world does a great job of capturing that.
While social networking technologies become ever more powerful, the networks of peopl…
-
Imagine A World: What if narrow AI fractured our shared reality?
10 oktober 2023 · 51 min
Let’s imagine a future where AGI is developed but kept at a distance from practically impacting the world, while narrow AI remakes the world completely. Most people don’t know or care about the difference and have no idea how they could distinguish between a human or artificial stranger. Inequality sticks around and AI fractures society into separate media bubbles with irreconcilable perspectives. But it's not all bad. AI markedly improves the general quality of life, enhancing medicine and therapy, and those bubbles help to sustain their inhabitants. Can you get excited about a world with these tradeoffs? Lees meer
Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last year
In the seventh episode of Imagine A World we explore a fictional worldbuild titled 'Hall of Mirrors', which was a third-place winner of FLI's worldbuilding contest.
Michael Vasser joins Guillaume Riesen to discuss his imagined future, which he created with the help of Matija Franklin and Bryce Hidysmith. Vassar was formerly the president of the Singularity Institute, and co-founded Metamed; more recently he has worked on communication across political divisions. Franklin is a PhD student at UCL working on AI Ethics and Alignment. Finally, Hidysmith began in fashion design, passed through fortune-telling before winding up in finance and policy research, at places like Numerai, the Median Group, Bismarck Analysis, and Eco.com.
Hall of Mirrors is a deeply unstable world where nothing is as it seems. The structures of power that we know today have eroded away, survived only by shells of expectation and appearance. People are isolated by perceptual bubbles and struggle to agree on what is real.
This team put a lot of effort into creating a plausible, empirically grounded world,…
-
Steve Omohundro on Provably Safe AGI
5 oktober 2023 · 2 uur 3 min
Steve Omohundro joins the podcast to discuss Provably Safe Systems, a paper he co-authored with FLI President Max Tegmark. You can read the paper here: https://arxiv.org/pdf/2309.01933.pdf Lees meer
Timestamps:
00:00 Provably safe AI systems
12:17 Alignment and evaluations
21:08 Proofs about language model behavior
27:11 Can we formalize safety?
30:29 Provable contracts
43:13 Digital replicas of actual systems
46:32 Proof-carrying code
56:25 Can language models think logically?
1:00:44 Can AI do proofs for us?
1:09:23 Hard to proof, easy to verify
1:14:31 Digital neuroscience
1:20:01 Risks of totalitarianism
1:22:29 Can we guarantee safety?
1:25:04 Real-world provable safety
1:29:29 Tamper-proof hardware
1:35:35 Mortal and throttled AI
1:39:23 Least-privilege guarantee
1:41:53 Basic AI drives
1:47:47 AI agency and world models
1:52:08 Self-improving AI
1:58:21 Is AI overhyped now? -
Imagine A World: What if AI enabled us to communicate with animals?
3 oktober 2023 · 1 uur 4 min
What if AI allowed us to communicate with animals? Could interspecies communication lead to new levels of empathy? How might communicating with animals lead humans to reimagine our place in the natural world? Lees meer
Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last year.
In the sixth episode of Imagine A World we explore the fictional worldbuild titled 'AI for the People', a third place winner of the worldbuilding contest.
Our host Guillaume Riesen welcomes Chi Rainer Bornfree, part of this three-person worldbuilding team alongside her husband Micah White, and their collaborator, J.R. Harris. Chi has a PhD in Rhetoric from UC Berkeley and has taught at Bard, Princeton, and NY State Correctional facilities, in the meantime writing fiction, essays, letters, and more. Micah, best-known as the co-creator of the 'Occupy Wall Street' movement and the author of 'The End of Protest', now focuses primarily on the social potential of cryptocurrencies, while Harris is a freelance illustrator and comic artist.
The name 'AI for the People' does a great job of capturing this team's activist perspective and their commitment to empowerment. They imagine social and political shifts that bring power back into the hands of individuals, whether that means serving as lawmakers on randomly selected committees, or gaining income by choosing to sell their personal data online. But this world isn't just about human people. Its biggest bombshell is an AI breakthrough that allows humans to communicate with other animals. What follows is an existential reconsideration of humanity's place in the universe. This team has created an intimate, complex portrait of a world shared by multiple parties: AIs, humans, other animals, and the environment itself. As these entiti…
-
Imagine A World: What if some people could live forever?
26 september 2023 · 59 min
If you could extend your life, would you? How might life extension technologies create new social and political divides? How can the world unite to solve the great problems of our time, like AI risk? What if AI creators could agree on an inspection process to expose AI dangers before they're unleashed? Lees meer
Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last year
In the fifth episode of Imagine A World, we explore the fictional worldbuild titled 'To Light’. Our host Guillaume Riesen speaks to Mako Yass, the first place winner of the FLI Worldbuilding Contest we ran last year. Mako lives in Auckland, New Zealand. He describes himself as a 'stray philosopher-designer', and has a background in computer programming and analytic philosophy.
Mako’s world is particularly imaginative, with richly interwoven narrative threads and high-concept sci fi inventions. By 2045, his world has been deeply transformed. There’s an AI-designed miracle pill that greatly extends lifespan and eradicates most human diseases. Sachets of this life-saving medicine are distributed freely by dove-shaped drones. There’s a kind of mind uploading which lets anyone become whatever they wish, live indefinitely and gain augmented intelligence. The distribution of wealth is almost perfectly even, with every human assigned a share of all resources. Some people move into space, building massive structures around the sun where they practice esoteric arts in pursuit of a more perfect peace.
While this peaceful, flourishing end state is deeply optimistic, Mako is also very conscious of the challenges facing humanity along the way. He sees a strong need for global collaboration and investment to avoid catastrophe as humanity develops more and more powerful technologies. He’s part…
-
Johannes Ackva on Managing Climate Change
21 september 2023 · 1 uur 40 min
Johannes Ackva joins the podcast to discuss the main drivers of climate change and our best technological and governmental options for managing it. You can read more about Johannes' work at http://founderspledge.com/climate Lees meer
Timestamps:
00:00 Johannes's journey as an environmentalist
13:21 The drivers of climate change
23:00 Oil, coal, and gas
38:05 Solar, wind, and hydro
49:34 Nuclear energy
57:03 Geothermal energy
1:00:41 Most promising technologies
1:05:40 Government subsidies
1:13:28 Carbon taxation
1:17:10 Planting trees
1:21:53 Influencing government policy
1:26:39 Different climate scenarios
1:34:49 Economic growth and emissions
1:37:23 Social stabilityReferences:
Emissions by sector: https://ourworldindata.org/emissions-by-sector
Energy density of different energy sources: https://www.nature.com/articles/s41598-022-25341-9
Emissions forecasts: https://www.lse.ac.uk/granthaminstitute/publication/the-unconditional-probability-distribution-of-future-emissions-and-temperatures/ and https://www.science.org/doi/10.1126/science.adg6248
Risk management: https://www.youtube.com/watch?v=6JJvIR1W-xI
Carbon pricing: https://www.cell.com/joule/pdf/S2542-4351(18)30567-1.pdf
Why not simply plant trees?: https://climate.mit.edu/ask-mit/how-many-new-trees-would-we-need-offset-our-carbon-emissions
Deforestation: https://www.science.org/doi/10.1126/science.ade3535
Decoupling of economic growth and emissions: https://www.globalcarbonproject.org/carbonbudget/22/highlights.htm
Premature deaths from air pollution: https://www.unep.org/interactives/air-pollution-note/ -
Imagine A World: What if we had digital nations untethered to geography?
19 september 2023 · 56 min
How do low income countries affected by climate change imagine their futures? How do they overcome these twin challenges? Will all nations eventually choose or be forced to go digital? Lees meer
Imagine a World is a podcast exploring a range of plausible and positive futures with advanced AI, produced by the Future of Life Institute. We interview the creators of 8 diverse and thought provoking imagined futures that we received as part of the worldbuilding contest FLI ran last year.
In the fourth episode of Imagine A World, we explore the fictional worldbuild titled 'Digital Nations'.
Conrad Whitaker and Tracey Kamande join Guillaume Riesen on 'Imagine a World' to talk about their worldbuild, 'Digital Nations', which they created with their teammate, Dexter Findley. All three worldbuilders were based in Kenya while crafting their entry, though Dexter has just recently moved to the UK. Conrad is a Nairobi-based startup advisor and entrepreneur, Dexter works in humanitarian aid, and Tracey is the Co-founder of FunKe Science, a platform that promotes interactive learning of science among school children.
As the name suggests, this world is a deep dive into virtual communities. It explores how people might find belonging and representation on the global stage through digital nations that aren't tied to any physical location. This world also features a fascinating and imaginative kind of artificial intelligence that they call 'digital persons'. These are inspired by biological brains and have a rich internal psychology. Rather than being trained on data, they're considered to be raised in digital nurseries. They have a nuanced but mostly loving relationship with humanity, with some even going on to found their own digital nations for us to join.
In an incredible turn of events, last year the South Pacific state of Tuvalu was the first to “go virtual” in response to sea levels threatening the island nation's physical territory. This happened in real life just months after it w…