Machine Learning Street Talk (MLST)
Alle afleveringen
272 afleveringen, van nieuw naar oud
-
Prof. Mark Solms - The Hidden Spring
18 september 2024 · 1 uur 27 min
Prof. Mark Solms, a neuroscientist and psychoanalyst, discusses his groundbreaking work on consciousness, challenging conventional cortex-centric views and emphasizing the role of brainstem structures in generating consciousness and affect. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
Key points discussed:
The limitations of vision-centric approaches to consciousness studies.
Evidence from decorticated animals and hydranencephalic children supporting the brainstem's role in consciousness.
The relationship between homeostasis, the free energy principle, and consciousness.
Critiques of behaviorism and modern theories of consciousness.
The importance of subjective experience in understanding brain function.
The discussion also explored broader topics:
The potential impact of affect-based theories on AI development.
The role of the SEEKING system in exploration and learning.
Connections between neuroscience, psychoanalysis, and philosophy of mind.
Challenges in studying consciousness and the limitations of current theories.
Mark Solms:
https://neuroscience.uct.ac.za/contacts/mark-solms
Show notes and transcript: https://www.dropbox.com/scl/fo/roipwmnlfmwk2e7kivzms/ACjZF-VIGC2-Suo30KcwVV0?rlkey=53y8v2cajfcgrf17p1h7v3suz&st=z8vu81hn&dl=0
TOC (*) are best bits
00:00:00 1. Intro: Challenging vision-centric approaches to consciousness *
00:02:20 2. Evidence from decorticated animals and hydranencephalic children *
00:07:40 3. Emotional responses in hydranencephalic children
00:10:40 4. Brainstem stimulation and affective states
00:15:00 5. Brainstem's role in generating affective consciousness *
00:21:50 6. Dual-aspect monism and the mind-brain relationship
00:29:37 7. Informat…
-
Patrick Lewis (Cohere) - Retrieval Augmented Generation
16 september 2024 · 1 uur 14 min
Dr. Patrick Lewis, who coined the term RAG (Retrieval Augmented Generation) and now works at Cohere, discusses the evolution of language models, RAG systems, and challenges in AI evaluation. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmented generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
Key topics covered:
- Origins and evolution of Retrieval Augmented Generation (RAG)
- Challenges in evaluating RAG systems and language models
- Human-AI collaboration in research and knowledge work
- Word embeddings and the progression to modern language models
- Dense vs sparse retrieval methods in information retrieval
The discussion also explored broader implications and applications:
- Balancing faithfulness and fluency in RAG systems
- User interface design for AI-augmented research tools
- The journey from chemistry to AI research
- Challenges in enterprise search compared to web search
- The importance of data quality in training AI models
Patrick Lewis: https://www.patricklewis.io/
Cohere Command Models, check them out - they are amazing for RAG!
https://cohere.com/command
TOC
00:00:00 1. Intro to RAG
00:05:30 2. RAG Evaluation: Poll framework & model performance
00:12:55 3. Data Quality: Cleanliness vs scale in AI training
00:15:13 4. Human-AI Collaboration: Research agents & UI design
00:22:57 5. RAG Origins: Open-domain QA to generative models
00:30:18 6. RAG Challenges: Info retrieval, tool use, faithfulness
00:42:01 7. Dense vs Sparse Retrieval: Techniques & trade-offs
00:47:02 8. RAG Applications: Grounding, attribution, hallucination prevention
00:54:04 9. UI for RAG: Human-computer interaction & model optimization
00:59:01 10. Word Embeddings: Word2Vec, GloVe, and semantic spaces
01:06:43 11. Language Model Evolution:…
-
Ashley Edwards - Genie Paper (DeepMind/Runway)
13 september 2024 · 25 min
Ashley Edwards, who was working at DeepMind when she co-authored the Genie paper and is now at Runway, covered several key aspects of the Genie AI system and its applications in video generation, robotics, and game creation. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
Genie's approach to learning interactive environments, balancing compression and fidelity.
The use of latent action models and VQE models for video processing and tokenization.
Challenges in maintaining action consistency across frames and integrating text-to-image models.
Evaluation metrics for AI-generated content, such as FID and PS&R diff metrics.
The discussion also explored broader implications and applications:
The potential impact of AI video generation on content creation jobs.
Applications of Genie in game generation and robotics.
The use of foundation models in robotics and the differences between internet video data and specialized robotics data.
Challenges in mapping AI-generated actions to real-world robotic actions.
Ashley Edwards: https://ashedwards.github.io/
TOC (*) are best bits
00:00:00 1. Intro to Genie & Brave Search API: Trade-offs & limitations *
00:02:26 2. Genie's Architecture: Latent action, VQE, video processing *
00:05:06 3. Genie's Constraints: Frame consistency & image model integration
00:07:26 4. Evaluation: FID, PS&R diff metrics & latent induction methods
00:09:44 5. AI Video Gen: Content creation impact, depth & parallax effects
00:11:39 6. Model Scaling: Training data impact & computational trade-offs
00:13:50 7. Game & Robotics Apps: Gamification & action mapping challenges *
00:16:16 8. Robotics Foundation Models: Action space & data considerations *
00:19:18 9. Mask-…
-
Cohere's SVP Technology - Saurabh Baji
12 september 2024 · 1 uur 30 min
Saurabh Baji discusses Cohere's approach to developing and deploying large language models (LLMs) for enterprise use. Lees meer
* Cohere focuses on pragmatic, efficient models tailored for business applications rather than pursuing the largest possible models.
* They offer flexible deployment options, from cloud services to on-premises installations, to meet diverse enterprise needs.
* Retrieval-augmented generation (RAG) is highlighted as a critical capability, allowing models to leverage enterprise data securely.
* Cohere emphasizes model customization, fine-tuning, and tools like reranking to optimize performance for specific use cases.
* The company has seen significant growth, transitioning from developer-focused to enterprise-oriented services.
* Major customers like Oracle, Fujitsu, and TD Bank are using Cohere's models across various applications, from HR to finance.
* Baji predicts a surge in enterprise AI adoption over the next 12-18 months as more companies move from experimentation to production.
* He emphasizes the importance of trust, security, and verifiability in enterprise AI applications.
The interview provides insights into Cohere's strategy, technology, and vision for the future of enterprise AI adoption.
https://www.linkedin.com/in/saurabhbaji/
https://x.com/sbaji
https://cohere.com/
https://cohere.com/business
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
TOC (*) are best bits
00:00:00 1. Introduction and Background
00:04:24 2. Cloud Infrastructure and LLM Optimization
00:06:43 2.1 Model deployment and fine-tuning strategies *
00:09:37 3. Enterprise AI Deployment Strategies
00:11:10 3.1 Retrieval-augmented generation in enterprise environments *
00:13:40 3.2 S…
-
David Hanson's Vision for Sentient Robots
10 september 2024 · 53 min
David Hanson, CEO of Hanson Robotics and creator of the humanoid robot Sofia, explores the intersection of artificial intelligence, ethics, and human potential. In this thought-provoking interview, Hanson discusses his vision for developing AI systems that embody the best aspects of humanity while pushing beyond our current limitations, aiming to achieve what he calls "super wisdom." Lees meer
YT version: https://youtu.be/LFCIEhlsozU
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
The interview with David Hanson covers:
The importance of incorporating biological drives and compassion into AI systems
Hanson's concept of "existential pattern ethics" as a basis for AI morality
The potential for AI to enhance human intelligence and wisdom
Challenges in developing artificial general intelligence (AGI)
The need to democratize AI technologies globally
Potential future advancements in human-AI integration and their societal impacts
Concerns about technological augmentation exacerbating inequality
The role of ethics in guiding AI development and deployment
Hanson advocates for creating AI systems that embody the best aspects of humanity while surpassing current human limitations, aiming for "super wisdom" rather than just artificial super intelligence.
David Hanson:
https://www.hansonrobotics.com/david-hanson/
https://www.youtube.com/watch?v=9u1O954cMmE
TOC
1. Introduction and Background [00:00:00]
1.1. David Hanson's interdisciplinary background [0:01:49]
1.2. Introduction to Sofia, the realistic robot [0:03:27]
2. Human Cognition and AI [0:03:50]
2.1. Importance of social interaction in cognition [0:03:50]
2.2. Compassion as distinguishing factor [0:05:55]
2.3. AI augmenting human intelli…
-
The Fabric of Knowledge - David Spivak
5 september 2024 · 46 min
David Spivak, a mathematician known for his work in category theory, discusses a wide range of topics related to intelligence, creativity, and the nature of knowledge. He explains category theory in simple terms and explores how it relates to understanding complex systems and relationships. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
We discuss abstract concepts like collective intelligence, the importance of embodiment in understanding the world, and how we acquire and process knowledge. Spivak shares his thoughts on creativity, discussing where it comes from and how it might be modeled mathematically.
A significant portion of the discussion focuses on the impact of artificial intelligence on human thinking and its potential role in the evolution of intelligence. Spivak also touches on the importance of language, particularly written language, in transmitting knowledge and shaping our understanding of the world.
David Spivak
http://www.dspivak.net/
TOC:
00:00:00 Introduction to category theory and functors
00:04:40 Collective intelligence and sense-making
00:09:54 Embodiment and physical concepts in knowledge acquisition
00:16:23 Creativity, open-endedness, and AI's impact on thinking
00:25:46 Modeling creativity and the evolution of intelligence
00:36:04 Evolution, optimization, and the significance of AI
00:44:14 Written language and its impact on knowledge transmission
REFS:
Mike Levin's work
https://scholar.google.com/citations?user=luouyakAAAAJ&hl=en
Eric Smith's videos on complexity and early life
https://www.youtube.com/watch?v=SpJZw-68QyE
Richard Dawkins' book "The Selfish Gene"
https://amzn.to/3X73X8w
Carl Sagan's statement about the cosmos knowing itself
htt…
-
Jürgen Schmidhuber - Neural and Non-Neural AI, Reasoning, Transformers, and LSTMs
28 augustus 2024 · 1 uur 40 min
Jürgen Schmidhuber, the father of generative AI shares his groundbreaking work in deep learning and artificial intelligence. In this exclusive interview, he discusses the history of AI, some of his contributions to the field, and his vision for the future of intelligent machines. Schmidhuber offers unique insights into the exponential growth of technology and the potential impact of AI on humanity and the universe. Lees meer
YT version: https://youtu.be/DP454c1K_vQ
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
TOC
00:00:00 Intro
00:03:38 Reasoning
00:13:09 Potential AI Breakthroughs Reducing Computation Needs
00:20:39 Memorization vs. Generalization in AI
00:25:19 Approach to the ARC Challenge
00:29:10 Perceptions of Chat GPT and AGI
00:58:45 Abstract Principles of Jurgen's Approach
01:04:17 Analogical Reasoning and Compression
01:05:48 Breakthroughs in 1991: the P, the G, and the T in ChatGPT and Generative AI
01:15:50 Use of LSTM in Language Models by Tech Giants
01:21:08 Neural Network Aspect Ratio Theory
01:26:53 Reinforcement Learning Without Explicit Teachers
Refs:
★ "Annotated History of Modern AI and Deep Learning" (2022 survey by Schmidhuber):
★ Chain Rule For Backward Credit Assignment (Leibniz, 1676)
★ First Neural Net / Linear Regression / Shallow Learning (Gauss & Legendre, circa 1800)
★ First 20th Century Pioneer of Practical AI (Quevedo, 1914)
★ First Recurrent NN (RNN) Architecture (Lenz, Ising, 1920-1925)
★ AI Theory: Fundamental Limitations of Computation and Computation-Based AI (Gödel, 1931-34)
★ Unpublished ideas about evolving RNNs (Turing, 1948)
★ Multilayer Feedforward NN Without Deep Learning (Rosenblatt, 1958)
★ First Published Learning RNNs (Amari and…
-
"AI should NOT be regulated at all!" - Prof. Pedro Domingos
25 augustus 2024 · 2 uur 12 min
Professor Pedro Domingos, is an AI researcher and professor of computer science. He expresses skepticism about current AI regulation efforts and argues for faster AI development rather than slowing it down. He also discusses the need for new innovations to fulfil the promises of current AI techniques. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmented generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
Show notes:
* Domingos' views on AI regulation and why he believes it's misguided
* His thoughts on the current state of AI technology and its limitations
* Discussion of his novel "2040", a satirical take on AI and tech culture
* Explanation of his work on "tensor logic", which aims to unify neural networks and symbolic AI
* Critiques of other approaches in AI, including those of OpenAI and Gary Marcus
* Thoughts on the AI "bubble" and potential future developments in the field
Prof. Pedro Domingos:
https://x.com/pmddomingos
2040: A Silicon Valley Satire [Pedro's new book]
https://amzn.to/3T51ISd
TOC:
00:00:00 Intro
00:06:31 Bio
00:08:40 Filmmaking skit
00:10:35 AI and the wisdom of crowds
00:19:49 Social Media
00:27:48 Master algorithm
00:30:48 Neurosymbolic AI / abstraction
00:39:01 Language
00:45:38 Chomsky
01:00:49 2040 Book
01:18:03 Satire as a shield for criticism?
01:29:12 AI Regulation
01:35:15 Gary Marcus
01:52:37 Copyright
01:56:11 Stochastic parrots come home to roost
02:00:03 Privacy
02:01:55 LLM ecosystem
02:05:06 Tensor logic
Refs:
The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World [Pedro Domingos]
https://amzn.to/3MiWs9B
Rebooting AI: Building Artificial Intelligence We Can Trust [Gary Marcus]
https://amzn.to/3AAywvL
Flash Boys [Michael Lewis]
https://amzn.to…
-
Adversarial Examples and Data Modelling - Andrew Ilyas (MIT)
22 augustus 2024 · 1 uur 28 min
Andrew Ilyas, a PhD student at MIT who is about to start as a professor at CMU. We discuss Data modeling and understanding how datasets influence model predictions, Adversarial examples in machine learning and why they occur, Robustness in machine learning models, Black box attacks on machine learning systems, Biases in data collection and dataset creation, particularly in ImageNet and Self-selection bias in data and methods to address it. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api
Andrew's site:
https://andrewilyas.com/
https://x.com/andrew_ilyas
TOC:
00:00:00 - Introduction and Andrew's background
00:03:52 - Overview of the machine learning pipeline
00:06:31 - Data modeling paper discussion
00:26:28 - TRAK: Evolution of data modeling work
00:43:58 - Discussion on abstraction, reasoning, and neural networks
00:53:16 - "Adversarial Examples Are Not Bugs, They Are Features" paper
01:03:24 - Types of features learned by neural networks
01:10:51 - Black box attacks paper
01:15:39 - Work on data collection and bias
01:25:48 - Future research plans and closing thoughts
References:
Adversarial Examples Are Not Bugs, They Are Features
https://arxiv.org/pdf/1905.02175
TRAK: Attributing Model Behavior at Scale
https://arxiv.org/pdf/2303.14186
Datamodels: Predicting Predictions from Training Data
https://arxiv.org/pdf/2202.00622
Adversarial Examples Are Not Bugs, They Are Features
https://arxiv.org/pdf/1905.02175
IMAGENET-TRAINED CNNS
https://arxiv.org/pdf/1811.12231
ZOO: Zeroth Order Optimization Based Black-box
https://arxiv.org/pdf/1708.03999
A Spline Theory of Deep Networks
https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf
Scaling Monosemanticity
…
-
Joscha Bach - AGI24 Keynote (Cyberanimism)
21 augustus 2024 · 57 min
Dr. Joscha Bach introduces a surprising idea called "cyber animism" in his AGI-24 talk - the notion that nature might be full of self-organizing software agents, similar to the spirits in ancient belief systems. Bach suggests that consciousness could be a kind of software running on our brains, and wonders if similar "programs" might exist in plants or even entire ecosystems. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
Joscha takes us on a tour de force through history, philosophy, and cutting-edge computer science, teasing us to rethink what we know about minds, machines, and the world around us. Joscha believes we should blur the lines between human, artificial, and natural intelligence, and argues that consciousness might be more widespread and interconnected than we ever thought possible.
Dr. Joscha Bach
https://x.com/Plinz
This is video 2/9 from our coverage of AGI-24 in Seattle https://agi-conf.org/2024/
Watch the official MLST interview with Joscha which we did right after this talk on our Patreon now on early access - https://www.patreon.com/posts/joscha-bach-110199676 (you also get access to our private discord and biweekly calls)
TOC:
00:00:00 Introduction: AGI and Cyberanimism
00:03:57 The Nature of Consciousness
00:08:46 Aristotle's Concepts of Mind and Consciousness
00:13:23 The Hard Problem of Consciousness
00:16:17 Functional Definition of Consciousness
00:20:24 Comparing LLMs and Human Consciousness
00:26:52 Testing for Consciousness in AI Systems
00:30:00 Animism and Software Agents in Nature
00:37:02 Plant Consciousness and Ecosystem Intelligence
00:40:36 The California Institute for Machine Consciousness
00:44:52 Ethics of Conscious AI and Suffering
00:…
-
Gary Marcus' keynote at AGI-24
17 augustus 2024 · 1 uur 12 min
Prof Gary Marcus revisited his keynote from AGI-21, noting that many of the issues he highlighted then are still relevant today despite significant advances in AI. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
Gary Marcus criticized current large language models (LLMs) and generative AI for their unreliability, tendency to hallucinate, and inability to truly understand concepts.
Marcus argued that the AI field is experiencing diminishing returns with current approaches, particularly the "scaling hypothesis" that simply adding more data and compute will lead to AGI.
He advocated for a hybrid approach to AI that combines deep learning with symbolic AI, emphasizing the need for systems with deeper conceptual understanding.
Marcus highlighted the importance of developing AI with innate understanding of concepts like space, time, and causality.
He expressed concern about the moral decline in Silicon Valley and the rush to deploy potentially harmful AI technologies without adequate safeguards.
Marcus predicted a possible upcoming "AI winter" due to inflated valuations, lack of profitability, and overhyped promises in the industry.
He stressed the need for better regulation of AI, including transparency in training data, full disclosure of testing, and independent auditing of AI systems.
Marcus proposed the creation of national and global AI agencies to oversee the development and deployment of AI technologies.
He concluded by emphasizing the importance of interdisciplinary collaboration, focusing on robust AI with deep understanding, and implementing smart, agile governance for AI and AGI.
YT Version (very high quality filmed)
https://youtu.be/91SK90SahHc
Pre-order Gary's new book here…
-
Is ChatGPT an N-gram model on steroids?
15 augustus 2024 · 33 min
DeepMind Research Scientist / MIT scholar Dr. Timothy Nguyen discusses his recent paper on understanding transformers through n-gram statistics. Nguyen explains his approach to analyzing transformer behavior using a kind of "template matching" (N-grams), providing insights into how these models process and predict language. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
Key points covered include:
A method for describing transformer predictions using n-gram statistics without relying on internal mechanisms.
The discovery of a technique to detect overfitting in large language models without using holdout sets.
Observations on curriculum learning, showing how transformers progress from simpler to more complex rules during training.
Discussion of distance measures used in the analysis, particularly the variational distance.
Exploration of model sizes, training dynamics, and their impact on the results.
We also touch on philosophical aspects of describing versus explaining AI behavior, and the challenges in understanding the abstractions formed by neural networks. Nguyen concludes by discussing potential future research directions, including attempts to convert descriptions of transformer behavior into explanations of internal mechanisms.
Timothy Nguyen's earned his B.S. and Ph.D. in mathematics from Caltech and MIT, respectively. He held positions as Research Assistant Professor at the Simons Center for Geometry and Physics (2011-2014) and Visiting Assistant Professor at Michigan State University (2014-2017). During this time, his research expanded into high-energy physics, focusing on mathematical problems in quantum field theory. His work notably provided a simplified and corrected for…
-
Jay Alammar on LLMs, RAG, and AI Engineering
11 augustus 2024 · 57 min
Jay Alammar, renowned AI educator and researcher at Cohere, discusses the latest developments in large language models (LLMs) and their applications in industry. Jay shares his expertise on retrieval augmented generation (RAG), semantic search, and the future of AI architectures. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
Cohere Command R model series: https://cohere.com/command
Jay Alamaar:
https://x.com/jayalammar
Buy Jay's new book here!
Hands-On Large Language Models: Language Understanding and Generation
https://amzn.to/4fzOUgh
TOC:
00:00:00 Introduction to Jay Alammar and AI Education
00:01:47 Cohere's Approach to RAG and AI Re-ranking
00:07:15 Implementing AI in Enterprise: Challenges and Solutions
00:09:26 Jay's Role at Cohere and the Importance of Learning in Public
00:15:16 The Evolution of AI in Industry: From Deep Learning to LLMs
00:26:12 Expert Advice for Newcomers in Machine Learning
00:32:39 The Power of Semantic Search and Embeddings in AI Systems
00:37:59 Jay Alammar's Journey as an AI Educator and Visualizer
00:43:36 Visual Learning in AI: Making Complex Concepts Accessible
00:47:38 Strategies for Keeping Up with Rapid AI Advancements
00:49:12 The Future of Transformer Models and AI Architectures
00:51:40 Evolution of the Transformer: From 2017 to Present
00:54:19 Preview of Jay's Upcoming Book on Large Language Models
Disclaimer: This is the fourth video from our Cohere partnership. We were not told what to say in the interview, and didn't edit anything out from the interview. Note also that this combines several previously unpublished interviews from Jay into one, the earlier one at Tim's house was shot in Aug 2023, and the more recent one in Toro…
-
Can AI therapy be more effective than drugs?
8 augustus 2024 · 2 uur 14 min
Daniel Cahn, co-founder of Slingshot AI, on the potential of AI in therapy. Why is anxiety and depression affecting a large population? To what extent are these real categories? Why is the mental health getting worse? How often do you want an AI to agree with you? What are the ethics of persuasive AI? You will discover all in this conversation. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
Daniel Cahn (who is also hiring ML engineers by the way!)
https://x.com/thecahnartist?lang=en
/ cahnd
https://thinkingmachinespodcast.com/
TOC:
00:00:00 Intro
00:01:56 Therapy effectiveness vs drugs and societal implications
00:04:02 Mental health categories: Iatrogenesis and social constructs
00:10:19 Psychiatric treatment models and cognitive perspectives
00:13:30 AI design and human-like interactions: Intentionality debates
00:20:04 AI in therapy: Ethics, anthropomorphism, and loneliness mitigation
00:28:13 Therapy efficacy: Neuroplasticity, suffering, and AI placebos
00:33:29 AI's impact on human agency and cognitive modeling
00:41:17 Social media's effects on brain structure and behavior
00:50:46 AI ethics: Altering values and free will considerations
01:00:00 Work value perception and personal identity formation
01:13:37 Free will, agency, and mutable personal identity in therapy
01:24:27 AI in healthcare: Challenges, ethics, and therapy improvements
01:53:25 AI development: Societal impacts and cultural implications
Full references on YT VD: https://www.youtube.com/watch?v=7hwX6OZyNC0 (and baked into mp3 metadata)
-
Prof. Subbarao Kambhampati - LLMs don't reason, they memorize (ICML2024 2/13)
29 juli 2024 · 1 uur 42 min
Prof. Subbarao Kambhampati argues that while LLMs are impressive and useful tools, especially for creative tasks, they have fundamental limitations in logical reasoning and cannot provide guarantees about the correctness of their outputs. He advocates for hybrid approaches that combine LLMs with external verification systems. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.
TOC (sorry the ones baked into the MP3 were wrong apropos due to LLM hallucination!)
[00:00:00] Intro
[00:02:06] Bio
[00:03:02] LLMs are n-gram models on steroids
[00:07:26] Is natural language a formal language?
[00:08:34] Natural language is formal?
[00:11:01] Do LLMs reason?
[00:19:13] Definition of reasoning
[00:31:40] Creativity in reasoning
[00:50:27] Chollet's ARC challenge
[01:01:31] Can we reason without verification?
[01:10:00] LLMs cant solve some tasks
[01:19:07] LLM Modulo framework
[01:29:26] Future trends of architecture
[01:34:48] Future research directions
Youtube version: https://www.youtube.com/watch?v=y1WnHpedi2A
Refs: (we didn't have space for URLs here, check YT video description instead)
Can LLMs Really Reason and Plan?
On the Planning Abilities of Large Language Models : A Critical Investigation
Chain of Thoughtlessness? An Analysis of CoT in Planning
On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks
LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks
Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve
"Task Success" is not Enough
Partition function (number theory) (Srinivasa Ramanujan and G.H. Hardy's work)
Poincaré conjecture
Gödel's incompleteness theorems
ROT1…
-
Sayash Kapoor - How seriously should we take AI X-risk? (ICML 1/13)
28 juli 2024 · 50 min
How seriously should governments take the threat of existential risk from AI, given the lack of consensus among researchers? On the one hand, existential risks (x-risks) are necessarily somewhat speculative: by the time there is concrete evidence, it may be too late. On the other hand, governments must prioritize — after all, they don’t worry too much about x-risk from alien invasions. Lees meer
MLST is sponsored by Brave:
The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at brave.com/api.
Sayash Kapoor is a computer science Ph.D. candidate at Princeton University's Center for Information Technology Policy. His research focuses on the societal impact of AI. Kapoor has previously worked on AI in both industry and academia, with experience at Facebook, Columbia University, and EPFL Switzerland. He is a recipient of a best paper award at ACM FAccT and an impact recognition award at ACM CSCW. Notably, Kapoor was included in TIME's inaugural list of the 100 most influential people in AI.
Sayash Kapoor
https://x.com/sayashk
https://www.cs.princeton.edu/~sayashk/
Arvind Narayanan (other half of the AI Snake Oil duo)
https://x.com/random_walker
AI existential risk probabilities are too unreliable to inform policy
https://www.aisnakeoil.com/p/ai-existential-risk-probabilities
Pre-order AI Snake Oil Book
https://amzn.to/4fq2HGb
AI Snake Oil blog
https://www.aisnakeoil.com/
AI Agents That Matter
https://arxiv.org/abs/2407.01502
Shortcut learning in deep neural networks
https://www.semanticscholar.org/paper/Shortcut-learning-in-deep-neural-networks-Geirhos-Jacobsen/1b04936c2599e59b120f743fbb30df2eed3fd782
77% Of Employees Report AI Has Increased Workloads And Hampered Productivity, Study Finds
https://www.forbes.com/sites/bryanrobinson/2024/07/23/empl…
-
Sara Hooker - Why US AI Act Compute Thresholds Are Misguided
19 juli 2024 · 1 uur 6 min
Sara Hooker is VP of Research at Cohere and leader of Cohere for AI. We discuss her recent paper critiquing the use of compute thresholds, measured in FLOPs (floating point operations), as an AI governance strategy. Lees meer
We explore why this approach, recently adopted in both US and EU AI policies, may be problematic and oversimplified. Sara explains the limitations of using raw computational power as a measure of AI capability or risk, and discusses the complex relationship between compute, data, and model architecture.
Equally important, we go into Sara's work on "The AI Language Gap." This research highlights the challenges and inequalities in developing AI systems that work across multiple languages. Sara discusses how current AI models, predominantly trained on English and a handful of high-resource languages, fail to serve the linguistic diversity of our global population. We explore the technical, ethical, and societal implications of this gap, and discuss potential solutions for creating more inclusive and representative AI systems.
We broadly discuss the relationship between language, culture, and AI capabilities, as well as the ethical considerations in AI development and deployment.
YT Version: https://youtu.be/dBZp47999Ko
TOC:
[00:00:00] Intro
[00:02:12] FLOPS paper
[00:26:42] Hardware lottery
[00:30:22] The Language gap
[00:33:25] Safety
[00:38:31] Emergent
[00:41:23] Creativity
[00:43:40] Long tail
[00:44:26] LLMs and society
[00:45:36] Model bias
[00:48:51] Language and capabilities
[00:52:27] Ethical frameworks and RLHF
Sara Hooker
https://www.sarahooker.me/
https://www.linkedin.com/in/sararosehooker/
https://scholar.google.com/citations?user=2xy6h3sAAAAJ&hl=en
https://x.com/sarahookr
Interviewer: Tim Scarfe
Refs
The AI Language gap
https://cohere.com/research/papers/the-AI-language-gap.pdf
On the Limitations of Compute Thresholds as a Governance Strategy.
https://arxiv.org/pdf/2407.05694v1
The Multilingual Alignment Prism: Al…
-
Prof. Murray Shanahan - Machines Don't Think Like Us
14 juli 2024 · 2 uur 15 min
Murray Shanahan is a professor of Cognitive Robotics at Imperial College London and a senior research scientist at DeepMind. He challenges our assumptions about AI consciousness and urges us to rethink how we talk about machine intelligence. Lees meer
We explore the dangers of anthropomorphizing AI, the limitations of current language in describing AI capabilities, and the fascinating intersection of philosophy and artificial intelligence.
Show notes and full references: https://docs.google.com/document/d/1ICtBI574W-xGi8Z2ZtUNeKWiOiGZ_DRsp9EnyYAISws/edit?usp=sharing
Prof Murray Shanahan:
https://www.doc.ic.ac.uk/~mpsha/ (look at his selected publications)
https://scholar.google.co.uk/citations?user=00bnGpAAAAAJ&hl=en
https://en.wikipedia.org/wiki/Murray_Shanahan
https://x.com/mpshanahan
Interviewer: Dr. Tim Scarfe
Refs (links in the Google doc linked above):
Role play with large language models
Waluigi effect
"Conscious Exotica" - Paper by Murray Shanahan (2016)
"Simulators" - Article by Janis from LessWrong
"Embodiment and the Inner Life" - Book by Murray Shanahan (2010)
"The Technological Singularity" - Book by Murray Shanahan (2015)
"Simulacra as Conscious Exotica" - Paper by Murray Shanahan (newer paper of the original focussed on LLMs)
A recent paper by Anthropic on using autoencoders to find features in language models (referring to the "Scaling Monosemanticity" paper)
Work by Peter Godfrey-Smith on octopus consciousness
"Metaphors We Live By" - Book by George Lakoff (1980s)
Work by Aaron Sloman on the concept of "space of possible minds" (1984 article mentioned)
Wittgenstein's "Philosophical Investigations" (posthumously published)
Daniel Dennett's work on the "intentional stance"
Alan Turing's original paper on the Turing Test (1950)
Thomas Nagel's paper "What is it like to be a bat?" (1974)
John Searle's Chinese Room Argument (mentioned but not detailed)
Work by Richard Evans on tackling reasoning problems
Claude Shannon's quote on knowled…
-
David Chalmers - Reality+
8 juli 2024 · 1 uur 18 min
In the coming decades, the technology that enables virtual and augmented reality will improve beyond recognition. Within a century, world-renowned philosopher David J. Chalmers predicts, we will have virtual worlds that are impossible to distinguish from non-virtual worlds. But is virtual reality just escapism? Lees meer
In a highly original work of 'technophilosophy', Chalmers argues categorically, no: virtual reality is genuine reality. Virtual worlds are not second-class worlds. We can live a meaningful life in virtual reality - and increasingly, we will.
What is reality, anyway? How can we lead a good life? Is there a god? How do we know there's an external world - and how do we know we're not living in a computer simulation? In Reality+, Chalmers conducts a grand tour of philosophy, using cutting-edge technology to provide invigorating new answers to age-old questions.
David J. Chalmers is an Australian philosopher and cognitive scientist specializing in the areas of philosophy of mind and philosophy of language. He is Professor of Philosophy and Neural Science at New York University, as well as co-director of NYU's Center for Mind, Brain, and Consciousness. Chalmers is best known for his work on consciousness, including his formulation of the "hard problem of consciousness."
Reality+: Virtual Worlds and the Problems of Philosophy
https://amzn.to/3RYyGD2
https://consc.net/
https://x.com/davidchalmers42
00:00:00 Reality+ Intro
00:12:02 GPT conscious? 10/10
00:14:19 The consciousness processor thought experiment (11/10)
00:20:34 Intelligence and Consciousness entangled? 10/10
00:22:44 Karl Friston / Meta Problem 10/10
00:29:05 Knowledge argument / subjective experience (6/10)
00:32:34 Emergence 11/10 (best chapter)
00:42:45 Working with Douglas Hofstadter 10/10
00:46:14 Intelligence is analogy making? 10/10
00:50:47 Intelligence explosion 8/10
00:58:44 Hypercomputation 10/10
01:09:44 Who designed the designer? (7/10)
01:13:57 Experience machine (7/10)
-
Ryan Greenblatt - Solving ARC with GPT4o
6 juli 2024 · 2 uur 18 min
Ryan Greenblatt from Redwood Research recently published "Getting 50% on ARC-AGI with GPT-4.0," where he used GPT4o to reach a state-of-the-art accuracy on Francois Chollet's ARC Challenge by generating many Python programs. Lees meer
Sponsor:
Sign up to Kalshi here https://kalshi.onelink.me/1r91/mlst -- the first 500 traders who deposit $100 will get a free $20 credit! Important disclaimer - In case it's not obvious - this is basically gambling and a *high risk* activity - only trade what you can afford to lose.
We discuss:
- Ryan's unique approach to solving the ARC Challenge and achieving impressive results.
- The strengths and weaknesses of current AI models.
- How AI and humans differ in learning and reasoning.
- Combining various techniques to create smarter AI systems.
- The potential risks and future advancements in AI, including the idea of agentic AI.
https://x.com/RyanPGreenblatt
https://www.redwoodresearch.org/
Refs:
Getting 50% (SoTA) on ARC-AGI with GPT-4o [Ryan Greenblatt]
https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt
On the Measure of Intelligence [Chollet]
https://arxiv.org/abs/1911.01547
Connectionism and Cognitive Architecture: A Critical Analysis [Jerry A. Fodor and Zenon W. Pylyshyn]
https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/proseminars/Proseminar13/ConnectionistArchitecture.pdf
Software 2.0 [Andrej Karpathy]
https://karpathy.medium.com/software-2-0-a64152b37c35
Why Greatness Cannot Be Planned: The Myth of the Objective [Kenneth Stanley]
https://amzn.to/3Wfy2E0
Biographical account of Terence Tao’s mathematical development. [M.A.(KEN) CLEMENTS]
https://gwern.net/doc/iq/high/smpy/1984-clements.pdf
Model Evaluation and Threat Research (METR)
https://metr.org/
Why Tool AIs Want to Be Agent AIs
https://gwern.net/tool-ai
Simulators - Janus
https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators
AI Control: Improving Safety Despite Intentional Subversion
https://www.lesswrong.com/…
-
Aiden Gomez - CEO of Cohere (AI's 'Inner Monologue' – Crucial for Reasoning)
29 juni 2024 · 1 uur
Aidan Gomez, CEO of Cohere, reveals how they're tackling AI hallucinations and improving reasoning abilities. He also explains why Cohere doesn't use any output from GPT-4 for training their models. Lees meer
Aidan shares his personal insights into the world of AI and LLMs and Cohere's unique approach to solving real-world business problems, and how their models are set apart from the competition. Aidan reveals how they are making major strides in AI technology, discussing everything from last mile customer engineering to the robustness of prompts and future architectures.
He also touches on the broader implications of AI for society, including potential risks and the role of regulation. He discusses Cohere's guiding principles and the health the of startup scene. With a particular focus on enterprise applications. Aidan provides a rare look into the internal workings of Cohere and their vision for driving productivity and innovation.
https://cohere.com/
https://x.com/aidangomez
Check out Cohere's amazing new Command R* models here
https://cohere.com/command
Disclaimer: This is the second video from our Cohere partnership. We were not told what to say in the interview, and didn't edit anything out from the interview.
-
New "50%" ARC result and current winners interviewed
18 juni 2024 · 2 uur 14 min
The ARC Challenge, created by Francois Chollet, tests how well AI systems can generalize from a few examples in a grid-based intelligence test. We interview the current winners of the ARC Challenge—Jack Cole, Mohammed Osman and their collaborator Michael Hodel. They discuss how they tackled ARC (Abstraction and Reasoning Corpus) using language models. We also discuss the new "50%" public set approach announced today from Redwood Research (Ryan Greenblatt). Lees meer
Jack and Mohammed explain their winning approach, which involves fine-tuning a language model on a large, specifically-generated dataset and then doing additional fine-tuning at test-time, a technique known in this context as "active inference". They use various strategies to represent the data for the language model and believe that with further improvements, the accuracy could reach above 50%. Michael talks about his work on generating new ARC-like tasks to help train the models.
They also debate whether their methods stay true to the "spirit" of Chollet's measure of intelligence. Despite some concerns, they agree that their solutions are promising and adaptable for other similar problems.
Note:
Jack's team is still the current official winner at 33% on the private set. Ryan's entry is not on the private leaderboard or eligible.
Chollet invented ARC in 2019 (not 2017 as stated)"Ryan's entry is not a new state of the art. We don't know exactly how well it does since it was only evaluated on 100 tasks from the evaluation set and does 50% on those, reportedly. Meanwhile Jacks team i.e. MindsAI's solution does 54% on the entire eval set and it is seemingly possible to do 60-70% with an ensemble"
Jack Cole:
https://x.com/Jcole75Cole
https://lab42.global/community-interview-jack-cole/Mohamed Osman:
Mohamed is looking to do a PhD in AI/ML, can you help him?
Email: mothman198@outlook.com
https://www.linkedin.com/in/mohamedosman1905/Michael Hodel:
https://arxiv.org/pdf/2404.07353v1
https://www.linkedin.com/in/… -
Cohere co-founder Nick Frosst on building LLM apps for business
16 juni 2024 · 41 min
Nick Frosst, co-founder of Cohere, on the future of LLMs, and AGI. Learn how Cohere is solving real problems for business with their new AI models. Lees meer
This is the first podcast from our new Cohere partnership!
Nick talks about his journey at Google Brain, working with AI legends like Geoff Hinton, and the amazing things his company, Cohere, is doing. From creating the must useful language models for businesses to making tools for developers, Nick shares a lot of interesting insights. He even talks about his band, Good Kid! Nick said that RAG is one of the best features of Cohere's new Command R* models. We are about to release a deep-dive on RAG with Patrick Lewis from Cohere, keep an eye out for that - he explains why their models are specifically optimised for RAG use cases.
Learn more about Cohere Command R* models here:
https://cohere.com/commandhttps://github.com/cohere-ai/cohere-toolkit
Nick's band Good Kid:
https://goodkidofficial.com/
Nick on Twitter:
https://x.com/nickfrosst
Disclaimer: We are in a partnership with Cohere to release content for them. We were not told what to say in the interview, and didn't edit anything out from the interview. We are currently planning to release 2 shows per month under the partnership about their AI platform, research and strategy.
-
What’s the Magic Word? A Control Theory of LLM Prompting.
5 juni 2024 · 1 uur 17 min
These two scientists have mapped out the insides or “reachable space” of a language model using control theory, what they discovered was extremely surprising. Lees meer
Please support us on Patreon to get access to the private Discord server, bi-weekly calls, early access and ad-free listening.
https://patreon.com/mlst
YT version: https://youtu.be/Bpgloy1dDn0
Aman Bhargava from Caltech and Cameron Witkowski from the University of Toronto to discuss their groundbreaking paper, “What’s the Magic Word? A Control Theory of LLM Prompting.” (the main theorem on self-attention controllability was developed in collaboration with Dr. Shi-Zhuo Looi from Caltech).
They frame LLM systems as discrete stochastic dynamical systems. This means they look at LLMs in a structured way, similar to how we analyze control systems in engineering. They explore the “reachable set” of outputs for an LLM. Essentially, this is the range of possible outputs the model can generate from a given starting point when influenced by different prompts. The research highlights that prompt engineering, or optimizing the input tokens, can significantly influence LLM outputs. They show that even short prompts can drastically alter the likelihood of specific outputs. Aman and Cameron’s work might be a boon for understanding and improving LLMs. They suggest that a deeper exploration of control theory concepts could lead to more reliable and capable language models.
We dropped an additional, more technical video on the research on our Twitter account here: https://x.com/MLStreetTalk/status/1795093759471890606
Additional 20 minutes of unreleased footage on our Patreon here: https://www.patreon.com/posts/whats-magic-word-104922629
What's the Magic Word? A Control Theory of LLM Prompting (Aman Bhargava, Cameron Witkowski, Manav Shah, Matt Thomson)
https://arxiv.org/abs/2310.04444
LLM Control Theory Seminar (April 2024)
https://www.youtube.com/watch?v=9QtS9sVBFM0
Society for the pursuit of AGI (Cameron founded …
-
CAN MACHINES REPLACE US? (AI vs Humanity) - Maria Santacaterina
6 mei 2024 · 2 uur 32 min
Maria Santacaterina, with her background in the humanities, brings a critical perspective on the current state and future implications of AI technology, its impact on society, and the nature of human intelligence and creativity. She emphasizes that despite technological advancements, AI lacks fundamental human traits such as consciousness, empathy, intuition, and the ability to engage in genuine creative processes. Maria argues that AI, at its core, processes data but does not have the capability to understand or generate new, intrinsic meaning or ideas as humans do. Lees meer
Throughout the conversation, Maria highlights her concern about the overreliance on AI in critical sectors such as healthcare, the justice system, and business. She stresses that while AI can serve as a tool, it should not replace human judgment and decision-making. Maria points out that AI systems often operate on past data, which may lead to outdated or incorrect decisions if not carefully managed.
The discussion also touches upon the concept of "adaptive resilience", which Maria describes in her book. She explains adaptive resilience as the capacity for individuals and enterprises to evolve and thrive amidst challenges by leveraging technology responsibly, without undermining human values and capabilities.
A significant portion of the conversation focussed on ethical considerations surrounding AI. Tim and Maria agree that there's a pressing need for strong governance and ethical frameworks to guide AI development and deployment. They discuss how AI, without proper ethical considerations, risks exacerbating issues like privacy invasion, misinformation, and unintended discrimination.
Maria is skeptical about claims of achieving Artificial General Intelligence (AGI) or a technological singularity where machines surpass human intelligence in all aspects. She argues that such scenarios neglect the complex, dynamic nature of human intelligence and consciousness, which cannot be fully replicated or rep…
-
Dr. Thomas Parr - Active Inference Book
1 mei 2024 · 1 uur 37 min
Thomas Parr and his collaborators wrote a book titled "Active Inference: The Free Energy Principle in Mind, Brain and Behavior" which introduces Active Inference from both a high-level conceptual perspective and a low-level mechanistic, mathematical perspective. Lees meer
Active inference, developed by the legendary neuroscientist Prof. Karl Friston - is a unifying mathematical framework which frames living systems as agents which minimize surprise and free energy in order to resist entropy and persist over time. It unifies various perspectives from physics, biology, statistics, and psychology - and allows us to explore deep questions about agency, biology, causality, modelling, and consciousness.
Buy Active Inference: The Free Energy Principle in Mind, Brain, and Behavior
https://amzn.to/4dj0iMj
YT version: https://youtu.be/lbb-Si5wa_o
Please support us on Patreon to get access to the private Discord server, bi-weekly calls, early access and ad-free listening.
https://patreon.com/mlst
Chapters should be embedded in the mp3, let me me know if issues
-
Connor Leahy - e/acc, AGI and the future.
21 april 2024 · 1 uur 20 min
Connor is the CEO of Conjecture and one of the most famous names in the AI alignment movement. This is the "behind the scenes footage" and bonus Patreon interviews from the day of the Beff Jezos debate, including an interview with Daniel Clothiaux. It's a great insight into Connor's philosophy. At the end there is an unreleased additional interview with Beff. Lees meer
Support MLST:
Please support us on Patreon. We are entirely funded from Patreon donations right now. Patreon supports get private discord access, biweekly calls, very early-access + exclusive content and lots more.
https://patreon.com/mlst
Donate: https://www.paypal.com/donate/?hosted_button_id=K2TYRVPBGXVNA
If you would like to sponsor us, so we can tell your story - reach out on mlstreettalk at gmail
Topics:
Externalized cognition and the role of society and culture in human intelligence
The potential for AI systems to develop agency and autonomy
The future of AGI as a complex mixture of various components
The concept of agency and its relationship to power
The importance of coherence in AI systems
The balance between coherence and variance in exploring potential upsides
The role of dynamic, competent, and incorruptible institutions in handling risks and developing technology
Concerns about AI widening the gap between the haves and have-nots
The concept of equal access to opportunity and maintaining dynamism in the system
Leahy's perspective on life as a process that "rides entropy"
The importance of distinguishing between epistemological, decision-theoretic, and aesthetic aspects of morality (inc ref to Hume's Guillotine)
The concept of continuous agency and the idea that the first AGI will be a messy admixture of various components
The potential for AI systems to become more physically embedded in the future
The challenges of aligning AI systems and the societal impacts of AI technologies like ChatGPT and Bing
The importance of humility in the face of complexity when considering the fu…
-
Prof. Chris Bishop's NEW Deep Learning Textbook!
10 april 2024 · 1 uur 23 min
Professor Chris Bishop is a Technical Fellow and Director at Microsoft Research AI4Science, in Cambridge. He is also Honorary Professor of Computer Science at the University of Edinburgh, and a Fellow of Darwin College, Cambridge. In 2004, he was elected Fellow of the Royal Academy of Engineering, in 2007 he was elected Fellow of the Royal Society of Edinburgh, and in 2017 he was elected Fellow of the Royal Society. Chris was a founding member of the UK AI Council, and in 2019 he was appointed to the Prime Minister’s Council for Science and Technology. Lees meer
At Microsoft Research, Chris oversees a global portfolio of industrial research and development, with a strong focus on machine learning and the natural sciences.
Chris obtained a BA in Physics from Oxford, and a PhD in Theoretical Physics from the University of Edinburgh, with a thesis on quantum field theory.
Chris's contributions to the field of machine learning have been truly remarkable. He has authored (what is arguably) the original textbook in the field - 'Pattern Recognition and Machine Learning' (PRML) which has served as an essential reference for countless students and researchers around the world, and that was his second textbook after his highly acclaimed first textbook Neural Networks for Pattern Recognition.
Recently, Chris has co-authored a new book with his son, Hugh, titled 'Deep Learning: Foundations and Concepts.' This book aims to provide a comprehensive understanding of the key ideas and techniques underpinning the rapidly evolving field of deep learning. It covers both the foundational concepts and the latest advances, making it an invaluable resource for newcomers and experienced practitioners alike.
Buy Chris' textbook here:
https://amzn.to/3vvLcCh
More about Prof. Chris Bishop:
https://en.wikipedia.org/wiki/Christopher_Bishop
https://www.microsoft.com/en-us/research/people/cmbishop/
Support MLST:
Please support us on Patreon. We are entirely funded from Patreon donations right no…
-
Philip Ball - How Life Works
7 april 2024 · 2 uur 9 min
Dr. Philip Ball is a freelance science writer. He just wrote a book called "How Life Works", discussing the how the science of Biology has advanced in the last 20 years. We focus on the concept of Agency in particular. Lees meer
He trained as a chemist at the University of Oxford, and as a physicist at the University of Bristol. He worked previously at Nature for over 20 years, first as an editor for physical sciences and then as a consultant editor. His writings on science for the popular press have covered topical issues ranging from cosmology to the future of molecular biology.
YT: https://www.youtube.com/watch?v=n6nxUiqiz9I
Transcript link on YT description
Philip is the author of many popular books on science, including H2O: A Biography of Water, Bright Earth: The Invention of Colour, The Music Instinct and Curiosity: How Science Became Interested in Everything. His book Critical Mass won the 2005 Aventis Prize for Science Books, while Serving the Reich was shortlisted for the Royal Society Winton Science Book Prize in 2014.
This is one of Tim's personal favourite MLST shows, so we have designated it a special edition. Enjoy!
Buy Philip's book "How Life Works" here: https://amzn.to/3vSmNqp
Support MLST:
Please support us on Patreon. We are entirely funded from Patreon donations right now. Patreon supports get private discord access, biweekly calls, early-access + exclusive content and lots more.
https://patreon.com/mlst
Donate: https://www.paypal.com/donate/?hosted...
If you would like to sponsor us, so we can tell your story - reach out on mlstreettalk at gmail -
Dr. Paul Lessard - Categorical/Structured Deep Learning
1 april 2024 · 1 uur 49 min
Dr. Paul Lessard and his collaborators have written a paper on "Categorical Deep Learning and Algebraic Theory of Architectures". They aim to make neural networks more interpretable, composable and amenable to formal reasoning. The key is mathematical abstraction, as exemplified by category theory - using monads to develop a more principled, algebraic approach to structuring neural networks. Lees meer
We also discussed the limitations of current neural network architectures in terms of their ability to generalise and reason in a human-like way. In particular, the inability of neural networks to do unbounded computation equivalent to a Turing machine. Paul expressed optimism that this is not a fundamental limitation, but an artefact of current architectures and training procedures.
The power of abstraction - allowing us to focus on the essential structure while ignoring extraneous details. This can make certain problems more tractable to reason about. Paul sees category theory as providing a powerful "Lego set" for productively thinking about many practical problems.
Towards the end, Paul gave an accessible introduction to some core concepts in category theory like categories, morphisms, functors, monads etc. We explained how these abstract constructs can capture essential patterns that arise across different domains of mathematics.
Paul is optimistic about the potential of category theory and related mathematical abstractions to put AI and neural networks on a more robust conceptual foundation to enable interpretability and reasoning. However, significant theoretical and engineering challenges remain in realising this vision.
Please support us on Patreon. We are entirely funded from Patreon donations right now.
https://patreon.com/mlst
If you would like to sponsor us, so we can tell your story - reach out on mlstreettalk at gmail
Links:
Categorical Deep Learning: An Algebraic Theory of Architectures
Bruno Gavranović, Paul Lessard, Andrew Dudzik,
Tamara von Glehn, …
-
Can we build a generalist agent? Dr. Minqi Jiang and Dr. Marc Rigter
20 maart 2024 · 1 uur 57 min
Dr. Minqi Jiang and Dr. Marc Rigter explain an innovative new method to make the intelligence of agents more general-purpose by training them to learn many worlds before their usual goal-directed training, which we call "reinforcement learning". Lees meer
Their new paper is called "Reward-free curricula for training robust world models" https://arxiv.org/pdf/2306.09205.pdf
https://twitter.com/MinqiJiang
https://twitter.com/MarcRigterInterviewer: Dr. Tim Scarfe
Please support us on Patreon, Tim is now doing MLST full-time and taking a massive financial hit. If you love MLST and want this to continue, please show your support! In return you get access to shows very early and private discord and networking. https://patreon.com/mlst
We are also looking for show sponsors, please get in touch if interested mlstreettalk at gmail.
MLST Discord: https://discord.gg/machine-learning-street-talk-mlst-937356144060530778
-
Prof. Nick Chater - The Language Game (Part 1)
1 maart 2024 · 1 uur 44 min
Nick Chater is Professor of Behavioural Science at Warwick Business School, who works on rationality and language using a range of theoretical and experimental approaches. We discuss his books The Mind is Flat, and the Language Game. Lees meer
Please support me on Patreon (this is now my main job!) - https://patreon.com/mlst - Access the private Discord, networking, and early access to content.
MLST Discord: https://discord.gg/machine-learning-street-talk-mlst-937356144060530778
https://twitter.com/MLStreetTalk
Buy The Language Game:
https://amzn.to/3SRHjPm
Buy The Mind is Flat:
https://amzn.to/3P3BUUC
YT version: https://youtu.be/5cBS6COzLN4
https://www.wbs.ac.uk/about/person/nick-chater/
https://twitter.com/nickjchater?lang=en
-
Kenneth Stanley created a new social network based on serendipity and divergence
28 februari 2024 · 3 uur 15 min
See what Sam Altman advised Kenneth when he left OpenAI! Professor Kenneth Stanley has just launched a brand new type of social network, which he calls a "Serendipity network". The idea is that you follow interests, NOT people. It's a social network without the popularity contest. We discuss the phgilosophy and technology behind the venture in great detail. The main ideas of which came from Kenneth's famous book "Why greatness cannot be planned". Lees meer
See what Sam Altman advised Kenneth when he left OpenAI! Professor Kenneth Stanley has just launched a brand new type of social network, which he calls a "Serendipity network".The idea is that you follow interests, NOT people. It's a social network without the popularity contest.
YT version: https://www.youtube.com/watch?v=pWIrXN-yy8g
Chapters should be baked into the MP3 file now
MLST public Discord: https://discord.gg/machine-learning-street-talk-mlst-937356144060530778
Please support our work on Patreon - get access to interviews months early, private Patreon, networking, exclusive content and regular calls with Tim and Keith.
https://patreon.com/mlstGet Maven here:
https://www.heymaven.com/Kenneth:
https://twitter.com/kenneth0stanley
https://www.kenstanley.net/homeHost - Tim Scarfe:
https://www.linkedin.com/in/ecsquizor/
https://www.mlst.ai/Original MLST show with Kenneth:
https://www.youtube.com/watch?v=lhYGXYeMq_ETim explains the book more here:
https://www.youtube.com/watch?v=wNhaz81OOqw
-
Dr. Brandon Rohrer - Robotics, Creativity and Intelligence
13 februari 2024 · 1 uur 32 min
Brandon Rohrer who obtained his Ph.D from MIT is driven by understanding algorithms ALL the way down to their nuts and bolts, so he can make them accessible to everyone by first explaining them in the way HE himself would have wanted to learn! Lees meer
Please support us on Patreon for loads of exclusive content and private Discord:
https://patreon.com/mlst (public discord)
https://discord.gg/aNPkGUQtc5
https://twitter.com/MLStreetTalk
Brandon Rohrer is a seasoned data science leader and educator with a rich background in creating robust, efficient machine learning algorithms and tools. With a Ph.D. in Mechanical Engineering from MIT, his expertise encompasses a broad spectrum of AI applications — from computer vision and natural language processing to reinforcement learning and robotics. Brandon's career has seen him in Principle-level roles at Microsoft and Facebook. An educator at heart, he also shares his knowledge through detailed tutorials, courses, and his forthcoming book, "How to Train Your Robot."
YT version: https://www.youtube.com/watch?v=4Ps7ahonRCY
Brandon's links:
https://github.com/brohrer
https://www.youtube.com/channel/UCsBKTrp45lTfHa_p49I2AEQ
https://www.linkedin.com/in/brohrer/
How transformers work:
https://e2eml.school/transformers
Brandon's End-to-End Machine Learning school courses, posts, and tutorials
https://e2eml.school
Free course:
https://end-to-end-machine-learning.teachable.com/p/complete-course-library-full-end-to-end-machine-learning-catalog
Blog: https://e2eml.school/blog.html
Ziptie: Learning Useful Features [Brandon Rohrer]
https://www.brandonrohrer.com/ziptie
TOC should be baked into the MP3 file now
00:00:00 - Intro to Brandon
00:00:36 - RLHF
00:01:09 - Limitations of transformers
00:07:23 - Agency - we are all GPTs
00:09:07 - BPE / representation bias
00:12:00 - LLM true believers
00:16:42 - Brandon's style of teaching
00:19:50 - ML vs real world = Robotics
00:29:59 - Reward shaping
00:37:08 - No true Scot…
-
Showdown Between e/acc Leader And Doomer - Connor Leahy + Beff Jezos
3 februari 2024 · 3 uur
The world's second-most famous AI doomer Connor Leahy sits down with Beff Jezos, the founder of the e/acc movement debating technology, AI policy, and human values. As the two discuss technology, AI safety, civilization advancement, and the future of institutions, they clash on their opposing perspectives on how we steer humanity towards a more optimal path. Lees meer
Watch behind the scenes, get early access and join the private Discord by supporting us on Patreon. We have some amazing content going up there with Max Bennett and Kenneth Stanley this week!
https://patreon.com/mlst (public discord)
https://discord.gg/aNPkGUQtc5
https://twitter.com/MLStreetTalkPost-interview with Beff and Connor: https://www.patreon.com/posts/97905213
Pre-interview with Connor and his colleague Dan Clothiaux: https://www.patreon.com/posts/connor-leahy-and-97631416
Leahy, known for his critical perspectives on AI and technology, challenges Jezos on a variety of assertions related to the accelerationist movement, market dynamics, and the need for regulation in the face of rapid technological advancements. Jezos, on the other hand, provides insights into the e/acc movement's core philosophies, emphasizing growth, adaptability, and the dangers of over-legislation and centralized control in current institutions.
Throughout the discussion, both speakers explore the concept of entropy, the role of competition in fostering innovation, and the balance needed to mediate order and chaos to ensure the prosperity and survival of civilization. They weigh up the risks and rewards of AI, the importance of maintaining a power equilibrium in society, and the significance of cultural and institutional dynamism.
Beff Jezos (Guillaume Verdon):
https://twitter.com/BasedBeffJezos
https://twitter.com/GillVerdConnor Leahy:
https://twitter.com/npcollapseYT: https://www.youtube.com/watch?v=0zxi0xSBOaQ
TOC:
00:00:00 - Intro
00:03:05 - Society library reference
00:03:35 - Debate starts
00:05:08 - Should an…
-
Mahault Albarracin - Cognitive Science
14 januari 2024 · 1 uur 7 min
Watch behind the scenes, get early access and join the private Discord by supporting us on Patreon: Lees meer
https://patreon.com/mlst (public discord)
https://discord.gg/aNPkGUQtc5
https://twitter.com/MLStreetTalk
YT version: https://youtu.be/n8G50ynU0Vg
In this interview on MLST, Dr. Tim Scarfe interviews Mahault Albarracin, who is the director of product for R&D at VERSES and also a PhD student in cognitive computing at the University of Quebec in Montreal. They discuss a range of topics related to consciousness, cognition, and machine learning.
Throughout the conversation, they touch upon various philosophical and computational concepts such as panpsychism, computationalism, and materiality. They consider the "hard problem" of consciousness, which is the question of how and why we have subjective experiences.
Albarracin shares her views on the controversial Integrated Information Theory and the open letter of opposition it received from the scientific community. She reflects on the nature of scientific critique and rivalry, advising caution in declaring entire fields of study as pseudoscientific.
A substantial part of the discussion is dedicated to the topic of science itself, where Albarracin talks about thresholds between legitimate science and pseudoscience, the role of evidence, and the importance of validating scientific methods and claims.
They touch upon language models, discussing whether they can be considered as having a "theory of mind" and the implications of assigning such properties to AI systems. Albarracin challenges the idea that there is a pure form of intelligence independent of material constraints and emphasizes the role of sociality in the development of our cognitive abilities.
Albarracin offers her thoughts on scientific endeavors, the predictability of systems, the nature of intelligence, and the processes of learning and adaptation. She gives insights into the concept of using degeneracy as a way to increase resilience within systems a…
-
$450M AI Startup In 3 Years | Chai AI
9 januari 2024 · 30 min
Chai AI is the leading platform for conversational chat artificial intelligence. Lees meer
Note: this is a sponsored episode of MLST.
William Beauchamp is the founder of two $100M+ companies - Chai Research, an AI startup, and Seamless Capital, a hedge fund based in Cambridge, UK.
Chaiverse is the Chai AI developer platform, where developers can train, submit and evaluate on millions of real users to win their share of $1,000,000.
https://www.chai-research.com
https://www.chaiverse.com
https://twitter.com/chai_research
https://facebook.com/chairesearch/
https://www.instagram.com/chairesearch/
Download the app on iOS and Android (https://onelink.to/kqzhy9 )#chai #chai_ai #chai_research #chaiverse #generative_ai #LLMs
-
DOES AI HAVE AGENCY? With Professor. Karl Friston and Riddhi J. Pitliya
7 januari 2024 · 1 uur 3 min
Watch behind the scenes, get early access and join the private Discord by supporting us on Patreon: Lees meer
https://patreon.com/mlst (public discord)
https://discord.gg/aNPkGUQtc5
https://twitter.com/MLStreetTalk
DOES AI HAVE AGENCY? With Professor. Karl Friston and Riddhi J. Pitliya
Agency in the context of cognitive science, particularly when considering the free energy principle, extends beyond just human decision-making and autonomy. It encompasses a broader understanding of how all living systems, including non-human entities, interact with their environment to maintain their existence by minimising sensory surprise.
According to the free energy principle, living organisms strive to minimize the difference between their predicted states and the actual sensory inputs they receive. This principle suggests that agency arises as a natural consequence of this process, particularly when organisms appear to plan ahead many steps in the future.
Riddhi J. Pitliya is based in the computational psychopathology lab doing her Ph.D at the University of Oxford and works with Professor Karl Friston at VERSES.
https://twitter.com/RiddhiJP
References:
THE FREE ENERGY PRINCIPLE—A PRECIS [Ramstead]
https://www.dialecticalsystems.eu/contributions/the-free-energy-principle-a-precis/
Active Inference: The Free Energy Principle in Mind, Brain, and Behavior [Thomas Parr, Giovanni Pezzulo, Karl J. Friston]
https://direct.mit.edu/books/oa-monograph/5299/Active-InferenceThe-Free-Energy-Principle-in-Mind
The beauty of collective intelligence, explained by a developmental biologist | Michael Levin
https://www.youtube.com/watch?v=U93x9AWeuOA
Growing Neural Cellular Automata
https://distill.pub/2020/growing-ca
Carcinisation
https://en.wikipedia.org/wiki/Carcinisation
Prof. KENNETH STANLEY - Why Greatness Cannot Be Planned
https://www.youtube.com/watch?v=lhYGXYeMq_E
On Defining Artificial Intelligence [Pei Wang]
https://sciendo.com/article/10.2478/jagi-2019-0002
Why? The Purpo…
-
Understanding Deep Learning - Prof. SIMON PRINCE [STAFF FAVOURITE]
26 december 2023 · 2 uur 7 min
Watch behind the scenes, get early access and join private Discord by supporting us on Patreon: https://patreon.com/mlst Lees meer
https://discord.gg/aNPkGUQtc5
https://twitter.com/MLStreetTalk
In this comprehensive exploration of the field of deep learning with Professor Simon Prince who has just authored an entire text book on Deep Learning, we investigate the technical underpinnings that contribute to the field's unexpected success and confront the enduring conundrums that still perplex AI researchers.
Key points discussed include the surprising efficiency of deep learning models, where high-dimensional loss functions are optimized in ways which defy traditional statistical expectations. Professor Prince provides an exposition on the choice of activation functions, architecture design considerations, and overparameterization. We scrutinize the generalization capabilities of neural networks, addressing the seeming paradox of well-performing overparameterized models. Professor Prince challenges popular misconceptions, shedding light on the manifold hypothesis and the role of data geometry in informing the training process. Professor Prince speaks about how layers within neural networks collaborate, recursively reconfiguring instance representations that contribute to both the stability of learning and the emergence of hierarchical feature representations. In addition to the primary discussion on technical elements and learning dynamics, the conversation briefly diverts to audit the implications of AI advancements with ethical concerns.
Follow Prof. Prince:
https://twitter.com/SimonPrinceAI
https://www.linkedin.com/in/simon-prince-615bb9165/
Get the book now!
https://mitpress.mit.edu/9780262048644/understanding-deep-learning/
https://udlbook.github.io/udlbook/
Panel: Dr. Tim Scarfe -
https://www.linkedin.com/in/ecsquizor/
https://twitter.com/ecsquendor
TOC:
[00:00:00] Introduction
[00:11:03] General Book Discussion
[00:15:30] The Neural Metaphor
[00:17:56] …
-
Prof. BERT DE VRIES - ON ACTIVE INFERENCE
20 november 2023 · 2 uur 28 min
Watch behind the scenes with Bert on Patreon: https://www.patreon.com/posts/bert-de-vries-93230722
https://discord.gg/aNPkGUQtc5
https://twitter.com/MLStreetTalk Lees meerNote, there is some mild background music on chapter 1 (Least Action), 3 (Friston) and 5 (Variational Methods) - please skip ahead if annoying. It's a tiny fraction of the overall podcast.
YT version: https://youtu.be/2wnJ6E6rQsU
Bert de Vries is Professor in the Signal Processing Systems group at Eindhoven University. His research focuses on the development of intelligent autonomous agents that learn from in-situ interactions with their environment. His research draws inspiration from diverse fields including computational neuroscience, Bayesian machine learning, Active Inference and signal processing.
Bert believes that development of signal processing systems will in the future be largely automated by autonomously operating agents that learn purposeful from situated environmental interactions.
Bert received nis M.Sc. (1986) and Ph.D. (1991) degrees in Electrical Engineering from Eindhoven University of Technology (TU/e) and the University of Florida, respectively. From 1992 to 1999, he worked as a research scientist at Sarnoff Research Center in Princeton (NJ, USA). Since 1999, he has been employed in the hearing aids industry, both in engineering and managerial positions. De Vries was appointed part-time professor in the Signal Processing Systems Group at TU/e in 2012.
Contact:
https://twitter.com/bertdv0
https://www.tue.nl/en/research/researchers/bert-de-vries
https://www.verses.ai/about-usPanel: Dr. Tim Scarfe / Dr. Keith Duggar
TOC:
[00:00:00] Principle of Least Action
[00:05:10] Patreon Teaser
[00:05:46] On Friston
[00:07:34] Capm Peterson (VERSES)
[00:08:20] Variational Methods
[00:16:13] Dan Mapes (VERSES)
[00:17:12] Engineering with Active Inference
[00:20:23] Jason Fox (VERSES)
[00:20:51] Riddhi Jain Pitliya
[00:21:49] Hearing Aids as Adaptive Agents
[00:33:38] Steven Swanson (VERSES)
… -
MULTI AGENT LEARNING - LANCELOT DA COSTA
5 november 2023 · 50 min
Please support us https://www.patreon.com/mlst Lees meer
https://discord.gg/aNPkGUQtc5
https://twitter.com/MLStreetTalk
Lance Da Costa aims to advance our understanding of intelligent systems by modelling cognitive systems and improving artificial systems.
He's a PhD candidate with Greg Pavliotis and Karl Friston jointly at Imperial College London and UCL, and a student in the Mathematics of Random Systems CDT run by Imperial College London and the University of Oxford. He completed an MRes in Brain Sciences at UCL with Karl Friston and Biswa Sengupta, an MASt in Pure Mathematics at the University of Cambridge with Oscar Randal-Williams, and a BSc in Mathematics at EPFL and the University of Toronto.
Summary:
Lance did pure math originally but became interested in the brain and AI. He started working with Karl Friston on the free energy principle, which claims all intelligent agents minimize free energy for perception, action, and decision-making. Lance has worked to provide mathematical foundations and proofs for why the free energy principle is true, starting from basic assumptions about agents interacting with their environment. This aims to justify the principle from first physics principles. Dr. Scarfe and Da Costa discuss different approaches to AI - the free energy/active inference approach focused on mimicking human intelligence vs approaches focused on maximizing capability like deep reinforcement learning. Lance argues active inference provides advantages for explainability and safety compared to black box AI systems. It provides a simple, sparse description of intelligence based on a generative model and free energy minimization. They discuss the need for structured learning and acquiring core knowledge to achieve more human-like intelligence. Lance highlights work from Josh Tenenbaum's lab that shows similar learning trajectories to humans in a simple Atari-like environment.
Incorporating core knowledge constraints the space of possible generative models t…
-
THE HARD PROBLEM OF OBSERVERS - WOLFRAM & FRISTON [SPECIAL EDITION]
29 oktober 2023 · 1 uur 59 min
Please support us! https://www.patreon.com/mlst
https://discord.gg/aNPkGUQtc5
https://twitter.com/MLStreetTalk Lees meerYT version (with intro not found here) https://youtu.be/6iaT-0Dvhnc
This is the epic special edition show you have been waiting for! With two of the most brilliant scientists alive today.
Atoms, things, agents, ... observers. What even defines an "observer" and what properties must all observers share? How do objects persist in our universe given that their material composition changes over time? What does it mean for a thing to be a thing? And do things supervene on our lower-level physical reality? What does it mean for a thing to have agency? What's the difference between a complex dynamical system with and without agency? Could a rock or an AI catflap have agency? Can the universe be factorised into distinct agents, or is agency diffused? Have you ever pondered about these deep questions about reality?
Prof. Friston and Dr. Wolfram have spent their entire careers, some 40+ years each thinking long and hard about these very questions and have developed significant frameworks of reference on their respective journeys (the Wolfram Physics project and the Free Energy principle).
Panel: MIT Ph.D Keith Duggar
Production: Dr. Tim ScarfeRefs:
TED Talk with Stephen:
https://www.ted.com/talks/stephen_wolfram_how_to_think_computationally_about_ai_the_universe_and_everything
https://writings.stephenwolfram.com/2023/10/how-to-think-computationally-about-ai-the-universe-and-everything/TOC
00:00:00 - Show kickoff00:02:38 - Wolfram gets to grips with FEP
00:27:08 - How much control does an agent/observer have
00:34:52 - Observer persistence, what universe seems like to us
00:40:31 - Black holes
00:45:07 - Inside vs outside
00:52:20 - Moving away from the predictable path
00:55:26 - What can observers do
01:06:50 - Self modelling gives agency
01:11:26 - How do you know a thing has agency?
01:22:48 - Deep link between dynamics, ruliad and AI
01:25:5…
-
DR. JEFF BECK - THE BAYESIAN BRAIN
16 oktober 2023 · 1 uur 10 min
Support us! https://www.patreon.com/mlst Lees meer
MLST Discord: https://discord.gg/aNPkGUQtc5
YT version: https://www.youtube.com/watch?v=c4praCiy9qU
Dr. Jeff Beck is a computational neuroscientist studying probabilistic reasoning (decision making under uncertainty) in humans and animals with emphasis on neural representations of uncertainty and cortical implementations of probabilistic inference and learning. His line of research incorporates information theoretic and hierarchical statistical analysis of neural and behavioural data as well as reinforcement learning and active inference.
https://www.linkedin.com/in/jeff-beck...
https://scholar.google.com/citations?...
Interviewer: Dr. Tim Scarfe
TOC
00:00:00 Intro
00:00:51 Bayesian / Knowledge
00:14:57 Active inference
00:18:58 Mediation
00:23:44 Philosophy of mind / science
00:29:25 Optimisation
00:42:54 Emergence
00:56:38 Steering emergent systems
01:04:31 Work plan
01:06:06 Representations/Core knowledge
#activeinference
-
Prof. Melanie Mitchell 2.0 - AI Benchmarks are Broken!
10 september 2023 · 1 uur 2 min
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB Lees meerProf. Melanie Mitchell argues that the concept of "understanding" in AI is ill-defined and multidimensional - we can't simply say an AI system does or doesn't understand. She advocates for rigorously testing AI systems' capabilities using proper experimental methods from cognitive science. Popular benchmarks for intelligence often rely on the assumption that if a human can perform a task, an AI that performs the task must have human-like general intelligence. But benchmarks should evolve as capabilities improve.
Large language models show surprising skill on many human tasks but lack common sense and fail at simple things young children can do. Their knowledge comes from statistical relationships in text, not grounded concepts about the world. We don't know if their internal representations actually align with human-like concepts. More granular testing focused on generalization is needed.
There are open questions around whether large models' abilities constitute a fundamentally different non-human form of intelligence based on vast statistical correlations across text. Mitchell argues intelligence is situated, domain-specific and grounded in physical experience and evolution. The brain computes but in a specialized way honed by evolution for controlling the body. Extracting "pure" intelligence may not work.
Other key points:
- Need more focus on proper experimental method in AI research. Developmental psychology offers examples for rigorous testing of cognition.
- Reporting instance-level failures rather than just aggregate accuracy can provide insights.
- Scaling laws and complex systems science are an interesting area of complexity theory, with applications to understanding cities.
- Concepts like "understanding" and "intelligence" in AI force refinement of fuzzy definitions.
- Human intelligence may be more collective and social than we realize. AI forces us to rethink concepts we … -
Autopoitic Enactivism and the Free Energy Principle - Prof. Friston, Prof Buckley, Dr. Ramstead
5 september 2023 · 1 uur 35 min
We explore connections between FEP and enactivism, including tensions raised in a paper critiquing FEP from an enactivist perspective. Lees meer
Dr. Maxwell Ramstead provides background on enactivism emerging from autopoiesis, with a focus on embodied cognition and rejecting information processing/computational views of mind.
Chris shares his journey from robotics into FEP, starting as a skeptic but becoming convinced it's the right framework. He notes there are both "high road" and "low road" versions, ranging from embodied to more radically anti-representational stances. He doesn't see a definitive fork between dynamical systems and information theory as the source of conflict. Rather, the notion of operational closure in enactivism seems to be the main sticking point.
The group explores definitional issues around structure/organization, boundaries, and operational closure. Maxwell argues the generative model in FEP captures organizational dependencies akin to operational closure. The Markov blanket formalism models structural interfaces.
We discuss the concept of goals in cognitive systems - Chris advocates an intentional stance perspective - using notions of goals/intentions if they help explain system dynamics. Goals emerge from beliefs about dynamical trajectories. Prof Friston provides an elegant explanation of how goal-directed behavior naturally falls out of the FEP mathematics in a particular "goldilocks" regime of system scale/dynamics. The conversation explores the idea that many systems simply act "as if" they have goals or models, without necessarily possessing explicit representations. This helps resolve tensions between enactivist and computational perspectives.
Throughout the dialogue, Maxwell presses philosophical points about the FEP abolishing what he perceives as false dichotomies in cognitive science such as internalism/externalism. He is critical of enactivists' commitment to bright line divides between subject areas.
Prof. Karl Friston - Invento…
-
STEPHEN WOLFRAM 2.0 - Resolving the Mystery of the Second Law of Thermodynamics
15 augustus 2023 · 1 uur 24 min
Please check out Numerai - our sponsor @
http://numer.ai/mlst Lees meerPatreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CBThe Second Law: Resolving the Mystery of the Second Law of Thermodynamics
Buy Stephen's book here - https://tinyurl.com/2jj2t9waThe Language Game: How Improvisation Created Language and Changed the World by Morten H. Christiansen and Nick Chater
Buy here: https://tinyurl.com/35bvs8beStephen Wolfram starts by discussing the second law of thermodynamics - the idea that entropy, or disorder, tends to increase over time. He talks about how this law seems intuitively true, but has been difficult to prove. Wolfram outlines his decades-long quest to fully understand the second law, including failed early attempts to simulate particles mixing as a 12-year-old. He explains how irreversibility arises from the computational irreducibility of underlying physical processes coupled with our limited ability as observers to do the computations needed to "decrypt" the microscopic details.
The conversation then shifts to discussing language and how concepts allow us to communicate shared ideas between minds positioned in different parts of "rule space." Wolfram talks about the successes and limitations of using large language models to generate Wolfram Language code from natural language prompts. He sees it as a useful tool for getting started programming, but one still needs human refinement.
The final part of the conversation focuses on AI safety and governance. Wolfram notes uncontrolled actuation is where things can go wrong with AI systems. He discusses whether AI agents could have intrinsic experiences and goals, how we might build trust networks between AIs, and that managing a system of many AIs may be easier than a single AI. Wolfram emphasizes the need for more philosophical depth in thinking about AI aims, and draws connections between potential solutions and his work on computational irreducibility and physics.
Show notes: ht…
-
Prof. Jürgen Schmidhuber - FATHER OF AI ON ITS DANGERS
14 augustus 2023 · 1 uur 21 min
Please check out Numerai - our sponsor @
http://numer.ai/mlst Lees meerPatreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CBProfessor Jürgen Schmidhuber, the father of artificial intelligence, joins us today. Schmidhuber discussed the history of machine learning, the current state of AI, and his career researching recursive self-improvement, artificial general intelligence and its risks.
Schmidhuber pointed out the importance of studying the history of machine learning to properly assign credit for key breakthroughs. He discussed some of the earliest machine learning algorithms. He also highlighted the foundational work of Leibniz, who discovered the chain rule that enables training of deep neural networks, and the ancient Antikythera mechanism, the first known gear-based computer.
Schmidhuber discussed limits to recursive self-improvement and artificial general intelligence, including physical constraints like the speed of light and what can be computed. He noted we have no evidence the human brain can do more than traditional computing. Schmidhuber sees humankind as a potential stepping stone to more advanced, spacefaring machine life which may have little interest in humanity. However, he believes commercial incentives point AGI development towards being beneficial and that open-source innovation can help to achieve "AI for all" symbolised by his company's motto "AI∀".
Schmidhuber discussed approaches he believes will lead to more general AI, including meta-learning, reinforcement learning, building predictive world models, and curiosity-driven learning. His "fast weight programming" approach from the 1990s involved one network altering another network's connections. This was actually the first Transformer variant, now called an unnormalised linear Transformer. He also described the first GANs in 1990, to implement artificial curiosity.
Schmidhuber reflected on his career researching AI. He said his fondest memories were gaining insights tha…
-
Can We Develop Truly Beneficial AI? George Hotz and Connor Leahy
4 augustus 2023 · 1 uur 30 min
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB Lees meerGeorge Hotz and Connor Leahy discuss the crucial challenge of developing beneficial AI that is aligned with human values. Hotz believes truly aligned AI is impossible, while Leahy argues it's a solvable technical challenge.
Hotz contends that AI will inevitably pursue power, but distributing AI widely would prevent any single AI from dominating. He advocates open-sourcing AI developments to democratize access. Leahy counters that alignment is necessary to ensure AIs respect human values. Without solving alignment, general AI could ignore or harm humans.
They discuss whether AI's tendency to seek power stems from optimization pressure or human-instilled goals. Leahy argues goal-seeking behavior naturally emerges while Hotz believes it reflects human values. Though agreeing on AI's potential dangers, they differ on solutions. Hotz favors accelerating AI progress and distributing capabilities while Leahy wants safeguards put in place.
While acknowledging risks like AI-enabled weapons, they debate whether broad access or restrictions better manage threats. Leahy suggests limiting dangerous knowledge, but Hotz insists openness checks government overreach. They concur that coordination and balance of power are key to navigating the AI revolution. Both eagerly anticipate seeing whose ideas prevail as AI progresses.Transcript and notes: https://docs.google.com/document/d/1smkmBY7YqcrhejdbqJOoZHq-59LZVwu-DNdM57IgFcU/edit?usp=sharing
Note: this is not a normal episode i.e. the hosts are not part of the debate (and for the record don't agree with Connor or George).
TOC:
[00:00:00] Introduction to George Hotz and Connor Leahy
[00:03:10] George Hotz's Opening Statement: Intelligence and Power
[00:08:50] Connor Leahy's Opening Statement: Technical Problem of Alignment and Coordination
[00:15:18] George Hotz's Response: Nature of Cooperation and Individual Sovereignty
[00:17:32] Discussion on indivi… -
Dr. MAXWELL RAMSTEAD - The Physics of Survival
16 juli 2023 · 2 uur 6 min
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB Lees meerJoin us for a fascinating discussion of the free energy principle with Dr. Maxwell Ramsted, a leading thinker exploring the intersection of math, physics, and philosophy and Director of Research at VERSES. The FEP was proposed by renowned neuroscientist Karl Friston, this principle offers a unifying theory explaining how systems maintain order and their identity.
The free energy principle inverts traditional survival logic. Rather than asking what behaviors promote survival, it queries - given things exist, what must they do? The answer: minimizing free energy, or "surprise." Systems persist by constantly ensuring their internal states match anticipated states based on a model of the world. Failure to minimize surprise leads to chaos as systems dissolve into disorder.
Thus, the free energy principle elucidates why lifeforms relentlessly model and predict their surroundings. It is an existential imperative counterbalancing entropy. Essentially, this principle describes the mind's pursuit of harmony between expectations and reality. Its relevance spans from cells to societies, underlying order wherever longevity is found.
Our discussion explores the technical details and philosophical implications of this paradigm-shifting theory. How does it further our understanding of cognition and intelligence? What insights does it offer about the fundamental patterns and properties of existence? Can it precipitate breakthroughs in disciplines like neuroscience and artificial intelligence?
Dr. Ramstead completed his Ph.D. at McGill University in Montreal, Canada in 2019, with frequent research visits to UCL in London, under the supervision of the world’s most cited neuroscientist, Professor Karl Friston (UCL).
YT version: https://youtu.be/8qb28P7ksyE
https://scholar.google.ca/citations?user=ILpGOMkAAAAJ&hl=frhttps://spatialwebfoundation.org/team/maxwell-ramstead/https://www.linkedin.com/in/maxwel…
-
MUNK DEBATE ON AI (COMMENTARY) [DAVID FOSTER]
2 juli 2023 · 2 uur 8 min
Patreon: https://www.patreon.com/mlst Lees meer
Discord: https://discord.gg/ESrGqhf5CB
The discussion between Tim Scarfe and David Foster provided an in-depth critique of the arguments made by panelists at the Munk AI Debate on whether artificial intelligence poses an existential threat to humanity. While the panelists made thought-provoking points, Scarfe and Foster found their arguments largely speculative, lacking crucial details and evidence to support claims of an impending existential threat.
Scarfe and Foster strongly disagreed with Max Tegmark’s position that AI has an unparalleled “blast radius” that could lead to human extinction. Tegmark failed to provide a credible mechanism for how this scenario would unfold in reality. His arguments relied more on speculation about advanced future technologies than on present capabilities and trends. As Foster argued, we cannot conclude AI poses a threat based on speculation alone. Evidence is needed to ground discussions of existential risks in science rather than science fiction fantasies or doomsday scenarios.
They found Yann LeCun’s statements too broad and high-level, critiquing him for not providing sufficiently strong arguments or specifics to back his position. While LeCun aptly noted AI remains narrow in scope and far from achieving human-level intelligence, his arguments lacked crucial details on current limitations and why we should not fear superintelligence emerging in the near future. As Scarfe argued, without these details the discussion descended into “philosophy” rather than focusing on evidence and data.
Scarfe and Foster also took issue with Yoshua Bengio’s unsubstantiated speculation that machines would necessarily develop a desire for self-preservation that threatens humanity. There is no evidence today’s AI systems are developing human-like general intelligence or desires, let alone that these attributes would manifest in ways dangerous to humans. The question is not whether machines will eventually s…