Machine Learning Street Talk (MLST)
Alle afleveringen
272 afleveringen, van nieuw naar oud
-
[SPONSORED] The Digitized Self: AI, Identity and the Human Psyche (YouAi)
29 juni 2023 · 1 uur 46 min
Sponsored Episode - YouAi Lees meer
What if an AI truly knew you—your thoughts, values, aptitudes, and dreams? An AI that could enhance your life in profound ways by amplifying your strengths, augmenting your weaknesses, and connecting you with like-minded souls. That is the vision of YouAi.
YouAi founder Dmitri Shapiro believes digitizing our inner lives could unlock tremendous benefits. But mapping the human psyche also poses deep questions. As technology mediates our self-understanding, what risks rendering our minds in bits and algorithms? Could we gain a new means of flourishing or lose something intangible?
There are no easy answers, but YouAi offers a vision balanced by hard thinking. Shapiro discussed YouAi's app, which builds personalized AI assistants by learning how individuals think through interactive questions. As people share, YouAi develops a multidimensional model of their mind. Users get a tailored feed of prompts to continue engaging and teaching their AI.
YouAi's vision provides a glimpse into a future that could unsettle or fulfill our hopes. As technology mediates understanding ourselves and others, will we risk losing what makes us human or find new means of flourishing? YouAI believes that together, we can build a future where our minds contain infinite potential—and their technology helps unlock it. But we must proceed thoughtfully, upholding human dignity above all else. Our minds shape who we are. And who we can become.Digitise your mind today:
YouAi - https://YouAi.aiMIndStudio – https://YouAi.ai/mindstudioYouAi Mind Indexer - https://YouAi.ai/trainJoin the MLST discord and register for the YouAi event on July 13th: https://discord.gg/ESrGqhf5CBTOC:
0:00:00 - Introduction to Mind Digitization
0:09:31 - The YouAi Platform and Personal Applications
0:27:54 - The Potential of Group Alignment
0:30:28 - Applications in Human-to-Human Communication
0:35:43 - Applications in Interfacing with Digital Technology
0:43:41 - Introduction to the Project
… -
Joscha Bach and Connor Leahy on AI risk
20 juni 2023 · 1 uur 31 min
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
Twitter: https://twitter.com/MLStreetTalk Lees meerThe first 10 mins of audio from Joscha isn't great, it improves after.
Transcript and longer summary: https://docs.google.com/document/d/1TUJhlSVbrHf2vWoe6p7xL5tlTK_BGZ140QqqTudF8UI/edit?usp=sharing
Dr. Joscha Bach argued that general intelligence emerges from civilization, not individuals. Given our biological constraints, humans cannot achieve a high level of general intelligence on our own. Bach believes AGI may become integrated into all parts of the world, including human minds and bodies. He thinks a future where humans and AGI harmoniously coexist is possible if we develop a shared purpose and incentive to align. However, Bach is uncertain about how AI progress will unfold or which scenarios are most likely.
Bach argued that global control and regulation of AI is unrealistic. While regulation may address some concerns, it cannot stop continued progress in AI. He believes individuals determine their own values, so "human values" cannot be formally specified and aligned across humanity. For Bach, the possibility of building beneficial AGI is exciting but much work is still needed to ensure a positive outcome.
Connor Leahy believes we have more control over the future than the default outcome might suggest. With sufficient time and effort, humanity could develop the technology and coordination to build a beneficial AGI. However, the default outcome likely leads to an undesirable scenario if we do not actively work to build a better future. Leahy thinks finding values and priorities most humans endorse could help align AI, even if individuals disagree on some values.
Leahy argued a future where humans and AGI harmoniously coexist is ideal but will require substantial work to achieve. While regulation faces challenges, it remains worth exploring. Leahy believes limits to progress in AI exist but we are unlikely to reach them before …
-
Neel Nanda - Mechanistic Interpretability
18 juni 2023 · 4 uur 10 min
In this wide-ranging conversation, Tim Scarfe interviews Neel Nanda, a researcher at DeepMind working on mechanistic interpretability, which aims to understand the algorithms and representations learned by machine learning models. Neel discusses how models can represent their thoughts using motifs, circuits, and linear directional features which are often communicated via a "residual stream", an information highway models use to pass information between layers. Lees meer
Neel argues that "superposition", the ability for models to represent more features than they have neurons, is one of the biggest open problems in interpretability. This is because superposition thwarts our ability to understand models by decomposing them into individual units of analysis. Despite this, Neel remains optimistic that ambitious interpretability is possible, citing examples like his work reverse engineering how models do modular addition. However, Neel notes we must start small, build rigorous foundations, and not assume our theoretical frameworks perfectly match reality.
The conversation turns to whether models can have goals or agency, with Neel arguing they likely can based on heuristics like models executing long term plans towards some objective. However, we currently lack techniques to build models with specific goals, meaning any goals would likely be learned or emergent. Neel highlights how induction heads, circuits models use to track long range dependencies, seem crucial for phenomena like in-context learning to emerge.
On the existential risks from AI, Neel believes we should avoid overly confident claims that models will or will not be dangerous, as we do not understand them enough to make confident theoretical assertions. However, models could pose risks through being misused, having undesirable emergent properties, or being imperfectly aligned. Neel argues we must pursue rigorous empirical work to better understand and ensure model safety, avoid "philosophizing" about definition…
-
Prof. Daniel Dennett - Could AI Counterfeit People Destroy Civilization? (SPECIAL EDITION)
5 juni 2023 · 1 uur 15 min
Please check out Numerai - our sponsor using our link @ Lees meer
http://numer.ai/mlst
Numerai is a groundbreaking platform which is taking the data science world by storm. Tim has been using Numerai to build state-of-the-art models which predict the stock market, all while being a part of an inspiring community of data scientists from around the globe. They host the Numerai Data Science Tournament, where data scientists like us use their financial dataset to predict future stock market performance.
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
Twitter: https://twitter.com/MLStreetTalk
YT version: https://youtu.be/axJtywd9Tbo
In this fascinating interview, Dr. Tim Scarfe speaks with renowned philosopher Daniel Dennett about the potential dangers of AI and the concept of "Counterfeit People." Dennett raises concerns about AI being used to create artificial colleagues, and argues that preventing counterfeit AI individuals is crucial for societal trust and security.
They delve into Dennett's "Two Black Boxes" thought experiment, the Chinese Room Argument by John Searle, and discuss the implications of AI in terms of reversibility, reontologisation, and realism. Dr. Scarfe and Dennett also examine adversarial LLMs, mental trajectories, and the emergence of consciousness and semanticity in AI systems.
Throughout the conversation, they touch upon various philosophical perspectives, including Gilbert Ryle's Ghost in the Machine, Chomsky's work, and the importance of competition in academia. Dennett concludes by highlighting the need for legal and technological barriers to protect against the dangers of counterfeit AI creations.
Join Dr. Tim Scarfe and Daniel Dennett in this thought-provoking discussion about the future of AI and the potential challenges we face in preserving our civilization. Don't miss this insightful conversation!
TOC:
00:00:00 Intro
00:09:56 Main show kick off
00:12:04 Counterfeit People
00:16:03 Reversibilit…
-
Decoding the Genome: Unraveling the Complexities with AI and Creativity [Prof. Jim Hughes, Oxford]
1 juni 2023 · 43 min
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
Twitter: https://twitter.com/MLStreetTalk Lees meerIn this eye-opening discussion between Tim Scarfe and Prof. Jim Hughes, a professor of gene regulation at Oxford University, they explore the intersection of creativity, genomics, and artificial intelligence. Prof. Hughes brings his expertise in genomics and insights from his interdisciplinary research group, which includes machine learning experts, mathematicians, and molecular biologists.
The conversation begins with an overview of Prof. Hughes' background and the importance of creativity in scientific research. They delve into the challenges of unlocking the secrets of the human genome and how machine learning, specifically convolutional neural networks, can assist in decoding genome function.
As they discuss validation and interpretability concerns in machine learning, they acknowledge the need for experimental tests and ponder the complex nature of understanding the basic code of life. They touch upon the fascinating world of morphogenesis and emergence, considering the potential crossovers into AI and their implications for self-repairing systems in medicine.
Examining the ethical and regulatory aspects of genomics and AI, the duo explores the implications of having access to someone's genome, the potential to predict traits or diseases, and the role of AI in understanding complex genetic signals. They also consider the challenges of keeping up with the rapidly expanding body of scientific research and the pressures faced by researchers in academia.
To wrap up the discussion, Tim and Prof. Hughes shed light on the significance of creativity and diversity in scientific research, emphasizing the need for divergent processes and diverse perspectives to foster innovation and avoid consensus-driven convergence.
Filmed at https://www.creativemachine.io/Prof. Jim Hughes: https://www.rdm.ox.ac.uk/people/jim-hughesDr. Tim Scarfe: https://… -
ROBERT MILES - "There is a good chance this kills everyone"
21 mei 2023 · 2 uur 2 min
Please check out Numerai - our sponsor @ Lees meer
https://numerai.com/mlst
Numerai is a groundbreaking platform which is taking the data science world by storm. Tim has been using Numerai to build state-of-the-art models which predict the stock market, all while being a part of an inspiring community of data scientists from around the globe. They host the Numerai Data Science Tournament, where data scientists like us use their financial dataset to predict future stock market performance.
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
Twitter: https://twitter.com/MLStreetTalk
Welcome to an exciting episode featuring an outstanding guest, Robert Miles! Renowned for his extraordinary contributions to understanding AI and its potential impacts on our lives, Robert is an artificial intelligence advocate, researcher, and YouTube sensation. He combines engaging discussions with entertaining content, captivating millions of viewers from around the world.
With a strong computer science background, Robert has been actively involved in AI safety projects, focusing on raising awareness about potential risks and benefits of advanced AI systems. His YouTube channel is celebrated for making AI safety discussions accessible to a diverse audience through breaking down complex topics into easy-to-understand nuggets of knowledge, and you might also recognise him from his appearances on Computerphile.
In this episode, join us as we dive deep into Robert's journey in the world of AI, exploring his insights on AI alignment, superintelligence, and the role of AI shaping our society and future. We'll discuss topics such as the limits of AI capabilities and physics, AI progress and timelines, human-machine hybrid intelligence, AI in conflict and cooperation with humans, and the convergence of AI communities.
Robert Miles:
@RobertMilesAI
https://twitter.com/robertskmiles
https://aisafety.info/
YT version: https://www.youtube.com/watch?v=kMLKbhY0ji0
… -
AI Senate Hearing - Executive Summary (Sam Altman, Gary Marcus)
17 mei 2023 · 50 min
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
Twitter: https://twitter.com/MLStreetTalk Lees meerIn a historic and candid Senate hearing, OpenAI CEO Sam Altman, Professor Gary Marcus, and IBM's Christina Montgomery discussed the regulatory landscape of AI in the US. The discussion was particularly interesting due to its timing, as it followed the recent release of the EU's proposed AI Act, which could potentially ban American companies like OpenAI and Google from providing API access to generative AI models and impose massive fines for non-compliance.
The speakers openly addressed potential risks of AI technology and emphasized the need for precision regulation. This was a unique approach, as historically, US companies have tried their hardest to avoid regulation. The hearing not only showcased the willingness of industry leaders to engage in discussions on regulation but also demonstrated the need for a balanced approach to avoid stifling innovation.
The EU AI Act, scheduled to come into power in 2026, is still just a proposal, but it has already raised concerns about its impact on the American tech ecosystem and potential conflicts between US and EU laws. With extraterritorial jurisdiction and provisions targeting open-source developers and software distributors like GitHub, the Act could create more problems than it solves by encouraging unsafe AI practices and limiting access to advanced AI technologies.
One core issue with the Act is the designation of foundation models in the highest risk category, primarily due to their open-ended nature. A significant risk theme revolves around users creating harmful content and determining who should be held accountable – the users or the platforms. The Senate hearing served as an essential platform to discuss these pressing concerns and work towards a regulatory framework that promotes both safety and innovation in AI.
00:00 Show
01:35 Legals
03:44 Intro
10:33 Altman intro
14:16 Chri…
-
Future of Generative AI [David Foster]
11 mei 2023 · 2 uur 32 min
Generative Deep Learning, 2nd Edition [David Foster] Lees meer
https://www.oreilly.com/library/view/generative-deep-learning/9781098134174/
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
Twitter: https://twitter.com/MLStreetTalk
In this conversation, Tim Scarfe and David Foster, the author of 'Generative Deep Learning,' dive deep into the world of generative AI, discussing topics ranging from model families and auto regressive models to the democratization of AI technology and its potential impact on various industries. They explore the connection between language and true intelligence, as well as the limitations of GPT and other large language models. The discussion also covers the importance of task-independent world models, the concept of active inference, and the potential of combining these ideas with transformer and GPT-style models.
Ethics and regulation in AI development are also discussed, including the need for transparency in data used to train AI models and the responsibility of developers to ensure their creations are not destructive. The conversation touches on the challenges posed by AI-generated content on copyright laws and the diminishing role of effort and skill in copyright due to generative models.
The impact of AI on education and creativity is another key area of discussion, with Tim and David exploring the potential benefits and drawbacks of using AI in the classroom, the need for a balance between traditional learning methods and AI-assisted learning, and the importance of teaching students to use AI tools critically and responsibly.
Generative AI in music is also explored, with David and Tim discussing the potential for AI-generated music to change the way we create and consume art, as well as the challenges in training AI models to generate music that captures human emotions and experiences.
Throughout the conversation, Tim and David touch on the potential risks and consequences of AI becoming too powe…
-
PERPLEXITY AI - The future of search.
8 mei 2023 · 1 uur
https://www.perplexity.ai/ Lees meer
https://www.perplexity.ai/iphone
https://www.perplexity.ai/android
Interview with Aravind Srinivas, CEO and Co-Founder of Perplexity AI – Revolutionizing Learning with Conversational Search Engines
Dr. Tim Scarfe talks with Dr. Aravind Srinivas, CEO and Co-Founder of Perplexity AI, about his journey from studying AI and reinforcement learning at UC Berkeley to launching Perplexity – a startup that aims to revolutionize learning through the power of conversational search engines. By combining the strengths of large language models like GPT-* with search engines, Perplexity provides users with direct answers to their questions in a decluttered user interface, making the learning process not only more efficient but also enjoyable.
Aravind shares his insights on how advertising can be made more relevant and less intrusive with the help of large language models, emphasizing the importance of transparency in relevance ranking to improve user experience. He also discusses the challenge of balancing the interests of users and advertisers for long-term success.
The interview delves into the challenges of maintaining truthfulness and balancing opinions and facts in a world where algorithmic truth is difficult to achieve. Aravind believes that opinionated models can be useful as long as they don't spread misinformation and are transparent about being opinions. He also emphasizes the importance of allowing users to correct or update information, making the platform more adaptable and dynamic.
Lastly, Aravind shares his thoughts on embracing a digital society with large language models, stressing the need for frequent and iterative deployments of these models to reduce fear of AI and misinformation. He envisions a future where using AI tools effectively requires clear thinking and first-principle reasoning, ultimately benefiting society as a whole. Education and transparency are crucial to counter potential misuse of AI for political or malicio…
-
#114 - Secrets of Deep Reinforcement Learning (Minqi Jiang)
16 april 2023 · 2 uur 47 min
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB
Twitter: https://twitter.com/MLStreetTalk Lees meerIn this exclusive interview, Dr. Tim Scarfe sits down with Minqi Jiang, a leading PhD student at University College London and Meta AI, as they delve into the fascinating world of deep reinforcement learning (RL) and its impact on technology, startups, and research. Discover how Minqi made the crucial decision to pursue a PhD in this exciting field, and learn from his valuable startup experiences and lessons.
Minqi shares his insights into balancing serendipity and planning in life and research, and explains the role of objectives and Goodhart's Law in decision-making. Get ready to explore the depths of robustness in RL, two-player zero-sum games, and the differences between RL and supervised learning.
As they discuss the role of environment in intelligence, emergence, and abstraction, prepare to be blown away by the possibilities of open-endedness and the intelligence explosion. Learn how language models generate their own training data, the limitations of RL, and the future of software 2.0 with interpretability concerns.
From robotics and open-ended learning applications to learning potential metrics and MDPs, this interview is a goldmine of information for anyone interested in AI, RL, and the cutting edge of technology. Don't miss out on this incredible opportunity to learn from a rising star in the AI world!
TOC
Tech & Startup Background [00:00:00]
Pursuing PhD in Deep RL [00:03:59]
Startup Lessons [00:11:33]
Serendipity vs Planning [00:12:30]
Objectives & Decision Making [00:19:19]
Minimax Regret & Uncertainty [00:22:57]
Robustness in RL & Zero-Sum Games [00:26:14]
RL vs Supervised Learning [00:34:04]
Exploration & Intelligence [00:41:27]
Environment, Emergence, Abstraction [00:46:31]
Open-endedness & Intelligence Explosion [00:54:28]
Language Models & Training Data [01:04:59]
RLHF & Language Models [01:16:37]
Creativity in …
-
Unlocking the Brain's Mysteries: Chris Eliasmith on Spiking Neural Networks and the Future of Human-Machine Interaction
10 april 2023 · 1 uur 50 min
Patreon: https://www.patreon.com/mlst Lees meer
Discord: https://discord.gg/ESrGqhf5CB
Twitter: https://twitter.com/MLStreetTalk
Chris Eliasmith is a renowned interdisciplinary researcher, author, and professor at the University of Waterloo, where he holds the prestigious Canada Research Chair in Theoretical Neuroscience. As the Founding Director of the Centre for Theoretical Neuroscience, Eliasmith leads the Computational Neuroscience Research Group in exploring the mysteries of the brain and its complex functions. His groundbreaking work, including the Neural Engineering Framework, Neural Engineering Objects software environment, and the Semantic Pointer Architecture, has led to the development of Spaun, the most advanced functional brain simulation to date. Among his numerous achievements, Eliasmith has received the 2015 NSERC "Polany-ee" Award and authored two influential books, "How to Build a Brain" and "Neural Engineering."
Chris' homepage:
http://arts.uwaterloo.ca/~celiasmi/
Interviewers: Dr. Tim Scarfe and Dr. Keith Duggar
TOC:
Intro to Chris [00:00:00]
Continuous Representation in Biologically Plausible Neural Networks [00:06:49]
Legendre Memory Unit and Spatial Semantic Pointer [00:14:36]
Large Contexts and Data in Language Models [00:20:30]
Spatial Semantic Pointers and Continuous Representations [00:24:38]
Auto Convolution [00:30:12]
Abstractions and the Continuity [00:36:33]
Compression, Sparsity, and Brain Representations [00:42:52]
Continual Learning and Real-World Interactions [00:48:05]
Robust Generalization in LLMs and Priors [00:56:11]
Chip design [01:00:41]
Chomsky + Computational Power of NNs and Recursion [01:04:02]
Spiking Neural Networks and Applications [01:13:07]
Limits of Empirical Learning [01:22:43]
Philosophy of Mind, Consciousness etc [01:25:35]
Future of human machine interaction [01:41:28]
Future research and advice to young researchers [01:45:06]
Refs:
http://compneuro.uwaterloo.ca/publications/dumont2023.html
htt…
-
#112 AVOIDING AGI APOCALYPSE - CONNOR LEAHY
2 april 2023 · 2 uur 40 min
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5 Lees meerIn this podcast with the legendary Connor Leahy (CEO Conjecture) recorded in Dec 2022, we discuss various topics related to artificial intelligence (AI), including AI alignment, the success of ChatGPT, the potential threats of artificial general intelligence (AGI), and the challenges of balancing research and product development at his company, Conjecture. He emphasizes the importance of empathy, dehumanizing our thinking to avoid anthropomorphic biases, and the value of real-world experiences in learning and personal growth. The conversation also covers the Orthogonality Thesis, AI preferences, the mystery of mode collapse, and the paradox of AI alignment.
Connor Leahy expresses concern about the rapid development of AI and the potential dangers it poses, especially as AI systems become more powerful and integrated into society. He argues that we need a better understanding of AI systems to ensure their safe and beneficial development. The discussion also touches on the concept of "futuristic whack-a-mole," where futurists predict potential AGI threats, and others try to come up with solutions for those specific scenarios. However, the problem lies in the fact that there could be many more scenarios that neither party can think of, especially when dealing with a system that's smarter than humans.
https://www.linkedin.com/in/connor-j-leahy/https://twitter.com/NPCollapse
Interviewer: Dr. Tim Scarfe (Innovation CTO @ XRAI Glass https://xrai.glass/)
TOC:
The success of ChatGPT and its impact on the AI field [00:00:00]
Subjective experience [00:15:12]
AI Architectural discussion including RLHF [00:18:04]
The paradox of AI alignment and the future of AI in society [00:31:44]
The impact of AI on society and politics [00:36:11]
Future shock levels and the challenges of predicting the future [00:45:58]
Long termism and existential risk [00:48:23]
Consequentialism vs. deontology in rat… -
#111 - AI moratorium, Eliezer Yudkowsky, AGI risk etc
1 april 2023 · 27 min
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5 Lees meerSend us a voice message which you want us to publish: https://podcasters.spotify.com/pod/show/machinelearningstreettalk/message
In a recent open letter, over 1500 individuals called for a six-month pause on the development of advanced AI systems, expressing concerns over the potential risks AI poses to society and humanity. However, there are issues with this approach, including global competition, unstoppable progress, potential benefits, and the need to manage risks instead of avoiding them.
Decision theorist Eliezer Yudkowsky took it a step further in a Time magazine article, calling for an indefinite and worldwide moratorium on Artificial General Intelligence (AGI) development, warning of potential catastrophe if AGI exceeds human intelligence. Yudkowsky urged for an immediate halt to all large AI training runs and the shutdown of major GPU clusters, calling for international cooperation to enforce these measures.
However, several counterarguments question the validity of Yudkowsky's concerns:
1. Hard limits on AGI
2. Dismissing AI extinction risk
3. Collective action problem
4. Misplaced focus on AI threatsWhile the potential risks of AGI cannot be ignored, it is essential to consider various arguments and potential solutions before making drastic decisions. As AI continues to advance, it is crucial for researchers, policymakers, and society as a whole to engage in open and honest discussions about the potential consequences and the best path forward. With a balanced approach to AGI development, we may be able to harness its power for the betterment of humanity while mitigating its risks.
Eliezer Yudkowsky: https://en.wikipedia.org/wiki/Eliezer_Yudkowsky
Connor Leahy: https://twitter.com/NPCollapse (we will release that interview soon)
Gary Marcus: http://garymarcus.com/index.html
Tim Scarfe is the innovation CTO of XRAI Glass: https://xrai.glass/Gary clip filmed at …
-
#110 Dr. STEPHEN WOLFRAM - HUGE ChatGPT+Wolfram announcement!
23 maart 2023 · 57 min
HUGE ANNOUNCEMENT, CHATGPT+WOLFRAM! You saw it HERE first! Lees meer
YT version: https://youtu.be/z5WZhCBRDpU
Support us! https://www.patreon.com/mlstMLST Discord: https://discord.gg/aNPkGUQtc5
Stephen's announcement post: https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/
OpenAI's announcement post: https://openai.com/blog/chatgpt-pluginsIn an era of technology and innovation, few individuals have left as indelible a mark on the fabric of modern science as our esteemed guest, Dr. Steven Wolfram.
Dr. Wolfram is a renowned polymath who has made significant contributions to the fields of physics, computer science, and mathematics. A prodigious young man too, Wolfram earned a Ph.D. in theoretical physics from the California Institute of Technology by the age of 20. He became the youngest recipient of the prestigious MacArthur Fellowship at the age of 21.
Wolfram's groundbreaking computational tool, Mathematica, was launched in 1988 and has become a cornerstone for researchers and innovators worldwide. In 2002, he published "A New Kind of Science," a paradigm-shifting work that explores the foundations of science through the lens of computational systems.
In 2009, Wolfram created Wolfram Alpha, a computational knowledge engine utilized by millions of users worldwide. His current focus is on the Wolfram Language, a powerful programming language designed to democratize access to cutting-edge technology.
Wolfram's numerous accolades include honorary doctorates and fellowships from prestigious institutions. As an influential thinker, Dr. Wolfram has dedicated his life to unraveling the mysteries of the universe and making computation accessible to all.
First of all... we have an announcement to make, you heard it FIRST here on MLST! ....
Intro [00:00:00]
Big announcement! Wolfram + ChatGPT! [00:02:57]
What does it mean to understand? [00:05:33]
Feeding information back into the model [00:13:48]
Semantics and cognitive categories [00:20:09]
… -
#109 - Dr. DAN MCQUILLAN - Resisting AI
20 maart 2023 · 2 uur 51 min
YT version: https://youtu.be/P1j3VoKBxbc (references in pinned comment)
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5 Lees meerDan McQuillan, a visionary in digital culture and social innovation, emphasizes the importance of understanding technology's complex relationship with society. As an academic at Goldsmiths, University of London, he fosters interdisciplinary collaboration and champions data-driven equity and ethical technology. Dan's career includes roles at Amnesty International and Social Innovation Camp, showcasing technology's potential to empower and bring about positive change. In this conversation, we discuss the challenges and opportunities at the intersection of technology and society, exploring the profound impact of our digital world.
Interviewer: Dr. Tim Scarfe
[00:00:00] Dan's background and journey to academia
[00:03:30] Dan's background and journey to academia
[00:04:10] Writing the book "Resisting AI"
[00:08:30] Necropolitics and its relation to AI
[00:10:06] AI as a new form of colonization
[00:12:57] LLMs as a new form of neo-techno-imperialism
[00:15:47] Technology for good and AGI's skewed worldview
[00:17:49] Transhumanism, eugenics, and intelligence
[00:20:45] Valuing differences (disability) and challenging societal norms
[00:26:08] Re-ontologizing and the philosophy of information
[00:28:19] New materialism and the impact of technology on society
[00:30:32] Intelligence, meaning, and materiality
[00:31:43] The constraints of physical laws and the importance of science
[00:32:44] Exploring possibilities to reduce suffering and increase well-being
[00:33:29] The division between meaning and material in our experiences
[00:35:36] Machine learning, data science, and neoplatonic approach to understanding reality
[00:37:56] Different understandings of cognition, thought, and consciousness
[00:39:15] Enactivism and its variants in cognitive science
[00:40:58] Jordan Peterson
[00:44:47] Rela…
-
#108 - Dr. JOEL LEHMAN - Machine Love [Staff Favourite]
16 maart 2023 · 2 uur 10 min
Support us! https://www.patreon.com/mlst Lees meer
MLST Discord: https://discord.gg/aNPkGUQtc5
We are honoured to welcome Dr. Joel Lehman, an eminent machine learning research scientist, whose work in AI safety, reinforcement learning, creative open-ended search algorithms, and indeed the philosophy of open-endedness and abandoning objectives has paved the way for innovative ideas that challenge our preconceptions and inspire new visions for the future.
Dr. Lehman's thought-provoking book, "Why Greatness Cannot Be Planned" penned with with our MLST favourite Professor Kenneth Stanley has left an indelible mark on the field and profoundly impacted the way we view innovation and the serendipitous nature of discovery. Those of you who haven't watched our special edition show on that, should do so at your earliest convenience! Building upon this foundation, Dr. Lehman has ventured into the domain of AI systems that embody principles of love, care, responsibility, respect, and knowledge, drawing from the works of Maslow, Erich Fromm, and positive psychology.
YT version: https://youtu.be/23-TXgJEv-Q
http://joellehman.com/
https://twitter.com/joelbot3000
Interviewer: Dr. Tim Scarfe
TOC:
Intro [00:00:00]
Model [00:04:26]
Intro and Paper Intro [00:08:52]
Subjectivity [00:16:07]
Reflections on Greatness Book [00:19:30]
Representing Subjectivity [00:29:24]
Nagal's Bat [00:31:49]
Abstraction [00:38:58]
Love as Action Rather Than Feeling [00:42:58]
Reontologisation [00:57:38]
Self Help [01:04:15]
Meditation [01:09:02]
The Human Reward Function / Effective... [01:16:52]
Machine Hate [01:28:32]
Societal Harms [01:31:41]
Lenses We Use Obscuring Reality [01:56:36]
Meta Optimisation and Evolution [02:03:14]
Conclusion [02:07:06]
References:
What Is It Like to Be a Bat? (Thomas Nagel)
https://warwick.ac.uk/fac/cross_fac/iatl/study/ugmodules/humananimalstudies/lectures/32/nagel_bat.pdf
Why Greatness Cannot Be Planned: The Myth of the Objective (Kenneth O. Stanley a…
-
#107 - Dr. RAPHAËL MILLIÈRE - Linguistics, Theory of Mind, Grounding
13 maart 2023 · 1 uur 44 min
Support us! https://www.patreon.com/mlst Lees meer
MLST Discord: https://discord.gg/aNPkGUQtc5
Dr. Raphaël Millière is the 2020 Robert A. Burt Presidential Scholar in Society and Neuroscience in the Center for Science and Society, and a Lecturer in the Philosophy Department at Columbia University. His research draws from his expertise in philosophy and cognitive science to explore the implications of recent progress in deep learning for models of human cognition, as well as various issues in ethics and aesthetics. He is also investigating what underlies the capacity to represent oneself as oneself at a fundamental level, in humans and non-human animals; as well as the role that self-representation plays in perception, action, and memory. In a world where technology is rapidly advancing, Dr. Millière is striving to gain a better understanding of how artificial neural networks work, and to establish fair and meaningful comparisons between humans and machines in various domains in order to shed light on the implications of artificial intelligence for our lives.
https://www.raphaelmilliere.com/
https://twitter.com/raphaelmilliere
Here is a version with hesitation sounds like "um" removed if you prefer (I didn't notice them personally): https://share.descript.com/view/aGelyTl2xpN
YT: https://www.youtube.com/watch?v=fhn6ZtD6XeE
TOC:
Intro to Raphael [00:00:00]
Intro: Moving Beyond Mimicry in Artificial Intelligence (Raphael Millière) [00:01:18]
Show Kick off [00:07:10]
LLMs [00:08:37]
Semantic Competence/Understanding [00:18:28]
Forming Analogies/JPG Compression Article [00:30:17]
Compositional Generalisation [00:37:28]
Systematicity [00:47:08]
Language of Thought [00:51:28]
Bigbench (Conceptual Combinations) [00:57:37]
Symbol Grounding [01:11:13]
World Models [01:26:43]
Theory of Mind [01:30:57]
Refs (this is truncated, full list on YT video description):
Moving Beyond Mimicry in Artificial Intelligence (Raphael Millière)
https://nautil.us/moving-beyond-mim…
-
#106 - Prof. KARL FRISTON 3.0 - Collective Intelligence [Special Edition]
11 maart 2023 · 2 uur 59 min
This show is sponsored by Numerai, please visit them here with our sponsor link (we would really appreciate it) http://numer.ai/mlst Lees meer
Prof. Karl Friston recently proposed a vision of artificial intelligence that goes beyond machines and algorithms, and embraces humans and nature as part of a cyber-physical ecosystem of intelligence. This vision is based on the principle of active inference, which states that intelligent systems can learn from their observations and act on their environment to reduce uncertainty and achieve their goals. This leads to a formal account of collective intelligence that rests on shared narratives and goals.
To realize this vision, Friston suggests developing a shared hyper-spatial modelling language and transaction protocol, as well as novel methods for measuring and optimizing collective intelligence. This could harness the power of artificial intelligence for the common good, without compromising human dignity or autonomy. It also challenges us to rethink our relationship with technology, nature, and each other, and invites us to join a global community of sense-makers who are curious about the world and eager to improve it.
YT version: https://www.youtube.com/watch?v=V_VXOdf1NMw
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
TOC:
Intro [00:00:00]
Numerai (Sponsor segment) [00:07:10]
Designing Ecosystems of Intelligence from First Principles (Friston et al) [00:09:48]
Information / Infosphere and human agency [00:18:30]
Intelligence [00:31:38]
Reductionism [00:39:36]
Universalism [00:44:46]
Emergence [00:54:23]
Markov blankets [01:02:11]
Whole part relationships / structure learning [01:22:33]
Enactivism [01:29:23]
Knowledge and Language [01:43:53]
ChatGPT [01:50:56]
Ethics (is-ought) [02:07:55]
Can people be evil? [02:35:06]
Ethics in Al, subjectiveness [02:39:05]
Final thoughts [02:57:00]
References:
Designing Ecosystems of Intelligence from First Principles (Friston et al…
-
#105 - Dr. MICHAEL OLIVER [CSO - Numerai]
4 maart 2023 · 1 uur 21 min
Access Numerai here: http://numer.ai/mlst Lees meer
Michael Oliver is the Chief Scientist at Numerai, a hedge fund that crowdsources machine learning models from data scientists. He has a PhD in Computational Neuroscience from UC Berkeley and was a postdoctoral researcher at the Allen Institute for Brain Science before joining Numerai in 2020. He is also the host of Numerai Quant Club, a YouTube series where he discusses Numerai’s research, data and challenges.
YT version: https://youtu.be/61s8lLU7sFg
TOC:
[00:00:00] Introduction to Michael and Numerai
[00:02:03] Understanding / new Bing
[00:22:47] Quant vs Neuroscience
[00:36:43] Role of language in cognition and planning, and subjective...
[00:45:47] Boundaries in finance modelling
[00:48:00] Numerai
[00:57:37] Aggregation systems
[01:00:52] Getting started on Numeral
[01:03:21] What models are people using
[01:04:23] Numerai Problem Setup
[01:05:49] Regimes in financial data and quant talk
[01:11:18] Esoteric approaches used on Numeral?
[01:13:59] Curse of dimensionality
[01:16:32] Metrics
[01:19:10] Outro
References:
Growing Neural Cellular Automata (Alexander Mordvintsev)
https://distill.pub/2020/growing-ca/
A Thousand Brains: A New Theory of Intelligence (Jeff Hawkins)
https://www.amazon.fr/Thousand-Brains-New-Theory-Intelligence/dp/1541675819
Perceptual Neuroscience: The Cerebral Cortex (Vernon B. Mountcastle)
https://www.amazon.ca/Perceptual-Neuroscience-Cerebral-Vernon-Mountcastle/dp/0674661885
Numerai Quant Club with Michael Oliver
https://www.youtube.com/watch?v=eLIxarbDXuQ&list=PLz3D6SeXhT3tTu8rhZmjwDZpkKi-UPO1F
Numerai YT channel
https://www.youtube.com/@Numerai/featured
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
-
#104 - Prof. CHRIS SUMMERFIELD - Natural General Intelligence [SPECIAL EDITION]
22 februari 2023 · 1 uur 29 min
Support us! https://www.patreon.com/mlst Lees meer
MLST Discord: https://discord.gg/aNPkGUQtc5
Christopher Summerfield, Department of Experimental Psychology, University of Oxford is a Professor of Cognitive Neuroscience at the University of Oxford and a Research Scientist at Deepmind UK. His work focusses on the neural and computational mechanisms by which humans make decisions.
Chris has just released an incredible new book on AI called "Natural General Intelligence". It's my favourite book on AI I have read so so far.
The book explores the algorithms and architectures that are driving progress in AI research, and discusses intelligence in the language of psychology and biology, using examples and analogies to be comprehensible to a wide audience. It also tackles longstanding theoretical questions about the nature of thought and knowledge.
With Chris' permission, I read out a summarised version of Chapter 2 from his book on which was on Intelligence during the 30 minute MLST introduction.
Buy his book here:
https://global.oup.com/academic/product/natural-general-intelligence-9780192843883?cc=gb&lang=en&
YT version: https://youtu.be/31VRbxAl3t0
Interviewer: Dr. Tim Scarfe
TOC:
[00:00:00] Walk and talk with Chris on Knowledge and Abstractions
[00:04:08] Intro to Chris and his book
[00:05:55] (Intro) Tim reads Chapter 2: Intelligence
[00:09:28] Intro continued: Goodhart's law
[00:15:37] Intro continued: The "swiss cheese" situation
[00:20:23] Intro continued: On Human Knowledge
[00:23:37] Intro continued: Neats and Scruffies
[00:30:22] Interview kick off
[00:31:59] What does it mean to understand?
[00:36:18] Aligning our language models
[00:40:17] Creativity
[00:41:40] "Meta" AI and basins of attraction
[00:51:23] What can Neuroscience impart to AI
[00:54:43] Sutton, neats and scruffies and human alignment
[01:02:05] Reward is enough
[01:19:46] Jon Von Neumann and Intelligence
[01:23:56] Compositionality
References:
The Language Game (Morten H. Ch…
-
#103 - Prof. Edward Grefenstette - Language, Semantics, Philosophy
11 februari 2023 · 1 uur 2 min
Support us! https://www.patreon.com/mlst Lees meer
MLST Discord: https://discord.gg/aNPkGUQtc5
YT: https://youtu.be/i9VPPmQn9HQ
Edward Grefenstette is a Franco-American computer scientist who currently serves as Head of Machine Learning at Cohere and Honorary Professor at UCL. He has previously been a research scientist at Facebook AI Research and staff research scientist at DeepMind, and was also the CTO of Dark Blue Labs. Prior to his move to industry, Edward was a Fulford Junior Research Fellow at Somerville College, University of Oxford, and was lecturing at Hertford College. He obtained his BSc in Physics and Philosophy from the University of Sheffield and did graduate work in the philosophy departments at the University of St Andrews. His research draws on topics and methods from Machine Learning, Computational Linguistics and Quantum Information Theory, and has done work implementing and evaluating compositional vector-based models of natural language semantics and empirical semantic knowledge discovery.
https://www.egrefen.com/
https://cohere.ai/
TOC:
[00:00:00] Introduction
[00:02:52] Differential Semantics
[00:06:56] Concepts
[00:10:20] Ontology
[00:14:02] Pragmatics
[00:16:55] Code helps with language
[00:19:02] Montague
[00:22:13] RLHF
[00:31:54] Swiss cheese problem / retrieval augmented
[00:37:06] Intelligence / Agency
[00:43:33] Creativity
[00:46:41] Common sense
[00:53:46] Thinking vs knowing
References:
Large language models are not zero-shot communicators (Laura Ruis)
https://arxiv.org/abs/2210.14986
Some remarks on Large Language Models (Yoav Goldberg)
https://gist.github.com/yoavg/59d174608e92e845c8994ac2e234c8a9
Quantum Natural Language Processing (Bob Coecke)
https://www.cs.ox.ac.uk/people/bob.coecke/QNLP-ACT.pdf
Constitutional AI: Harmlessness from AI Feedback
https://www.anthropic.com/constitutional.pdf
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Patrick Lewis)
https://www.patricklewis.io/publication…
-
#102 - Prof. MICHAEL LEVIN, Prof. IRINA RISH - Emergence, Intelligence, Transhumanism
11 februari 2023 · 55 min
Support us! https://www.patreon.com/mlst Lees meer
MLST Discord: https://discord.gg/aNPkGUQtc5
YT: https://youtu.be/Vbi288CKgis
Michael Levin is a Distinguished Professor in the Biology department at Tufts University, and the holder of the Vannevar Bush endowed Chair. He is the Director of the Allen Discovery Center at Tufts and the Tufts Center for Regenerative and Developmental Biology. His research focuses on understanding the biophysical mechanisms of pattern regulation and harnessing endogenous bioelectric dynamics for rational control of growth and form.
The capacity to generate a complex, behaving organism from the single cell of a fertilized egg is one of the most amazing aspects of biology. Levin' lab integrates approaches from developmental biology, computer science, and cognitive science to investigate the emergence of form and function. Using biophysical and computational modeling approaches, they seek to understand the collective intelligence of cells, as they navigate physiological, transcriptional, morphognetic, and behavioral spaces. They develop conceptual frameworks for basal cognition and diverse intelligence, including synthetic organisms and AI.
Also joining us this evening is Irina Rish. Irina is a Full Professor at the Université de Montréal's Computer Science and Operations Research department, a core member of Mila - Quebec AI Institute, as well as the holder of the Canada CIFAR AI Chair and the Canadian Excellence Research Chair in Autonomous AI. She has a PhD in AI from UC Irvine. Her research focuses on machine learning, neural data analysis, neuroscience-inspired AI, continual lifelong learning, optimization algorithms, sparse modelling, probabilistic inference, dialog generation, biologically plausible reinforcement learning, and dynamical systems approaches to brain imaging analysis.
Interviewer: Dr. Tim Scarfe
TOC:
[00:00:00] Introduction
[00:02:09] Emergence
[00:13:16] Scaling Laws
[00:23:12] Intelligence
[00:44:36] Transhumanism
… -
#100 Dr. PATRICK LEWIS (co:here) - Retrieval Augmented Generation
10 februari 2023 · 26 min
Dr. Patrick Lewis is a London-based AI and Natural Language Processing Research Scientist, working at co:here. Prior to this, Patrick worked as a research scientist at the Fundamental AI Research Lab (FAIR) at Meta AI. During his PhD, Patrick split his time between FAIR and University College London, working with Sebastian Riedel and Pontus Stenetorp. Lees meer
Patrick’s research focuses on the intersection of information retrieval techniques (IR) and large language models (LLMs). He has done extensive work on Retrieval-Augmented Language Models. His current focus is on building more powerful, efficient, robust, and update-able models that can perform well on a wide range of NLP tasks, but also excel on knowledge-intensive NLP tasks such as Question Answering and Fact Checking.
YT version: https://youtu.be/Dm5sfALoL1Y
MLST Discord: https://discord.gg/aNPkGUQtc5
Support us! https://www.patreon.com/mlst
References:
Patrick Lewis (Natural Language Processing Research Scientist @ co:here)
https://www.patricklewis.io/
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Patrick Lewis et al)
https://arxiv.org/abs/2005.11401
Atlas: Few-shot Learning with Retrieval Augmented Language Models (Gautier Izacard, Patrick Lewis, et al)
https://arxiv.org/abs/2208.03299
Improving language models by retrieving from trillions of tokens (RETRO) (Sebastian Borgeaud et al)
https://arxiv.org/abs/2112.04426
-
#99 - CARLA CREMER & IGOR KRAWCZUK - X-Risk, Governance, Effective Altruism
5 februari 2023 · 1 uur 40 min
YT version (with references): https://www.youtube.com/watch?v=lxaTinmKxs0 Lees meer
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
Carla Cremer and Igor Krawczuk argue that AI risk should be understood as an old problem of politics, power and control with known solutions, and that threat models should be driven by empirical work. The interaction between FTX and the Effective Altruism community has sparked a lot of discussion about the dangers of optimization, and Carla's Vox article highlights the need for an institutional turn when taking on a responsibility like risk management for humanity.
Carla's “Democratizing Risk” paper found that certain types of risks fall through the cracks if they are just categorized into climate change or biological risks. Deliberative democracy has been found to be a better way to make decisions, and AI tools can be used to scale this type of democracy and be used for good, but the transparency of these algorithms to the citizens using the platform must be taken into consideration.
Aggregating people’s diverse ways of thinking about a problem and creating a risk-averse procedure gives a likely, highly probable outcome for having converged on the best policy. There needs to be a good reason to trust one organization with the risk management of humanity and all the different ways of thinking about risk must be taken into account. AI tools can help to scale this type of deliberative democracy, but the transparency of these algorithms must be taken into consideration.
The ambition of the EA community and Altruism Inc. is to protect and do risk management for the whole of humanity and this requires an institutional turn in order to do it effectively. The dangers of optimization are real, and it is essential to ensure that the risk management of humanity is done properly and ethically. By understanding the importance of aggregating people’s diverse ways of thinking about a problem, and creating a risk-aver…
-
[NO MUSIC] #98 - Prof. LUCIANO FLORIDI - ChatGPT, Singularitarians, Ethics, Philosophy of Information
3 februari 2023 · 1 uur 6 min
Support us! https://www.patreon.com/mlst Lees meer
MLST Discord: https://discord.gg/aNPkGUQtc5
YT version: https://youtu.be/YLNGvvgq3eg
We are living in an age of rapid technological advancement, and with this growth comes a digital divide. Professor Luciano Floridi of the Oxford Internet Institute / Oxford University believes that this divide not only affects our understanding of the implications of this new age, but also the organization of a fair society.
The Information Revolution has been transforming the global economy, with the majority of global GDP now relying on intangible goods, such as information-related services. This in turn has led to the generation of immense amounts of data, more than humanity has ever seen in its history. With 95% of this data being generated by the current generation, Professor Floridi believes that we are becoming overwhelmed by this data, and that our agency as humans is being eroded as a result.
According to Professor Floridi, the digital divide has caused a lack of balance between technological growth and our understanding of this growth. He believes that the infosphere is becoming polluted and the manifold of the infosphere is increasingly determined by technology and AI. Identifying, anticipating and resolving these problems has become essential, and Professor Floridi has dedicated his research to the Philosophy of Information, Philosophy of Technology and Digital Ethics.
We must equip ourselves with a viable philosophy of information to help us better understand and address the risks of this new information age. Professor Floridi is leading the charge, and his research on Digital Ethics, the Philosophy of Information and the Philosophy of Technology is helping us to better anticipate, identify and resolve problems caused by the digital divide.
TOC:
[00:00:00] Introduction to Luciano and his ideas
[00:14:00] Chat GPT / language models
[00:28:45] AI risk / "Singularitarians"
[00:37:15] Forms of governance
[00:43:56] Re-on…
-
#98 - Prof. LUCIANO FLORIDI - ChatGPT, Superintelligence, Ethics, Philosophy of Information
3 februari 2023 · 1 uur 7 min
Support us! https://www.patreon.com/mlst Lees meer
MLST Discord: https://discord.gg/aNPkGUQtc5
YT version: https://youtu.be/YLNGvvgq3eg
(If music annoying, skip to main interview @ 14:14)
We are living in an age of rapid technological advancement, and with this growth comes a digital divide. Professor Luciano Floridi of the Oxford Internet Institute / Oxford University believes that this divide not only affects our understanding of the implications of this new age, but also the organization of a fair society.
The Information Revolution has been transforming the global economy, with the majority of global GDP now relying on intangible goods, such as information-related services. This in turn has led to the generation of immense amounts of data, more than humanity has ever seen in its history. With 95% of this data being generated by the current generation, Professor Floridi believes that we are becoming overwhelmed by this data, and that our agency as humans is being eroded as a result.
According to Professor Floridi, the digital divide has caused a lack of balance between technological growth and our understanding of this growth. He believes that the infosphere is becoming polluted and the manifold of the infosphere is increasingly determined by technology and AI. Identifying, anticipating and resolving these problems has become essential, and Professor Floridi has dedicated his research to the Philosophy of Information, Philosophy of Technology and Digital Ethics.
We must equip ourselves with a viable philosophy of information to help us better understand and address the risks of this new information age. Professor Floridi is leading the charge, and his research on Digital Ethics, the Philosophy of Information and the Philosophy of Technology is helping us to better anticipate, identify and resolve problems caused by the digital divide.
TOC:
[00:00:00] Introduction to Luciano and his ideas
[00:14:40] Chat GPT / language models
[00:29:24] AI risk / "Singularitaria…
-
#97 SREEJAN KUMAR - Human Inductive Biases in Machines from Language
28 januari 2023 · 25 min
Research has shown that humans possess strong inductive biases which enable them to quickly learn and generalize. In order to instill these same useful human inductive biases into machines, a paper was presented by Sreejan Kumar at the NeurIPS conference which won the Outstanding Paper of the Year award. The paper is called Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines. Lees meer
This paper focuses on using a controlled stimulus space of two-dimensional binary grids to define the space of abstract concepts that humans have and a feedback loop of collaboration between humans and machines to understand the differences in human and machine inductive biases.
It is important to make machines more human-like to collaborate with them and understand their behavior. Synthesised discrete programs running on a turing machine computational model instead of a neural network substrate offers promise for the future of artificial intelligence. Neural networks and program induction should both be explored to get a well-rounded view of intelligence which works in multiple domains, computational substrates and which can acquire a diverse set of capabilities.
Natural language understanding in models can also be improved by instilling human language biases and programs into AI models. Sreejan used an experimental framework consisting of two dual task distributions, one generated from human priors and one from machine priors, to understand the differences in human and machine inductive biases. Furthermore, he demonstrated that compressive abstractions can be used to capture the essential structure of the environment for more human-like behavior. This means that emergent language-based inductive priors can be distilled into artificial neural networks, and AI models can be aligned to the us, world and indeed, our values.
Humans possess strong inductive biases which enable them to quickly learn to perform various tasks. This is in contrast to neura…
-
#96 Prof. PEDRO DOMINGOS - There are no infinities, utility functions, neurosymbolic
30 december 2022 · 2 uur 49 min
Pedro Domingos, Professor Emeritus of Computer Science and Engineering at the University of Washington, is renowned for his research in machine learning, particularly for his work on Markov logic networks that allow for uncertain inference. He is also the author of the acclaimed book "The Master Algorithm". Lees meer
Panel: Dr. Tim Scarfe
TOC:
[00:00:00] Introduction
[00:01:34] Galaxtica / misinformation / gatekeeping
[00:12:31] Is there a master algorithm?
[00:16:29] Limits of our understanding
[00:21:57] Intentionality, Agency, Creativity
[00:27:56] Compositionality
[00:29:30] Digital Physics / It from bit / Wolfram
[00:35:17] Alignment / Utility functions
[00:43:36] Meritocracy
[00:45:53] Game theory
[01:00:00] EA/consequentialism/Utility
[01:11:09] Emergence / relationalism
[01:19:26] Markov logic
[01:25:38] Moving away from anthropocentrism
[01:28:57] Neurosymbolic / infinity / tensor algerbra
[01:53:45] Abstraction
[01:57:26] Symmetries / Geometric DL
[02:02:46] Bias variance trade off
[02:05:49] What seen at neurips
[02:12:58] Chalmers talk on LLMs
[02:28:32] Definition of intelligence
[02:32:40] LLMs
[02:35:14] On experts in different fields
[02:40:15] Back to intelligence
[02:41:37] Spline theory / extrapolation
YT version: https://www.youtube.com/watch?v=C9BH3F2c0vQ
References;
The Master Algorithm [Domingos]
https://www.amazon.co.uk/s?k=master+algorithm&i=stripbooks&crid=3CJ67DCY96DE8&sprefix=master+algorith%2Cstripbooks%2C82&ref=nb_sb_noss_2
INFORMATION, PHYSICS, QUANTUM: THE SEARCH FOR LINKS [John Wheeler/It from Bit]
https://philpapers.org/archive/WHEIPQ.pdf
A New Kind Of Science [Wolfram]
https://www.amazon.co.uk/New-Kind-Science-Stephen-Wolfram/dp/1579550088
The Rationalist's Guide to the Galaxy: Superintelligent AI and the Geeks Who Are Trying to Save Humanity's Future [Tom Chivers]
https://www.amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795
The Status Game: On Social Position and How We Use It [Will Storr…
-
#95 - Prof. IRINA RISH - AGI, Complex Systems, Transhumanism
26 december 2022 · 39 min
Canadian Excellence Research Chair in Autonomous AI. Irina holds an MSc and PhD in AI from the University of California, Irvine as well as an MSc in Applied Mathematics from the Moscow Gubkin Institute. Her research focuses on machine learning, neural data analysis, and neuroscience-inspired AI. In particular, she is exploring continual lifelong learning, optimization algorithms for deep neural networks, sparse modelling and probabilistic inference, dialog generation, biologically plausible reinforcement learning, and dynamical systems approaches to brain imaging analysis. Prof. Rish holds 64 patents, has published over 80 research papers, several book chapters, three edited books, and a monograph on Sparse Modelling. She has served as a Senior Area Chair for NeurIPS and ICML. Irina's research is focussed on taking us closer to the holy grail of Artificial General Intelligence. She continues to push the boundaries of machine learning, continually striving to make advancements in neuroscience-inspired AI. Lees meer
In a conversation about artificial intelligence (AI), Irina and Tim discussed the idea of transhumanism and the potential for AI to improve human flourishing. Irina suggested that instead of looking at AI as something to be controlled and regulated, people should view it as a tool to augment human capabilities. She argued that attempting to create an AI that is smarter than humans is not the best approach, and that a hybrid of human and AI intelligence is much more beneficial. As an example, she mentioned how technology can be used as an extension of the human mind, to track mental states and improve self-understanding. Ultimately, Irina concluded that transhumanism is about having a symbiotic relationship with technology, which can have a positive effect on both parties.
Tim then discussed the contrasting types of intelligence and how this could lead to something interesting emerging from the combination. He brought up the Trolley Problem and how difficult moral…
-
#94 - ALAN CHAN - AI Alignment and Governance #NEURIPS
26 december 2022 · 14 min
Support us! https://www.patreon.com/mlst Lees meer
Alan Chan is a PhD student at Mila, the Montreal Institute for Learning Algorithms, supervised by Nicolas Le Roux. Before joining Mila, Alan was a Masters student at the Alberta Machine Intelligence Institute and the University of Alberta, where he worked with Martha White. Alan's expertise and research interests encompass value alignment and AI governance. He is currently exploring the measurement of harms from language models and the incentives that agents have to impact the world. Alan's research focuses on understanding and controlling the values expressed by machine learning models. His projects have examined the regulation of explainability in algorithmic systems, scoring rules for performative binary prediction, the effects of global exclusion in AI development, and the role of a graduate student in approaching ethical impacts in AI research. In addition, Alan has conducted research into inverse policy evaluation for value-based sequential decision-making, and the concept of "normal accidents" and AI systems. Alan's research is motivated by the need to align AI systems with human values, and his passion for scientific and governance work in this field. Alan's energy and enthusiasm for his field is infectious.
This was a discussion at NeurIPS. It was in quite a loud environment so the audio quality could have been better.
References:
The Rationalist's Guide to the Galaxy: Superintelligent AI and the Geeks Who Are Trying to Save Humanity's Future [Tim Chivers]
https://www.amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795
The implausibility of intelligence explosion [Chollet]
https://medium.com/@francois.chollet/the-impossibility-of-intelligence-explosion-5be4a9eda6ec
Superintelligence: Paths, Dangers, Strategies [Bostrom]
https://www.amazon.co.uk/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111
A Theory of Universal Artificial Intelligence based on Algorithmic Complexity [Hutter]
h…
-
#93 Prof. MURRAY SHANAHAN - Consciousness, Embodiment, Language Models
24 december 2022 · 1 uur 20 min
Support us! https://www.patreon.com/mlst Lees meer
Professor Murray Shanahan is a renowned researcher on sophisticated cognition and its implications for artificial intelligence. His 2016 article ‘Conscious Exotica’ explores the Space of Possible Minds, a concept first proposed by philosopher Aaron Sloman in 1984, which includes all the different forms of minds from those of other animals to those of artificial intelligence. Shanahan rejects the idea of an impenetrable realm of subjective experience and argues that the majority of the space of possible minds may be occupied by non-natural variants, such as the ‘conscious exotica’ of which he speaks. In his paper ‘Talking About Large Language Models’, Shanahan discusses the capabilities and limitations of large language models (LLMs). He argues that prompt engineering is a key element for advanced AI systems, as it involves exploiting prompt prefixes to adjust LLMs to various tasks. However, Shanahan cautions against ascribing human-like characteristics to these systems, as they are fundamentally different and lack a shared comprehension with humans. Even though LLMs can be integrated into embodied systems, it does not mean that they possess human-like language abilities. Ultimately, Shanahan concludes that although LLMs are formidable and versatile, we must be wary of over-simplifying their capacities and limitations.
YT version: https://youtu.be/BqkWpP3uMMU
Full references on the YT description.
[00:00:00] Introduction
[00:08:51] Consciousness and Consciousness Exotica
[00:34:59] Slightly Consciousness LLMs
[00:38:05] Embodiment
[00:51:32] Symbol Grounding
[00:54:13] Emergence
[00:57:09] Reasoning
[01:03:16] Intentional Stance
[01:07:06] Digression on Chomsky show and Andrew Lampinen
[01:10:31] Prompt Engineering
Find Murray online:
https://www.doc.ic.ac.uk/~mpsha/
https://twitter.com/mpshanahan?lang=en
https://scholar.google.co.uk/citations?user=00bnGpAAAAAJ&hl=en
MLST Discord: https://discord.gg/aNPkGUQt…
-
#92 - SARA HOOKER - Fairness, Interpretability, Language Models
23 december 2022 · 52 min
Support us! https://www.patreon.com/mlst Lees meer
Sara Hooker is an exceptionally talented and accomplished leader and research scientist in the field of machine learning. She is the founder of Cohere For AI, a non-profit research lab that seeks to solve complex machine learning problems. She is passionate about creating more points of entry into machine learning research and has dedicated her efforts to understanding how progress in this field can be translated into reliable and accessible machine learning in the real-world.
Sara is also the co-founder of the Trustworthy ML Initiative, a forum and seminar series related to Trustworthy ML. She is on the advisory board of Patterns and is an active member of the MLC research group, which has a focus on making participation in machine learning research more accessible.
Before starting Cohere For AI, Sara worked as a research scientist at Google Brain. She has written several influential research papers, including "The Hardware Lottery", "The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation", "Moving Beyond “Algorithmic Bias is a Data Problem”" and "Characterizing and Mitigating Bias in Compact Models".
In addition to her research work, Sara is also the founder of the local Bay Area non-profit Delta Analytics, which works with non-profits and communities all over the world to build technical capacity and empower others to use data. She regularly gives tutorials on machine learning fundamentals, interpretability, model compression and deep neural networks and is dedicated to collaborating with independent researchers around the world.
Sara Hooker is famous for writing a paper introducing the concept of the 'hardware lottery', in which the success of a research idea is determined not by its inherent superiority, but by its compatibility with available software and hardware. She argued that choices about software and hardware have had a substantial impact in deciding the outcomes of…
-
#91 - HATTIE ZHOU - Teaching Algorithmic Reasoning via In-context Learning #NeurIPS
20 december 2022 · 21 min
Support us! https://www.patreon.com/mlst Lees meer
Hattie Zhou, a PhD student at Université de Montréal and Mila, has set out to understand and explain the performance of modern neural networks, believing it a key factor in building better, more trusted models. Having previously worked as a data scientist at Uber, a private equity analyst at Radar Capital, and an economic consultant at Cornerstone Research, she has recently released a paper in collaboration with the Google Brain team, titled ‘Teaching Algorithmic Reasoning via In-context Learning’. In this work, Hattie identifies and examines four key stages for successfully teaching algorithmic reasoning to large language models (LLMs): formulating algorithms as skills, teaching multiple skills simultaneously, teaching how to combine skills, and teaching how to use skills as tools. Through the application of algorithmic prompting, Hattie has achieved remarkable results, with an order of magnitude error reduction on some tasks compared to the best available baselines. This breakthrough demonstrates algorithmic prompting’s viability as an approach for teaching algorithmic reasoning to LLMs, and may have implications for other tasks requiring similar reasoning capabilities.
TOC
[00:00:00] Hattie Zhou
[00:19:49] Markus Rabe [Google Brain]
Hattie's Twitter - https://twitter.com/oh_that_hat
Website - http://hattiezhou.com/
Teaching Algorithmic Reasoning via In-context Learning [Hattie Zhou, Azade Nova, Hugo Larochelle, Aaron Courville, Behnam Neyshabur, and Hanie Sedghi]
https://arxiv.org/pdf/2211.09066.pdf
Markus Rabe [Google Brain]:
https://twitter.com/markusnrabe
https://research.google/people/106335/
https://www.linkedin.com/in/markusnrabe
Autoformalization with Large Language Models [Albert Jiang Charles Edgar Staats Christian Szegedy Markus Rabe Mateja Jamnik Wenda Li Yuhuai Tony Wu]
https://research.google/pubs/pub51691/
Discord: https://discord.gg/aNPkGUQtc5
YT: https://youtu.be/80i6D2TJdQ4
-
(Music Removed) #90 - Prof. DAVID CHALMERS - Consciousness in LLMs [Special Edition]
19 december 2022 · 54 min
Support us! https://www.patreon.com/mlst Lees meer
(On the main version we released; the music was a tiny bit too loud in places, and some pieces had percussion which was a bit distracting -- here is a version with all music removed so you have the option! )
David Chalmers is a professor of philosophy and neural science at New York University, and an honorary professor of philosophy at the Australian National University. He is the co-director of the Center for Mind, Brain, and Consciousness, as well as the PhilPapers Foundation. His research focuses on the philosophy of mind, especially consciousness, and its connection to fields such as cognitive science, physics, and technology. He also investigates areas such as the philosophy of language, metaphysics, and epistemology. With his impressive breadth of knowledge and experience, David Chalmers is a leader in the philosophical community.
The central challenge for consciousness studies is to explain how something immaterial, subjective, and personal can arise out of something material, objective, and impersonal. This is illustrated by the example of a bat, whose sensory experience is much different from ours, making it difficult to imagine what it's like to be one. Thomas Nagel's "inconceivability argument" has its advantages and disadvantages, but ultimately it is impossible to solve the mind-body problem due to the subjective nature of experience. This is further explored by examining the concept of philosophical zombies, which are physically and behaviorally indistinguishable from conscious humans yet lack conscious experience. This has implications for the Hard Problem of Consciousness, which is the attempt to explain how mental states are linked to neurophysiological activity. The Chinese Room Argument is used as a thought experiment to explain why physicality may be insufficient to be the source of the subjective, coherent experience we call consciousness. Despite much debate, the Hard Problem of Consciousness remains…
-
#90 - Prof. DAVID CHALMERS - Consciousness in LLMs [Special Edition]
19 december 2022 · 54 min
Support us! https://www.patreon.com/mlst Lees meer
David Chalmers is a professor of philosophy and neural science at New York University, and an honorary professor of philosophy at the Australian National University. He is the co-director of the Center for Mind, Brain, and Consciousness, as well as the PhilPapers Foundation. His research focuses on the philosophy of mind, especially consciousness, and its connection to fields such as cognitive science, physics, and technology. He also investigates areas such as the philosophy of language, metaphysics, and epistemology. With his impressive breadth of knowledge and experience, David Chalmers is a leader in the philosophical community.
The central challenge for consciousness studies is to explain how something immaterial, subjective, and personal can arise out of something material, objective, and impersonal. This is illustrated by the example of a bat, whose sensory experience is much different from ours, making it difficult to imagine what it's like to be one. Thomas Nagel's "inconceivability argument" has its advantages and disadvantages, but ultimately it is impossible to solve the mind-body problem due to the subjective nature of experience. This is further explored by examining the concept of philosophical zombies, which are physically and behaviorally indistinguishable from conscious humans yet lack conscious experience. This has implications for the Hard Problem of Consciousness, which is the attempt to explain how mental states are linked to neurophysiological activity. The Chinese Room Argument is used as a thought experiment to explain why physicality may be insufficient to be the source of the subjective, coherent experience we call consciousness. Despite much debate, the Hard Problem of Consciousness remains unsolved. Chalmers has been working on a functional approach to decide whether large language models are, or could be conscious.
Filmed at #neurips22
Discord: https://discord.gg/aNPkGUQtc5
YT: https://yout…
-
#88 Dr. WALID SABA - Why machines will never rule the world [UNPLUGGED]
16 december 2022 · 1 uur 22 min
Support us! https://www.patreon.com/mlst Lees meer
Dr. Walid Saba recently reviewed the book Machines Will Never Rule The World, which argues that strong AI is impossible. He acknowledges the complexity of modeling mental processes and language, as well as interactive dialogues, and questions the authors' use of "never." Despite his skepticism, he is impressed with recent developments in large language models, though he questions the extent of their success.
We then discussed the successes of cognitive science. Walid believes that something has been achieved which many cognitive scientists would never accept, namely the ability to learn from data empirically. Keith agrees that this is a huge step, but notes that there is still much work to be done to get to the "other 5%" of accuracy. They both agree that the current models are too brittle and require much more data and parameters to get to the desired level of accuracy.
Walid then expresses admiration for deep learning systems' ability to learn non-trivial aspects of language from ingesting text only. He argues that this is an "existential proof" of language competency and that it would be impossible for a group of luminaries such as Montague, Marvin Minsky, John McCarthy, and a thousand other bright engineers to replicate the same level of competency as we have now with LLMs. He then discusses the problem of semantics and pragmatics, as well as symbol grounding, and expresses skepticism about grounded meaning and embodiment. He believes that artificial intelligence should be used to solve real-world problems which require human intelligence but not believe that robots should be built to understand love or other subjective feelings.
We discussed the unique properties of natural human language. Walid believes that the core unique property is the ability to do abductive reasoning, which is the process of reasoning to the best explanation or understanding. Keith adds that there are two types of abduction - one for generati…
-
#86 - Prof. YANN LECUN and Dr. RANDALL BALESTRIERO - SSL, Data Augmentation, Reward isn't enough [NEURIPS2022]
11 december 2022 · 30 min
Yann LeCun is a French computer scientist known for his pioneering work on convolutional neural networks, optical character recognition and computer vision. He is a Silver Professor at New York University and Vice President, Chief AI Scientist at Meta. Along with Yoshua Bengio and Geoffrey Hinton, he was awarded the 2018 Turing Award for their work on deep learning, earning them the nickname of the "Godfathers of Deep Learning". Lees meer
Dr. Randall Balestriero has been researching learnable signal processing since 2013, with a focus on learnable parametrized wavelets and deep wavelet transforms. His research has been used by NASA, leading to applications such as Marsquake detection. During his PhD at Rice University, Randall explored deep networks from a theoretical perspective and improved state-of-the-art methods such as batch-normalization and generative networks. Later, when joining Meta AI Research (FAIR) as a postdoc with Prof. Yann LeCun, Randall further broadened his research interests to include self-supervised learning and the biases emerging from data-augmentation and regularization, resulting in numerous publications.
Episode recorded live at NeurIPS.
YT: https://youtu.be/9dLd6n9yT8U (references are there)
Support us! https://www.patreon.com/mlst
Host: Dr. Tim Scarfe
TOC:
[00:00:00] LeCun interview
[00:18:25] Randall Balestriero interview (mostly on spectral SSL paper, first ref)
-
#85 Dr. Petar Veličković (Deepmind) - Categories, Graphs, Reasoning [NEURIPS22 UNPLUGGED]
9 december 2022 · 37 min
Dr. Petar Veličković is a Staff Research Scientist at DeepMind, he has firmly established himself as one of the most significant up and coming researchers in the deep learning space. He invented Graph Attention Networks in 2017 and has been a leading light in the field ever since pioneering research in Graph Neural Networks, Geometric Deep Learning and also Neural Algorithmic reasoning. If you haven’t already, you should check out our video on the Geometric Deep learning blueprint, featuring Petar. I caught up with him last week at NeurIPS. In this show, from NeurIPS 2022 we discussed his recent work on category theory and graph neural networks. Lees meer
https://petar-v.com/
https://twitter.com/PetarV_93/
TOC:
Categories (Cats for AI) [00:00:00]
Reasoning [00:14:44]
Extrapolation [00:19:09]
Ishan Misra Skit [00:27:50]
Graphs (Expander Graph Propagation) [00:29:18]
YT: https://youtu.be/1lkdWduuN14
MLST Discord: https://discord.gg/V25vQeFwhS
Support us! https://www.patreon.com/mlst
References on YT description, lots of them!
Host: Dr. Tim Scarfe
-
#84 LAURA RUIS - Large language models are not zero-shot communicators [NEURIPS UNPLUGGED]
6 december 2022 · 28 min
In this NeurIPSs interview, we speak with Laura Ruis about her research on the ability of language models to interpret language in context. She has designed a simple task to evaluate the performance of widely used state-of-the-art language models and has found that they struggle to make pragmatic inferences (implicatures). Tune in to learn more about her findings and what they mean for the future of conversational AI. Lees meer
Laura Ruis
https://www.lauraruis.com/
https://twitter.com/LauraRuis
BLOOM
https://bigscience.huggingface.co/blog/bloom
Large language models are not zero-shot communicators [Laura Ruis, Akbir Khan, Stella Biderman, Sara Hooker, Tim Rocktäschel, Edward Grefenstette]
https://arxiv.org/abs/2210.14986
[Zhang et al] OPT: Open Pre-trained Transformer Language Models
https://arxiv.org/pdf/2205.01068.pdf
[Lampinen] Can language models handle recursively nested grammatical structures? A case study on comparing models and humans
https://arxiv.org/pdf/2210.15303.pdf
[Gary Marcus] Horse rides astronaut
https://garymarcus.substack.com/p/horse-rides-astronaut
[Gary Marcus] GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about
https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/
[Bender et al] On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?
https://dl.acm.org/doi/10.1145/3442188.3445922
[janus] Simulators (Less Wrong)
https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators
-
#83 Dr. ANDREW LAMPINEN (Deepmind) - Natural Language, Symbols and Grounding [NEURIPS2022 UNPLUGGED]
4 december 2022 · 21 min
First in our unplugged series live from #NeurIPS2022 Lees meer
We discuss natural language understanding, symbol meaning and grounding and Chomsky with Dr. Andrew Lampinen from DeepMind.
We recorded a LOT of material from NeurIPS, keep an eye out for the uploads.
YT version: https://youtu.be/46A-BcBbMnA
References
[Paul Cisek] Beyond the computer metaphor: Behaviour as interaction
https://philpapers.org/rec/CISBTC
Linguistic Competence (Chomsky reference)
https://en.wikipedia.org/wiki/Linguistic_competence
[Andrew Lampinen] Can language models handle recursively nested grammatical structures? A case study on comparing models and humans
https://arxiv.org/abs/2210.15303
[Fodor et al] Connectionism and Cognitive Architecture: A Critical Analysis
https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/proseminars/Proseminar13/ConnectionistArchitecture.pdf
[Melanie Mitchell et al] The Debate Over Understanding in AI's Large Language Models
https://arxiv.org/abs/2210.13966
[Gary Marcus] GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about
https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/
[Bender et al] On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?
https://dl.acm.org/doi/10.1145/3442188.3445922
[Adam Santoro, Andrew Lampinen et al] Symbolic Behaviour in Artificial Intelligence
https://arxiv.org/abs/2102.03406
[Ishita Dasgupta, Lampinen et al] Language models show human-like content effects on reasoning
https://arxiv.org/abs/2207.07051
REACT - Synergizing Reasoning and Acting in Language Models
https://arxiv.org/pdf/2210.03629.pdf
https://ai.googleblog.com/2022/11/react-synergizing-reasoning-and-acting.html
[Fabian Paischer] HELM - History Compression via Language Models in Reinforcement Learning
https://ml-jku.github.io/blog/2022/helm/
https://arxiv.org/abs/2205.12258
[Laura Ruis] Large language models are not zero-shot communicator…
-
#82 - Dr. JOSCHA BACH - Digital Physics, DL and Consciousness [UNPLUGGED]
27 november 2022 · 1 uur 15 min
AI Helps Ukraine - Charity Conference Lees meer
A charity conference on AI to raise funds for medical and humanitarian aid for Ukraine
https://aihelpsukraine.cc/
YT version: https://youtu.be/LgwjcqhkOA4
Support us!
https://www.patreon.com/mlst
Dr. Joscha Bach (born 1973 in Weimar, Germany) is a German artificial intelligence researcher and cognitive scientist focusing on cognitive architectures, mental representation, emotion, social modelling, and multi-agent systems.
http://bach.ai/
https://twitter.com/plinz
TOC:
[00:00:00] Ukraine Charity Conference and NeurIPS 2022
[00:03:40] Theory of computation, Godel, Penrose
[00:11:44] Modelling physical reality
[00:15:19] Is our universe infinite?
[00:24:30] Large language models, and on DL / is Gary Marcus hitting a wall?
[00:45:17] Generative models / Codex / Language of thought
[00:58:46] Consciousness (with Friston references)
References:
Am I Self-Conscious? (Or Does Self-Organization Entail Self-Consciousness?) [Friston]
https://www.frontiersin.org/articles/10.3389/fpsyg.2018.00579/full
Impact of Pretraining Term Frequencies on Few-Shot Reasoning [Yasaman Razeghi]
https://arxiv.org/abs/2202.07206
Deep Learning Is Hitting a Wall [Gary Marcus]
https://nautil.us/deep-learning-is-hitting-a-wall-238440/
Turing machines
https://en.wikipedia.org/wiki/Turing_machine
Lambda Calculus
https://en.wikipedia.org/wiki/Lambda_calculus
Godel's incompletness theorem
https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems
Oracle machine
https://en.wikipedia.org/wiki/Oracle_machine
-
#81 JULIAN TOGELIUS, Prof. KEN STANLEY - AGI, Games, Diversity & Creativity [UNPLUGGED]
20 november 2022 · 1 uur 10 min
Support us (and please rate on podcast app) Lees meer
https://www.patreon.com/mlst
In this show tonight with Prof. Julian Togelius (NYU) and Prof. Ken Stanley we discuss open-endedness, AGI, game AI and reinforcement learning.
[Prof Julian Togelius]
https://engineering.nyu.edu/faculty/julian-togelius
https://twitter.com/togelius
[Prof Ken Stanley]
https://www.cs.ucf.edu/~kstanley/
https://twitter.com/kenneth0stanley
TOC:
[00:00:00] Introduction
[00:01:07] AI and computer games
[00:12:23] Intelligence
[00:21:27] Intelligence Explosion
[00:25:37] What should we be aspiring towards?
[00:29:14] Should AI contribute to culture?
[00:32:12] On creativity and open-endedness
[00:36:11] RL overfitting
[00:44:02] Diversity preservation
[00:51:18] Empiricism vs rationalism , in gradient descent the data pushes you around
[00:55:49] Creativity and interestingness (does complexity / information increase)
[01:03:20] What does a population give us?
[01:05:58] Emergence / generalisation snobbery
References;
[Hutter/Legg] Universal Intelligence: A Definition of Machine Intelligence
https://arxiv.org/abs/0712.3329
https://en.wikipedia.org/wiki/Artificial_general_intelligence
https://en.wikipedia.org/wiki/I._J._Good
https://en.wikipedia.org/wiki/G%C3%B6del_machine
[Chollet] Impossibility of intelligence explosion
https://medium.com/@francois.chollet/the-impossibility-of-intelligence-explosion-5be4a9eda6ec
[Alex Irpan] - RL is hard
https://www.alexirpan.com/2018/02/14/rl-hard.html
https://nethackchallenge.com/
Map elites
https://arxiv.org/abs/1504.04909
Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space
https://arxiv.org/abs/1912.02400
[Stanley] - Why greatness cannot be planned
https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237
[Lehman/Stanley] Abandoning Objectives: Evolution through the Search for Novelty Alone
https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/lehman_ecj11.pdf
-
#80 AIDAN GOMEZ [CEO Cohere] - Language as Software
15 november 2022 · 52 min
We had a conversation with Aidan Gomez, the CEO of language-based AI platform Cohere. Cohere is a startup which uses artificial intelligence to help users build the next generation of language-based applications. It's headquartered in Toronto. The company has raised $175 million in funding so far. Lees meer
Language may well become a key new substrate for software building, both in its representation and how we build the software. It may democratise software building so that more people can build software, and we can build new types of software. Aidan and I discuss this in detail in this episode of MLST.
Check out Cohere -- https://dashboard.cohere.ai/welcome/register?utm_source=influencer&utm_medium=social&utm_campaign=mlst
Support us!
https://www.patreon.com/mlst
YT version: https://youtu.be/ooBt_di8DLs
TOC:
[00:00:00] Aidan Gomez intro
[00:02:12] What's it like being a CEO?
[00:02:52] Transformers
[00:09:33] Deepmind Chomsky Hierarchy
[00:14:58] Cohere roadmap
[00:18:18] Friction using LLMs for startups
[00:25:31] How different from OpenAI / GPT-3
[00:29:31] Engineering questions on Cohere
[00:35:13] Francois Chollet says that LLMs are like databases
[00:38:34] Next frontier of language models
[00:42:04] Different modes of understanding in LLMs
[00:47:04] LLMs are the new extended mind
[00:50:03] Is language the next interface, and why might that be bad?
References:
[Balestriero] Spine theory of NNs
https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf
[Delétang et al] Neural Networks and the Chomsky Hierarchy
https://arxiv.org/abs/2207.02098
[Fodor, Pylyshyn] Connectionism and Cognitive Architecture: A Critical Analysis
https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/docs/jaf.pdf
[Chalmers, Clark] The extended mind
https://icds.uoregon.edu/wp-content/uploads/2014/06/Clark-and-Chalmers-The-Extended-Mind.pdf
[Melanie Mitchell et al] The Debate Over Understanding in AI's Large Language Models
https://arxiv.org/abs/2210.13…
-
#79 Consciousness and the Chinese Room [Special Edition] (CHOLLET, BISHOP, CHALMERS, BACH)
8 november 2022 · 2 uur 10 min
This video is demonetised on music copyright so we would appreciate support on our Patreon! https://www.patreon.com/mlst Lees meer
We would also appreciate it if you rated us on your podcast platform.
YT: https://youtu.be/_KVAzAzO5HU
Panel: Dr. Tim Scarfe, Dr. Keith Duggar
Guests: Prof. J. Mark Bishop, Francois Chollet, Prof. David Chalmers, Dr. Joscha Bach, Prof. Karl Friston, Alexander Mattick, Sam Roffey
The Chinese Room Argument was first proposed by philosopher John Searle in 1980. It is an argument against the possibility of artificial intelligence (AI) – that is, the idea that a machine could ever be truly intelligent, as opposed to just imitating intelligence.
The argument goes like this:
Imagine a room in which a person sits at a desk, with a book of rules in front of them. This person does not understand Chinese.
Someone outside the room passes a piece of paper through a slot in the door. On this paper is a Chinese character. The person in the room consults the book of rules and, following these rules, writes down another Chinese character and passes it back out through the slot.
To someone outside the room, it appears that the person in the room is engaging in a conversation in Chinese. In reality, they have no idea what they are doing – they are just following the rules in the book.
The Chinese Room Argument is an argument against the idea that a machine could ever be truly intelligent. It is based on the idea that intelligence requires understanding, and that following rules is not the same as understanding.
in this detailed investigation into the Chinese Room, Consciousness and Syntax vs Semantics, we interview luminaries J.Mark Bishop and Francois Chollet and use unreleased footage from our interviews with David Chalmers, Joscha Bach and Karl Friston. We also cover material from Walid Saba and interview Alex Mattick from Yannic's Discord.
This is probably my favourite ever episode of MLST. I hope you enjoy it! With Keith Duggar.
Note that we are u…
-
MLST #78 - Prof. NOAM CHOMSKY (Special Edition)
9 juli 2022 · 3 uur 37 min
Patreon: https://www.patreon.com/mlst Lees meer
Discord: https://discord.gg/ESrGqhf5CB
In this special edition episode, we have a conversation with Prof. Noam Chomsky, the father of modern linguistics and the most important intellectual of the 20th century.
With a career spanning the better part of a century, we took the chance to ask Prof. Chomsky his thoughts not only on the progress of linguistics and cognitive science but also the deepest enduring mysteries of science and philosophy as a whole - exploring what may lie beyond our limits of understanding. We also discuss the rise of connectionism and large language models, our quest to discover an intelligible world, and the boundaries between silicon and biology.
We explore some of the profound misunderstandings of linguistics in general and Chomsky’s own work specifically which have persisted, at the highest levels of academia for over sixty years.
We have produced a significant introduction section where we discuss in detail Yann LeCun’s recent position paper on AGI, a recent paper on emergence in LLMs, empiricism related to cognitive science, cognitive templates, “the ghost in the machine” and language.
Panel:
Dr. Tim Scarfe
Dr. Keith Duggar
Dr. Walid Saba
YT version: https://youtu.be/-9I4SgkHpcA
00:00:00 Kick off
00:02:24 C1: LeCun's recent position paper on AI, JEPA, Schmidhuber, EBMs
00:48:38 C2: Emergent abilities in LLMs paper
00:51:32 C3: Empiricism
01:25:33 C4: Cognitive Templates
01:35:47 C5: The Ghost in the Machine
01:59:21 C6: Connectionism and Cognitive Architecture: A Critical Analysis by Fodor and Pylyshyn
02:19:25 C7: We deep-faked Chomsky
02:29:11 C8: Language
02:34:41 C9: Chomsky interview kick-off!
02:35:39 Large Language Models such as GPT-3
02:39:14 Connectionism and radical empiricism
02:44:44 Hybrid systems such as neurosymbolic
02:48:47 Computationalism silicon vs biological
02:53:28 Limits of human understanding
03:00:46 Semantics state-of-the-art
03:06:43 Universal gr…
-
#77 - Vitaliy Chiley (Cerebras)
16 juni 2022 · 1 uur 8 min
Vitaliy Chiley is a Machine Learning Research Engineer at the next-generation computing hardware company Cerebras Systems. We spoke about how DL workloads including sparse workloads can run faster on Cerebras hardware. Lees meer
[00:00:00] Housekeeping
[00:01:08] Preamble
[00:01:50] Vitaliy Chiley Introduction
[00:03:11] Cerebrus architecture
[00:08:12] Memory management and FLOP utilisation
[00:18:01] Centralised vs decentralised compute architecture
[00:21:12] Sparsity
[00:23:47] Does Sparse NN imply Heterogeneous compute?
[00:29:21] Cost of distributed memory stores?
[00:31:01] Activation vs weight sparsity
[00:37:52] What constitutes a dead weight to be pruned?
[00:39:02] Is it still a saving if we have to choose between weight and activation sparsity?
[00:41:02] Cerebras is a cool place to work
[00:44:05] What is sparsity? Why do we need to start dense?
[00:46:36] Evolutionary algorithms on Cerebras?
[00:47:57] How can we start sparse? Google RIGL
[00:51:44] Inductive priors, why do we need them if we can start sparse?
[00:56:02] Why anthropomorphise inductive priors?
[01:02:13] Could Cerebras run a cyclic computational graph?
[01:03:16] Are NNs locality sensitive hashing tables?
References;
Rigging the Lottery: Making All Tickets Winners [RIGL]
https://arxiv.org/pdf/1911.11134.pdf
[D] DanNet, the CUDA CNN of Dan Ciresan in Jurgen Schmidhuber's team, won 4 image recognition challenges prior to AlexNet
https://www.reddit.com/r/MachineLearning/comments/dwnuwh/d_dannet_the_cuda_cnn_of_dan_ciresan_in_jurgen/
A Spline Theory of Deep Learning [Balestriero]
https://proceedings.mlr.press/v80/balestriero18b.html
-
#76 - LUKAS BIEWALD (Weights and Biases CEO)
9 juni 2022 · 58 min
Check out Weights and Biases here! Lees meer
https://wandb.me/MLST
Lukas Biewald is an entrepreneur living in San Francisco. He was the founder and CEO of Figure Eight an Internet company that collects training data for machine learning. In 2018, he founded Weights and Biases, a company that creates developer tools for machine learning. Recently WandB got a cash injection of 15 million dollars in its second funding round.
Lukas has a bachelors and masters in mathematics and computer science respectively from Stanford university. He was a research student under the tutelage of the legendary Daphne Koller.
Lukas Biewald
https://twitter.com/l2k
[00:00:00] Preamble
[00:01:27] Intro to Lukas
[00:02:46] How did Lukas build 2 sucessful startups?
[00:05:49] Rebalancing games with ML
[00:08:14] Elevator pitch for WandB
[00:10:38] Science vs Engineering divide in ML DevOps
[00:14:11] Too much focus on the minutiae?
[00:18:03] Vertical information sharing in large enterprises (metrics)
[00:20:37] Centralised vs Decentralised topology
[00:24:02] Generalisation vs specialisation
[00:28:59] Enhancing explainability
[00:33:14] Should we try and understand "the machine" or is testing / behaviourism enough?
[00:36:55] WandB roadmap
[00:39:06] WandB / ML Ops competitor space?
[00:44:10] How is WandB differentiated over Sagemaker / AzureML
[00:46:02] WandB Sponsorship of ML YT channels
[00:48:43] Alternatives to deep learning?
[00:53:47] How to build a business like WandB
Panel: Tim Scarfe Ph.D and Keith Duggar Ph.D
Note we didn't get paid by Weights and Biases to conduct this interview.
-
#75 - Emergence [Special Edition] with Dr. DANIELE GRATTAROLA
29 april 2022 · 1 uur 55 min
An emergent behavior or emergent property can appear when a number of simple entities operate in an environment, forming more complex behaviours as a collective. If emergence happens over disparate size scales, then the reason is usually a causal relation across different scales. Weak emergence describes new properties arising in systems as a result of the low-level interactions, these might be interactions between components of the system or the components and their environment. Lees meer
In our epic introduction we focus a lot on the concept of self-organisation, complex systems, cellular automata and strong vs weak emergence. In the main show we discuss this more in detail with Dr. Daniele Grattarola and cover his recent NeurIPS paper on learning graph cellular automata.
YT version: https://youtu.be/MDt2e8XtUcA
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB
Featuring;
Dr. Daniele Grattarola
Dr. Tim Scarfe
Dr. Keith Duggar
Prof. David Chalmers
Prof. Ken Stanley
Prof. Julian Togelius
Dr. Joscha Bach
David Ha
Dr. Pei Wang
[00:00:00] Special Edition Intro: Emergence and Cellular Automata
[00:49:02] Intro to Daniele and CAs
[00:57:23] Numerical analysis link with CA (PDEs)
[00:59:50] The representational dichotomy of discrete and continuous at different scales
[01:05:21] Universal computation in CAs
[01:10:27] Computational irreducibility
[01:16:33] Is the universe discrete?
[01:20:49] Emergence but with the same computational principle
[01:23:10] How do you formalise the emergent phenomenon
[01:25:44] Growing cellular automata
[01:33:53] Openeded and unbounded computation is required for this kind of behaviour
[01:37:31] Graph cellula automata
[01:43:40] Connection to protein folding
[01:46:24] Are CAs the best tool for the job?
[01:49:37] Where to go to find more information
-
#74 Dr. ANDREW LAMPINEN - Symbolic behaviour in AI [UNPLUGGED]
14 april 2022 · 1 uur 6 min
Please note that in this interview Dr. Lampinen was expressing his personal opinions and they do not necessarily represent those of DeepMind. Lees meer
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB
YT version: https://youtu.be/yPMtSXXn4OY
Dr. Andrew Lampinen is a Senior Research Scientist at DeepMind, and he thinks that symbols are subjective in the relativistic sense. Dr. Lampinen completed his PhD in Cognitive Psychology at Stanford University. His background is in mathematics, physics, and machine learning. Andrew has said that his research interests are in cognitive flexibililty and generalization, and how these abilities are enabled by factors like language, memory, and embodiment. Andrew with his coauthors has just released a paper called symbolic behaviour in artificial intelligence. Andrew lead in the paper by saying the human ability to use symbols has yet to be replicated in machines. He thinks that one of the key areas to bridge the gap here is considering how symbol meaning is established, and he strongly believes it is the symbol users themselves who agree upon the symbol meaning, And that the use of symbols entails behaviours which coalesce agreements about their meaning. Which in plain English means that symbols are defined by behaviours rather than their content.
[00:00:00] Intro to Andrew and Symbolic Behaviour paper
[00:07:01] Semantics underpins the unreasonable effectiveness of symbols
[00:12:56] The Depth of Subjectivity
[00:21:03] Walid Saba - universal cognitive templates
[00:27:47] Insufficiently Darwinian
[00:30:52] Discovered vs invented
[00:34:19] Does language have primacy
[00:35:59] Research directions
[00:39:43] Comparison to BenG OpenCog and human compatible AI
[00:42:53] Aligning AI with our culture
[00:47:55] Do we need to model the worst aspects of human behaviour?
[00:50:57] Fairness
[00:54:24] Memorisatation on LLMs
[01:00:38] Wason selection task
[01:03:45] Would an Andrew hashtable robot …
-
#73 - YASAMAN RAZEGHI & Prof. SAMEER SINGH - NLP benchmarks
7 april 2022 · 56 min
Patreon: https://www.patreon.com/mlst Lees meer
Discord: https://discord.gg/ESrGqhf5CB
YT version: https://youtu.be/RzGaI7vXrkk
This week we speak with Yasaman Razeghi and Prof. Sameer Singh from UC Urvine. Yasaman recently published a paper called Impact of Pretraining Term Frequencies on Few-Shot Reasoning where she demonstrated comprehensively that large language models only perform well on reasoning tasks because they memorise the dataset. For the first time she showed the accuracy was linearly correlated to the occurance rate in the training corpus, something which OpenAI should have done in the first place!
We also speak with Sameer who has been a pioneering force in the area of machine learning interpretability for many years now, he created LIME with Marco Riberio and also had his hands all over the famous Checklist paper and many others.
We also get into the metric obsession in the NLP world and whether metrics are one of the principle reasons why we are failing to make any progress in NLU.
[00:00:00] Impact of Pretraining Term Frequencies on Few-Shot Reasoning
[00:14:59] Metrics
[00:18:55] Definition of reasoning
[00:25:12] Metrics (again)
[00:28:52] On true believers
[00:33:04] Sameers work on model explainability / LIME
[00:36:58] Computational irreducability
[00:41:07] ML DevOps and Checklist
[00:45:58] Future of ML devops
[00:49:34] Thinking about future
Prof. Sameer Singh
https://sameersingh.org/
Yasaman Razeghi
https://yasamanrazeghi.com/
References;
Impact of Pretraining Term Frequencies on Few-Shot Reasoning [Razeghi et al with Singh]
https://arxiv.org/pdf/2202.07206.pdf
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList [Riberio et al with Singh]
https://arxiv.org/pdf/2005.04118.pdf
“Why Should I Trust You?” Explaining the Predictions of Any Classifier (LIME) [Riberio et al with Singh]
https://arxiv.org/abs/1602.04938
Tim interviewing LIME Creator Marco Ribeiro in 2019
https://www.youtube.com/watch?v=6aUU-Ob4a8I
T…