Machine Learning Street Talk (MLST)
Alle afleveringen
272 afleveringen, van nieuw naar oud
-
#72 Prof. KEN STANLEY 2.0 - On Art and Subjectivity [UNPLUGGED]
29 maart 2022 · 1 uur 25 min
YT version: https://youtu.be/DxBZORM9F-8 Lees meer
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB
Prof. Ken Stanley argued in his book that our world has become saturated with objectives. The process of setting an objective, attempting to achieve it, and measuring progress along the way has become the primary route to achievement in our culture. He’s not saying that objectives are bad per se, especially if they’re modest, but he thinks that when goals are ambitious then the search space becomes deceptive.
Is the key to artificial intelligence really related to intelligence? Does taking a job with a higher salary really bring you closer to being a millionaire? The problem is that the stepping stones which lead to ambitious objectives tend to be pretty strange, they don't resemble the final end state at all. Vaccum tubes led to computers for example and Youtube started as a dating website.
What fascinated us about this conversation with Ken is that we got a much deeper understanding of his philosophy. He lead by saying that he thought it's worth questioning whether artificial intelligence is even a science or not. Ken thinks that the secret to future progress is for us to embrace more subjectivity.
[00:00:00] Tim Intro
[00:12:54] Intro
[00:17:08] Seeing ideas everywhere - AI and art are highly connected
[00:28:40] Creativity in Mathematics
[00:30:14] Where is the intelligence in art?
[00:38:49] Is AI disappointingly simple to mechanise?
[00:42:48] Slightly conscious
[00:46:27] Do we have subjective experience?
[00:50:23] Fear of the unknown
[00:51:48] Free Will
[00:54:22] Chalmers
[00:55:08] What's happening now in open-endedness
[00:58:31] Generalisation
[01:06:34] Representation primitives and what it means to understand
[01:12:37] Appeal to definitions, knowledge itself blocks discovery
Make sure you buy Kenneth's book!
Why Greatness Cannot Be Planned: The Myth of the Objective [Stanley, Lehman]
https://www.amazon.co.uk/W…
-
#71 - ZAK JOST (Graph Neural Networks + Geometric DL) [UNPLUGGED]
25 maart 2022 · 1 uur 3 min
Special discount link for Zak's GNN course - https://bit.ly/3uqmYVq Lees meer
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB
YT version: https://youtu.be/jAGIuobLp60 (there are lots of helper graphics there, recommended if poss)
Want to sponsor MLST!? Let us know on Linkedin / Twitter.
[00:00:00] Preamble
[00:03:12] Geometric deep learning
[00:10:04] Message passing
[00:20:42] Top down vs bottom up
[00:24:59] All NN architectures are different forms of information diffusion processes (squashing and smoothing problem)
[00:29:51] Graph rewiring
[00:31:38] Back to information diffusion
[00:42:43] Transformers vs GNNs
[00:47:10] Equivariant subgraph aggregation networks + WL test
[00:55:36] Do equivariant layers aggregate too?
[00:57:49] Zak's GNN course
Exhaustive list of references on the YT show URL (https://youtu.be/jAGIuobLp60)
-
#70 - LETITIA PARCALABESCU - Symbolics, Linguistics [UNPLUGGED]
19 maart 2022 · 1 uur 18 min
Today we are having a discussion with Letitia Parcalabescu from the AI Coffee Break youtube channel! We discuss linguistics, symbolic AI and our respective Youtube channels. Make sure you subscribe to her channel! In the first 15 minutes Tim dissects the recent article from Gary Marcus "Deep learning has hit a wall". Lees meer
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB
YT: https://youtu.be/p2D2duT-R2E
[00:00:00] Comments on Gary Marcus Article / Symbolic AI
[00:14:57] Greetings
[00:17:40] Introduction
[00:18:48] A shared journey towards computation
[00:22:10] A linguistics outsider
[00:24:11] Is computational linguistics AI?
[00:28:23] swinging pendulums of dogma and resource allocation
[00:31:16] the road less travelled
[00:34:35] pitching grants with multimodality ... and then the truth
[00:40:50] some aspects of language are statistically learnable
[00:44:58] ... and some aspects of language are dimensionally cursed
[00:48:24] it's good to have both approaches to machine intelligence
[00:51:14] the world runs on symbols
[00:54:28] there is much more to learn biology
[00:59:26] Letitia's creation process
[01:02:23] don't overfit content, instead publish and iterate
[01:07:48] merging the big picture arrow from the small direction arrows
[01:11:02] use passion to drive through failure to success
[01:12:56] stay positive
[01:16:02] closing remarks
-
#69 DR. THOMAS LUX - Interpolation of Sparse High-Dimensional Data
12 maart 2022 · 51 min
Today we are speaking with Dr. Thomas Lux, a research scientist at Meta in Silicon Valley. Lees meer
In some sense, all of supervised machine learning can be framed through the lens of geometry. All training data exists as points in euclidean space, and we want to predict the value of a function at all those points. Neural networks appear to be the modus operandi these days for many domains of prediction. In that light; we might ask ourselves — what makes neural networks better than classical techniques like K nearest neighbour from a geometric perspective. Our guest today has done research on exactly that problem, trying to define error bounds for approximations in terms of directions, distances, and derivatives.
The insights from Thomas's work point at why neural networks are so good at problems which everything else fails at, like image recognition. The key is in their ability to ignore parts of the input space, do nonlinear dimension reduction, and concentrate their approximation power on important parts of the function.
[00:00:00] Intro to Show
[00:04:11] Intro to Thomas (Main show kick off)
[00:04:56] Interpolation of Sparse High-Dimensional Data
[00:12:19] Where does one place the basis functions to partition the space, the perennial question
[00:16:20] The sampling phenomenon -- where did all those dimensions come from?
[00:17:40] The placement of the MLP basis functions, they are not where you think they are
[00:23:15] NNs only extrapolate when given explicit priors to do so, CNNs in the translation domain
[00:25:31] Transformers extrapolate in the permutation domain
[00:28:26] NN priors work by creating space junk everywhere
[00:36:44] Are vector spaces the way to go? On discrete problems
[00:40:23] Activation functioms
[00:45:57] What can we prove about NNs? Gradients without backprop
Interpolation of Sparse High-Dimensional Data [Lux]
https://tchlux.github.io/papers/tchlux-2020-NUMA.pdf
A Spline Theory of Deep Learning [_Balestriero_]
https://pr…
-
#68 DR. WALID SABA 2.0 - Natural Language Understanding [UNPLUGGED]
7 maart 2022 · 1 uur 42 min
Patreon: https://www.patreon.com/mlst Lees meer
Discord: https://discord.gg/HNnAwSduud
YT version: https://youtu.be/pMtk-iUaEuQ
Dr. Walid Saba is an old-school polymath. He has a background in cognitive psychology, linguistics, philosophy, computer science and logic and he’s is now a Senior Scientist at Sorcero.
Walid is perhaps the most outspoken critic of BERTOLOGY, which is to say trying to solve the problem of natural language understanding with application of large statistical language models. Walid thinks this approach is cursed to failure because it’s analogous to memorising infinity with a large hashtable. Walid thinks that the various appeals to infinity by some deep learning researchers are risible.
[00:00:00] MLST Housekeeping
[00:08:03] Dr. Walid Saba Intro
[00:11:56] AI Cannot Ignore Symbolic Logic, and Here’s Why
[00:23:39] Main show - Proposition: Statistical learning doesn't work
[01:04:44] Discovering a sorting algorithm bottom-up is hard
[01:17:36] The axioms of nature (universal cognitive templates)
[01:31:06] MLPs are locality sensitive hashing tables
References;
The Missing Text Phenomenon, Again: the case of Compound Nominals
https://ontologik.medium.com/the-missing-text-phenomenon-again-the-case-of-compound-nominals-abb6ece3e205
A Spline Theory of Deep Networks
https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf
The Defeat of the Winograd Schema Challenge
https://arxiv.org/pdf/2201.02387.pdf
Impact of Pretraining Term Frequencies on Few-Shot Reasoning
https://twitter.com/yasaman_razeghi/status/1495112604854882304?s=21
https://arxiv.org/abs/2202.07206
AI Cannot Ignore Symbolic Logic, and Here’s Why
https://medium.com/ontologik/ai-cannot-ignore-symbolic-logic-and-heres-why-1f896713525b
Learnability can be undecidable
http://gtts.ehu.es/German/Docencia/1819/AC/extras/s42256-018-0002-3.pdf
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
https://arxiv.org/pdf/2112.11446.pdf
DreamCoder: G…
-
#67 Prof. KARL FRISTON 2.0
2 maart 2022 · 1 uur 42 min
We engage in a bit of epistemic foraging with Prof. Karl Friston! In this show; we discuss the free energy principle in detail, also emergence, cognition, consciousness and Karl's burden of knowledge! Lees meer
YT: https://youtu.be/xKQ-F2-o8uM
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/HNnAwSduud
[00:00:00] Introduction to FEP/Friston
[00:06:53] Cheers to Epistemic Foraging!
[00:09:17] The Burden of Knowledge Across Disciplines
[00:12:55] On-show introduction to Friston
[00:14:23] Simple does NOT mean Easy
[00:21:25] Searching for a Mathematics of Cognition
[00:26:44] The Low Road and The High Road to the Principle
[00:28:27] What's changed for the FEP in the last year
[00:39:36] FEP as stochastic systems with a pullback attractor
[00:44:03] An attracting set at multiple time scales and time infinity
[00:53:56] What about fuzzy Markov boundaries?
[00:59:17] Is reality densely or sparsely coupled?
[01:07:00] Is a Strong and Weak Emergence distinction useful?
[01:13:25] a Philosopher, a Zombie, and a Sentient Consciousness walk into a bar ...
[01:24:28] Can we recreate consciousness in silico? Will it have qualia?
[01:28:29] Subjectivity and building hypotheses
[01:34:17] Subject specific realizations to minimize free energy
[01:37:21] Free will in a deterministic Universe
The free energy principle made simpler but not too simple
https://arxiv.org/abs/2201.06387
-
#66 ALEXANDER MATTICK - [Unplugged / Community Edition]
28 februari 2022 · 51 min
We have a chat with Alexander Mattick aka ZickZack from Yannic's Discord community. Alex is one of the leading voices in that community and has an impressive technical depth. Don't forget MLST has now started it's own Discord server too, come and join us! We are going to run regular events, our first big event on Wednesday 9th 1700-1900 UK time. Lees meer
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/HNnAwSduud
YT version: https://youtu.be/rGOOLC8cIO4
[00:00:00] Introduction to Alex
[00:02:16] Spline theory of NNs
[00:05:19] Do NNs abstract?
[00:08:27] Tim's exposition of spline theory of NNs
[00:11:11] Semantics in NNs
[00:13:37] Continuous vs discrete
[00:19:00] Open-ended Search
[00:22:54] Inductive logic programming
[00:25:00] Control to gain knowledge and knowledge to gain control
[00:30:22] Being a generalist with a breadth of knowledge and knowledge transfer
[00:36:29] Causality
[00:43:14] Discrete program synthesis + theorem solvers
-
#65 Prof. PEDRO DOMINGOS [Unplugged]
26 februari 2022 · 1 uur 28 min
Note: there are no politics discussed in this show and please do not interpret this show as any kind of a political statement from us. We have decided not to discuss politics on MLST anymore due to its divisive nature. Lees meer
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/HNnAwSduud
[00:00:00] Intro
[00:01:36] What we all need to understand about machine learning
[00:06:05] The Master Algorithm Target Audience
[00:09:50] Deeply Connected Algorithms seen from Divergent Frames of Reference
[00:12:49] There is a Master Algorithm; and it's mine!
[00:14:59] The Tribe of Evolution
[00:17:17] Biological Inspirations and Predictive Coding
[00:22:09] Shoe-Horning Gradient Descent
[00:27:12] Sparsity at Training Time vs Prediction Time
[00:30:00] World Models and Predictive Coding
[00:33:24] The Cartoons of System 1 and System 2
[00:40:37] AlphaGo Searching vs Learning
[00:45:56] Discriminative Models evolve into Generative Models
[00:50:36] Generative Models, Predictive Coding, GFlowNets
[00:55:50] Sympathy for a Thousand Brains
[00:59:05] A Spectrum of Tribes
[01:04:29] Causal Structure and Modelling
[01:09:39] Entropy and The Duality of Past vs Future, Knowledge vs Control
[01:16:14] A Discrete Universe?
[01:19:49] And yet continuous models work so well
[01:23:31] Finding a Discretised Theory of Everything
-
#64 Prof. Gary Marcus 3.0
24 februari 2022 · 52 min
Patreon: https://www.patreon.com/mlst Lees meer
Discord: https://discord.gg/HNnAwSduud
YT: https://www.youtube.com/watch?v=ZDY2nhkPZxw
We have a chat with Prof. Gary Marcus about everything which is currently top of mind for him, consciousness
[00:00:00] Gary intro
[00:01:25] Slightly conscious
[00:24:59] Abstract, compositional models
[00:32:46] Spline theory of NNs
[00:36:17] Self driving cars / algebraic reasoning
[00:39:43] Extrapolation
[00:44:15] Scaling laws
[00:49:50] Maximum likelihood estimation
References:
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
https://arxiv.org/abs/2201.02177
DEEP DOUBLE DESCENT: WHERE BIGGER MODELS AND MORE DATA HURT
https://arxiv.org/pdf/1912.02292.pdf
Bayesian Deep Learning and a Probabilistic Perspective of Generalization
https://arxiv.org/pdf/2002.08791.pdf
-
#063 - Prof. YOSHUA BENGIO - GFlowNets, Consciousness & Causality
22 februari 2022 · 1 uur 33 min
We are now sponsored by Weights and Biases! Please visit our sponsor link: http://wandb.me/MLST Lees meer
Patreon: https://www.patreon.com/mlst
For Yoshua Bengio, GFlowNets are the most exciting thing on the horizon of Machine Learning today. He believes they can solve previously intractable problems and hold the key to unlocking machine abstract reasoning itself. This discussion explores the promise of GFlowNets and the personal journey Prof. Bengio traveled to reach them.
Panel:
Dr. Tim Scarfe
Dr. Keith Duggar
Dr. Yannic Kilcher
Our special thanks to:
- Alexander Mattick (Zickzack)
References:
Yoshua Bengio @ MILA (https://mila.quebec/en/person/bengio-yoshua/)
GFlowNet Foundations (https://arxiv.org/pdf/2111.09266.pdf)
Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation (https://arxiv.org/pdf/2106.04399.pdf)
Interpolation Consistency Training for Semi-Supervised Learning (https://arxiv.org/pdf/1903.03825.pdf)
Towards Causal Representation Learning (https://arxiv.org/pdf/2102.11107.pdf)
Causal inference using invariant prediction: identification and confidence intervals (https://arxiv.org/pdf/1501.01332.pdf)
-
#062 - Dr. Guy Emerson - Linguistics, Distributional Semantics
3 februari 2022 · 1 uur 30 min
Dr. Guy Emerson is a computational linguist and obtained his Ph.D from Cambridge university where he is now a research fellow and lecturer. On panel we also have myself, Dr. Tim Scarfe, as well as Dr. Keith Duggar and the veritable Dr. Walid Saba. We dive into distributional semantics, probability theory, fuzzy logic, grounding, vagueness and the grammar/cognition connection. Lees meer
The aim of distributional semantics is to design computational techniques that can automatically learn the meanings of words from a body of text. The twin challenges are: how do we represent meaning, and how do we learn these representations? We want to learn the meanings of words from a corpus by exploiting the fact that the context of a word tells us something about its meaning. This is known as the distributional hypothesis. In his Ph.D thesis, Dr. Guy Emerson presented a distributional model which can learn truth-conditional semantics which are grounded by objects in the real world.
Hope you enjoy the show!
https://www.cai.cam.ac.uk/people/dr-guy-emerson
https://www.repository.cam.ac.uk/handle/1810/284882?show=full
https://www.semanticscholar.org/paper/Computational-linguistics-and-grammar-engineering-Bender-Emerson/bbd6f3b92a0f1ea8212f383cc4719bfe86b3588c
Patreon: https://www.patreon.com/mlst
-
061: Interpolation, Extrapolation and Linearisation (Prof. Yann LeCun, Dr. Randall Balestriero)
4 januari 2022 · 3 uur 20 min
We are now sponsored by Weights and Biases! Please visit our sponsor link: http://wandb.me/MLST Lees meer
Patreon: https://www.patreon.com/mlst
Yann LeCun thinks that it's specious to say neural network models are interpolating because in high dimensions, everything is extrapolation. Recently Dr. Randall Balestriero, Dr. Jerome Pesente and prof. Yann LeCun released their paper learning in high dimensions always amounts to extrapolation. This discussion has completely changed how we think about neural networks and their behaviour.
[00:00:00] Pre-intro
[00:11:58] Intro Part 1: On linearisation in NNs
[00:28:17] Intro Part 2: On interpolation in NNs
[00:47:45] Intro Part 3: On the curse
[00:48:19] LeCun
[01:40:51] Randall B
YouTube version: https://youtu.be/86ib0sfdFtw
-
#60 Geometric Deep Learning Blueprint (Special Edition)
19 september 2021 · 3 uur 33 min
Patreon: https://www.patreon.com/mlst Lees meer
The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Many high-dimensional learning tasks previously thought to be beyond reach -- such as computer vision, playing Go, or protein folding -- are in fact tractable given enough computational horsepower. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning and second, learning by local gradient-descent type methods, typically implemented as backpropagation.
While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not uniform and have strong repeating patterns as a result of the low-dimensionality and structure of the physical world.
Geometric Deep Learning unifies a broad class of ML problems from the perspectives of symmetry and invariance. These principles not only underlie the breakthrough performance of convolutional neural networks and the recent success of graph neural networks but also provide a principled way to construct new types of problem-specific inductive biases.
This week we spoke with Professor Michael Bronstein (head of graph ML at Twitter) and Dr.
Petar Veličković (Senior Research Scientist at DeepMind), and Dr. Taco Cohen and Prof. Joan Bruna about their new proto-book Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges.
See the table of contents for this (long) show at https://youtu.be/bIZB1hIJ4u8
-
#59 - Jeff Hawkins (Thousand Brains Theory)
3 september 2021 · 2 uur 35 min
Patreon: https://www.patreon.com/mlst Lees meer
The ultimate goal of neuroscience is to learn how the human brain gives rise to human intelligence and what it means to be intelligent. Understanding how the brain works is considered one of humanity’s greatest challenges.
Jeff Hawkins thinks that the reality we perceive is a kind of simulation, a hallucination, a confabulation. He thinks that our brains are a model reality based on thousands of information streams originating from the sensors in our body. Critically - Hawkins doesn’t think there is just one model but rather; thousands.
Jeff has just released his new book, A thousand brains: a new theory of intelligence. It’s an inspiring and well-written book and I hope after watching this show; you will be inspired to read it too.
https://numenta.com/a-thousand-brains-by-jeff-hawkins/
https://numenta.com/blog/2019/01/16/the-thousand-brains-theory-of-intelligence/
Panel:
Dr. Keith Duggar https://twitter.com/DoctorDuggar
Connor Leahy https://twitter.com/npcollapse
-
#58 Dr. Ben Goertzel - Artificial General Intelligence
11 augustus 2021 · 2 uur 28 min
The field of Artificial Intelligence was founded in the mid 1950s with the aim of constructing “thinking machines” - that is to say, computer systems with human-like general intelligence. Think of humanoid robots that not only look but act and think with intelligence equal to and ultimately greater than that of human beings. But in the intervening years, the field has drifted far from its ambitious old-fashioned roots. Lees meer
Dr. Ben Goertzel is an artificial intelligence researcher, CEO and founder of SingularityNET. A project combining artificial intelligence and blockchain to democratize access to artificial intelligence. Ben seeks to fulfil the original ambitions of the field. Ben graduated with a PhD in Mathematics from Temple University in 1990. Ben’s approach to AGI over many decades now has been inspired by many disciplines, but in particular from human cognitive psychology and computer science perspective. To date Ben’s work has been mostly theoretically-driven. Ben thinks that most of the deep learning approaches to AGI today try to model the brain. They may have a loose analogy to human neuroscience but they have not tried to derive the details of an AGI architecture from an overall conception of what a mind is. Ben thinks that what matters for creating human-level (or greater) intelligence is having the right information processing architecture, not the underlying mechanics via which the architecture is implemented.
Ben thinks that there is a certain set of key cognitive processes and interactions that AGI systems must implement explicitly such as; working and long-term memory, deliberative and reactive processing, perc biological systems tend to be messy, complex and integrative; searching for a single “algorithm of general intelligence” is an inappropriate attempt to project the aesthetics of physics or theoretical computer science into a qualitatively different domain.
TOC is on the YT show description https://www.youtube.com/watch?v=sw8IE3MX1SY
Panel: …
-
#57 - Prof. Melanie Mitchell - Why AI is harder than we think
25 juli 2021 · 2 uur 31 min
Since its beginning in the 1950s, the field of artificial intelligence has vacillated between periods of optimistic predictions and massive investment and periods of disappointment, loss of confidence, and reduced funding. Even with today’s seemingly fast pace of AI breakthroughs, the development of long-promised technologies such as self-driving cars, housekeeping robots, and conversational companions has turned out to be much harder than many people expected. Professor Melanie Mitchell thinks one reason for these repeating cycles is our limited understanding of the nature and complexity of intelligence itself. Lees meer
YT vid- https://www.youtube.com/watch?v=A8m1Oqz2HKc
Main show kick off [00:26:51]
Panel: Dr. Tim Scarfe, Dr. Keith Duggar, Letitia Parcalabescu (https://www.youtube.com/c/AICoffeeBreak/)
-
#56 - Dr. Walid Saba, Gadi Singer, Prof. J. Mark Bishop (Panel discussion)
8 juli 2021 · 1 uur 11 min
It has been over three decades since the statistical revolution overtook AI by a storm and over two decades since deep learning (DL) helped usher the latest resurgence of artificial intelligence (AI). However, the disappointing progress in conversational agents, NLU, and self-driving cars, has made it clear that progress has not lived up to the promise of these empirical and data-driven methods. DARPA has suggested that it is time for a third wave in AI, one that would be characterized by hybrid models – models that combine knowledge-based approaches with data-driven machine learning techniques. Lees meer
Joining us on this panel discussion is polymath and linguist Walid Saba - Co-founder ONTOLOGIK.AI, Gadi Singer - VP & Director, Cognitive Computing Research, Intel Labs and J. Mark Bishop - Professor of Cognitive Computing (Emeritus), Goldsmiths, University of London and Scientific Adviser to FACT360.
Moderated by Dr. Keith Duggar and Dr. Tim Scarfe
https://www.linkedin.com/in/gadi-singer/
https://www.linkedin.com/in/walidsaba/
https://www.linkedin.com/in/profjmarkbishop/
#machinelearning #artificialintelligence
-
#55 Self-Supervised Vision Models (Dr. Ishan Misra - FAIR).
21 juni 2021 · 1 uur 36 min
Dr. Ishan Misra is a Research Scientist at Facebook AI Research where he works on Computer Vision and Machine Learning. His main research interest is reducing the need for human supervision, and indeed, human knowledge in visual learning systems. He finished his PhD at the Robotics Institute at Carnegie Mellon. He has done stints at Microsoft Research, INRIA and Yale. His bachelors is in computer science where he achieved the highest GPA in his cohort. Lees meer
Ishan is fast becoming a prolific scientist, already with more than 3000 citations under his belt and co-authoring with Yann LeCun; the godfather of deep learning. Today though we will be focusing an exciting cluster of recent papers around unsupervised representation learning for computer vision released from FAIR. These are; DINO: Emerging Properties in Self-Supervised Vision Transformers, BARLOW TWINS: Self-Supervised Learning via Redundancy Reduction and PAWS: Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with
Support Samples. All of these papers are hot off the press, just being officially released in the last month or so. Many of you will remember PIRL: Self-Supervised Learning of Pretext-Invariant Representations which Ishan was the primary author of in 2019.
References;
Shuffle and Learn - https://arxiv.org/abs/1603.08561
DepthContrast - https://arxiv.org/abs/2101.02691
DINO - https://arxiv.org/abs/2104.14294
Barlow Twins - https://arxiv.org/abs/2103.03230
SwAV - https://arxiv.org/abs/2006.09882
PIRL - https://arxiv.org/abs/1912.01991
AVID - https://arxiv.org/abs/2004.12943 (best paper candidate at CVPR'21 (just announced over the weekend) - http://cvpr2021.thecvf.com/node/290)
Alexei (Alyosha) Efros
http://people.eecs.berkeley.edu/~efros/
http://www.cs.cmu.edu/~tmalisie/projects/nips09/
Exemplar networks
https://arxiv.org/abs/1406.6909
The bitter lesson - Rich Sutton
http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Machine Teaching: A…
-
#54 Gary Marcus and Luis Lamb - Neurosymbolic models
4 juni 2021 · 2 uur 24 min
Professor Gary Marcus is a scientist, best-selling author, and entrepreneur. He is Founder and CEO of Robust.AI, and was Founder and CEO of Geometric Intelligence, a machine learning company acquired by Uber in 2016. Gary said in his recent next decade paper that — without us, or other creatures like us, the world would continue to exist, but it would not be described, distilled, or understood. Human lives are filled with abstraction and causal description. This is so powerful. Francois Chollet the other week said that intelligence is literally sensitivity to abstract analogies, and that is all there is to it. It's almost as if one of the most important features of intelligence is to be able to abstract knowledge, this drives the generalisation which will allow you to mine previous experience to make sense of many future novel situations. Also joining us today is Professor Luis Lamb — Secretary of Innovation for Science and Technology of the State of Rio Grande do Sul, Brazil. His Research Interests are Machine Learning and Reasoning, Neuro-Symbolic Computing, Logic in Computation and Artificial Intelligence, Cognitive and Neural Computation and also AI Ethics and Social Computing. Luis released his new paper Neurosymbolic AI: the third wave at the end of last year. It beautifully articulated the key ingredients needed in the next generation of AI systems, integrating type 1 and type 2 approaches to AI and it summarises all the of the achievements of the last 20 years of research. We cover a lot of ground in today's show. Explaining the limitations of deep learning, Rich Sutton's the bitter lesson and "reward is enough", and the semantic foundation which is required for us to build robust AI. Lees meer
-
#53 Quantum Natural Language Processing - Prof. Bob Coecke (Oxford)
19 mei 2021 · 2 uur 18 min
Bob Coercke is a celebrated physicist, he's been a Physics and Quantum professor at Oxford University for the last 20 years. He is particularly interested in Structure which is to say, Logic, Order, and Category Theory. He is well known for work involving compositional distributional models of natural language meaning and he is also fascinated with understanding how our brains work. Bob was recently appointed as the Chief Scientist at Cambridge Quantum Computing. Lees meer
Bob thinks that interactions between systems in Quantum Mechanics carries naturally over to how word meanings interact in natural language. Bob argues that this interaction embodies the phenomenon of quantum teleportation.
Bob invented ZX-calculus, a graphical calculus for revealing the compositional structure inside quantum circuits - to show entanglement states and protocols in a visually succinct but logically complete way. Von Neumann himself didn't even like his own original symbolic formalism of quantum theory, despite it being widely used!
We hope you enjoy this fascinating conversation which might give you a lot of insight into natural language processing.
Tim Intro [00:00:00]
The topological brain (Post-record button skit) [00:13:22]
Show kick off [00:19:31]
Bob introduction [00:22:37]
Changing culture in universities [00:24:51]
Machine Learning is like electricity [00:31:50]
NLP -- what is Bob's Quantum conception? [00:34:50]
The missing text problem [00:52:59]
Can statistical induction be trusted? [00:59:49]
On pragmatism and hybrid systems [01:04:42]
Parlour tricks, parsing and information flows [01:07:43]
How much human input is required with Bob's method? [01:11:29]
Reality, meaning, structure and language [01:14:42]
Replacing complexity with quantum entanglement, emergent complexity [01:17:45]
Loading quantum data requires machine learning [01:19:49]
QC is happy math coincidence for NLP [01:22:30]
The Theory of English (ToE) [01:28:23]
... or can we learn the ToE? [01:29…
-
#52 - Unadversarial Examples (Hadi Salman, MIT)
1 mei 2021 · 1 uur 48 min
Performing reliably on unseen or shifting data distributions is a difficult challenge for modern vision systems, even slight corruptions or transformations of images are enough to slash the accuracy of state-of-the-art classifiers. When an adversary is allowed to modify an input image directly, models can be manipulated into predicting anything even when there is no perceptible change, this is known an adversarial example. The ideal definition of an adversarial example is when humans consistently say two pictures are the same but a machine disagrees. Hadi Salman, a Ph.D student at MIT (ex-Uber and Microsoft Research) started thinking about how adversarial robustness could be leveraged beyond security. Lees meer
He realised that the phenomenon of adversarial examples could actually be turned upside down to lead to more robust models instead of breaking them. Hadi actually utilized the brittleness of neural networks to design unadversarial examples or robust objects which_ are objects designed specifically to be robustly recognized by neural networks.
Introduction [00:00:00]
DR KILCHER'S PHD HAT [00:11:18]
Main Introduction [00:11:38]
Hadi's Introduction [00:14:43]
More robust models == transfer better [00:46:41]
Features not bugs paper [00:49:13]
Manifolds [00:55:51]
Robustness and Transferability [00:58:00]
Do non-robust features generalize worse than robust? [00:59:52]
The unreasonable predicament of entangled features [01:01:57]
We can only find adversarial examples in the vicinity [01:09:30]
Certifiability of models for robustness [01:13:55]
Carlini is coming for you! And we are screwed [01:23:21]
Distribution shift and corruptions are a bigger problem than adversarial examples [01:25:34]
All roads lead to generalization [01:26:47]
Unadversarial examples [01:27:26]
-
#51 Francois Chollet - Intelligence and Generalisation
16 april 2021 · 2 uur 2 min
In today's show we are joined by Francois Chollet, I have been inspired by Francois ever since I read his Deep Learning with Python book and started using the Keras library which he invented many, many years ago. Francois has a clarity of thought that I've never seen in any other human being! He has extremely interesting views on intelligence as generalisation, abstraction and an information conversation ratio. He wrote on the measure of intelligence at the end of 2019 and it had a huge impact on my thinking. He thinks that NNs can only model continuous problems, which have a smooth learnable manifold and that many "type 2" problems which involve reasoning and/or planning are not suitable for NNs. He thinks that many problems have type 1 and type 2 enmeshed together. He thinks that the future of AI must include program synthesis to allow us to generalise broadly from a few examples, but the search could be guided by neural networks because the search space is interpolative to some extent. Lees meer
https://youtu.be/J0p_thJJnoo
Tim's Whimsical notes; https://whimsical.com/chollet-show-QQ2atZUoRR3yFDsxKVzCbj
-
#50 Christian Szegedy - Formal Reasoning, Program Synthesis
4 april 2021 · 1 uur 33 min
Dr. Christian Szegedy from Google Research is a deep learning heavyweight. He invented adversarial examples, one of the first object detection algorithms, the inceptionnet architecture, and co-invented batchnorm. He thinks that if you bet on computers and software in 1990 you would have been as right as if you bet on AI now. But he thinks that we have been programming computers the same way since the 1950s and there has been a huge stagnation ever since. Mathematics is the process of taking a fuzzy thought and formalising it. But could we automate that? Could we create a system which will act like a super human mathematician but you can talk to it in natural language? This is what Christian calls autoformalisation. Christian thinks that automating many of the things we do in mathematics is the first step towards software synthesis and building human-level AGI. Mathematics ability is the litmus test for general reasoning ability. Christian has a fascinating take on transformers too. Lees meer
With Yannic Lightspeed Kilcher and Dr. Mathew Salvaris
Whimsical Canvas with Tim's Notes:
https://whimsical.com/mar-26th-christian-szegedy-CpgGhnEYDBrDMFoATU6XYC
YouTube version (with detailed table of contents) https://youtu.be/ehNGGYFO6ms
-
#49 - Meta-Gradients in RL - Dr. Tom Zahavy (DeepMind)
23 maart 2021 · 1 uur 25 min
The race is on, we are on a collective mission to understand and create artificial general intelligence. Dr. Tom Zahavy, a Research Scientist at DeepMind thinks that reinforcement learning is the most general learning framework that we have today, and in his opinion it could lead to artificial general intelligence. He thinks there are no tasks which could not be solved by simply maximising a reward. Lees meer
Back in 2012 when Tom was an undergraduate, before the deep learning revolution he attended an online lecture on how CNNs automatically discover representations. This was an epiphany for Tom. He decided in that very moment that he was going to become an ML researcher. Tom's view is that the ability to recognise patterns and discover structure is the most important aspect of intelligence. This has been his quest ever since. He is particularly focused on using diversity preservation and metagradients to discover this structure.
In this discussion we dive deep into meta gradients in reinforcement learning.
Video version and TOC @ https://www.youtube.com/watch?v=hfaZwgk_iS0
-
#48 Machine Learning Security - Andy Smith
16 maart 2021 · 37 min
First episode in a series we are doing on ML DevOps. Starting with the thing which nobody seems to be talking about enough, security! We chat with cyber security expert Andy Smith about threat modelling and trust boundaries for an ML DevOps system. Lees meer
Intro [00:00:00]
ML DevOps - a security perspective [00:00:50]
Threat Modelling [00:03:03]
Adversarial examples? [00:11:27]
Nobody understands the whole stack [00:13:53]
On the size of the state space, the element of unpredictability [00:18:32]
Threat modelling in more detail [00:21:17]
Trust boundaries for an ML DevOps system [00:25:45]
Andy has a YouTube channel on cyber security! Check it out @
https://www.youtube.com/channel/UCywP24ly6h6NTusX88TQKTQ
https://www.linkedin.com/in/andysmith-uk/
Video version:
https://youtu.be/7Tz-3S4lypI
-
047 Interpretable Machine Learning - Christoph Molnar
14 maart 2021 · 1 uur 40 min
Christoph Molnar is one of the main people to know in the space of interpretable ML. In 2018 he released the first version of his incredible online book, interpretable machine learning. Interpretability is often a deciding factor when a machine learning (ML) model is used in a product, a decision process, or in research. Interpretability methods can be used to discover knowledge, to debug or justify the model and its predictions, and to control and improve the model, reason about potential bias in models as well as increase the social acceptance of models. But Interpretability methods can also be quite esoteric, add an additional layer of complexity and potential pitfalls and requires expert knowledge to understand. Is it even possible to understand complex models or even humans for that matter in any meaningful way? Lees meer
Introduction to IML [00:00:00]
Show Kickoff [00:13:28]
What makes a good explanation? [00:15:51]
Quantification of how good an explanation is [00:19:59]
Knowledge of the pitfalls of IML [00:22:14]
Are linear models even interpretable? [00:24:26]
Complex Math models to explain Complex Math models? [00:27:04]
Saliency maps are glorified edge detectors [00:28:35]
Challenge on IML -- feature dependence [00:36:46]
Don't leap to using a complex model! Surrogate models can be too dumb [00:40:52]
On airplane pilots. Seeking to understand vs testing [00:44:09]
IML Could help us make better models or lead a better life [00:51:53]
Lack of statistical rigor and quantification of uncertainty [00:55:35]
On Causality [01:01:09]
Broadening out the discussion to the process or institutional level [01:08:53]
No focus on fairness / ethics? [01:11:44]
Is it possible to condition ML model training on IML metrics ? [01:15:27]
Where is IML going? Some of the esoterica of the IML methods [01:18:35]
You can't compress information without common knowledge, the latter becomes the bottleneck [01:23:25]
IML methods used non-interactively? Making IML an engineer…
-
#046 The Great ML Stagnation (Mark Saroufim and Dr. Mathew Salvaris)
6 maart 2021 · 1 uur 40 min
Academics think of themselves as trailblazers, explorers — seekers of the truth. Lees meer
Any fundamental discovery involves a significant degree of risk. If an idea is guaranteed to work then it moves from the realm of research to engineering. Unfortunately, this also means that most research careers will invariably be failures at least if failures are measured via “objective” metrics like citations. Today we discuss the recent article from Mark Saroufim called Machine Learning: the great stagnation. We discuss the rise of gentleman scientists, fake rigor, incentives in ML, SOTA-chasing, "graduate student descent", distribution of talent in ML and how to learn effectively.
With special guest interviewer Mat Salvaris.
Machine learning: the great stagnation [00:00:00]
Main show kick off [00:16:30]
Great stagnation article / Bad incentive systems in academia [00:18:24]
OpenAI is a media business [00:19:48]
Incentive structures in academia [00:22:13]
SOTA chasing [00:24:47]
F You Money [00:28:53]
Research grants and gentlemen scientists [00:29:13]
Following your own gradient of interest and making a contribution [00:33:27]
Marketing yourself to be successful [00:37:07]
Tech companies create the bad incentives [00:42:20]
GPT3 was sota chasing but it seemed really... "good"? Scaling laws? [00:51:09]
Dota / game AI [00:58:39]
Hard to go it alone? [01:02:08]
Reaching out to people [01:09:21]
Willingness to be wrong [01:13:14]
Distribution of talent / tech interviews [01:18:30]
What should you read online and how to learn? Sharing your stuff online and finding your niece [01:25:52]
Mark Saroufim:
https://marksaroufim.substack.com/
http://robotoverlordmanual.com/
https://twitter.com/marksaroufim
https://www.youtube.com/marksaroufim
Dr. Mathew Salvaris:
https://www.linkedin.com/in/drmathewsalvaris/
https://twitter.com/MSalvaris
-
#045 Microsoft's Platform for Reinforcement Learning (Bonsai)
28 februari 2021 · 2 uur 30 min
Microsoft has an interesting strategy with their new “autonomous systems” technology also known as Project Bonsai. They want to create an interface to abstract away the complexity and esoterica of deep reinforcement learning. They want to fuse together expert knowledge and artificial intelligence all on one platform, so that complex problems can be decomposed into simpler ones. They want to take machine learning Ph.Ds out of the equation and make autonomous systems engineering look more like a traditional software engineering process. It is an ambitious undertaking, but interesting. Reinforcement learning is extremely difficult (as I cover in the video), and if you don’t have a team of RL Ph.Ds with tech industry experience, you shouldn’t even consider doing it yourself. This is our take on it! Lees meer
There are 3 chapters in this video;
Chapter 1: Tim's intro and take on RL being hard, intro to Bonsai and machine teaching
Chapter 2: Interview with Scott Stanfield [recorded Jan 2020] 00:56:41
Chapter 3: Traditional street talk episode [recorded Dec 2020] 01:38:13
This is *not* an official communication from Microsoft, all personal opinions. There is no MS-confidential information in this video.
With:
Scott Stanfield
https://twitter.com/seesharp
Megan Bloemsma
https://twitter.com/BloemsmaMegan
Gurdeep Pall (he has not validated anything we have said in this video or been involved in the creation of it)
https://www.linkedin.com/in/gurdeep-pall-0aa639bb/
Panel:
Dr. Keith Duggar
Dr. Tim Scarfe
Yannic Kilcher
-
#044 - Data-efficient Image Transformers (Hugo Touvron)
25 februari 2021 · 52 min
Today we are going to talk about the *Data-efficient image Transformers paper or (DeiT) which Hugo is the primary author of. One of the recipes of success for vision models since the DL revolution began has been the availability of large training sets. CNNs have been optimized for almost a decade now, including through extensive architecture search which is prone to overfitting. Motivated by the success of transformers-based models Lees meer
in Natural Language Processing there has been increasing attention in applying these approaches to vision models. Hugo and his collaborators used a different training strategy and a new distillation token to get a massive increase in sample efficiency with image transformers.
00:00:00 Introduction
00:06:33 Data augmentation is all you need
00:09:53 Now the image patches are the convolutions though?
00:12:16 Where are those inductive biases hiding?
00:15:46 Distillation token
00:21:01 Why different resolutions on training
00:24:14 How data efficient can we get?
00:26:47 Out of domain generalisation
00:28:22 Why are transformers data efficient at all? Learning invariances
00:32:04 Is data augmentation cheating?
00:33:25 Distillation strategies - matching the intermediatae teacher representation as well as output
00:35:49 Do ML models learn the same thing for a problem?
00:39:01 How is it like at Facebook AI?
00:41:17 How long is the PhD programme?
00:42:03 Other interests outside of transformers?
00:43:18 Transformers for Vision and Language
00:47:40 Could we improve transformers models? (Hybrid models)
00:49:03 Biggest challenges in AI?
00:50:52 How far can we go with data driven approach?
-
#043 Prof J. Mark Bishop - Artificial Intelligence Is Stupid and Causal Reasoning won't fix it.
19 februari 2021 · 1 uur 35 min
Professor Mark Bishop does not think that computers can be conscious or have phenomenological states of consciousness unless we are willing to accept panpsychism which is idea that mentality is fundamental and ubiquitous in the natural world, or put simply, that your goldfish and everything else for that matter has a mind. Panpsychism postulates that distinctions between intelligences are largely arbitrary. Lees meer
Mark’s work in the ‘philosophy of AI’ led to an influential critique of computational approaches to Artificial Intelligence through a thorough examination of John Searle's 'Chinese Room Argument'
Mark just published a paper called artificial intelligence is stupid and causal reasoning wont fix it. He makes it clear in this paper that in his opinion computers will never be able to compute everything, understand anything, or feel anything.
00:00:00 Tim Intro
00:15:04 Intro
00:18:49 Introduction to Marks ideas
00:25:49 Some problems are not computable
00:29:57 the dancing was Pixies fallacy
00:32:36 The observer relative problem, and its all in the mapping
00:43:03 Conscious Experience
00:53:30 Intelligence without representation, consciousness is something that we do
01:02:36 Consciousness helps us to act autonomously
01:05:13 The Chinese room argument
01:14:58 Simulation argument and computation doesn't have phenomenal consciousness
01:17:44 Language informs our colour perception
01:23:11 We have our own distinct ontologies
01:27:12 Kurt Gödel, Turing and Penrose and the implications of their work
-
#042 - Pedro Domingos - Ethics and Cancel Culture
11 februari 2021 · 1 uur 34 min
Today we have professor Pedro Domingos and we are going to talk about activism in machine learning, cancel culture, AI ethics and kernels. In Pedro's book the master algorithm, he segmented the AI community into 5 distinct tribes with 5 unique identities (and before you ask, no the irony of an anti-identitarian doing do was not lost on us!). Pedro recently published an article in Quillette called Beating Back Cancel Culture: A Case Study from the Field of Artificial Intelligence. Domingos has railed against political activism in the machine learning community and cancel culture. Recently Pedro was involved in a controversy where he asserted the NeurIPS broader impact statements are an ideological filter mechanism. Lees meer
Important Disclaimer: All views expressed are personal opinions.
00:00:00 Caveating
00:04:08 Main intro
00:07:44 Cancelling culture is a culture and intellectual weakness
00:12:26 Is cancel culture a post-modern religion?
00:24:46 Should we have gateways and gatekeepers?
00:29:30 Does everything require broader impact statements?
00:33:55 We are stifling diversity (of thought) not promoting it.
00:39:09 What is fair and how to do fair?
00:45:11 Models can introduce biases by compressing away minority data
00:48:36 Accurate but unequal soap dispensers
00:53:55 Agendas are not even self-consistent
00:56:42 Is vs Ought: all variables should be used for Is
01:00:38 Fighting back cancellation with cancellation?
01:10:01 Intent and degree matter in right vs wrong.
01:11:08 Limiting principles matter
01:15:10 Gradient descent and kernels
01:20:16 Training Journey matter more than Destination
01:24:36 Can training paths teach us about symmetry?
01:28:37 What is the most promising path to AGI?
01:31:29 Intelligence will lose its mystery
-
#041 - Biologically Plausible Neural Networks - Dr. Simon Stringer
3 februari 2021 · 1 uur 27 min
Dr. Simon Stringer. Obtained his Ph.D in mathematical state space control theory and has been a Senior Research Fellow at Oxford University for over 27 years. Simon is the director of the the Oxford Centre for Theoretical Neuroscience and Artificial Intelligence, which is based within the Oxford University Department of Experimental Psychology. His department covers vision, spatial processing, motor function, language and consciousness -- in particular -- how the primate visual system learns to make sense of complex natural scenes. Dr. Stringers laboratory houses a team of theoreticians, who are developing computer models of a range of different aspects of brain function. Simon's lab is investigating the neural and synaptic dynamics that underpin brain function. An important matter here is the The feature-binding problem which concerns how the visual system represents the hierarchical relationships between features. the visual system must represent hierarchical binding relations across the entire visual field at every spatial scale and level in the hierarchy of visual primitives. Lees meer
We discuss the emergence of self-organised behaviour, complex information processing, invariant sensory representations and hierarchical feature binding which emerges when you build biologically plausible neural networks with temporal spiking dynamics.
00:00:09 Tim Intro
00:09:31 Show kickoff
00:14:37 Hierarchical Feature binding and timing of action potentials
00:30:16 Hebb to Spike-timing-dependent plasticity (STDP)
00:35:27 Encoding of shape primitives
00:38:50 Is imagination working in the same place in the brain
00:41:12 Compare to supervised CNNs
00:45:59 Speech recognition, motor system, learning mazes
00:49:28 How practical are these spiking NNs
00:50:19 Why simulate the human brain
00:52:46 How much computational power do you gain from differential timings
00:55:08 Adversarial inputs
00:59:41 Generative / causal component needed?
01:01:46 Modalities of processing i.…
-
#040 - Adversarial Examples (Dr. Nicholas Carlini, Dr. Wieland Brendel, Florian Tramèr)
31 januari 2021 · 1 uur 36 min
Adversarial examples have attracted significant attention in machine learning, but the reasons for their existence and pervasiveness remain unclear. there's good reason to believe neural networks look at very different features than we would have expected. As articulated in the 2019 "features not bugs" paper Adversarial examples can be directly attributed to the presence of non-robust features: features derived from patterns in the data distribution that are highly predictive, yet brittle and incomprehensible to humans. Lees meer
Adversarial examples don't just affect deep learning models. A cottage industry has sprung up around Threat Modeling in AI and ML Systems and their dependencies. Joining us this evening are some of currently leading researchers in adversarial examples;
Florian Tramèr - A fifth year PhD student in Computer Science at Stanford University
https://floriantramer.com/
https://twitter.com/florian_tramer
Dr. Wieland Brendel - Machine Learning Researcher at the University of Tübingen & Co-Founder of layer7.ai
https://medium.com/@wielandbr
https://twitter.com/wielandbr
Dr. Nicholas Carlini - Research scientist at Google Brain working in that exciting space between machine learning and computer security.
https://nicholas.carlini.com/
We really hope you enjoy the conversation, remember to subscribe!
Yannic Intro [00:00:00]
Tim Intro [00:04:07]
Threat Taxonomy [00:09:00]
Main show intro [00:11:30]
Whats wrong with Neural Networks? [00:14:52]
The role of memorization [00:19:51]
Anthropomorphization of models [00:22:42]
Whats the harm really though / focusing on actual ML security risks [00:27:03]
Shortcut learning / OOD generalization [00:36:18]
Human generalization [00:40:11]
An existential problem in DL getting the models to learn what we want? [00:41:39]
Defenses to adversarial examples [00:47:15]
What if we had all the data and the labels? Still problems? [00:54:28]
Defenses are easily broken [01:00:24]
Self decepti…
-
#039 - Lena Voita - NLP
24 januari 2021 · 1 uur 58 min
ena Voita is a Ph.D. student at the University of Edinburgh and University of Amsterdam. Previously, She was a research scientist at Yandex Research and worked closely with the Yandex Translate team. She still teaches NLP at the Yandex School of Data Analysis. She has created an exciting new NLP course on her website lena-voita.github.io which you folks need to check out! She has one of the most well presented blogs we have ever seen, where she discusses her research in an easily digestable manner. Lena has been investigating many fascinating topics in machine learning and NLP. Today we are going to talk about three of her papers and corresponding blog articles; Lees meer
Source and Target Contributions to NMT Predictions -- Where she talks about the influential dichotomy between the source and the prefix of neural translation models.
https://arxiv.org/pdf/2010.10907.pdf
https://lena-voita.github.io/posts/source_target_contributions_to_nmt.html
Information-Theoretic Probing with MDL -- Where Lena proposes a technique of evaluating a model using the minimum description length or Kolmogorov complexity of labels given representations rather than something basic like accuracy
https://arxiv.org/pdf/2003.12298.pdf
https://lena-voita.github.io/posts/mdl_probes.html
Evolution of Representations in the Transformer - Lena investigates the evolution of representations of individual tokens in Transformers -- trained with different training objectives (MT, LM, MLM)
https://arxiv.org/abs/1909.01380
https://lena-voita.github.io/posts/emnlp19_evolution.html
Panel Dr. Tim Scarfe, Yannic Kilcher, Sayak Paul
00:00:00 Kenneth Stanley / Greatness can not be planned house keeping
00:21:09 Kilcher intro
00:28:54 Hello Lena
00:29:21 Tim - Lenas NMT paper
00:35:26 Tim - Minimum Description Length / Probe paper
00:40:12 Tim - Evolution of representations
00:46:40 Lenas NLP course
00:49:18 The peppermint tea situation
00:49:28 Main Show Kick Off
00:50:22 Hallucination vs exposure b…
-
#038 - Professor Kenneth Stanley - Why Greatness Cannot Be Planned
20 januari 2021 · 2 uur 46 min
Professor Kenneth Stanley is currently a research science manager at OpenAI in San Fransisco. We've Been dreaming about getting Kenneth on the show since the very begininning of Machine Learning Street Talk. Some of you might recall that our first ever show was on the enhanced POET paper, of course Kenneth had his hands all over it. He's been cited over 16000 times, his most popular paper with over 3K citations was the NEAT algorithm. His interests are neuroevolution, open-endedness, NNs, artificial life, and AI. He invented the concept of novelty search with no clearly defined objective. His key idea is that there is a tyranny of objectives prevailing in every aspect of our lives, society and indeed our algorithms. Crucially, these objectives produce convergent behaviour and thinking and distract us from discovering stepping stones which will lead to greatness. He thinks that this monotonic objective obsession, this idea that we need to continue to improve benchmarks every year is dangerous. He wrote about this in detail in his recent book "greatness can not be planned" which will be the main topic of discussion in the show. We also cover his ideas on open endedness in machine learning. Lees meer
00:00:00 Intro to Kenneth
00:01:16 Show structure disclaimer
00:04:16 Passionate discussion
00:06:26 WHy greatness cant be planned and the tyranny of objectives
00:14:40 Chinese Finger Trap
00:16:28 Perverse Incentives and feedback loops
00:18:17 Deception
00:23:29 Maze example
00:24:44 How can we define curiosity or interestingness
00:26:59 Open endedness
00:33:01 ICML 2019 and Yannic, POET, first MSLST
00:36:17 evolutionary algorithms++
00:43:18 POET, the first MLST
00:45:39 A lesson to GOFAI people
00:48:46 Machine Learning -- the great stagnation
00:54:34 Actual scientific successes are usually luck, and against the odds -- Biontech
00:56:21 Picbreeder and NEAT
01:10:47 How Tim applies these ideas to his life and why he runs MLST
01:14:58 Keith Skit about UCF…
-
#037 - Tour De Bayesian with Connor Tann
11 januari 2021 · 1 uur 35 min
Connor Tan is a physicist and senior data scientist working for a multinational energy company where he co-founded and leads a data science team. He holds a first-class degree in experimental and theoretical physics from Cambridge university. With a master's in particle astrophysics. He specializes in the application of machine learning models and Bayesian methods. Today we explore the history, pratical utility, and unique capabilities of Bayesian methods. We also discuss the computational difficulties inherent in Bayesian methods along with modern methods for approximate solutions such as Markov Chain Monte Carlo. Finally, we discuss how Bayesian optimization in the context of automl may one day put Data Scientists like Connor out of work. Lees meer
Panel: Dr. Keith Duggar, Alex Stenlake, Dr. Tim Scarfe
00:00:00 Duggars philisophical ramblings on Bayesianism
00:05:10 Introduction
00:07:30 small datasets and prior scientific knowledge
00:10:37 Bayesian methods are probability theory
00:14:00 Bayesian methods demand hard computations
00:15:46 uncertainty can matter more than estimators
00:19:29 updating or combining knowledge is a key feature
00:25:39 Frequency or Reasonable Expectation as the Primary Concept
00:30:02 Gambling and coin flips
00:37:32 Rev. Thomas Bayes's pool table
00:40:37 ignorance priors are beautiful yet hard
00:43:49 connections between common distributions
00:49:13 A curious Universe, Benford's Law
00:55:17 choosing priors, a tale of two factories
01:02:19 integration, the computational Achilles heel
01:35:25 Bayesian social context in the ML community
01:10:24 frequentist methods as a first approximation
01:13:13 driven to Bayesian methods by small sample size
01:18:46 Bayesian optimization with automl, a job killer?
01:25:28 different approaches to hyper-parameter optimization
01:30:18 advice for aspiring Bayesians
01:33:59 who would connor interview next?
Connor Tann: https://www.linkedin.com/in/connor-tann-a92906a1/
https://t…
-
#036 - Max Welling: Quantum, Manifolds & Symmetries in ML
3 januari 2021 · 1 uur 43 min
Today we had a fantastic conversation with Professor Max Welling, VP of Technology, Qualcomm Technologies Netherlands B.V. Lees meer
Max is a strong believer in the power of data and computation and its relevance to artificial intelligence. There is a fundamental blank slate paradgm in machine learning, experience and data alone currently rule the roost. Max wants to build a house of domain knowledge on top of that blank slate. Max thinks there are no predictions without assumptions, no generalization without inductive bias. The bias-variance tradeoff tells us that we need to use additional human knowledge when data is insufficient.
Max Welling has pioneered many of the most sophistocated inductive priors in DL models developed in recent years, allowing us to use Deep Learning with non-euclidean data i.e. on graphs/topology (a field we now called "geometric deep learning") or allowing network architectures to recognise new symmetries in the data for example gauge or SE(3) equivariance. Max has also brought many other concepts from his physics playbook into ML, for example quantum and even Bayesian approaches.
This is not an episode to miss, it might be our best yet!
Panel: Dr. Tim Scarfe, Yannic Kilcher, Alex Stenlake
00:00:00 Show introduction
00:04:37 Protein Fold from DeepMind -- did it use SE(3) transformer?
00:09:58 How has machine learning progressed
00:19:57 Quantum Deformed Neural Networks paper
00:22:54 Probabilistic Numeric Convolutional Neural Networks paper
00:27:04 Ilia Karmanov from Qualcomm interview mini segment
00:32:04 Main Show Intro
00:35:21 How is Max known in the community?
00:36:35 How Max nurtures talent, freedom and relationship is key
00:40:30 Selecting research directions and guidance
00:43:42 Priors vs experience (bias/variance trade-off)
00:48:47 Generative models and GPT-3
00:51:57 Bias/variance trade off -- when do priors hurt us
00:54:48 Capsule networks
01:03:09 Which old ideas whould we revive
01:04:36 Hardware lottery pa…
-
#035 Christmas Community Edition!
27 december 2020 · 2 uur 56 min
Welcome to the Christmas special community edition of MLST! We discuss some recent and interesting papers from Pedro Domingos (are NNs kernel machines?), Deepmind (can NNs out-reason symbolic machines?), Anna Rodgers - When BERT Plays The Lottery, All Tickets Are Winning, Prof. Mark Bishop (even causal methods won't deliver understanding), We also cover our favourite bits from the recent Montreal AI event run by Prof. Gary Marcus (including Rich Sutton, Danny Kahneman and Christof Koch). We respond to a reader mail on Capsule networks. Then we do a deep dive into Type Theory and Lambda Calculus with community member Alex Mattick. In the final hour we discuss inductive priors and label information density with another one of our discord community members. Lees meer
Panel: Dr. Tim Scarfe, Yannic Kilcher, Alex Stenlake, Dr. Keith Duggar
Enjoy the show and don't forget to subscribe!
00:00:00 Welcome to Christmas Special!
00:00:44 SoTa meme
00:01:30 Happy Christmas!
00:03:11 Paper -- DeepMind - Outperforming neuro-symbolic models with NNs (Ding et al)
00:08:57 What does it mean to understand?
00:17:37 Paper - Prof. Mark Bishop Artificial Intelligence is stupid and causal reasoning
wont fix it
00:25:39 Paper -- Pedro Domingos - Every Model Learned by Gradient Descent Is Approximately a Kernel Machine
00:31:07 Paper - Bengio - Inductive Biases for Deep Learning of Higher-Level Cognition
00:32:54 Anna Rodgers - When BERT Plays The Lottery, All Tickets Are Winning
00:37:16 Montreal AI event - Gary Marcus on reasoning
00:40:37 Montreal AI event -- Rich Sutton on universal theory of AI
00:49:45 Montreal AI event -- Danny Kahneman, System 1 vs 2 and Generative Models ala free energy principle
01:02:57 Montreal AI event -- Christof Koch - Neuroscience is hard
01:10:55 Markus Carr -- reader letter on capsule networks
01:13:21 Alex response to Marcus Carr
01:22:06 Type theory segment -- with Alex Mattick from Discord
01:24:45 Type theory segment -- What is Type Theory
…
-
#034 Eray Özkural- AGI, Simulations & Safety
20 december 2020 · 2 uur 39 min
Dr. Eray Ozkural is an AGI researcher from Turkey, he is the founder of Celestial Intellect Cybernetics. Eray is extremely critical of Max Tegmark, Nick Bostrom and MIRI founder Elizier Yodokovsky and their views on AI safety. Eray thinks that these views represent a form of neoludditism and they are capturing valuable research budgets with doomsday fear-mongering and effectively want to prevent AI from being developed by those they don't agree with. Eray is also sceptical of the intelligence explosion hypothesis and the argument from simulation. Lees meer
Panel -- Dr. Keith Duggar, Dr. Tim Scarfe, Yannic Kilcher
00:00:00 Show teaser intro with added nuggets and commentary
00:48:39 Main Show Introduction
00:53:14 Doomsaying to Control
00:56:39 Fear the Basilisk!
01:08:00 Intelligence Explosion Ethics
01:09:45 Fear the Automous Drone! ... or spam
01:11:25 Infinity Point Hypothesis
01:15:26 Meat Level Intelligence
01:21:25 Defining Intelligence ... Yet Again
01:27:34 We'll make brains and then shoot them
01:31:00 The Universe likes deep learning
01:33:16 NNs are glorified hash tables
01:38:44 Radical behaviorists
01:41:29 Omega Architecture, possible AGI?
01:53:33 Simulation hypothesis
02:09:44 No one cometh unto Simulation, but by Jesus Christ
02:16:47 Agendas, Motivations, and Mind Projections
02:23:38 A computable Universe of Bulk Automata
02:30:31 Self-Organized Post-Show Coda
02:31:29 Investigating Intelligent Agency is Science
02:36:56 Goodbye and cheers!
https://www.youtube.com/watch?v=pZsHZDA9TJU
-
#033 Prof. Karl Friston - The Free Energy Principle
13 december 2020 · 1 uur 51 min
This week Dr. Tim Scarfe, Dr. Keith Duggar and Connor Leahy chat with Prof. Karl Friston. Professor Friston is a British neuroscientist at University College London and an authority on brain imaging. In 2016 he was ranked the most influential neuroscientist on Semantic Scholar. His main contribution to theoretical neurobiology is the variational Free energy principle, also known as active inference in the Bayesian brain. The FEP is a formal statement that the existential imperative for any system which survives in the changing world can be cast as an inference problem. Bayesian Brain Hypothesis states that the brain is confronted with ambiguous sensory evidence, which it interprets by making inferences about the hidden states which caused the sensory data. So is the brain an inference engine? The key concept separating Friston's idea from traditional stochastic reinforcement learning methods and even Bayesian reinforcement learning is moving away from goal-directed optimisation. Lees meer
Remember to subscribe! Enjoy the show!
00:00:00 Show teaser intro
00:16:24 Main formalism for FEP
00:28:29 Path Integral
00:30:52 How did we feel talking to friston?
00:34:06 Skit - on cultures (checked, but maybe make shorter)
00:36:02 Friston joins
00:36:33 Main show introduction
00:40:51 Is prediction all it takes for intelligence?
00:48:21 balancing accuracy with flexibility
00:57:36 belief-free vs belief-based; beliefs are crucial
01:04:53 Fuzzy Markov Blankets and Wandering Sets
01:12:37 The Free Energy Principle conforms to itself
01:14:50 useful false beliefs
01:19:14 complexity minimization is the heart of free energy [01:19:14 ]Keith:
01:23:25 An Alpha to tip the scales? Absoute not! Absolutely yes!
01:28:47 FEP applied to brain anatomy
01:36:28 Are there multiple non-FEP forms in the brain?
01:43:11 a positive conneciton to backpropagation
01:47:12 The FEP does not explain the origin of FEP systems
01:49:32 Post-show banter
https://www.fil.ion.ucl.ac.uk/~kar…
-
#032- Simon Kornblith / GoogleAI - SimCLR and Paper Haul!
6 december 2020 · 1 uur 30 min
This week Dr. Tim Scarfe, Sayak Paul and Yannic Kilcher speak with Dr. Simon Kornblith from Google Brain (Ph.D from MIT). Simon is trying to understand how neural nets do what they do. Simon was the second author on the seminal Google AI SimCLR paper. We also cover "Do Wide and Deep Networks learn the same things?", "Whats in a Loss function for Image Classification?", and "Big Self-supervised models are strong semi-supervised learners". Simon used to be a neuroscientist and also gives us the story of his unique journey into ML. Lees meer
00:00:00 Show Teaser / or "short version"
00:18:34 Show intro
00:22:11 Relationship between neuroscience and machine learning
00:29:28 Similarity analysis and evolution of representations in Neural Networks
00:39:55 Expressability of NNs
00:42:33 Whats in a loss function for image classification
00:46:52 Loss function implications for transfer learning
00:50:44 SimCLR paper
01:00:19 Contrast SimCLR to BYOL
01:01:43 Data augmentation
01:06:35 Universality of image representations
01:09:25 Universality of augmentations
01:23:04 GPT-3
01:25:09 GANs for data augmentation??
01:26:50 Julia language
@skornblith
https://www.linkedin.com/in/simon-kornblith-54b2033a/
https://arxiv.org/abs/2010.15327
Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth
https://arxiv.org/abs/2010.16402
What's in a Loss Function for Image Classification?
https://arxiv.org/abs/2002.05709
A Simple Framework for Contrastive Learning of Visual Representations
https://arxiv.org/abs/2006.10029
Big Self-Supervised Models are Strong Semi-Supervised Learners
-
#031 WE GOT ACCESS TO GPT-3! (With Gary Marcus, Walid Saba and Connor Leahy)
28 november 2020 · 2 uur 44 min
In this special edition, Dr. Tim Scarfe, Yannic Kilcher and Keith Duggar speak with Gary Marcus and Connor Leahy about GPT-3. We have all had a significant amount of time to experiment with GPT-3 and show you demos of it in use and the considerations. Lees meer
Note that this podcast version is significantly truncated, watch the youtube version for the TOC and experiments with GPT-3 https://www.youtube.com/watch?v=iccd86vOz3w
-
#030 Multi-Armed Bandits and Pure-Exploration (Wouter M. Koolen)
20 november 2020 · 1 uur 48 min
This week Dr. Tim Scarfe, Dr. Keith Duggar and Yannic Kilcher discuss multi-arm bandits and pure exploration with Dr. Wouter M. Koolen, Senior Researcher, Machine Learning group, Centrum Wiskunde & Informatica. Lees meer
Wouter specialises in machine learning theory, game theory, information theory, statistics and optimisation. Wouter is currently interested in pure exploration in multi-armed bandit models, game tree search, and accelerated learning in sequential decision problems. His research has been cited 1000 times, and he has been published in NeurIPS, the number 1 ML conference 14 times as well as lots of other exciting publications.
Today we are going to talk about two of the most studied settings in control, decision theory, and learning in unknown environment which are the multi-armed bandit (MAB) and reinforcement learning (RL) approaches
- when can an agent stop learning and start exploiting using the knowledge it obtained
- which strategy leads to minimal learning time
00:00:00 What are multi-arm bandits/show trailer
00:12:55 Show introduction
00:15:50 Bandits
00:18:58 Taxonomy of decision framework approaches
00:25:46 Exploration vs Exploitation
00:31:43 the sharp divide between modes
00:34:12 bandit measures of success
00:36:44 connections to reinforcement learning
00:44:00 when to apply pure exploration in games
00:45:54 bandit lower bounds, a pure exploration renaissance
00:50:21 pure exploration compiler dreams
00:51:56 what would the PX-compiler DSL look like
00:57:13 the long arms of the bandit
01:00:21 causal models behind the curtain of arms
01:02:43 adversarial bandits, arms trying to beat you
01:05:12 bandits as an optimization problem
01:11:39 asymptotic optimality vs practical performance
01:15:38 pitfalls hiding under asymptotic cover
01:18:50 adding features to bandits
01:27:24 moderate confidence regimes
01:30:33 algorithms choice is highly sensitive to bounds
01:46:09 Post script: Keith interesting piece on n quantum
h…
-
#029 GPT-3, Prompt Engineering, Trading, AI Alignment, Intelligence
8 november 2020 · 1 uur 51 min
This week Dr. Tim Scarfe, Dr. Keith Duggar, Yannic Kilcher and Connor Leahy cover a broad range of topics, ranging from academia, GPT-3 and whether prompt engineering could be the next in-demand skill, markets and economics including trading and whether you can predict the stock market, AI alignment, utilitarian philosophy, randomness and intelligence and even whether the universe is infinite! Lees meer
00:00:00 Show Introduction
00:12:49 Academia and doing a Ph.D
00:15:49 From academia to wall street
00:17:08 Quants -- smoke and mirrors? Tail Risk
00:19:46 Previous results dont indicate future success in markets
00:23:23 Making money from social media signals?
00:24:41 Predicting the stock market
00:27:20 Things which are and are not predictable
00:31:40 Tim postscript comment on predicting markets
00:32:37 Connor take on markets
00:35:16 As market become more efficient..
00:36:38 Snake oil in ML
00:39:20 GPT-3, we have changed our minds
00:52:34 Prompt engineering a new form of software development?
01:06:07 GPT-3 and prompt engineering
01:12:33 Emergent intelligence with increasingly weird abstractions
01:27:29 Wireheading and the economy
01:28:54 Free markets, dragon story and price vs value
01:33:59 Utilitarian philosophy and what does good look like?
01:41:39 Randomness and intelligence
01:44:55 Different schools of thought in ML
01:46:09 Is the universe infinite?
Thanks a lot for Connor Leahy for being a guest on today's show. https://twitter.com/NPCollapse -- you can join his EleutherAI community discord here: https://discord.com/invite/vtRgjbM
-
NLP is not NLU and GPT-3 - Walid Saba
4 november 2020 · 2 uur 21 min
#machinelearning Lees meer
This week Dr. Tim Scarfe, Dr. Keith Duggar and Yannic Kilcher speak with veteran NLU expert Dr. Walid Saba.
Walid is an old-school AI expert. He is a polymath, a neuroscientist, psychologist, linguist, philosopher, statistician, and logician. He thinks the missing information problem and lack of a typed ontology is the key issue with NLU, not sample efficiency or generalisation. He is a big critic of the deep learning movement and BERTology. We also cover GPT-3 in some detail in today's session, covering Luciano Floridi's recent article "GPT‑3: Its Nature, Scope, Limits, and Consequences" and a commentary on the incredible power of GPT-3 to perform tasks with just a few examples including the Yann LeCun commentary on Facebook and Hackernews.
Time stamps on the YouTube version
0:00:00 Walid intro
00:05:03 Knowledge acquisition bottleneck
00:06:11 Language is ambiguous
00:07:41 Language is not learned
00:08:32 Language is a formal language
00:08:55 Learning from data doesn’t work
00:14:01 Intelligence
00:15:07 Lack of domain knowledge these days
00:16:37 Yannic Kilcher thuglife comment
00:17:57 Deep learning assault
00:20:07 The way we evaluate language models is flawed
00:20:47 Humans do type checking
00:23:02 Ontologic
00:25:48 Comments On GPT3
00:30:54 Yann lecun and reddit
00:33:57 Minds and machines - Luciano
00:35:55 Main show introduction
00:39:02 Walid introduces himself
00:40:20 science advances one funeral at a time
00:44:58 Deep learning obsession syndrome and inception
00:46:14 BERTology / empirical methods are not NLU
00:49:55 Pattern recognition vs domain reasoning, is the knowledge in the data
00:56:04 Natural language understanding is about decoding and not compression, it's not learnable.
01:01:46 Intelligence is about not needing infinite amounts of time
01:04:23 We need an explicit ontological structure to understand anything
01:06:40 Ontological concepts
01:09:38 Word embeddings
01:12:20 There is pow…
-
AI Alignment & AGI Fire Alarm - Connor Leahy
1 november 2020 · 2 uur 5 min
This week Dr. Tim Scarfe, Alex Stenlake and Yannic Kilcher speak with AGI and AI alignment specialist Connor Leahy a machine learning engineer from Aleph Alpha and founder of EleutherAI. Lees meer
Connor believes that AI alignment is philosophy with a deadline and that we are on the precipice, the stakes are astronomical. AI is important, and it will go wrong by default. Connor thinks that the singularity or intelligence explosion is near. Connor says that AGI is like climate change but worse, even harder problems, even shorter deadline and even worse consequences for the future. These problems are hard, and nobody knows what to do about them.
00:00:00 Introduction to AI alignment and AGI fire alarm
00:15:16 Main Show Intro
00:18:38 Different schools of thought on AI safety
00:24:03 What is intelligence?
00:25:48 AI Alignment
00:27:39 Humans dont have a coherent utility function
00:28:13 Newcomb's paradox and advanced decision problems
00:34:01 Incentives and behavioural economics
00:37:19 Prisoner's dilemma
00:40:24 Ayn Rand and game theory in politics and business
00:44:04 Instrumental convergence and orthogonality thesis
00:46:14 Utility functions and the Stop button problem
00:55:24 AI corrigibality - self alignment
00:56:16 Decision theory and stability / wireheading / robust delegation
00:59:30 Stop button problem
01:00:40 Making the world a better place
01:03:43 Is intelligence a search problem?
01:04:39 Mesa optimisation / humans are misaligned AI
01:06:04 Inner vs outer alignment / faulty reward functions
01:07:31 Large corporations are intelligent and have no stop function
01:10:21 Dutch booking / what is rationality / decision theory
01:16:32 Understanding very powerful AIs
01:18:03 Kolmogorov complexity
01:19:52 GPT-3 - is it intelligent, are humans even intelligent?
01:28:40 Scaling hypothesis
01:29:30 Connor thought DL was dead in 2017
01:37:54 Why is GPT-3 as intelligent as a human
01:44:43 Jeff Hawkins on intelligence as compressi…
-
Kaggle, ML Community / Engineering (Sanyam Bhutani)
28 oktober 2020 · 1 uur 27 min
Join Dr Tim Scarfe, Sayak Paul, Yannic Kilcher, and Alex Stenlake have a conversation with Mr. Chai Time Data Science; Sanyam Bhutani! Lees meer
00:00:00 Introduction
00:03:42 Show kick off
00:06:34 How did Sanyam get started into ML
00:07:46 Being a content creator
00:09:01 Can you be self taught without a formal education in ML?
00:22:54 Kaggle
00:33:41 H20 product / job
00:40:58 Intepretability / bias / engineering skills
00:43:22 Get that first job in DS
00:46:29 AWS ML Ops architecture / ml engineering
01:14:19 Patterns
01:18:09 Testability
01:20:54 Adversarial examples
Sanyam's blog -- https://sanyambhutani.com/tag/chaitimedatascience/
Chai Time Data Science -- https://www.youtube.com/c/ChaiTimeDataScience
-
Sara Hooker - The Hardware Lottery, Sparsity and Fairness
21 oktober 2020 · 1 uur 31 min
Dr. Tim Scarfe, Yannic Kilcher and Sayak Paul chat with Sara Hooker from the Google Brain team! We discuss her recent hardware lottery paper, pruning / sparsity, bias mitigation and intepretability. Lees meer
The hardware lottery -- what causes inertia or friction in the marketplace of ideas? Is there a meritocracy of ideas or do the previous decisions we have made enslave us? Sara Hooker calls this a lottery because she feels that machine learning progress is entirely beholdant to the hardware and software landscape. Ideas succeed if they are compatible with the hardware and software at the time and also the existing inventions. The machine learning community is exceptional because the pace of innovation is fast and we operate largely in the open, this is largely because we don't build anything physical which is expensive, slow and the cost of being scooped is high. We get stuck in basins of attraction based on our technology decisions and it's expensive to jump outside of these basins. So is this story unique to hardware and AI algorithms or is it really just the story of all innovation? Every great innovation must wait for the right stepping stone to be in place before it can really happen. We are excited to bring you Sara Hooker to give her take.
YouTube version (including TOC): https://youtu.be/sQFxbQ7ade0
Show notes; https://drive.google.com/file/d/1S_rHnhaoVX4Nzx_8e3ESQq4uSswASNo7/view?usp=sharing
Sara Hooker page; https://www.sarahooker.me
-
The Social Dilemma Part 3 - Dr. Rebecca Roache
12 oktober 2020 · 1 uur 16 min
This week join Dr. Tim Scarfe, Yannic Kilcher, and Keith Duggar have a conversation with Dr. Rebecca Roache in the last of our 3-part series on the social dilemma Netflix film. Rebecca is a senior lecturer in philosophy at Royal Holloway, university of London and has written extensively about the future of friendship. Lees meer
People claim that friendships are not what they used to be. People are always staring at their phones, even when in public Social media has turned us into narcissists who are always managing our own PR rather than being present with each other. Anxiety about the negative effects of technology are as old as the written word. Is technology bad for friendships? Can you have friends through screens? Does social media cause polarization? And is that a bad thing? Does it promote quantity over quality? Rebecca thinks that social media and echo chambers are less ominous to friendship on closer inspection.
00:00:32 Teaser clip from Rebecca and her new manuscript on friendship
00:02:52 Introduction
00:04:56 Memorisation vs reasoning / is technology enhancing friendships
00:09:29 Word of warcraft / gaming communities / echo chambers / polarisation
00:12:34 Horizontal vs Vertical social attributes
00:17:18 Exclusion of others opinions
00:20:36 The power to silence others / truth verification
00:23:58 Misinformation
00:27:28 Norms / memes / political terms and co-opting / bullying
00:31:57 Redefinition of political terms i.e. racism
00:36:13 Virtue signalling
00:38:57 How many friends can you have / spread thin / Dunbars 150
00:42:54 Is it morally objectionable to believe or contemplate objectionable ideas, punishment
00:50:52 Is speaking the same thing as acting
00:52:24 Punishment - deterrence vs retribution / historical
00:53:59 Yannic: contemplating is a form of speaking
00:57:32 silencing/blocking is intellectual laziness - what ideas are we allowed to talk about
01:04:53 Corporate AI ethics frameworks
01:09:14 Autonomous Vehicles
01:10:5…
-
The Social Dilemma - Part 1
3 oktober 2020 · 1 uur 7 min
In this first part of our three part series on the Social Dilemma Netflix film, Dr. Tim Scarfe, Yannic "Lightspeed" Kilcher and Zak Jost gang up with Cybersecurity expert Andy Smith. We give you our take on the film. We are super excited to get your feedback on this one! Hope you enjoy. Lees meer
00:00:00 Introduction
00:06:11 Moral hypocrisy
00:12:38 Road to hell is paved with good intentions, attention economy
00:15:04 They know everything about you
00:18:02 Addiction
00:21:22 Differential realities
00:26:12 Self determination and Monetisation
00:29:08 AI: Overwhelm human strengths undermine human vulnerabilities
00:31:51 Conspiracy theory / fake news
00:34:23 Overton window / polarisation
00:39:12 Short attention span / convergent behaviour
00:41:26 Is social media good for you
00:45:17 Your attention time is linear, the things you can pay attention to are a volume, anonymity
00:51:32 Andy question on security: social engineering
00:56:32 Is it a security risk having your information in social media
00:58:02 Retrospective judgement
01:03:06 Free speech and censorship
01:06:06 Technology accelerator