Since the release of ChatGPT late last year, the public has become enamored with AI. I have too (though my interest predates this). In fact, I believe we are at the precipice of an AI-fueled technology explosion unlike anything humanity has experienced. This is exciting news and I have a sister post to this one that attempts to consolidate information on AI tools and educational resources. That’s the good news about AI. Unfortunately, as with anything so powerful, there is a downside to AI which is real and should not be dismissed.
AI alignment involves shaping AIs to have goals and motivations aligned with those of humanity. This way when they become superintelligent (much much MUCH more intelligent than the smartest human), they don’t decide there are better ways to use our atoms and extinguish the human race. This seems like science fiction but is one of many scenarios where AI can lead to our extinction.
Below is a collection of materials that I think are worth consuming if this interests you. If you’re skeptical, start small with one of the podcasts, and if you become more convinced (as I have), you can dig deeper.
Some resources on AI Alignment
The following are a few podcasts with Eliezer Yudkowsky. Eliezer is a long-time AI researcher and the creator the excellent rationalist site lesswrong. He’s been talking about AI alignment for longer than most, and now feels that AGI (Artificial General Intelligence). AGI is essentially AIs that are at least as smart as humans, which is a very short way from superintelligence since AGIs will be able to improve themselves at the speed of computers. He is the most pessimistic of the AI alignment thought leaders. Most give us better odds (though very few 0% chance of extinction).
We’re all Gonna Die with Eliezer Yudkowsky – A shortish interview with him from the Bankless podcast (which typically focusses on cryptocurrency).
Eliezer Yudkowsky: Dangers of AI and the End of Human Civilization – A longer and more nuanced interview with Lex Fridman. Lex, being an AI researcher himself, pushes back on some of Yudkowsky’s claims.
Eliezer paints a grim picture of the future, giving humanity about a 2% chance of surviving an AI. Many of his colleagues are less pessimistic. Here are a few interviews with alignment researchers that give us better odds.
Robin Hanson believes it us unlikely that we will have an intelligence explosion. He’s debated Eliezer before. Lesswrong has aggregated information on The Hanson-Yudkowsky AI-Foom Debate.
FOOM is the event where AI undergoes explosive, recursive self-improvement. See here.
Another researcher that it more optimistic (though he still puts the odds of an AI takeover at ~ 20%) is Paul Christiano.
How to Solve AI Alignment with Paul Christiano. This is an interview he gave with Bankless (they become a bit obsessed with the topic after the Eliezer interview scared the dickens out of them).
Max Tegmark, the famous physicist and AI researcher at MIT is calling for a halt to AI research and was interviewed by Lex Friedman.
Max Tegmark: The Case for Halting AI Development. Max at times gets quite emotional during the interview. He genuinely believes we are on a bad trajectory.
Some good books on superintelligence and AI are listed below:
Superintelligence, Paths Dangers, Strategies by Nick Bostron. This book came out way back in 2014. This is a prescient book and one that is quite thorough in its arguments (Bostron is a philosopher and professor at Oxford).
The Alignment Problem, Machine Learning and Human Values by Brian Christian. A very readable modern overview of the problem that includes a nice history of AI progress.
Scary Smart: The Future of Artificial Intelligence and How You Can Save Our World As the former Chief Business Officer for Google X he has a unique perspective on the problem. He was there to witness ML technology leaps in LLM as well as robotics. He believes we should treat AIs like they’re are children. Raise them well and they’ll take care of us in our old age.