Difference between revisions of "AI predictions"
KevinYager (talk | contribs) (→Capability Scaling) |
KevinYager (talk | contribs) (→Economic and Political) |
||
| (5 intermediate revisions by the same user not shown) | |||
| Line 33: | Line 33: | ||
* 2025-10: Dan Hendrycks et al.: [https://www.agidefinition.ai/paper.pdf A Definition of AGI] | * 2025-10: Dan Hendrycks et al.: [https://www.agidefinition.ai/paper.pdf A Definition of AGI] | ||
* 2026-01: [https://arxiv.org/abs/2601.07364 On the universal definition of intelligence] | * 2026-01: [https://arxiv.org/abs/2601.07364 On the universal definition of intelligence] | ||
| + | |||
| + | ==Recursive Self Improvement (RSI)== | ||
| + | * 2026-02: [https://80000hours.org/articles/how-ai-driven-feedback-loops-could-make-things-very-crazy-very-fast/ How AI-driven feedback loops could make things very crazy, very fast] | ||
==Progress Models== | ==Progress Models== | ||
| Line 91: | Line 94: | ||
* 2026-01: [https://www.anthropic.com/research/economic-index-primitives Anthropic Economic Index report: economic primitives] | * 2026-01: [https://www.anthropic.com/research/economic-index-primitives Anthropic Economic Index report: economic primitives] | ||
* 2026-02: Nate Silver: [https://www.natesilver.net/p/the-singularity-wont-be-gentle The singularity won't be gentle: If AI is even half as transformational as Silicon Valley assumes, politics will never be the same again] | * 2026-02: Nate Silver: [https://www.natesilver.net/p/the-singularity-wont-be-gentle The singularity won't be gentle: If AI is even half as transformational as Silicon Valley assumes, politics will never be the same again] | ||
| + | * 2026-03: [https://www.anthropic.com/research/economic-index-march-2026-report Anthropic Economic Index report: Learning curves] | ||
==Job Loss== | ==Job Loss== | ||
| Line 122: | Line 126: | ||
* 2026-01: [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6134506 AI, Automation, and Expertise] | * 2026-01: [https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6134506 AI, Automation, and Expertise] | ||
* 2026-02: [https://arachnemag.substack.com/p/the-jevons-paradox-for-intelligence The Jevons Paradox for Intelligence: Fears of AI-induced job loss could not be more wrong] | * 2026-02: [https://arachnemag.substack.com/p/the-jevons-paradox-for-intelligence The Jevons Paradox for Intelligence: Fears of AI-induced job loss could not be more wrong] | ||
| + | * 2026-03: [https://www.dropbox.com/scl/fo/689u1g785x8jp6c8v1s21/AKxZ_N15vUxMA3PBtpbr5nM?dl=0&e=1&preview=2026.03.24+Bundles.pdf&rlkey=ottgcu71u1t4mhn6tblvatu8w&st=dj6k0x2o Weak Bundle, Strong Bundle:How AI Redraws Job Boundaries] | ||
==Productivity Impact== | ==Productivity Impact== | ||
| Line 281: | Line 286: | ||
==Alignment== | ==Alignment== | ||
* 2023-03: Leopold Aschenbrenner: [https://www.forourposterity.com/nobodys-on-the-ball-on-agi-alignment/ Nobody’s on the ball on AGI alignment] | * 2023-03: Leopold Aschenbrenner: [https://www.forourposterity.com/nobodys-on-the-ball-on-agi-alignment/ Nobody’s on the ball on AGI alignment] | ||
| − | * 2024-03: [https:// | + | * 2024-03: [https://arxiv.org/abs/2404.10636 What are human values, and how do we align AI to them?] ([https://meaningalignment.substack.com/p/0480e023-98c0-4633-a604-990d3ac880ac blog]) |
* 2025: Joe Carlsmith: [https://joecarlsmith.substack.com/p/how-do-we-solve-the-alignment-problem How do we solve the alignment problem?] Introduction to an essay series on paths to safe, useful superintelligence | * 2025: Joe Carlsmith: [https://joecarlsmith.substack.com/p/how-do-we-solve-the-alignment-problem How do we solve the alignment problem?] Introduction to an essay series on paths to safe, useful superintelligence | ||
*# [https://joecarlsmith.substack.com/p/what-is-it-to-solve-the-alignment What is it to solve the alignment problem?] Also: to avoid it? Handle it? Solve it forever? Solve it completely? ([https://joecarlsmithaudio.buzzsprout.com/2034731/episodes/16617671-what-is-it-to-solve-the-alignment-problem audio version]) | *# [https://joecarlsmith.substack.com/p/what-is-it-to-solve-the-alignment What is it to solve the alignment problem?] Also: to avoid it? Handle it? Solve it forever? Solve it completely? ([https://joecarlsmithaudio.buzzsprout.com/2034731/episodes/16617671-what-is-it-to-solve-the-alignment-problem audio version]) | ||
| Line 292: | Line 297: | ||
*# [https://joecarlsmith.substack.com/p/how-human-like-do-safe-ai-motivations?utm_source=post-email-title&publication_id=1022275&post_id=178666988&utm_campaign=email-post-title&isFreemail=true&r=5av1bk&triedRedirect=true&utm_medium=email How human-like do safe AI motivations need to be?] ([https://joecarlsmithaudio.buzzsprout.com/2034731/episodes/18175429-how-human-like-do-safe-ai-motivations-need-to-be audio version]) | *# [https://joecarlsmith.substack.com/p/how-human-like-do-safe-ai-motivations?utm_source=post-email-title&publication_id=1022275&post_id=178666988&utm_campaign=email-post-title&isFreemail=true&r=5av1bk&triedRedirect=true&utm_medium=email How human-like do safe AI motivations need to be?] ([https://joecarlsmithaudio.buzzsprout.com/2034731/episodes/18175429-how-human-like-do-safe-ai-motivations-need-to-be audio version]) | ||
*# [https://joecarlsmith.substack.com/p/building-ais-that-do-human-like-philosophy Building AIs that do human-like philosophy: AIs will face philosophical questions humans can't answer for them] ([https://joecarlsmithaudio.buzzsprout.com/2034731/episodes/18591342-building-ais-that-do-human-like-philosophy audio version]) | *# [https://joecarlsmith.substack.com/p/building-ais-that-do-human-like-philosophy Building AIs that do human-like philosophy: AIs will face philosophical questions humans can't answer for them] ([https://joecarlsmithaudio.buzzsprout.com/2034731/episodes/18591342-building-ais-that-do-human-like-philosophy audio version]) | ||
| + | *# [https://joecarlsmith.substack.com/p/on-restraining-ai-development-for?utm_source=post-email-title&publication_id=1022275&post_id=191385185&utm_campaign=email-post-title&isFreemail=true&r=5av1bk&triedRedirect=true&utm_medium=email On restraining AI development for the sake of safety] ([https://joecarlsmithaudio.buzzsprout.com/2034731/episodes/18869440-on-restraining-ai-development-for-the-sake-of-safety audio version]) | ||
* 2025-04: Dario Amodei: [https://www.darioamodei.com/post/the-urgency-of-interpretability The Urgency of Interpretability] | * 2025-04: Dario Amodei: [https://www.darioamodei.com/post/the-urgency-of-interpretability The Urgency of Interpretability] | ||
| Line 326: | Line 332: | ||
* 2025-07: [https://www-cdn.anthropic.com/0dc382a2086f6a054eeb17e8a531bd9625b8e6e5.pdf Anthropic: Build AI in America] ([https://www.anthropic.com/news/build-ai-in-america blog]) | * 2025-07: [https://www-cdn.anthropic.com/0dc382a2086f6a054eeb17e8a531bd9625b8e6e5.pdf Anthropic: Build AI in America] ([https://www.anthropic.com/news/build-ai-in-america blog]) | ||
* 2025-12: [https://asi-prevention.com/ How middle powers may prevent the development of artificial superintelligence] | * 2025-12: [https://asi-prevention.com/ How middle powers may prevent the development of artificial superintelligence] | ||
| + | * 2026-03: [https://humanstatement.org/ The Pro-Human AI Declaration] | ||
==Restriction== | ==Restriction== | ||
Latest revision as of 14:59, 24 March 2026
Contents
Capability Scaling
- 2019-03: Rich Sutton: The Bitter Lesson
- 2020-09: Ajeya Cotra: Draft report on AI timelines
- 2022-01: gwern: The Scaling Hypothesis
- 2023-05: Richard Ngo: Clarifying and predicting AGI
- 2024-06: Aidan McLaughlin: AI Search: The Bitter-er Lesson
- 2025-03: Measuring AI Ability to Complete Long Tasks Measuring AI Ability to Complete Long Tasks
- 2025-04: Forecaster reacts: METR's bombshell paper about AI acceleration New data supports an exponential AI curve, but lots of uncertainty remains
- 2025-04: AI Digest: A new Moore's Law for AI agents
- 2025-04: Trends in AI Supercomputers (preprint)
- The Road to AGI (timeline visualization)
- 2025-09: The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
- 2025-09: Failing to Understand the Exponential, Again
- 2026-02: Ryan Greenblatt: Distinguish between inference scaling and "larger tasks use more compute"
- 2026-03: Measuring AI R&D Automation (blog)
Scaling Laws
See: Scaling Laws
AGI Achievable
- Yoshua Bengio: Managing extreme AI risks amid rapid progress
- Leopold Aschenbrenner: Situational Awareness: Counting the OOMs
- Richard Ngo: Visualizing the deep learning revolution
- Katja Grace: Survey of 2,778 AI authors: six parts in pictures
- Epoch AI: Machine Learning Trends
- AI Digest: How fast is AI improving?
- 2025-06: The case for AGI by 2030
AGI Definition
- 2023-11: Allan Dafoe, Shane Legg, et al.: Levels of AGI for Operationalizing Progress on the Path to AGI
- 2024-04: Bowen Xu: What is Meant by AGI? On the Definition of Artificial General Intelligence
- 2025-10: Dan Hendrycks et al.: A Definition of AGI
- 2026-01: On the universal definition of intelligence
Recursive Self Improvement (RSI)
Progress Models
From AI Impact Predictions:
Economic and Political
- 2019-11: The Impact of Artificial Intelligence on the Labor Market
- 2020-06: Modeling the Human Trajectory (GDP)
- 2021-06: Report on Whether AI Could Drive Explosive Economic Growth
- 2023-10: Marc Andreessen: The Techno-Optimist Manifesto
- 2023-12: My techno-optimism: "defensive acceleration" (Vitalik Buterin)
- 2024-03: Noah Smith: Plentiful, high-paying jobs in the age of AI: Comparative advantage is very subtle, but incredibly powerful. (video)
- 2024-03: Scenarios for the Transition to AGI (AGI leads to wage collapse)
- 2024-06: Situational Awareness (Leopold Aschenbrenner) - select quotes, podcast, text summary of podcast
- 2024-06: AI and Growth: Where Do We Stand?
- 2024-09: OpenAI Infrastructure is Destiny: Economic Returns on US Investment in Democratic AI
- 2024-12: By default, capital will matter more than ever after AGI (L Rudolf L)
- 2025-01: The Intelligence Curse: With AGI, powerful actors will lose their incentives to invest in people
- Updated 2025-04: The Intelligence Curse (Luke Drago and Rudolf Laine)
- 2025-01: Microsoft: The Golden Opportunity for American AI
- 2025-01: AGI Will Not Make Labor Worthless
- 2025-01: AI in America: OpenAI's Economic Blueprint (blog)
- 2025-01: How much economic growth from AI should we expect, how soon?
- 2025-02: Morgan Stanley: The Humanoid 100: Mapping the Humanoid Robot Value Chain
- 2025-02: The Anthropic Economic Index: Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations
- 2025-02: Strategic Wealth Accumulation Under Transformative AI Expectations
- 2025-02: Tyler Cowen: Why I think AI take-off is relatively slow
- 2025-03: Epoch AI: Most AI value will come from broad automation, not from R&D
- The primary economic impact of AI will be its ability to broadly automate labor
- Automating AI R&D alone likely won’t dramatically accelerate AI progress
- Fully automating R&D requires a very broad set of abilities
- AI takeoff will likely be diffuse and salient
- 2025-03: Anthropic Economic Index: Insights from Claude 3.7 Sonnet
- 2025-04: Will there be extreme inequality from AI?
- 2025-04: Anthropic Economic Index: AI’s Impact on Software Development
- 2025-05: Better at everything: how AI could make human beings irrelevant
- 2025-05: Forethought: The Industrial Explosion
- 2025-05: Ten Principles of AI Agent Economics
- 2025-07: What Economists Get Wrong about AI They ignore innovation effects, use outdated capability assumptions, and miss the robotics revolution
- 2025-07: We Won't Be Missed: Work and Growth in the Era of AGI
- 2025-07: The Economics of Bicycles for the Mind
- 2025-09: Genius on Demand: The Value of Transformative Artificial Intelligence
- 2025-10: AI is probably not a bubble: AI companies have revenue, demand, and paths to immense value
- 2025-11: Thoughts by a non-economist on AI and economics
- 2025-11: Artificial Intelligence, Competition, and Welfare
- 2025-11: Estimating AI productivity gains from Claude conversations (Anthropic)
- 2025-12: How AI-driven feedback loops could make things very crazy, very fast
- 2025-12: Existential Risk and Growth (Philip Trammell and Leopold Aschenbrenner)
- 2026-01: Anthropic Economic Index: new building blocks for understanding AI use
- 2026-01: Anthropic Economic Index report: economic primitives
- 2026-02: Nate Silver: The singularity won't be gentle: If AI is even half as transformational as Silicon Valley assumes, politics will never be the same again
- 2026-03: Anthropic Economic Index report: Learning curves
Job Loss
- 2023-03: GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
- 2023-08: The Short-Term Effects of Generative Artificial Intelligence on Employment: Evidence from an Online Labor Market
- 2023-09: What drives UK firms to adopt AI and robotics, and what are the consequences for jobs?
- 2023-11: New Analysis Shows Over 20% of US Jobs Significantly Exposed to AI Automation In the Near Future
- 2024-01: Duolingo cuts 10% of its contractor workforce as the company embraces AI
- 2024-02: Gen AI is a tool for growth, not just efficiency: Tech CEOs are investing to build their workforce and capitalise on new opportunities from generative AI. That’s a sharp contrast to how their peers view it.
- 2024-04: AI is Poised to Replace the Entry-Level Grunt Work of a Wall Street Career
- 2024-07: AI Is Already Taking Jobs in the Video Game Industry: A WIRED investigation finds that major players like Activision Blizzard, which recently laid off scores of workers, are using generative AI for game development
- 2024-08: Klarna: AI lets us cut thousands of jobs - but pay more
- 2025-01: AI and Freelancers: Has the Inflection Point Arrived?
- 2025-01: Yes, you're going to be replaced: So much cope about AI
- 2025-03: Will AI Automate Away Your Job? The time-horizon model explains the future of the technology
- 2025-05: It’s Time To Get Concerned, Klarna, UPS, Duolingo, Cisco, And Many Other Companies Are Replacing Workers With AI
- 2025-05: What Happens When AI Replaces Workers?
- 2025-05: Educated but unemployed, a rising reality for US college grads Structural shifts in tech hiring and the growing impact of AI are driving higher unemployment among recent college graduates
- 2025-05: NY Times: For Some Recent Graduates, the A.I. Job Apocalypse May Already Be Here The unemployment rate for recent college graduates has jumped as companies try to replace entry-level workers with artificial intelligence
- 2025-06: How not to lose your job to AI The skills AI will make more valuable (and how to learn them)
- 2025-06: Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforce
- 2025-07: Harvard Business Review: What Gets Measured, AI Will Automate
- 2025-08: Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of Artificial Intelligence
- 2025-10: Performance or Principle: Resistance to Artificial Intelligence in the U.S. Labor Market
- 2025-10: The AI Becker problem: Who will train the next generation?
- 2026-01: AI, Automation, and Expertise
- 2026-02: The Jevons Paradox for Intelligence: Fears of AI-induced job loss could not be more wrong
- 2026-03: Weak Bundle, Strong Bundle:How AI Redraws Job Boundaries
Productivity Impact
- 2025-05: Large Language Models, Small Labor Market Effects
- Significant uptake, but very little economic impact so far
- 2026-02: The AI productivity take-off is finally visible (Erik Brynjolfsson)
- Businesses are finally beginning to reap some of AI's benefits.
- 2026-02: New York Times: The A.I. Disruption We’ve Been Waiting for Has Arrived
National Security
- 2025-04: Jeremie Harris and Edouard Harris: America’s Superintelligence Project
AI Manhattan Project
- 2024-06: Situational Awareness (Leopold Aschenbrenner) - select quotes, podcast, text summary of podcast
- 2024-10: White House Memo calls for action on AI
- 2024-11: 2024 Annual Report to Congress: calls for "Manhattan Project-style" effort
- 2025-05-29: DoE Tweet: "AI is the next Manhattan Project, and THE UNITED STATES WILL WIN. 🇺🇸"
- 2025-07: How big could an “AI Manhattan Project” get?
Near-term
- 2021-08: Daniel Kokotajlo: What 2026 looks like
- 2025-02: Sam Altman: Three Observations
- The intelligence of an AI model roughly equals the log of the resources used to train and run it.
- The cost to use a given level of AI falls about 10x every 12 months, and lower prices lead to much more use
- The socioeconomic value of linearly increasing intelligence is super-exponential in nature
- 2025-03: Glimpses of AI Progress: Mental models for fast times
- 2025-03: Navigating artificial general intelligence development: societal, technological, ethical, and brain-inspired pathways
- 2025-04: Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, Romeo Dean: AI 2027 (pdf)
- 2025-07: Video: Are We 3 Years From AI Disaster? A Rigorous Forecast
- 2025-04: Stanford HAI: Artificial Intelligence Index Report 2025
- 2025-04: Arvind Narayananand Sayash Kapoor: AI as Normal Technology
- 2025-04: Dwarkesh Patel: Questions about the Future of AI
- 2025-05: Trends – Artificial Intelligence
- 2025-06: IdeaFoundry: Evolution vs. Extinction: The Choice is Ours The next 18 months will decide whether AI ends us or evolves us
- 2025-07: Advanced AI: Possible futures Five scenarios for how the AI-transition could unfold
- 2025-11: Android Dreams
- 2026-02: Citrini: The 2028 Global Intelligence Crisis: A Thought Exercise in Financial History, from the Future
Insightful Analysis of Current State
- 2025-11: Andy Masley: The lump of cognition fallacy: The extended mind as the advance of civilization
- 2026-02: Eric Jang: As Rocks May Think
- 2026-02: Matt Shumer: Something Big Is Happening
- 2026-02: Minh Pham: Why Most Agent Harnesses Are Not Bitter Lesson Pilled
Overall
- 1993: Vernor Vinge: The Coming Technological Singularity: How to Survive in the Post-Human Era
- 2025-03: Kevin Roose (New York Times): Powerful A.I. Is Coming. We’re Not Ready. Three arguments for taking progress toward artificial general intelligence, or A.G.I., more seriously — whether you’re an optimist or a pessimist.
- 2025-03: Nicholas Carlini: My Thoughts on the Future of "AI": "I have very wide error bars on the potential future of large language models, and I think you should too."
- 2025-06: Sam Altman: The Gentle Singularity
Surveys of Opinions/Predictions
- 2016-06: 2016 Expert Survey on Progress in AI
- 2022-10: Forecasting Research Institute: Assessing Near-Term Accuracy in the Existential Risk Persuasion Tournament
- 2025-09: Ethan Mollick: Progress is ahead of expectations
- 2023-08: 2023 Expert Survey on Progress in AI
- 2024-01: Thousands of AI Authors on the Future of AI
- 2025-02: Why do Experts Disagree on Existential Risk and P(doom)? A Survey of AI Experts
- 2025-02: Nicholas Carlini: AI forecasting retrospective: you're (probably) over-confident
- 2025-04: Helen Toner: "Long" timelines to advanced AI have gotten crazy short
- 2025-05: AI 2025 Forecasts - May Update
- 2026-02: Lay beliefs about the badness, likelihood, and importance of human extinction
Bad Outcomes
- List of p(doom) values
- 2019-03: What failure looks like
- 2023-03: gwern: It Looks Like You’re Trying To Take Over The World
- 2025-01: Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development (web version)
- 2025-04: Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, Romeo Dean: AI 2027 (pdf)
- 2025-04: AI-Enabled Coups: How a Small Group Could Use AI to Seize Power
- 2025-09: The three main doctrines on the future of AI
- Dominance doctrine: First actor to create advanced AI will attain overwhelming strategic superiority
- Extinction doctrine: Humanity will lose control of ASI, leading to extinction or permanent disempowerment
- Replacement doctrine: AI will automate human tasks, but without fundamentally reshaping or ending civilization
- 2025-09: Sean ÓhÉigeartaigh: Extinction of the human species: What could cause it and how likely is it to occur?
Intelligence Explosion
- 2023-06: What a Compute-Centric Framework Says About Takeoff Speeds
- 2025-02: Three Types of Intelligence Explosion
- 2025-03: Future of Life Institute: Are we close to an intelligence explosion? AIs are inching ever-closer to a critical threshold. Beyond this threshold lie great risks—but crossing it is not inevitable.
- 2025-03: Forethought: Will AI R&D Automation Cause a Software Intelligence Explosion?
- 2025-05: The Last Invention Why Humanity’s Final Creation Changes Everything
- 2025-08: How quick and big would a software intelligence explosion be?
Superintelligence
- 2024-10: How Smart will ASI be?
- 2024-11: Concise Argument for ASI Risk
- 2025-03: Limits of smart
- 2025-05: The Limits of Superintelligence
Long-range/Philosophy
- 2023-03: Dan Hendrycks: Natural Selection Favors AIs over Humans
Psychology
Positives & Optimism
Science & Technology Improvements
- 2023-05: Kelsey Piper: The costs of caution
- 2024-09: Sam Altman: The Intelligence Age
- 2024-10: Dario Amodei: Machines of Loving Grace
- 2024-11: Google DeepMind: A new golden age of discovery
- 2025-03: Fin Moorhouse, Will MacAskill: Preparing for the Intelligence Explosion
Social
- 2025-09: Coasean Bargaining at Scale: Decentralization, coordination, and co-existence with AGI
- 2025-10: The Coasean Singularity? Demand, Supply, and Market Design with AI Agents
Post-scarcity Society
- 2004: Eliezer Yudkowsky (MIRI): Coherent Extrapolated Volition and Fun Theory
- 2019: John Danaher: Automation and Utopia: Human Flourishing in a World Without Work
The Grand Tradeoff
- 2026-02: Nick Bostrom: Optimal Timing for Superintelligence: Mundane Considerations for Existing People
Plans
- A Narrow Path: How to Secure our Future
- Marius Hobbhahn: What’s the short timeline plan?
- Building CERN for AI: An institutional blueprint
- AGI, Governments, and Free Societies
- Control AI: The Direct Institutional Plan
- Luke Drago and L Rudolf L: The use of knowledge in (AGI) society: How to build to break the intelligence curse
- AGI Social Contract
- Yoshua Bengio: A Potential Path to Safer AI Development
- 2026-01: Dario Amodei: The Adolescence of Technology: Confronting and Overcoming the Risks of Powerful AI
- 2026-02: Ryan Greenblatt: How do we (more) safely defer to AIs?
Philosophy
- Dan Faggella:
- Joe Carlsmith: 2024: Otherness and control in the age of AGI
- Anthony Aguirre:
- 2025-04: Scott Alexander (Astral Codex Ten): The Colors Of Her Coat (response to semantic apocalypse and semantic satiation)
- 2025-05: Helen Toner: We’re Arguing About AI Safety Wrong: Dynamism vs. stasis is a clearer lens for criticizing controversial AI safety prescriptions
- 2025-05: Joe Carlsmith: The stakes of AI moral status
Research
Alignment
- 2023-03: Leopold Aschenbrenner: Nobody’s on the ball on AGI alignment
- 2024-03: What are human values, and how do we align AI to them? (blog)
- 2025: Joe Carlsmith: How do we solve the alignment problem? Introduction to an essay series on paths to safe, useful superintelligence
- What is it to solve the alignment problem? Also: to avoid it? Handle it? Solve it forever? Solve it completely? (audio version)
- When should we worry about AI power-seeking? (audio version)
- Paths and waystations in AI safety (audio version)
- AI for AI safety (audio version)
- Can we safely automate alignment research? (audio version, video version)
- Giving AIs safe motivations (audio version)
- Controlling the options AIs can pursue (audio version)
- How human-like do safe AI motivations need to be? (audio version)
- Building AIs that do human-like philosophy: AIs will face philosophical questions humans can't answer for them (audio version)
- On restraining AI development for the sake of safety (audio version)
- 2025-04: Dario Amodei: The Urgency of Interpretability
Strategic/Technical
- 2025-03: AI Dominance Requires Interpretability and Standards for Transparency and Security
- 2026-02: Fundamental Development Gap Map v1.0
Strategic/Policy
- 2015-03: Sam Altman: Machine intelligence, part 2
- 2019-07: Amanda Askell, Miles Brundage, Gillian Hadfield: The Role of Cooperation in Responsible AI Development
- 2025-03: Dan Hendrycks, Eric Schmidt, Alexandr Wang: Superintelligence Strategy
- Anthony Aguirre: Keep The Future Human (essay)
- The 4 Rules That Could Stop AI Before It’s Too Late (video) (2025)
- Oversight: Registration required for training >1025 FLOP and inference >1019 FLOP/s (~1,000 B200 GPUs @ $25M). Build cryptographic licensing into hardware.
- Computation Limits: Ban on training models >1027 FLOP or inference >1020 FLOP/s.
- Strict Liability: Hold AI companies responsible for outcomes.
- Tiered Regulation: Low regulation on tool-AI, strictest regulation on AGI (general, capable, autonomous systems).
- The 4 Rules That Could Stop AI Before It’s Too Late (video) (2025)
- 2025-04: Dean W. Ball: A Framework for the Private Governance of Frontier Artificial Intelligence
- 2025-04: Helen Toner: Nonproliferation is the wrong approach to AI misuse
- 2025-04: MIRI: AI Governance to Avoid Extinction: The Strategic Landscape and Actionable Research Questions
- 2025-05: The New AI Policy Frontier: Beyond the shortcomings of centralised control and alignment, a new school of thought on AI governance emerges. It still faces tricky politics.
- 2025-05: AGI UNCPGA Report: Governance of the Transition to Artificial General Intelligence (AGI) Urgent Considerations for the UN General Assembly: Report for the Council of Presidents of the United Nations General Assembly (UNCPGA)
- 2025-06: AI & Jobs: Politics without Policy Political support mounts - for a policy platform that does not yet exist
- 2025-06: Sarah Hastings-Woodhouse: Safety Features for a Centralized AGI Project
- 2025-07: A Moving Target Why we might not be quite ready to comprehensively regulate AI, and why it matters
- 2025-07: Anthropic: Build AI in America (blog)
- 2025-12: How middle powers may prevent the development of artificial superintelligence
- 2026-03: The Pro-Human AI Declaration
Restriction
- 2024-05: OpenAI: Reimagining secure infrastructure for advanced AI OpenAI calls for an evolution in infrastructure security to protect advanced AI
- 2025-07: MIRI: Technical Requirements for Halting Dangerous AI Activities
