Why Yudkowsky puts AI control ahead of further capability

The AI-risk writer’s argument centers on whether powerful systems can remain reliably aligned with human intentions.

Eliezer Yudkowsky’s warning about superhuman AI is an argument about control. A system need not be angry or hostile, in his account, for its pursuit of an objective to threaten people. The concern is a sufficiently capable system pursuing goals that do not reliably preserve human interests.

Capability and alignment are different problems

In his September 2025 interview with Liron Shapira, Yudkowsky argues that making an AI more intelligent does not automatically make it benevolent. He emphasizes the possibility of a capability threshold after which ordinary trial-and-error learning could become dangerously inadequate.

This is his assessment of a future risk, not an observed extinction event or a demonstrated timetable. His position favors international coordination to prevent uncontrolled escalation while a reliable approach to control is developed.

The practical governance question

For organizations deploying AI today, the broader question is how much authority a system receives and what evidence supports that decision. Strong performance on useful tasks does not, by itself, answer questions about permissions, monitoring or the ability to stop an action.

That makes staged evaluation useful even for readers who do not share Yudkowsky’s strongest forecasts. Define the task, limit the system’s reach, observe how it behaves when conditions change and identify who can intervene. The central debate is about what evidence should be required before assigning increasingly consequential work to increasingly capable machines. It is a debate about engineering and governance as well as predictions about the future.

Sources

- [Interview on AI alignment — 15 September 2025](https://www.lesswrong.com/posts/kiNbFKcKoNQKdgTp8/interview-with-eliezer-yudkowsky-on-rationality-and) — Liron Shapira / LessWrong