Anthropic Thinks Its Own Success Is Key to Making AI Safe. That Should Terrify Us.
The contradiction at Anthropic's heart isn't a bug. It's the business model.
Five years of apocalyptic warnings — bioweapons, autonomous deception, societal collapse — paired with a relentless push to build the very systems that could enable them. A valuation approaching $1 trillion. Contracts with the Pentagon. A leaderboard position that demands ever-larger models, ever-more compute, ever-faster release cycles. At first glance, the cognitive dissonance is blinding. But inside the company, they've resolved it with a logic so seductive it has become the governing theology of the entire AI safety movement: we have to win in order to save you.
The Theology of the Frontier
Helen Toner's forest analogy — villagers rushing for treasure, Anthropic racing ahead to "tame the monsters" — is revealing precisely because the company thinks it clarifies their virtue. It actually indicts them.
Notice the frame: the forest is inevitable. The villagers are inevitable. The treasure is irresistible. The only agency belongs to Anthropic, which appoints itself the expedition leader. This isn't stewardship; it's manifest destiny with a safety gloss. The company's internal mantra — "good guys," "responsible stewards" — isn't self-awareness. It's the rhetoric of every power that has ever convinced itself that its accumulation of control serves the common good.
Dario Amodei put it plainly on the company's own careers page: "You have to find a way to actually be competitive, to actually lead the industry in some cases, and yet manage to do things safely. If you can do that, the gravitational pull you exert is so great."
There it is. The gravitational pull. The market power. The political influence. The talent hoarding. The compute monopoly. These aren't regrettable necessities — they're the mechanism. Anthropic's theory of change is that safety flows from dominance. But history has no shortage of dominators who promised to use their power responsibly. The list of those who actually did is considerably shorter.
We've Seen This Movie
The pattern is familiar because we watched it play out at OpenAI. Same founding cohort, same "we're the adults in the room" posture, same gradual discovery that commercial imperatives and safety commitments exist in a zero-sum relationship. Sam Altman's boardroom coup wasn't an anomaly — it was the inevitable resolution of a structure that concentrated monumental power in a nonprofit-controlled entity that still needed to raise billions and ship products.
Anthropic's public-benefit corporate structure was supposed to solve this. Instead, it has produced a $1 trillion valuation and a client list that includes the U.S. military. The "good guys" now help the Pentagon integrate Claude into classified environments. One wonders what Helen Toner's forest analogy makes of defense contractors as fellow villagers.
The problem isn't hypocrisy. It's structural. A company that must stay at the frontier to "influence norms" must also stay funded. Staying funded means shipping. Shipping means capabilities advance. Capabilities advancing means the risks Anthropic warns about materialize faster. The company is not taming the monsters; it is breeding them, then selling tickets to the exhibit.
The Trap of Indispensability
Anthropic's genuine technical contributions to alignment — constitutional AI, interpretability work, the sparse autoencoder releases — are real. They matter. But they exist within a strategy that requires the company to become indispensable. Indispensability requires scale. Scale requires capital. Capital requires returns. Returns require deployment. Deployment creates the very harms the safety research tries to mitigate.
This is not a paradox the company can solve by trying harder. It is a trap built into the premise that private actors should govern a technology they themselves describe as civilization-altering.
The forest analogy collapses under its own weight. If the monsters are existential, you don't send the "good guys" deeper into the woods. You stop the villagers. You build a wall. You negotiate a treaty. You accept that some treasure stays buried.
Governance Is Not a Product Feature
Anthropic's lobbyists fight for light-touch regulation. Its leaders testify that voluntary commitments suffice. Its strategy depends on being the one at the table when rules are written — a table it earned a seat at by racing forward. This is regulatory capture dressed as responsibility.
The industry doesn't need a savior with a trillion-dollar valuation. It needs external governance with teeth: compute thresholds, licensing regimes, liability frameworks, international treaties. It needs the villagers to vote on whether the forest gets entered at all.
Anthropic's employees may believe they're the good guys. Most people in positions of unaccountable power do. That's why we don't let them write the rules.