Chris Olah is not exactly central casting for a papal ceremony. He's an atheist who walked away from evangelical Christianity at 15. He took a Thiel Fellowship—money from the same guy who apparently believes AI skeptics are servants of the antichrist. And he's a cofounder of Anthropic, a company barreling toward a public offering at a valuation that could brush a trillion dollars.

Yet there he was, speaking at the Vatican following Pope Leo XIV's landmark encyclical on artificial intelligence, Magnifica Humanitas—a document that explicitly called for "disarming" the technology. The irony wasn't lost on Olah. He opened his remarks by acknowledging the elephant in the room: that every frontier AI lab, Anthropic included, operates inside a system of incentives that can and does conflict with actually doing the right thing. Not hedged corporate speak. A straight admission, at the Vatican, from someone who helped build one of the most powerful AI systems on the planet.

That's a remarkable sentence to say out loud. It's also the entire point of why Leo wanted him there.

What the Pope Actually Said—and What He Didn't

Let's be clear about what this encyclical is and isn't. It's not a kill switch. Pope Leo XIV isn't going to halt GPU production or convince Sam Altman to pivot to artisanal cheese. Magnifica Humanitas won't stop defense contractors from wiring AI into weapons systems, and it certainly won't guilt-trip a CEO out of using "AI efficiency" as cover for the next round of layoffs.

The document's goals are more modest and arguably more durable: generate dialogue, create moral friction, and maybe—just maybe—manufacture enough institutional shame that some of the people building these systems start taking seriously what they already suspect in the back of their minds. That the outcome could be genuinely terrible.

The core tension Leo identifies isn't subtle. Silicon Valley's dominant mythology holds that AI creates abundance that lifts all boats. The Pope's counternarrative is grimmer: a world where a privileged slice of humanity accumulates unimaginable wealth while the rest get optimized, surveilled, and quietly discarded by systems that were never designed with them in mind. You don't have to be Catholic—or religious at all—to recognize that's a legitimate concern worth naming loudly.

For reference, Pope Francis made a similar move in 2015 with his climate encyclical, Laudato Si'. Did it stop fossil fuel production? Obviously not. Did it shift the moral framing around climate in ways that had lasting institutional effects? Arguably yes. That's the playbook here.

How the Church Got Its Man Inside the Machine

The Vatican didn't just cold-call Anthropic. This was years of groundwork. The Church has been seriously engaging with AI since at least 2016, when it launched the Minerva Dialogues—a series of closed-door meetings held at a church in Rome where Galileo was once sanctioned for saying the Earth orbits the sun, which is either deeply ironic or perfectly appropriate depending on your disposition. Tech luminaries including Reid Hoffman and Eric Schmidt attended. Pope Francis used the 2023 session to preview what would become Leo's central themes: human dignity, social inclusion, the necessity of voices outside the industry.

The more direct path to Olah ran through Silicon Valley's own backyard. In 2025, a cluster of Catholic ethicists and clergy based in San Jose began deliberately cultivating relationships with the AI industry growing around them. Two figures from Santa Clara University—ethicist Brian Patrick Green and pastor Brendan McGuire—started meeting with Olah in the fall to work through the moral dimensions of what his company was building. By January, they had brought Cardinal Paul Tigue, the Vatican's designated AI point person, into those conversations.

Olah is, by all accounts, someone who takes this stuff seriously in a way that isn't performative. The worm rescue story—apparently he actually picks earthworms off wet sidewalks after rain—tells you something about the man's moral operating system that no press release could.

Anthropic's Constitution Gets a Theological Commentary

Here's where it gets genuinely interesting from a technical and institutional standpoint. When Anthropic was revising the behavioral guidelines for Claude—the document that defines how the model is supposed to reason about ethics, harm, and its own role in the world—Olah sent a draft to the San Jose group for feedback.

McGuire responded with a 28-page commentary. Not a red-team exercise. Not a RLHF critique. By his own description, it was "wisdom from the mystics in the dark ages, from the perspective of the tension between knowing and not knowing." That framing might make some engineers instinctively reach for the skip button, but consider what that tradition actually encodes: centuries of hard-won thinking about epistemic humility, the limits of human agency, and what happens when institutions become untethered from accountability.

Both Green and McGuire are acknowledged in Claude's constitution. A Catholic mystic tradition left fingerprints on one of the most widely used AI systems in the world. Whether that makes you hopeful or unsettled probably says something about your own priors.

The Blowback Was Predictable, and Probably Worth It

Inviting an AI company cofounder to speak after a document calling for technological restraint was always going to annoy someone. People who found Leo's message genuinely compelling were irritated that the microphone went to an industry insider. AI accelerationists—the crowd who treat any friction on development timelines as heresy—felt that Olah had stabbed the field in the back by legitimizing a document that gestured toward a pause.

Both reactions miss the point. The Vatican wasn't looking for a villain or a convert. It needed someone who could stand inside the machine and say, honestly, that the machine's incentive structure is broken. Olah did that. That's more valuable than any amount of outside criticism, because it removes the easy dismissal: "Those people don't understand how this works."

He understands exactly how it works. That's the whole point.

What Any of This Actually Changes

Realistically? Not much, fast. The capital is already deployed. The compute clusters are already running. The product roadmaps are already locked. No encyclical rewrites a VC term sheet.

But the conditions under which AI gets built are not purely technical. They're social, political, and yes, moral. The industry has spent years insisting it can self-regulate—and has produced a track record that makes that claim increasingly hard to defend with a straight face. External pressure, whether from regulators, civil society, or an 88-year-old institution with a two-millennia track record of playing long games, changes the environment in which decisions get made.

Olah's appearance at the Vatican didn't fix anything. But it made visible something that usually stays hidden: that the people building the most consequential technology in a generation are not uniformly convinced it's going to go well. Some of them are genuinely frightened. And they're looking, however awkwardly, for frameworks that can hold that fear and do something useful with it.

Whether the mystics of the dark ages have better answers than a loss function is an open question. I wouldn't dismiss it out of hand.