I’ve read several big sections of Pope Leo XIV’s Magnifica Humanitas — magnificent humanity. The whole thing is essentially a book at 40k words. It’s fascinating and at times even great. Some of the most interesting parts of the text aren’t really about AI, but more about what it means to be human and to care for others.

There is also a theme of concentrated power and who will control AI:

In many cases within the digital context, control over platforms, infrastructure, data and computing power does not rest with States, but with major economic and technological actors. These entities effectively set the conditions for access, determine the rules of visibility and shape the very possibilities for participation. When such power is concentrated in the hands of a few, it tends to become opaque and evade public oversight, increasing the risk of distorted forms of development that give rise to new dependencies, exclusions, manipulations and inequalities.

The encyclical also argues that we can’t accept better alignment without oversight:

We cannot be satisfied with merely calling for the moralization of machines — the so-called “alignment” of AI with human values — without also having the courage to insist on a further condition: the possibility of openly discussing the ethical frameworks involved and subjecting them to shared standards of social justice. Otherwise, those who control AI will impose their own moral vision, which will become the invisible infrastructure of these systems. A more moral AI is not enough if that morality is determined by a few.

I found it a little discomforting that the pope appeared alongside Anthropic co-founder Chris Olah, as if this was a political event where the most influential lobbyist was granted a privileged role in framing the discussion. It feels like something this significant should be above corporate needs and beyond current politics.

If we are talking about decentralization, as I’ve blogged before OpenAI has attempted to do more with open models and broad access to their API. Anthropic worries so much about the ill effects of AI and yet they trust no one but themselves with the power, turning safety into a marketing pitch. It rings more and more hollow for me. Still, Anthropic clearly has people who are thinking deeply about what intelligence and consciousness mean, and I appreciate that.