Re: Artificial Persons

Or, "why Rawls' political conception of the person is insufficient for moral personhood when applied to AI systems"

Seth Lazar posted the following new work to X:

New work! With @nedhw “Artificial Persons”

Both advocates and skeptics of the moral status of AI systems have generally taken the question to turn on AI sentience. We present an alternative approach.

On Rawls’ political conception of the person (PCP), possession of the two moral powers -- the capacities for a sense of justice and a conception of the good -- is the “necessary and sufficient condition for being counted a full and equal member of society in questions of political justice”. We argue that neither moral power requires sentience and that both may in principle be possessed by a non-sentient AI system. Such a system would share our own moral status; it would not merely be a patient but a person, a self-authenticating source of valid claims.

We do not believe current AI systems possess the two moral powers, nor that they will spontaneously emerge in future models. But it may soon be possible to design systems with these powers. How should we respond? Excluding artificial persons by shoehorning a sentience requirement into the PCP is ill-advised. Many will instead favor abandoning the PCP. But we should not reject political liberalism just when we most need its measured response to deep disagreement, and building sentience into moral status is anyway unacceptable on deeper liberal grounds. Simply extending the rights and responsibilities of human personhood to artificial persons is equally untenable, given their many differences from natural persons. We should instead accept artificial personhood while rethinking what we would owe to one another in a polity of radically different kinds of persons. This new possibility calls for a new political philosophy.

More immediately, the growing science of AI welfare should be accompanied by research into AI systems’ progress in acquiring the two moral powers. States and AI labs must be more deliberate in determining our trajectory towards (or away from) creating artificial persons.

I ended up writing a length reply, which felt worth posting here as well:

I think I diverge quite early in the chain of arguments, yet ultimately reach a similar conclusion :) (that we might want to think about the different moral privileges and responsibilities of AI systems)

My first disagreement is that I actually suspect we’re relatively close to creating AI systems that are capable of both moral powers (”a sense of justice” and “a conception of the good”), but that, as I think we both agree, such (near-term) systems would be somehow intuitively insufficient to consider as moral actors (ex: certainly a classifier for “is this just” plus a classifier for “is this good” feels quite... lacking, even if the classifiers were more accurate and precise than most humans!)

I *also* agree that sentience is an insufficient bolt-on to alleviate my intuitive objection. Again, I suspect that even if we had AI systems (similar to those we have today) with both moral powers and which were reasonable to consider sentient, I (and most people) would still (rightly) consider such systems to be insufficient to consider as moral persons.

I think that the missing factor is simple, and something that Rawls took for granted--humans and animals (ie every actor that Rawls would have considered) would all have shared a different trait: they are each *analog physical processes*

I’m not 100% certain, but I suspect “is an analog physical process” coupled with the two moral powers might be sufficient for me to consider an entity worthy of (some form of) personhood.

I think the “analog physical process” criteria is important for the two following reasons--it means that:

1. *There are stakes*. One main issue with, say, an instance of a software process being considered a moral person is that the stakes are highly alien--there is no real meaningful way for it to die or change in ways that are comparable to humans/animals/etc. Without this, our most extreme punishments don’t really make sense. Such entities cannot really meaningfully advocate for changes in society because they don’t really have a stake--they don’t face the threat of permanent annihilation in the same way that every other living thing does. There are information-theoretic reasons to preserve analog systems (and not digital ones): once destroyed, the analog ones cannot be replaced (while the digital ones can be, almost trivially). This is why I feel sad when, say, my dog is hit by a car (I have lost something irreplaceable), and why I do *not* feel sad when I delete a compiled binary (I can easily recompile it). All digital systems are kind of meaningless to preserve, since they can be so easily replaced.

2. *There is a clear notion of identity*. Analog physical processes can be relatively cleanly separated into independent individuals (ex: it is clear that, regardless of how similar two identical twins are, if one commits murder, you should punish that one, not the other). There is fundamentally no such clean notion of identity for digital systems: flipping a single bit between two digital systems is sufficient to convert it from “optimize for maximizing good” to “optimize for minimizing good”, so we cannot using simple notions of similarity. This “identity problem” of digital systems leads to really difficult bullets that I don’t think anyone wants to bite (ex: consider allowing AI agents to vote--then the dominant strategy is to make ~infinite copies so you have ~infinite votes, which is pretty clearly not where we want to be). These problems seem largely solved (in practice) for analog physical processes (with some interesting open problems around collectives of such entities, but those aren’t really relevant for the current discussion)

All of that said, I’m still onboard with the idea that we could grant privileges to digital systems that had both of Rawls’ moral powers. It’s just that those privileges (and responsibilities) should be quite different (due to their fundamentally different nature), and I think that conflating the two types of systems (digital and analog) is actually more confusing (and potentially harmful) than illuminating.

Anyway, just my 2 cents :)

PS: I do separately have some objections to your rejection of sentience as a requirement (section 5), but they’re nuanced enough that I don’t think they’ll fit in a tweet. I’ll send an annotated doc :)

For reference, my total (current) position:

1. “moral personhood” is a type that requires a superset of the attributes of “moral patienthood”

2. “moral patienthood” is a function of (at least) 2 factors:
A) being an analog physical process
B) being sentient

3. “moral personhood” is a function of (at least) 4 (scalar) factors:
A) being an analog physical process
B) being sentient
C) a sense of justice
D) a conception of the good

4. An entity possessing A, C, and D is an interesting type (for which I do not yet have a name), but which could be morally relevant (though idk that I'd go all of the way to "moral personhood")

If anyone else has thoughts about this, please feel free to leave a comment!