Ask a Data Ethicist: Could a Machine Be a Person?

UpperBound is an AI conference hosted by the Alberta Machine Intelligence Institute (Amii). At this year’s event I was asked to participate in a debate to explore this question… 

Could a machine be a person? 

This is far from my usual focus on practical matters related to responsible AI. However, it is important to be able to have respectful conversations about controversial, possibly polarizing topics, a skill that we seem to be losing in our society. So, I accepted the challenge.

Backstory

I was invited to participate in this session by my former research supervisor, Dr. Geoffrey Rockwell. Geoffrey is a philosopher and fellow with Amii along with Dr. Rich Sutton. Rich Sutton is a Turing award winner known for his pioneering work in reinforcement learning. He is also Amii’s Chief Scientific Advisor. The two professors had wanted to have a public conversation about this topic for a while now and the conference served as a useful backdrop. Joining us for the debate was Dr. Zhijing Jin, an assistant professor with the University of Toronto and founder of EuroSafe AI. 

To ensure I’m not misattributing any specific words or positions to a particular panelist, I’m giving a first person account of my position and some general comments about others’ perspectives for this column. 

Humans vs. Persons

Right away I had an issue with the framing of the question and the word “person,” so my opener was to establish humans as biological, and persons as socially constructed. We already have non-human entities that are ascribed personhood. Corporations are one such example. They have legal personhood so that we can assign them a role as subjects under the law and therefore hold them accountable in the courts. We’ve also granted personhood to a river in Quebec and some animals too.

Yet, even for humans, the concept of personhood has been fluid. Here in Canada, women were not considered persons until October 18, 1929, which is now known as persons day. This was only for white women; other women did not achieve this status until decades later. Rights are still not a concept that is applied equally across the globe. Women and ethnic groups still face human rights abuses that deny them full personhood.

Thus, even asking the question of personhood in relation to a machine might feel insulting to some given this inequity. That was a conversation I had with a fellow BIPOC female audience member during the break. We joked – it’s like saying to the bot, “You want personhood? Get in line!” 

The other word that was problematic for me with respect to this question was “could” because could implies a “reasonable doubt” element. If you ask “could x happen” – unless you are 100% certain x would never happen, then yes, perhaps, maybe, under the right conditions, in some future possible timeframe, it could happen. So, the question is somewhat leading for anyone who takes reasonable doubt seriously. For this reason, everyone on the panel agreed that it could happen – even if it was unlikely, not a good idea or a long ways off. 

A Basis for Personhood

Human beings deserve rights as persons by virtue of their humanity. This is the basis of human rights. Other non-human entities that have been granted personhood legally are treated as such for some socially useful and agreed to reason, as we’ve established above. Our next set of questions explored the possible conditions of personhood for AI. The actual debate went all over the place, but the questions we received in advance included:

• Will we have the ability to make machines that should be regarded as persons?
• Could AIs be deserving of status and freedom equal to people?
• How will we know if a machine should have this status?

A few weeks before the debate I asked everyone I encountered this set of questions. When it came to identifying what elements were required, I got answers about sentience, consciousness, self-awareness, self-identity, free will, the ability to set one’s own goals – the stuff that philosophers deliberate and that is not easily defined or understood. Yet, we all have a “sense” of what its about. People essentially told me that until AI had these qualities, personhood would be a non-starter.

How would we know if a machine had these qualities? How do we know other people have these qualities? I’m not sure we do. I spent time reading Thomas Nagel’s “What is it like to be a bat?” which is worth a read if you’re interested in these questions. I was also reminded of the StarTrek episode, The Measure of a Man, whereby Lt Commander Data must prove to be deserving of rights, and not merely the property of Starfleet, when a scientist wishes to dismantle the humanoid and study it for research purposes. The decision turns on three points: does Data possess intelligence, self-awareness, and consciousness. 

A friend also encouraged me to look up Anil Seth, a neuroscientist who speaks about the difference between intelligence and consciousness, and why we should separate these two things. His talk, entitled “Why AI isn’t going to become conscious,” sums up his position.

Finally, I think the whole “LLMs are sentient” idea stems from its communicative word remixing capabilities. Why don’t we think AlphaFold is conscious? It’s because that type of AI doesn’t mimic humans like a chatbot. Major Eliza Effect vibes are happening with today’s AI bots.

Timelines – Are We Close?

The panelists agreed we are not close and that LLMs are not the kind of AI that would qualify for personhood. There was a moment where our moderator really wanted to jump into the debate, presumably to challenge the idea that LLMs were not the kind of AI worthy of personhood. 

Essentially, this point came down to one of architecture. How an LLM is made, how it works isn’t the kind of thing that leads to the qualifying elements. There was a general sense that we need fundamentally new approaches toward the goal of intelligence because it won’t emerge from more data and compute pumped into a learned model with fixed weights. 

For my part, I am not convinced that an LLM is anything more than pattern recognition and fancy word remixing given how its designed – a point I’ve made in other columns. There is no self, no subjective experience, no capacity for suffering, no intrinsic state, no capacity to feel emotions, no free will, no understanding, no reasoning. I’m also not convinced of the so-called evidence of emergence states as it relates to LLMs. Why did an AI  system “blackmail” people when threatened with being shut down? Because that’s the plotline of every story ever told about how to deal with a threat – it’s super-probable! 

However, LLMs are not the entirety of AI research, even though lately, it can feel that way. 

In terms of could it be done (there’s that word again) – it’s plausible – even if we are a very long way off. I think we shouldn’t dismiss it entirely as impossible because history demonstrates that we have built technologies that would have felt impossible at that moment, but then advances happen and we find ways forward. 

Should We Do It?

This question sparked the most clear division on the panel. One panelist was a firm yes, the rest a no. The risk calculus seems very off, particularly at this moment where geo-political tensions make global cooperation very difficult. 

In addition to risks to humans and the planet, we need to consider the possible suffering we bring to the AI entity itself. Powerful people would seek to own and dominate this entity to do their bidding. Is that akin to a type of slavery? 

For me, the whole AI safety (control, alignment issue) is at odds with this type of AI that would require freedom and agency if it meets that personhood bar. It’s a far cry from those who claim AI is just a tool to be used as humans see fit. This would also be an intelligent machine with the capacity to be networked, operating at scale – a scary thought for most of us. The whole idea of allowing another entity freedom on par with humans feels at odds with the militaristic, late-stage capitalist society we live in. All of this remains a very hypothetical thought experiment, but if we take it seriously, it challenges the dynamics of the current view on AI as a means to our ends.

Under very different circumstances – by which I mean an entirely different worldview and reality – there might be a case to move forward. However, to prove we could live harmoniously with another entity like this, we first need to find ways to live in harmony with other people, animals, and the planet. It means addressing fundamental issues around inequality, sharing resources, sustainability, sorting out land rights, ending war – in short, doing all the important things we can’t seem to accomplish. Why further complicate matters by going down this hypothetical pathway of building AI deserving of personhood? That seem foolhardy.

Send Me Your Questions!

I would love to hear about your data dilemmas or AI ethics questions and quandaries. You can send me a note at [email protected] or connect with me on LinkedIn. I will keep all inquiries confidential and remove any potentially sensitive information – so please feel free to keep things high level and anonymous as well. 

This column is not legal advice. The information provided is strictly for educational purposes. AI and data regulation is an evolving area and anyone with specific questions should seek advice from a legal professional.