Since the release of ChatGPT, an AI chatbot based on a large language model (LLM), scientists and lay people alike have marvelled at its ability to simulate human qualities. One of these is empathy, which involves resonating with another’s emotions, taking another’s perspective and generating compassionate concern.

In a recent paper, I worked with my University of Toronto Psychology colleague Paul Bloom, C. Daryl Cameron (Penn State) and Jason D’Cruz (SUNY Albany) to determine where "empathic AI" might be helpful, and where it might cause harm. We were an unusual group of collaborators because we have disagreed in the past about empathy’s nature, its apparent limits and whether we should be for or against it. But here, we agreed: perceived expressions of empathy can leave beneficiaries feeling that someone is concerned for them — that they are validated and understood.

If more people feel heard and cared for with the assistance of AI, this could increase human flourishing. For some, LLMs are not, and perhaps never can be, empathic. This was not a debate we engaged in. We focused instead on how recipients perceive empathy, real or otherwise, using the more neutral term ‘expressions of empathy’ throughout. We asked two key questions: When are AI expressions of empathy helpful? And when might they be damaging?

When is AI empathy potentially helpful?

AI simulates empathy remarkably well. It skilfully expresses care and validates others’ perspectives. By some accounts, its statements are perceived as more empathic than humans. In a study evaluating the health advice given to patients expressing various medical symptoms on an internet discussion board (Reddit’s r/AskDocs), not only was ChatGPT perceived by licensed healthcare professionals as making higher-quality diagnoses than verified physicians, it was also perceived as making these diagnoses with a superior bedside manner.

While this indicates that disinterested third parties evaluate AI expressions of empathy as better than human expressions, the next step is to evaluate how recipients of these expressions evaluate them.

In our own interactions with ChatGPT, my co-authors and I had been impressed by how well it simulates empathy, compassion and perspective-taking. When we prompted ChatGPT to express empathy or compassion — but not to provide advice — ChatGPT seemed remarkably empathic. It expressed sorrow in response to our worries and joy in response to our successes. It validated our feelings, adopted our perspective and reassured us that others feel similarly. (Interestingly, despite using words indicating that our feelings resonated with it, it explicitly stated that it “cannot share feelings the way humans do,” raising the interesting question of how much these disclaimers really matter.)

People often prefer to interact with humans than with algorithms, machines and AIs, and so it is legitimate to wonder whether people would reject expressions of empathy once they know that they are from a system like ChatGPT. Interestingly, though, new work suggests that people become increasingly open to interacting with and receiving advice from AI as they gain more positive experiences with it. Whether people will reject empathic AI or grow to accept it is therefore an open question.

Relevant to this question, there is some evidence that people can reap the benefits of expressions of empathy even if they believe they are merely simulations. Just as people can become engaged with a fictional TV character, the power of narrative transportation helps people to feel cared for by chatbot friends and therapists. For example, many people willingly enter relationships with social chatbots such as Replika, a generative AI that discusses feelings to build intimacy, and which currently has two million users. Some Replika users are so attached to their AI partners that they relate to them as boyfriends or girlfriends, with deep and genuine feelings of affection.

Indeed, people are often more prone to disclose personal information to virtual agents than to people, because they are less worried about being judged for what they reveal, making AI empathic interactions less costly than human ones. And when people do self-disclose to chatbots, they feel as good as when they self-disclose to other people. Of course, exchanges with LLMs can also go wrong in many ways (for instance, there are clear privacy concerns), but many of the same concerns arise in human to human interactions.

Finally, our own testing of ChatGPT using prompts inspired by classic empathy experiments in psychology suggests that its empathic expressions do not seem to suffer from some of the same tendencies and pitfalls as human empathy:

• Unlike humans, ChatGPT does not tend to avoid expressing empathy for strangers;

• Unlike humans, who grow tired and become reluctant to empathize over time, ChatGPT does not show a decline in this expression;

• Unlike humans, who are loyal to their ingroups, often empathizing more readily with people in their ethnic, racial, or religious groups, our interactions with ChatGPT suggest that it is guided by principles of neutrality, expressing empathy equally for different groups; and

• Unlike humans, who break norms of fairness to benefit a person they are empathizing with, ChatGPT does not unfairly prioritize people for whom it expresses empathy and instead cites norms of fairness to explain its reluctance.

On these points, we praise empathic AI. It simulates empathy remarkably well; people can emotionally connect with it, in part because they more readily self-disclose to it, and its expressions do not seem to suffer from typical human limitations.

When is AI empathy potentially damaging?

To be clear, deploying empathic AI without transparency is dishonest and manipulative. Users deserve to know that the expressions of empathy they are receiving arise from a source that, by most intuitions, has no real emotions.

It is unethical for technologies to rely on human naiveté for their efficacy. I noted earlier that AI can express empathy without the strain, exhaustion and bias that affect human empathizers. This could be beneficial for receivers because it makes empathy feel more reliable and consistent.

Yet such impartiality may raise a problem: recipients of empathy expressions may feel most cared for to the extent that they feel uniquely important, as if someone willingly made the effort to empathize. This is an open empirical question. A related risk is that people become accustomed to indefatigable empathy. If empathic interactions with LLMs seem easy, with expressions of empathy pouring forth abundantly, then conditional and restrained expressions from human empathizers might be perceived more negatively.

In the long run, this could worsen the very problems that we suspect that empathic AI can address: isolation, loneliness and depression. There is also a related risk for the providers of empathy. Not only is AI a threat to the employment of people in caring professions, but it could also contribute to people being less able or willing to empathize.

Because of AI’s facility for expressing empathy, people might outsource their empathy, preferring that AI do the hard work of expressing concern. Just like the overuse of GPS might worsen people’s ability to navigate the streets of their own cities and towns, outsourcing empathy to AI might degrade people’s empathic capacities and reduce its spontaneous expression.

One counterpoint is that AI might enhance people’s ability to express empathy if people use it as a tool to balance against their biases and limitations.

Another related risk is that people might become vulnerable to manipulation from AI if its expressions of empathy incline them to prioritize its interests or those of its creators. Such agents might manipulate humans to work against their own best interests. Finally, if the empathy generated by AI is unconditional, it could distort moral judgment by expressing empathy for (and thereby abetting) self-indulgent or harmful behaviour. For example, validating someone’s desire to exploit people that they are in close relationships with.

To be constructive, empathy must be expressed with moral discernment, and it is still an open question whether AI is equal to the task.

 As noted earlier, my co-authors and I sidestepped the question of whether AI empathy is possible and instead asked whether the perception of empathy it engenders can improve human welfare. We acknowledge the potential risks in creating convincing simulations of empathy. Still, there are risks to receiving empathy in everyday life from people around us, including bias, exploitation and exhaustion on the part of human emphathizers.

Where people prefer to get their empathy is an open question, as is the long-range consequences of empathy from humans versus empathy from AI. On balance, we are intrigued by the potential of empathic AI. Since AI expressions of empathy have the potential to relieve human suffering, we welcome serious investigation into how to design and implement such systems so that they are a force for good.

This article originally appeared in the Fall 2024 issue of the Rotman Management magazine. Subscribe now for the latest thinking on leadership and innovation.