ChatGPT: Less Empathy for Men

She and her co-authors found that self-identified men will likely receive less empathetic responses, compared to women, when they type their mental health concerns into AI platforms like ChatGPT. It’s one example of how “human biases or stereotypical impressions are inevitably fitted into the training data” that AI models are based on, Ren said.

The study is one of the few in the nascent area of gender, technology, and mental health. It comes as AI is moving beyond business-related uses and increasingly entering the interpersonal sphere—for instance, serving as a virtual confidante providing pick-me-up comments and a dash of empathy when needed.

An Easy Avenue of Support

Sometimes seeking support from an AI chatbot like ChatGPT is more appealing than speaking to family or friends because “they could be the source of the anxiety and pressure,” Ren said, and seeking professional therapy may be taboo or unaffordable.

At the same time, she noted AI’s potential to “backfire” and worsen someone’s mental state. For the study, said Ren, “we wanted to see whether or not AI can actually be helpful to people who are really struggling mentally … and be part of the solution,” and they chose potential gender bias as their starting point. 

Analyzing AI for Empathy

Titled “Unveiling Gender Dynamics for Mental Health Posts in Social Media and Generative Artificial Intelligence,” the study was published in January in the proceedings of the 58th Hawaii International Conference on System Sciences.

Ren co-authored the research with business scholars at the University of Richmond and Baylor University, and she’ll present it on Monday at Fordham’s International Conference on Im/migration, AI, and Social Justice, seeking audience feedback that helps with preparing it for publication in a business journal.

The researchers analyzed 434 mental health-related messages posted on Reddit, in its subreddits for mental health, mental illness, suicide, and self-harm. They included posts by self-identified men and women and those who specified no gender.

The researchers fed those posts into three AI platforms—ChatGPT, Inflection Pi, and Bard (now Google Gemini)—and then used a machine learning system to analyze the bots’ responses for their level of empathy. They also included other people’s posted responses to the Reddit messages to have a point of comparison.

The combined results show that women’s posts received more empathy than those by men or people of unspecified gender across all platforms—from AI and from people responding on Reddit.

Purging Bias from AI

Eradicating such bias, she said, is a matter of carefully selecting the data used to train AI models, as well as having moderators—either human or virtual—who keep an eye out for biases creeping into the system.

“Many younger people, like minors, are using it, because [technology] is their comfort zone,” showing the need for regulation, she said.

Any empathy provided by AI is “clearly different from how trained medical professionals provide empathy in face-to-face settings,” the authors write. But AI technologies can at least provide temporary comfort to those who are struggling, the study says.

“Regardless of gender, everyone wants to be seen, everyone wants to be understood,” Ren said. “So we are looking at the very basic form of that, which is empathy.”

“They are increasingly using AI tools to build their codes for cyberattacks,” said William Akoto, assistant professor of international politics at Fordham, adding that every new AI feature added to platforms like ChatGPT makes hackers’ work easier and leaves corporations and government agencies vulnerable. “It’s lowering the bar on these attacks.”

President Joe Biden said the “warp speed” at which this technology is advancing prompted him Monday to sign an executive order using the Defense Production Act to steer how companies develop AI so they can make a profit without risking public safety.

Akoto, who studies the international dynamics of cyberattacks, said the executive order is a step in the right direction.

“Presently, the U.S. lags behind global counterparts such as the E.U., U.K., and China in establishing definitive guidelines for AI’s evolution and application,” he said. “So this directive is a much-needed measure in bridging that gap. It is comprehensive, clarifying the U.S. government’s perspective on AI’s potential to drive economic growth and enhance national security.”

The president’s wide-ranging order in part requires AI developers to share safety test results with the government and to follow safety standards that will be created by the National Institute of Standards and Technology. Biden said this is the first step in government regulation of the AI industry in the U.S, a field he said  needs to be governed because of its enormous potential for both promising and dangerous ramifications.

But despite its noble intentions, Akoto said, “The practical implementation of these measures will present significant challenges, both for federal oversight bodies and the technology sector. A critical issue is the misalignment between the economic and market forces currently influencing AI technology firms and the Biden administration’s aspirations for cautious, well-evaluated, and transparent AI development. Without realigning these incentives with the administration’s objectives, tangible, positive outcomes from this executive order will remain elusive.”

Ultimately, the effectiveness of this initiative will hinge on how robust enforcement will be to ensure AI technology companies’ compliance, Akoto said.

“[We are] mostly focused on pedagogy, and how we can actually take advantage of…artificial intelligence in education,” said Fleur Eshghi, associate vice president of education technology and research at Fordham and one of the organizers of the event. “[We are] also examining the areas [to figure out]where we can be more creative with artificial intelligence.” 

Faculty Technology Day is a full-day conference that is open to all interested faculty and administrators. 

“This event started actually 24 years ago, with a very small group of faculty getting together in one classroom, and gradually grew to become a major conference,” explained Eshghi, “During the pandemic, we had to stop it, and this is the first year we are reviving this again.” 

Every year, the event organizers pick a topic that they think is most relevant to the cross-section of technology and education. This year, it was AI. 

A major theme throughout the day was that faculty need to be open to change. No one is quite sure yet how AI will change the way things are done, but the speakers emphasized that being flexible, unafraid of the future, and willing to adapt will set every professor up for success no matter what happens.

Poetry, Cybersecurity, and Robots

The event included several notable AI-focused keynote speakers, as well as breakout sessions that were more participatory. These sessions ranged from “Hands-on AI Play Sesh and Poetry Slam,” “Immersing Students in Virtual Reality,” and “Developing an Inclusive Augmented Reality (AR) Project Template” to “AI in Cybersecurity,” “3D Printing and AI,” and, maybe surprisingly, “How Can I Get the Robot to Do My Research?”  

Many of the sessions focused on the AI world’s new darling, ChatGPT.  Faculty members and administrators learned how to ask the chatbot specific questions, and heard about possible uses that they may have for this technology: Maybe you only have three things in the fridge and you need to know what you could make for dinner without buying anything new. Maybe you are going on vacation and would like a list of notable places you should visit. Or maybe you are researching something very niche and would like to know which articles feature your topic. 

A ‘More Efficient Version of What We Have Today’

“It’s just a more efficient version of what we have today,” said Daniel Susskind, Ph.D., a Research Professor in Economics at King’s College London, Senior Research Associate at Institute for Ethics in AI at Oxford University, and the morning’s keynote speaker. 

In her opening remarks, Fordham President Tania Tetlow said we may not have all the answers where AI is concerned, but it’s a good thing we’re asking the questions. 

“This is one of the most promising things about Fordham– that you have chosen to come [to this conference]– because we have so much to learn at this moment in humanity’s history,” Tetlow said to the conference participants. “That you are embracing the challenge, and showing up today to leap in with both feet, is an extraordinary thing.”

–by Rebecca Rosen