PHILADELPHIA—Methods researchers developed to detect possible depression through language in social media posts don’t appear to work when applied to posts by Black people on social media, according to a new analysis by researchers from Penn’s Perelman School of Medicine and its School of Engineering and Applied Science. The research, published in PNAS, points to an area to focus on for significant improvement and amplifies the importance of considering the intersection of race, health risks, and social media.
Work in the past uncovered that using first-person pronouns in posts (“I”) and certain categories of words (self-deprecating terms and expressing outsider feelings) in social media posts was predictive of depression among people who use social media. However, in analyzing Facebook posts from more than 800 people—a sample that included equal numbers of Black and white individuals, some who reported having depression and some who did not—the researchers found that the predictive qualities of the “predictive” words applied mainly to white people on social media.
“We were surprised that these language associations found in numerous prior studies didn’t apply across the board,” said one of the study’s senior authors, Sharath Chandra Guntuku, PhD, a researcher in the Center for Insights to Outcomes at Penn Medicine and an assistant professor (research) of Computer and Information Science in Penn Engineering. “We need to have the understanding that, when thinking about mental health and devising interventions for treatment, we should account for the differences among racial groups and how they may talk about depression. We cannot put everyone in the same bucket.”
When the types of words identified in the past as predictive for depression were plugged into an artificial intelligence-guided model, the researchers found that it performed “strong[ly]” among white people. However, they found that the model was more than three times less predictive for depression when applied to Black people who use Facebook.
Even when the researchers trained the artificial intelligence (AI) model on language used by Black people in their posts, the model still performed poorly.
“Why? There could be multiple reasons,” said the study’s lead author, Sunny Rai, PhD, a postdoctoral researcher in Computer and Information Science. “It could be the case that we need more data to learn depression patterns in Black individuals compared to white individuals. It could also be the case that Black individuals do not exhibit markers of depression on social media platforms due to perceived stigma.”
Something that potentially confounded the existing depression-detection models, the researchers found, was that Black people tended to use “I” more overall in their posts. That includes participants in the study who did not report having depression.
And while words expressing self-deprecation (including “worthless” and “useless”) and feeling like an outsider (such as “weirdo” and “creep”) were associated with white people with depression, these groups of language were not specifically tied to depression in Black people.
Rai said that it’s clear that researchers need to increase representation of Black people and other races and ethnicities in studies to better understand the ways in which depression is expressed across different groups of people. Through this, the researchers hope that better predictive models can be established and mental health needs can be better addressed.
“AI-guided models that were developed using social media data can help in monitoring the prevalence of mental health disorders, especially depression, and their manifestations,” Rai said. “Such computational models hold promise in assisting policy-making as well as designing AI assistants that can provide affordable yet personalized healthcare options to citizens.”
Insights made through AI can also serve the education of professionals who help people manage depression.
"Understanding differences in how Black and white people with depression talk about themselves and their condition will be important when training psychotherapists who work across different communities,” said, Lyle Ungar, PhD, a co-author on the study and professor of Computer and Information Science.
In the vein of better understanding the differences in how mental health conditions are outwardly expressed, Guntuku, Rai, and Ungar are planning to study how depression is expressed in cultures beyond the United States.
The PNAS study was funded, in part, by the National Institute of Drug Abuse (ZIA-DA000628), the National Institute on Minority Health and Health Disparities (R01MD018340) and the National Institute on Alcohol Abuse and Alcoholism (R01 AA028032-01). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Penn Medicine is one of the world’s leading academic medical centers, dedicated to the related missions of medical education, biomedical research, excellence in patient care, and community service. The organization consists of the University of Pennsylvania Health System and Penn’s Raymond and Ruth Perelman School of Medicine, founded in 1765 as the nation’s first medical school.
The Perelman School of Medicine is consistently among the nation's top recipients of funding from the National Institutes of Health, with $550 million awarded in the 2022 fiscal year. Home to a proud history of “firsts” in medicine, Penn Medicine teams have pioneered discoveries and innovations that have shaped modern medicine, including recent breakthroughs such as CAR T cell therapy for cancer and the mRNA technology used in COVID-19 vaccines.
The University of Pennsylvania Health System’s patient care facilities stretch from the Susquehanna River in Pennsylvania to the New Jersey shore. These include the Hospital of the University of Pennsylvania, Penn Presbyterian Medical Center, Chester County Hospital, Lancaster General Health, Penn Medicine Princeton Health, and Pennsylvania Hospital—the nation’s first hospital, founded in 1751. Additional facilities and enterprises include Good Shepherd Penn Partners, Penn Medicine at Home, Lancaster Behavioral Health Hospital, and Princeton House Behavioral Health, among others.
Penn Medicine is an $11.1 billion enterprise powered by more than 49,000 talented faculty and staff.