An Innovative Penn Team Believes Tracking Patients’ Digital Footprints on Social Media Could Transform the Future of Personalized Medicine
By Carole Bernstein
Photos by Tommy Leonardi
In Facebook posts, Tweets, Instagram photos, and other social media, we continually leave clues about ourselves. Our words and images tell about our friends, families, and children; our business trips and where we vacationed, how we celebrated the holidays, what parties we attended; recent illnesses, political views, what we find funny or thought-provoking. We even say what we ate for breakfast or what movie we watched last night. And on popular sites like Yelp and TripAdvisor, we report detailed personal experiences with businesses, health care facilities, restaurants, and hotels.
We’re leaving a digital trail that leads to who we are and how we live our lives.
Advertisers and marketers have been hot on that digital trail for a number of years now, trying to find out as much as they can about our behavior in order to target their products and services. Could similar techniques be used to help patients and doctors better understand health and disease, and to improve medical care?
That’s the intriguing question posed by Raina Merchant, MD, MSHP’09, and her interdisciplinary team at the Penn Center for Digital Health.
The Center for Digital Health initially launched in 2013 as the Social Media and Health Innovation Lab. It is associated with the Penn Medicine Center for Health Care Innovation in partnership with the Leonard Davis Institute of Health Economics. Merchant, the center’s director, is an assistant professor of Emergency Medicine with secondary appointments in General Internal Medicine and Anesthesia and Critical Care.
Merchant runs the group like a startup, and she encourages studies of the connections between social media use, technology, and health with a broad, unique perspective, according to Elissa Klinger, SM, the center’s assistant director. Though social media and digital media are trendy topics, the lab is taking a long view of what it can achieve. They are already marking successes along the traditional metrics of research productivity, including grant awards from the National Institutes of Health and publications in high-impact journals. But they hope that, just as social media itself has transformed our social connectivity, their research will be a transformative force in the future of personalized medicine.
Mining the “Social Mediome”
In ongoing studies, Merchant and her team are comparing individuals’ social media posts with their electronic medical records to discover patterns in how people with already-diagnosed disorders talk about their lives online. “Our focus is on how we can learn by listening to patients through digital means,” she says.
The center refers to the research as “the social mediome,” which Merchant describes as a way of collectively describing people or groups based on their digital data merged with their health record data. The “–ome” suffix, she explains, is to suggest its place among the other –omes currently being explored under the umbrella of precision medicine: “the genome, the microbiome, the metabolome, the proteome.” (For more on this idea, see sidebar, “The ‘Social Mediome’: Comparable to the Human Genome?”)
Just as researchers need patients’ consent to sample and study their genomes, the team needs to get permission to collect and analyze social media data to study their health. A landmark finding of the Social Media and Health Innovation Lab was that this consent could, in fact, be obtained, and on a significant scale. In 2015, the team reported in a BMJ Quality & Safety paper that over 1,000 patients in the Penn Medicine health system had agreed to donate their Facebook data along with access to their electronic health record. More than 4,800 participants have now signed on, and the team is studying Twitter and Instagram activity as well. Some of the study participants the lab has recruited have even allowed access to their Google search histories.
“Most people do allow us to look in these ways for research purposes. That in itself is an interesting finding, because there’s so much attention being paid to privacy. And rightfully so,” notes David Asch, MD, MBA’89, GME’87, a Penn professor of Medicine, Medical Ethics and Health Policy, and executive director of the Center for Health Care Innovation, of which Merchant’s lab is a part. Asch is a co-founder of the lab with Kevin Mahoney, executive vice president and chief administrative officer for the University of Pennsylvania Health System.
When patients are concerned about privacy, this must be respected, Asch acknowledges. He points out, though, that some people find therapeutic value in blogging about a journey through cancer treatment, for example, or they discuss their health problems on Facebook or Twitter, “which is about as un-private as it gets.”
Raina Merchant, MD, MSHP'09,
and David Asch, MD, MBA'89,
GME'87, collaborate on research
on the "social mediome" to understand
and improve health, on
using online reviews to improve
hospital quality, and more.
Now that they have permission to collect data, the team is in an exploratory phase, where they are thinking about ways to make sense of all that information. Some intriguing connections have already come to light.
For example, people who are documented in their health record as being overweight have a common theme in their Facebook activity. You might guess it’s food, but it isn’t. It’s being sedentary. “They’re all about relaxing and chilling out,” Asch says. Another finding involves patients with hypertension: they tend to post about their families, which, Asch says, the team found surprising, despite the standard joke about your kids giving you high blood pressure.
In looking at patients with depression, the team has in fact found more topics overtly related to sadness than in people without this disorder. But according to Merchant, there may be other words and phrases linked to depression which are not as obvious. “This work has really been enabled by collaborating with computer scientists at Penn who have expertise in natural language processing, deep learning, and mental health” like Lyle Ungar, PhD of Penn Engineering, Merchant says.
A new area for the team is studying social media to trace language changes that may be associated with Alzheimer’s or other types of cognitive decline. For this effort, they are partnering with Penn Memory Center Co-Director Jason Karlawish, MD, a Penn professor of Geriatric Medicine, Medical Ethics and Health Policy, and Neurology.
“If there’s one organ that is uniquely tied to technology, and where the use of technology is affected early, it’s the brain,” Karlawish notes. A person’s use of social media could become a way to detect if they are experiencing changes in brain function.
He says that based on existing research into cognitive decline, one might see the affected person’s writing become less syntactically complex, and shorter overall. Another change might be that the individual simply posts on social media less and less as their condition worsens.
Merchant says the plan is to study the posts of caregivers as well as patients. Karlawish explains that the team will track changes in the emotional tone of caregivers’ social media. Such changes could signal the onset of the anxiety and depression that often plague people caring for family members with cognitive decline.
Of the collaboration, Karlawish comments that Merchant reaches “way outside the comfort zone of things she’s studied,” which he says is a “great testament” to her skills as a researcher.
Your Social Media Health Dashboard
An ultimate goal of collecting and analyzing all of this information is to provide useful reports to both patients and providers, which could be used in a preventive or diagnostic way. Such a report could become a standard part of the annual primary care visit.
“We don’t envision that we need to know that patient X posted on Facebook,” Merchant clarifies. “That would be overwhelming. Or that they liked something on Facebook.”
Instead, she says, the idea is that the doctor would look at a dashboard summary of what someone’s posts had been about over the past six months or so, perhaps along with the sites they’d visited and the questions they’d asked on Google. Merchant points out that this could jumpstart and add to the conversation between doctor and patient that typically begins, “So, what brings you in today?”
Such a report could spark conversations that truly address patient-generated data and not just data that patients fill out on forms during a clinical visit, Klinger notes. It could also promote shared decision-making between patient and provider. “Providers don’t have a lot of time to engage with patients, to go through very detailed discussions about their social and behavioral lives,” Klinger says. “Hopefully, these kinds of tools might help.”
The team is weighing ways that empowering patients with specific knowledge about how social media relates to health could offer further benefits. For example, preliminary findings by other researchers suggest that people who are not depressed tend to post photos with more vibrant colors and fewer muted hues compared to those who are depressed. Could patients and their loved ones someday benefit from learning about such trends from their physicians? “This work suggests that changes in image filters or treatments could suggest changes in mental health and signal to an individuals’ network potential decline or need for a check-in,” Merchant says.
In December of last year, Merchant became one of the eight inaugural recipients of an accelerator grant from the Penn Center for Precision Medicine Accelerator Fund. Her project is titled “Precision Medicine and Digital Media: Exploring Applications for Clinical Care.”
“We received that funding—which we’re really excited about—to think about how do we move beyond just the prediction to the clinical applications.” She says the focus will be on figuring out how to give social media analyses back—in an easily digestible form—to patients and providers, and discover whether they find that information useful and actionable. The team also plans to use the grant to explore more ways to recruit and connect with patients through existing Penn health system portals.
The “Social Mediome”: Comparable to the Human Genome?
“Social mediome” is the term used by experts in the Penn Center for Digital Health to describe the study of how people’s social media activity may yield clues about their health. Asch notes that the “–ome” suffix is an echo of “genome.” Could their work yield information similar to investigating the human genome, which has brought us revolutionary diagnostic and therapeutic tools?
“I think there are lots of parallels,” Asch says. He asserts that like the genome, the social mediome may reveal an enormous amount of information about individual health and disease. “And the social mediome is observable in ways that are actually easier than the genome,” he points out. There is no need to biopsy the patient or extract a DNA sample: all that is required is the individual’s permission, which can be obtained remotely without a visit to a medical facility.
One key difference, Asch says, is that while someone’s genetic makeup may cause them to develop a disease, their use of social media doesn’t. “We don’t actually think there’s something about tweets, for example, that causes heart attacks or cancer.” While there is good reason to believe the genome mediates health and disease, the social mediome is merely a marker for it. Nevertheless, he cautions, don’t underestimate that marker—it may turn out to be “incredibly predictive.”
An individual’s social mediome could be used almost like a screening test in preventive care. For example, if we know that “people who tweet about X also tend to have a higher incidence of depression, obesity, hypertension,” or other conditions, we and our doctors could be better attuned to the need for disease-specific preventive efforts. Asch theorizes that in addition to diagnostic uses, the social mediome might someday be used therapeutically: to correct behavioral errors the way gene therapy is used to correct harmful genetic mutations.
“I don’t know if any of this would work—it’s sort of new frontier stuff—but you could imagine [for example] for people who have misconceptions about the risks of heart disease, you could communicate back the truth, that cigarettes are bad for you.” Social media is bi-directional, he says. We can tweet at someone and send messages to them.
The messages could be sent from one’s physician, or from a social network, which he says have already been shown to be useful in the area of health. Perhaps, he suggests, they could be harnessed to create support groups for people with cancer, or give people advice on healthy eating.
However incongruous we may find the connection between our spur-of-the-moment, even humorous, social media posts and the researcher’s realm of useful health information, Asch points out that there is a vital pathway connecting them: “No matter how much we might have a biomedical view of disease, for so many conditions human behavior sits on the final common pathway between health and disease.”
He cites examples of widespread risky behaviors such as people failing to take prescribed lifesaving medicine, or not bothering to use the seatbelts provided in their cars. Regardless of the genes you’re born with, it’s behavior that can often make the difference between health and disease, life and death.
Merchant, at center right, collaborates with Center for Digital Health team members (L to R) Jesse Goldshear, MPH, Elissa Klinger, SM, Emily Seltzer, MPH,
Jeremy Asch, and Remi Gurak. A broad range of disciplines, including medicine, design, engineering, and business, contribute to the team's innovations.
Could Yelp Reviews Help to Improve Hospital Quality?
“Let’s say you’re a patient and you have a really great experience with your hospital stay,” Merchant says. “You might, ten years ago, have sent a letter to the department or the provider.” Same with a really negative experience, she says. You wouldn’t have any way to broadcast that letter so that other patients, or your friends, or your neighbors, could know about your experience. And you wouldn’t know what happened with that letter. Today’s online reviews have made that a whole different ball game.
In addition to her social mediome research, Merchant has been studying online reviews as a way of listening to patients to discover what really matters to them. What makes people post things in the first place, or to say the experience was a one-star versus a five-star?
She is well aware that Yelp reviews aren’t perfect: “They aren’t validated, and there are all these caveats… but we think there’s a really important opportunity to learn from patients through these channels.” For example, she points out, in Yelp reviews individuals volunteer unscripted information rather than responding to canned survey questions like those patients receive through Press Ganey surveys following doctors’ appointments and hospital stays.
Merchant and Asch recently studied 16,000 Yelp posts about U.S. hospitals and compared them to HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Systems), the traditional government-based survey related to hospital quality that is sent to patients. The team found that people on Yelp talk about all the areas that HCAHPS asks about, but also plenty of areas that it doesn’t ask about. The twelve areas they counted include the cost of the visit, insurance and billing issues, ancillary testing, amenities, the compassion of the staff, the quality of nursing, and more. The results were published in Health Affairs in April 2016. Merchant, Asch, and Kevin Volpp, MD’98, PhD’98, a professor of Medicine and Health Care Management in the Wharton School, also published about the potential value of online reviews in JAMA in December 2016 as “Learning by Listening—Improving Health Care in the Era of Yelp.”
One aspect of the Yelp study that Merchant finds intriguing is the ability to hear from caregivers who may contribute to online reviews themselves, although they are never included in the standardized surveys sent to patients. Merchant believes they are incredibly important. “They’re often the ones who will remember the most about the experience, because they weren’t under the influence of pain medications, they weren’t in pain, they weren’t as afraid or nervous, or all these emotions that you go through as a patient,” she says. “But we don’t traditionally ask them about their experience.”
Using Crowdsourcing to Restart the Heart
Even before the founding of the Penn Social Media and Health Innovation Lab, ideas were spinning in Merchant’s brain about how digital technology could be used to improve health care.
“I was excited by this problem that AEDs, or defibrillators, are these lifesaving devices that are located all over the place in public locations, and yet no one knows exactly where all of them are,” she says. (You’ve probably walked past many of these devices without noticing them, in shopping malls, gyms, airports, casinos, recreation centers, and other locations.)
An emergency medicine physician, Merchant points out that these devices can flip the odds of survival after sudden cardiac arrest from less than two percent to better than one in two. The devices are equipped with audio instructions for non-medical professionals, she notes. “You open the device and it starts talking to you and tells you how to use it.”
“AEDs are an essential part of the ‘chain of survival’ that's necessary to save cardiac arrest victims,” Merchant noted in a 2012 Penn Medicine press release. “Despite thousands of [devices] in the community, our results show they are usually not readily available during cardiac arrests. Without an AED, the minutes bystanders spend waiting for paramedics to arrive could mean the difference between life and death.”
Thus the MyHeartMap Challenge, which Merchant launched in 2012 to improve awareness of, and access to, these lifesaving portable devices. She partnered with people across Penn, including at Wharton, and several different schools of design and others to address the challenge of finding these devices so people could use them.
The team developed a mobile app and promoted a contest using crowdsourcing. The contest offered $10,000 to whoever could find, photograph, and supply the location of the most AEDs in Philadelphia County over a six-week period. Participants submitted the required data via the project website or mobile app.
The challenge garnered widespread media coverage including NPR, CNN, the Wall Street Journal, and many local Philadelphia news outlets. The university’s “Frankly Penn” blog ran a story at the time that described the contest in a nutshell: “Want to Win $10,000 and Save Lives By Using Your Cell Phone?”
The contest resulted in submissions of thousands of AED locations in Philadelphia, which were then validated and mapped by the team. “Before, there were close to 50 devices which were catalogued in the city, [but] we found over 1,400 of them. We were able to validate 99 percent of these devices, and we created a mobile app that people can use to report [where they found additional] devices and to locate them in an emergency.”
The MyHeartMap team continues to accept electronic submissions of AED locations toward the implementation of a nationwide database.
Leading in Precision Medicine
Not every emergency medicine physician branches out to take on social media and digital health analysis. Merchant may be the first. Her unorthodox foray into social media and digital health research owes its origins to the MyHeartMap Challenge, a project built on her expertise caring for and studying patients who’ve suffered from cardiac arrest. Merchant directed this innovative crowdsourcing project at Penn in 2012, aiming to raise public awareness and access to automated external defibrillators (AEDs) to boost survival from this leading killer. (See sidebar, “Using Crowdsourcing to Restart the Heart.”)
“We experimented with using social media in different ways for the contest,” she says. “That got me really excited about understanding more about how networks work, how to access people, how people share information, and their motivations.” She also was inspired to think about other ways to use social media for good.
Merchant was recognized by the Robert Wood Johnson Foundation in 2012 as one of ten young investigators “likely to have a significant impact on the future of health and healthcare in the U.S.”
Asch characterizes her as “brilliant, charismatic, high-energy, and fun.” Klinger describes her as “extremely innovative.”
All three believe that a key strength of the Penn Center for Digital Health is the diversity of the team—spanning from the Perelman School of Medicine and the Wharton School to the School of Design, School of Engineering and Applied Science, and Annenberg School for Communication—and from emeritus faculty to Penn undergraduates. (There were even two high school students who came on board to help last summer: “they were amazing,” Merchant says.) Asch credits the union of different intellectual viewpoints largely to Merchant’s vision and leadership.
The team’s work is intimately connected with health outcomes for Penn Medicine patients as well, according to Mahoney, the Health System executive vice president who co-founded the group with Asch in 2013. “The center is an important initiative for the health system as we focus on how new technologies and platforms can enhance our ability to understand and improve individual and population health behaviors and outcomes, as well as improve health care management and operations,” he says.
Both Merchant and Asch note that Penn is largely unique in its work on social media and individual health. “To the best of our knowledge, Penn is the only place really building a social mediome, where we are [collecting] digital footprints to validate or compare with data from your health records,” Merchant says. Asch agrees and adds, “I think it’ll be even more exciting as the field expands… Because of the uniqueness of the center, [it’s] beginning to define the field of social media, digital media, and health.”
According to Klinger, the center’s research cuts through the “buzz” around Big Data and predictive analytics “to start to make sense of this data in a research-oriented, knowledge-generation way.”
For Merchant, the lab’s research is in keeping with the nationwide focus on precision medicine as transforming the future of health care. “The precision medicine initiative really calls for being able to collect data that’s more personalized, and that includes mobile data, online data,” she says. “And we want to be part of the group that cracks this: that figures out what promise this data may hold.”
Sign up here to subscribe to the email edition of Penn Medicine magazine.