New AI tool helps doctors to sift and synthesize patient data
An AI-guided platform at Penn Medicine allows clinicians to quickly and easily unearth pertinent information from patients’ electronic health records that otherwise might have been difficult to find.
Before a patient visit, many doctors have a single, deceptively complicated question: “Why is this person seeing me?”
Patients are complex, so doctors have a lot of homework to do for every person they see. Everyone, from an infant to a centenarian, has a unique medical history that constantly grows with wide-ranging data from things like test results, scans, family histories, prescriptions, treatments, and referrals, all documented and integrated over time in their health records. What led a patient to stepping into their doctor’s office could be a combination of all, some, or none of that data.
Enter Chart Hero, a new generative artificial intelligence-powered, chat-based system created at Penn Medicine’s Center for Health Care Transformation and Innovation (CHTI) in partnership with the Penn Medicine Data and Technology Solutions (DTS) team and clinical leaders in the health system.
Clinicians can ask Chart Hero, which appears like a sidebar in the electronic health record, to help them automatically gather, arrange, synthesize, and assist in the interpretation of all the pertinent information they’d need for a patient they’re going to see, and do it in a minute or two with a couple simple queries instead of digging through piles of data in multiple areas of the electronic health record.
A clinician using Chart Hero is like a paleontologist who can just summon the bones of a Tyrannosaurus, pre-assembled, out of the rock in front of them instead of spending years digging into the side of a cliff.
“It saves so much time that I would spend very inefficiently searching for an answer to my questions,” said Tessa Cook, MD, PhD, an associate professor of Radiology and vice chair of Practice Transformation in that department. “There was a day when it wasn’t available, and I thought, ‘I have to figure all of this out by myself? Are you kidding?’”
Simplifying ‘a big pain’
Yevgeniy “Eugene” Gitelman, MD, a clinical associate professor of Hospital Medicine, head of custom software at CHTI and associate chief health information officer, began working on Chart Hero late last year.
“A doctor reached out to me saying he had to summarize all of these notes and it was a big pain,” Gitelman said. “There’s just so much data in the charts, and everyone was saying ‘Why don’t we make it as easy as ChatGPT to find information?’”
What Gitelman and his colleagues created is effectively a conversational AI chat that has considerable search abilities while also safeguarding patient information to the exacting standards of HIPAA privacy laws. In addition to building its search and organizational functions, the team also made Chart Hero able to suggest, if asked, potential courses of action for patients that doctors can use their knowledge to review.
“There’s so much documentation happening across different teams, all these different people looking at the same things, but we haven’t been easily able to make use of the work that everyone has done,” Gitelman said.
The tool has been developed to be as friendly for clinicians to use as possible. That includes making it “smart” enough to determine when someone might be confused.
“You can ask it, ‘Who placed this G tube?’ and it will tell you who, but also clarify, ‘It’s actually a J-tube,’ which is a different kind of feeding tube,” Gitelman said.
When patients have multiple overlapping or similar conditions, the ability to sort through detailed information is also extremely valuable.
“If a clinician asks, ‘What is this patient’s oncologic history,’ it can break things apart and clarify that the patient has something like two separate cancers and, ‘Here is the course and treatment for each one, separately,’” Gitelman said.
Comprehensive data builds a comprehensive tool
Other health systems have taken swings at something similar, but Penn Medicine’s may be the most comprehensive, functional tool in large part due to a specific advantage it has—a unique system that covers a lot of ground and people across the health system. With data from populations that range from Lancaster through Chester Counties, into Philadelphia, and up north to Princeton and Doylestown, with practices dotted in all the areas in between, the data it was built on is diverse and complex.
“We have most of our offices and hospitals on the same platform, on the same set of pipes that have powered all of the other apps our teams have created for years,” said Srinath Adusumalli, MD, MSHP, MBMI, chief health information officer of the University of Pennsylvania Health System and an associate professor of Cardiovascular Medicine. “Chart Hero is built on data that’s diverse and extensive. And when you combine our team’s experience with that data advantage, an EHR-embedded product like Chart Hero is going to deliver deeper, more contextualized insights.”
Day-to-day value
To this point, much of the artificial intelligence employed in medicine has been in the realm of interpreting and assisting with radiology results—catching hard-to-see tumors on CT scans, offering better measurements of organs on MRIs to track disease progression, and the like.
Much of the AI employed in the realm of large language models also “lacks clinical context,” Cook said, not understanding how different medical conditions fit together or how they influence each other.
But Cook, the radiologist, said Chart Hero is something different. It seems to understand the clinic and bring what is important—and what might escape her otherwise—to the forefront almost immediately.
“I can just type, ‘Why is the patient here for a cardiac MRI?’ and get a summary of their cardiac history and work-up to date. Or I can ask, ‘Have they had gastric bypass surgery?’ Because sometimes it’s not obvious from their chart or recent imaging, but it’ll sure look like they did on their imaging today, and that’s important to know,” Cook said. “And Chart Hero can figure that out for me right away. That helps me and the patient.”
Cook estimated that, some days, using Chart Hero saves her up to two hours that would have otherwise been spent work on her patients’ charts.
Adusumalli also saw time savings and values Chart Hero’s speed.
“Just recently, I had a patient schedule at 8:30 a.m. for a 10:00 a.m. visit,” Adusumalli said. “But all I had to do was click one button in Chart Hero and then I knew everything I had to know. Otherwise, the nurse I work with would need 30 minutes to pull relevant information, and then I’d be clicking through three or four different tabs, and it’d take me 10 times the amount of work to get what Chart Hero pulled for me.”
Making patients’ care easier and more personal
The tool fits into Penn Medicine’s wider strategic effort to use technology to simplify patient care and create more time for meaningful patient interactions. As such, Chart Hero slots in comfortably with other initiatives like ambient listening, the AI scribe technology more than 1,200 providers are using across Penn Medicine to automatically take notes during patient visits—so they can focus on listening, not typing.
“Imagine you’re a clinician, seeing a new patient: You want to get up to speed, so you can use Chart Hero to generate a summary, and then that is effectively the seed for the visit,” Adusumalli said.
The summary Chart Hero helped generate then guides the visit while ambient listening takes comprehensive notes, adding to that original note.
“At the end, you can review everything quickly, and you’ve had a productive visit with a new patient where you’re not just tied to your computer,” Adusumalli said.
The team envisions Chart Hero or similar tools being two-way in the future: not just to be used by care providers, but the patients, too. They can arrive at their visit with even more clarity about their top concerns.
Then, Adusumalli envisions, “the tech melts into the background and we have a natural interaction.”
“This is technology supporting a visit end-to-end,” Adusumalli said. “And it makes it so we can just take care of people.”
Not just expansion, but improvement
Chart Hero is not yet employed across the entire health system, but its users are growing. In August 2025, there were 10 clinicians testing it. Now, 30 are helping to stress-test it.
As they work to expand its use, the development team is also working carefully to smooth out some of the challenges they’ve encountered with implementation. Like many AI systems, there is risk of hallucination, where an AI generates false, incorrect, or misleading information that it presents as fact. The development team is working to eliminate these hallucinations, with the human doctors testing invaluable for discovering and correcting them.
“To combat errors, we’ve made it so that the AI ties every statement to chart data and clinicians have one-click access to source material,” said Damien Leri, senior principal engineer at CHTI. “A secondary AI audit is also being tested to improve our tool’s accuracy.”
They view Chart Hero as an ongoing project, and human review remains a critical safeguard in validating outputs.
“We’re not just providing tech and walking away,” said Lauren Hahn, MBA, senior director of product management at CHTI. “[We ask:] How do we introduce this into clinicians’ workflows, as well as potentially redesign care delivery, in a way that is human-centered and easy to approach?”
She said that the multidisciplinary expertise of the team developing this, along with their experience scaling projects, is important because they know what it takes to make an enduring solution, rather than a one-off.
“Technology only matters if people are willing to use it and use it over time,” said Raina Merchant, MD, MSHP, executive director of CHTI and chief transformation officer of Penn Medicine. “With Chart Hero, our goal isn’t just accuracy; it’s creating something clinicians reach for because it makes their day easier and helps them take better care of patients. That’s the kind of change that is sustainable and lasts.”