PHILADELPHIA – When health care providers order a test or prescribe a medicine, they want to be 100 percent confident in their decision. That means being able to explain their decision and study it over depending upon how a patient responds. As artificial intelligence’s footprint increases in medicine, that ability to check work and follow the path of a decision can become a bit muddied. That’s why the discovery of a once-hidden through-line between two popular predictive models used in artificial intelligence opens the door much wider to confidently spread machine learning further throughout health care. The discovery of the linking algorithm and the subsequent creation of the “additive tree” is now detailed in the Proceedings of the National Academy of Sciences (PNAS).
“In medicine, the cost of a wrong decision can be very high,” said one of the study’s authors, Lyle Ungar, PhD, a professor of Computer and Information Science at Penn. “In other industries, for example, if a company is deciding which advertisement to show its consumers, they likely don’t need to double-check why the computer selected a given ad. But in health care, since it’s possible to harm someone with a wrong decision, it’s best to know exactly how and why a decision was made.”
The team led by Jose Marcio Luna, PhD, a research associate in Radiation Oncology and member of the Computational Biomarker Imaging Group (CBIG) at Penn Medicine, and Gilmer Valdes, PhD, an assistant professor of Radiation Oncology at the University of California, San Francisco, uncovered an algorithm that runs from zero to one on a scale. When a predictive model is set to zero on the algorithm’s scale, its predictions are most accurate but also most difficult to decipher, similar to “gradient boosting” models. When a model is set to one, it is easier to interpret, though the predictions are less accurate, like “classification and regression trees” (CARTs). Luna and his co-authors subsequently developed their decision tree somewhere in the middle of the algorithm’s scale.
“Previously, people used CART and gradient boosting separately, as two different tools in the toolbox,” Luna said. “But the algorithm we developed shows that they both exist at the extreme ends of a spectrum. The additive tree uses that spectrum so that we get the best of both worlds: high accuracy and graphical interpretability.”
In the study, the researchers found that the additive tree displayed superior predictive performance to CART in 55 of 83 different tasks. On the other end, gradient boosting performed better in prediction in 46 of 83 scenarios. While this was not significantly better, it does show that the additive tree was competitive while still being more interpretable.
Moving forward, the additive tree provides an attractive option for health care systems, especially for diagnostics and the generation of prognoses in an era when there is more demand for precision medicine. Furthermore, the additive tree has the potential to assist in making informed decisions in other high-stakes domains such as criminal justice and finance, where interpreting the models could help overcoming possible serious risks.
The researchers’ work was funded partially by the Emerson Collective and also by the NIH’s National Institute of Biomedical Imaging and Bioengineering (Grant K08EB026500).
Other authors include Efstathios D. Gennatas, Eric Eaton, Eric S. Diffenderfer, Timothy Solberg, and Shane T. Jensen, of the University of Pennsylvania; Charles B. Simone II of the New York Proton Center; and Jerome H. Friedman of Stanford University.
Penn Medicine is one of the world’s leading academic medical centers, dedicated to the related missions of medical education, biomedical research, and excellence in patient care. Penn Medicine consists of the Raymond and Ruth Perelman School of Medicine at the University of Pennsylvania (founded in 1765 as the nation’s first medical school) and the University of Pennsylvania Health System, which together form a $7.8 billion enterprise.
The Perelman School of Medicine has been ranked among the top medical schools in the United States for more than 20 years, according to U.S. News & World Report's survey of research-oriented medical schools. The School is consistently among the nation's top recipients of funding from the National Institutes of Health, with $425 million awarded in the 2018 fiscal year.
The University of Pennsylvania Health System’s patient care facilities include: the Hospital of the University of Pennsylvania and Penn Presbyterian Medical Center—which are recognized as one of the nation’s top “Honor Roll” hospitals by U.S. News & World Report—Chester County Hospital; Lancaster General Health; Penn Medicine Princeton Health; and Pennsylvania Hospital, the nation’s first hospital, founded in 1751. Additional facilities and enterprises include Good Shepherd Penn Partners, Penn Home Care and Hospice Services, Lancaster Behavioral Health Hospital, and Princeton House Behavioral Health, among others.
Penn Medicine is powered by a talented and dedicated workforce of more than 40,000 people. The organization also has alliances with top community health systems across both Southeastern Pennsylvania and Southern New Jersey, creating more options for patients no matter where they live.
Penn Medicine is committed to improving lives and health through a variety of community-based programs and activities. In fiscal year 2018, Penn Medicine provided more than $525 million to benefit our community.