“When will this end?” is the question that Michael Z. Levy, PhD, is most often asked these days.

Levy, an associate professor of Epidemiology in the Perelman School of Medicine, is an expert in disease ecology — meaning he studies the ways in which pathogens spread through space and time. He and other researchers around the world have been gathering as much data as they can get their hands on, making calculations, and creating mathematical models — all in an effort to answer critical questions about COVID-19.

Epidemiologists and data scientists have become unlikely heroes during the coronavirus pandemic. Wave-shaped COVID-19 models that present “best” and “worst-case” scenarios for the virus trajectory are now commonplace in news stories and on screens behind governors during press conferences. “Flatten the curve” is scribbled on signs that hang in neighbors’ windows. Local governments are using algorithms to make decisions about stay-at-home orders and supplies.   

During a time of unprecedented uncertainty, it makes sense that we would put stock in these models. We are hoping they will be able to forecast the future and offer a glimmer of sunshine. 

But math cannot account for the unpredictability of human behavior, and epidemiologists are not in the business of fortune-telling, Levy cautions.

“A model isn’t a crystal ball to make predictions,” he said. “It’s more like a pensieve — you take what you already have in your brain, you pull it out, and you swirl it around, so that you can better understand the ramifications of your assumptions. That’s all these models are for — to get our head around what’s already going on and what we can do about it.”

How Far Can You Trust a Model’s Assumptions?

In the case of COVID-19, the virus’s behavior is particularly hard to predict. The first problem, Levy says, is that an infectious disease outbreak will always grow exponentially in its beginning stages, and therefore, scientists operate on that assumption when making their projections. However, the true spread of the coronavirus is impacted by dozens of different factors, including how quickly people can be tested and quarantined, how well populations are able to social distance, and whether those infected later become immune to the disease — all important determinants that are difficult to quantify.

For example, one type of SIR model — which splits a population into susceptible, infectious, and recovered individuals — has become a commonly-cited exponential projection of COVID-19. The early models that have been adopted by many health departments and health systems are useful for making very short-term predictions, Levy says. For instance, how many ICU beds will a particular hospital need next week?

“But don’t use it to play the stock market, or don’t even use it to estimate when the epidemic will peak,” Levy said during a presentation at Penn’s recent Coronavirus Outbreak Symposium. “That’s not what it’s made for.”

Why not?

Most of the early COVID-19 models operate on a “mass action” assumption, which means that they assume every individual is “bumping into” everyone else equally — “like billiard balls in a perfect vacuum,” Levy says. (Though he concedes that the analogy is not ideal. “I think it’s probably from a high school physics textbook.”)

But that’s not how people behave — especially now. Some states enacted stay-at-home orders weeks ago, while Florida’s governor did not issue one until early April. Doctors, nurses, and other essential workers are obviously more exposed to the virus, but other factors that affect social distancing — like income level — the mass action SIR model is also unable to account for.

“That kind of model is fine for short-term predictions, but it becomes problematic when you ask different questions,” Levy said.

Differentiating Effects of Social Distancing

Network models, by contrast, may be better at answering the question: “When will we see the effects of social distancing?” Though they’re not perfect.

These types of models (which look like giant balls of yarn connected through pegs) are able to reveal the effects of the variation in how we are connected to one another. When compared to mass action models, they will typically predict a smaller epidemic, which will affect different populations disproportionately. Those who are more connected are more likely to be infected, and those who are more isolated will be spared.

And we’re already seeing this prediction play out in U.S. cities. In Philadelphia, which had racial and ethnic data on 66 percent of confirmed positive cases of COVID-19 as of April 14, 40 percent of positive cases are black, 14 percent are white, and 6 percent are Hispanic, in a city where around 30 percent of black individuals are poor. Nationwide, black Americans are “overwhelmingly dying” from the virus.

This might be due, in large part, to the poor population’s inability to socially distance. According to a New York Times analysis of cell phone data, those in the top 10 percent of income have limited their movement more than those in the bottom 10 percent of the same metro areas. By March 16, when President Trump asked people to stay at home to slow the spread of the virus, those in the highest-income locations had already cut their movement by nearly half. Poorer areas did not see a similar drop until three days later.

This variation in social distancing — how soon different people are able to distance themselves from others, and to what extent — paired with the fact that most virus transmission is taking place within households, makes tracking and predicting the trajectory of COVID-19 extremely difficult, because we don’t have complete information about where and how that variation is occurring. 

“The chains of transmission that have already initiated will continue to play out. The virus is already through the door,” Levy said. “It could take over two weeks to see the effects of [today’s] social distancing and we still don’t have a very good way to measure and quantify that effect. We’re really making those methods up on the fly.”

Models as a Mental Aid

If the current epidemiological models we have are not necessarily a useful road map for what lies ahead, what purpose do they serve?

To answer that, Levy continuously points back to something that Robert Ross, another infectious disease researcher, said over 100 years ago: “Such calculations…are useful, not so much for the numerical estimates yielded by them, but because they give us more precision to our ideas, and a guide for future investigations.”

A model, Levy posits, is only an argument. It gives provisional answers to different kinds of what-if scenarios, which we can then examine and debate.

“Most of the time it’s an argument with ourselves.”