Yelp Ratings of Healthcare Facilities May Reveal Death Rate Disparities
By Carol Davis
U.S. counties with healthcare facilities with the greatest share of 1-star Yelp reviews had the highest death rates, and a difference of just one point—roughly one star—between counties’ average scores could indicate a mortality rate that is better or worse by dozens of lives, according to researchers at the Penn Medicine Center for Digital Health.
Yelp is a review website that uses a five-star rating system to evaluate businesses, with one start rating the lowest and five stars rating the highest.
“Many of the facilities that provide essential care may not otherwise have standardized measures or approaches to collect data about patients’ experience of care. This is a missed opportunity,” the study’s senior author, Raina Merchant, MD, director of the Center for Digital Health and a professor of emergency medicine in the Perelman School of Medicine at the University of Pennsylvania, said in a press release.
“Much of the focus in healthcare is on quality and outcomes,” she said. “Patient experience is also critically important and should be factored into how to improve care across the board. This appears to be one novel data source for doing that.”
More than 95,000 facilities that provided some form of care recognized by the Affordable Care Act were included in the study led by Merchant and its lead author, Daniel Stokes, MD, a researcher with the Center for Digital Health and an internal medicine resident at UCLA Health.
Each facility included in the study had at least three reviews between 2015 and 2019 on Yelp. Each healthcare facility’s ratings were also coded to the specific U.S. where it was located, resulting in more than 1,300 counties—roughly a third of the country—being represented in the study.
Overall, healthcare facilities achieved an average 2.9 score out of 5 stars, but reviews were weighted very heavily to either side of the scale: five-star reviews account for 52.9% of all reviews, while one-stars made up 33.3%.
But when researchers looked at the county-level data of reviews, they found that five-star reviews within the group with the lowest death rates made up 55.6 percent of their total, while one-star reviews were at just 29.1 percent. In the group of counties with the highest death rates, five-star reviews made up only 42.9 percent of the total, compared to 38.8 percent one-stars.
The researchers then determined that if a county’s health facilities’ reviews were a star higher than their average—one point on the scale—models indicated that it translated to 18 fewer deaths per 100,000 residents. But when the study was refined to include counties with three or more healthcare facilities, the impact was greater, indicating a reduction in roughly 53 deaths per 100,000. Refined even further to counties with five healthcare facilities or more, the impact grew to about 60 preventable deaths.
Yelp reviews provide narratives, which was particularly useful to the study. The researchers used natural language processing algorithms to gain special, categorical insights. They showed that the types of words most associated with one-star reviews related to time [such as “hours” and “waiting”], payment [“money” and “pay”] and interpersonal interactions [“rude” and “told”].
Common language in five-star reviews changed depending on location. In high-mortality counties, “friendly,” “nice,” and “staff” were all typical, while low-mortality counties were associated with “Dr.,” “helpful,” “question,” and “pain.”
The researchers believe their work, published this week in the JAMA Network Open, adds evidence that unfiltered online repositories like review sites and social media contain valuable patient feedback and are an untapped resource for informing healthcare providers about what they do.
“Online reviews of healthcare facilities provide direct insight into patients’ experiences of care and can be a powerful force in shaping the care we provide to be more patient-centered,” Stokes said. “This has important implications for both individual and community health.”
Carol Davis is the Nursing Editor at HealthLeaders, an HCPro brand.