Most people choose their doctor first, not their hospital. But there are times when you want to be sure the hospital your doctor uses is highly ranked, especially for the type of care you’ll be receiving.

As most of you know, I’m an orthopedic surgeon. In the field of orthopedics, there are many specialty areas and many specialty hospitals. Finding the best hospital for the specialty you need is important. You want a hospital that does a lot of the procedures you’re facing; not a place that only does that procedure once in a while. And you want a hospital that has a proven track-record of success with a low complication rate. How do you find such a hospital?

If you want to find a good car, you’ll probably check Edmunds or Kelley Blue Book. When you want to take a trip, you’ll probably check the ratings of Trip Advisor. Until recently, if you wanted to find the best hospital, many people relied upon U.S. News & World Report. After more than 30 years, U.S. News & World Report has become the most publicly visible ranking source for health care institutions.

Anthony A. Romeo, M.D. and Nikhil N. Verma, M.D., writing for Healio/Orthopedics, say this is changing. In recent years, the U.S. News rankings have been criticized by numerous leading institutions. Some top academic centers in the United States have withdrawn participation in the process with strong suggestions that the results provide artificially skewed perceptions and distorted incentives.

Romeo and Verma say the rankings are flawed and they give reasons. The first step of performing a ranking analysis is to obtain consistent data among various institutions that will comprise the source of the analysis and conclusions. The U.S. News data are largely obtained from the American Hospital Association, including patient satisfaction scores and CMS data. CMS data are limited to Medicare patients, with 86% of the patients being aged 65 years or older. Perhaps you’re looking for an ACL reconstruction or rotator cuff repair. These procedures are usually done in a younger population.

Even the Medicare age data are flawed. The most common orthopedic procedures within the Medicare population are hip and knee replacement. But strangely, these procedures are excluded from the U.S. News ranking algorithm. The outcomes and complications of these procedures are similar across the entire Medicare population, especially at the top health care institutions evaluated with U.S. News rankings.

The authors say, “Due to the limited variability, the results of the most common elective orthopedic procedures in the Medicare population are not reflected in the overall rankings. If 50% or more of the Medicare procedures being performed are removed from the analysis, along with the absence of procedures in the non-Medicare population, the analysis and subsequent ranking represent less than 10% of the overall orthopedic procedural volume, providing minimal evidence for conclusions related to overall orthopedic program quality and outcomes.”

Another flaw they point out is the use of risk-adjusted mortality as a proxy for quality. Mortality is a “worst-case” approximation of outcome. In contrast, understanding outcomes in orthopedic literature is largely determined by patient-reported outcome measures (PROM). These measure pain, function, satisfaction and quality of life – what patients really need to know about the procedures they’re having. But PROMs are not reported to CMS or any accrediting body and thus, are not available for use.

These are just a few of the flaws mentioned by the authors in this disturbing report. If you’re looking for the best hospital for orthopedics, try looking somewhere else than the U.S. News & World Report rankings.