But Is It Clinically Significant? (Part I)

While attending both undergraduate and graduate nursing schools, I loathed research classes. These classes were invariably taught by the oldest faculty members, and they were boring! The classes put much more emphasis on how to conduct research than on how to read, interpret, and implement research findings. After I graduated and entered practice, I did read some research studies, or at least some abstracts of research studies, so that I could keep up to date.

But then I realized that this effort was not enough, that I needed to look beyond the fact that certain study findings were statistically significant. What did the study findings really mean? What implications, if any, did these findings have for my patients as a group or for specific patients in my practice? And were these findings clinically significant? (Just because study findings are statistically significant doesn’t mean that a certain intervention will have a clinical impact on patients.) In addition, I wanted to be able to carry on an intelligent conversation with the pharmaceutical company sales representatives who visit my office. I’ve often felt intimidated by these sales reps, as if they knew so much more than I did about scientific research. Finally, I wanted to make thoughtful decisions about my patients’ treatment plans—to be able to explain to myself, as well as to them, why we were embarking on a certain treatment plan. In this column, I’d like to share some of my insight into one important aspect of interpreting research findings—the distinction between relative risk reduction and absolute risk reduction—so that we can better apply research findings to the care we deliver to our patients and to determine whether the information is clinically significant for our particular patients.

What do the statistics really mean? Let’s say a new study showed that a certain medication reduced the risk for a myocardial infarction (MI) by 50% in the study participants. That sounds great! According to the researchers, the reduction in MI risk was statistically significant. But was it clinically significant? Should all of my patients take this medication to reduce MI risk? Should some of my patients take it? If so, which ones? This 50% reduction is what is known as a relative risk reduction, or RRR, which is reported as a percentage or a proportion. This value indicates how much more or less of an effect the intervention had versus the placebo or the control. Is the RRR a big deal? Maybe!

Let’s put RRR in a different context. I’m a salesman now, and I tell you, my customer, that I’ve marked down the cost of an item you want to buy by 50%. That is, I’m selling the product at 50% of the initial price—a 50% reduction. How much money do you save? It depends on the initial price! If this item is a comb and the initial price of the comb is $1, you will pay $.50 and you will save $.50. It’s a great price for a comb, but you may not feel it’s worth your time and the cost of gas to drive across town to get this discount. If, by contrast, the item in question is a car, and the initial price is $20,000 and the price is cut by 50%, you will pay $10,000 and you will save $10,000. This 50% reduction may surely be worth a drive across town—especially if you need a new car. There’s a difference in how you perceive each of these sale prices, yet the price reduction is 50% in both cases.

Information about relative risk reduction is helpful only if you know the baseline risk (or the initial price). In the examples in the preceding paragraph, you saved $.50 on the cost of the comb or $10,000 on the cost of the car. Those savings aren’t the same, but they both represent relative reductions of 50%.

Now let’s take a look at absolute risk reduction, or ARR, which is the absolute difference between the baseline value (or the placebo or control value) and the intervention value (or between the initial price and the sale price). Rather than a percentage or a proportion, ARR represents the difference. Using the previous price reduction analogy, in the case of the comb, the initial price is $1 and the sale price following the intervention is $.50; the absolute reduction in price is the difference between these two values: $1 minus $.50, or $.50. In the case of the car, the initial price is $20,000 and the sale price following the intervention) is $10,000. The absolute reduction in price is $20,000 minus $10,000, or $10,000. Which sale price is going to draw your attention? Remember, both items feature 50% reductions in price. But the higher the initial price, the more money you save with the 50% discount. The same is true with clinical effects; the higher your baseline risk, the greater the effect of a risk reduction.

Using the Framingham risk assessment tool, my baseline risk for having an MI in the next 10 years is ~3%. If I cut this risk in half—a relative risk reduction of 50%—my risk for MI declines to 1.5%. Because my baseline risk is so low, this reduction is not a big deal. It’s not worth exposing myself to potential side effects of medication, not to mention incurring the cost of the medication and the follow-up lab tests and office visits, to reduce my MI risk from 3% to 1.5%, an absolute risk reduction of 1.5%. By contrast, for a 65-year-old smoker with elevated total cholesterol, low HDL-C, and borderline hypertension (baseline risk for MI, 24%), a 50% risk reduction would lower his 10-year MI risk to 12%; in his case, the upside, an absolute risk reduction of 12%, exceeds the downside. The benefit is clinically significant.

The numbers don’t lie, but they can be deceptive. When you read about a clinical trial, take a close look at the numbers, and try to understand what they mean in the real world, for your real patients. Make sure you know the baseline risk for a certain outcome for the population that was studied in the trial—a risk that may differ from that of your own patients or any given patient.

The statistics alone, even a 50% RRR or an ARR that seems large, don’t tell the whole story. Research articles in peer-reviewed journals don’t always show clinically significant interventions. When you read about a study in a journal, ask yourself questions about the RRR and the ARR and try to determine whether the findings pertain to any patients in your practice. Through some basic understanding of research, you can become more confident in prescribing medications and other therapies that are likely to have clinical benefit for our patients.

In addition to RRR and ARR, other tools and techniques are available to help you understand and interpret the research and determine the clinical significance of study findings for your patients. In this column, I’ve covered just one aspect of interpreting research findings. In future columns, I hope to cover more. Regardless, keep in mind that you won’t understand the statistical significance or the clinical relevance of a study by simply reading the abstract or the conclusion.