Statistics R Your Friends: Lies, Damned Lies and Statistics II

By Gail Garfield Schwartz - May 31, 2011

Suppose you are seeking the best way to prevent recurrence of breast or ovarian cancer. Your doctor might say something like "if you take this treatment there's a 50% greater likelihood that you will not have a recurrence of breast cancer for at least three years." Do you know what this really means?

Even more important, does your doctor pay attention to what it means for just one very special patient, you?

That doctor's "50% greater likelihood" is a comparative statement. Fifty percent greater than what? Standing alone, the statement is no different from "Use this toothpaste and you'll have a 50% whiter smile." Whiter than what?

There is more than one way to compare likelihood of cancer recurrence (or disease progression) in treatment groups and non-treatment (control) groups. Each method has particular implications. Let us compare the three statistics researchers use to describe the outcomes of treatment trials so we can focus on what we need to know about them.

In the following discussion using hypothetical examples, metastatic breast cancer is the disease. The "events" being measured in a clinical-trial Intent to treat group are new or growing breast cancer tumors anywhere in the body. The time period for the trial is three years.

Absolute Risk Reduction (ARR)

Absolute risk is the likelihood that a woman who has already had breast cancer will have a recurrence of breast cancer in the time period. Absolute risk is observed in the trial, by counting the number of events in the control group and in the treatment group. Absolute Risk Reduction is simply the difference in proportions of events between the treatment group and the control group. If in the control group 10% of patients have an event and in the treatment group only 6% of patients have an event, then treatment confers a reduction in risk of recurrence of 4 percentage points. This is called ARR of 4 percent.

The size of ARR of course depends on the number of events in each group. High ARR does not necessarily indicate a wonderfully low proportion of events in the treatment group. It may only indicate a woefully high percentage of events in the control group. If events are the same 6% in the treatment group but 15% in the control group instead of 10% as above, ARR is more than twice as large: 9 percentage points (15-6)

So if a doctor presents absolute risk reduction by saying "With this treatment, your chances of having a recurrence are cut by X percent," you should ask about the risk in the control group. The statement may be true, but it is meaningless without knowing the risks of having an event absent treatment. If that risk is very small, ARR might not even matter, especially if serious side effects can be expected with treatment.

Relative Risk Reduction (RRR)

Relative Risk Reduction is the absolute risk in the treatment group divided by the absolute risk in the control group. Where absolute risk in the treatment group is 6% and absolute risk in the control group is 10%, RRR is 60%. 60% seems like a much higher number than the ARR of 4 percentage points, but the same number of events has occurred in both the treatment group and the control group.

The decision-making usefulness of RRR can depend on the particular patient's baseline no-treatment risk, or on her age or general health. For instance, consider a very toxic treatment for metastatic breast cancer. An estimated RRR of 50% or more might be both statistically significant and clinically important for patients at high baseline risk of an event--such as someone that has already experienced significant recurrence of tumors. However, for patients with a lower probability of an event, even high RRR might not be great enough to recommend such toxic treatment. Failure to take into account differences between high-risk and low-risk patients can be a problem when using RRR for decision making in the doctor's office.*

Number Needed to Treat (NNT)

NNT is the number of patients to whom a clinician would need to give a particular treatment to prevent one patient from having an event over a specified period of time. If, for example, the NNT for a breast cancer drug is 10, a doctor would have to give the treatment to ten patients diagnosed with breast cancer to prevent one patient from having a new tumor over the time period. Each patient who received the treatment would have a 1 in 10 chance of being a beneficiary.

Arithmetically, the NNT is the reciprocal of the ARR. If ARR is ten percentage points, NNT is 1 /.10 =10. If ARR is only five percentage points, then NNT is higher: 1/.05=20. Twenty patients must be treated before one benefits. If ARR is higher, say 20 percentage points, NNT is lower: 1/.20 means that only five patients need treatment for one patient to benefit.

NNT is intuitively easy to understand: all you have to remember is, the lower the number, the better. Also it's easy to remember benchmarks: NNTs under 5 are unusual, so very good. NNTs over 20 are common. So, NNTs under 20 would seem more promising.

If a physician offers ARR or RRR, a patient can always ask for NNT in addition or instead. Just ask: How many patients must receive this therapy to make it probable that one recurrence will be prevented? If the doctor can't answer, you can figure it out yourself, and find another doctor.**

Statistics R Your Friends

I absolutely get how difficult it is to understand statistics when we are nervous or scared: I had the experience myself of completely misinterpreting the relative risk of adjuvant chemotherapies, and lacking a full explanation from my oncologist, came much too close to making a dangerously wrong choice. A person in the throes of decision making should always ask her doctor to fully explain everything that makes any statistic important for her. If the patient feels the explanation is cloudy, she can take ask for a written copy and ask someone knowledgeable.

Equally important, every patient should always ask whether the statistics her doctor uses when describing a treatment's effectiveness are better than similar statistics for other possible treatments.

Statistics cannot lie to us if we understand them. When we do, they become our friends because they are the only way to put probable benefits of treatments in a comparative context.

* One more thing to note about RRR. Since RRR "balloons" results, it is the preferred way that drug companies tout their products.

**See Alexandra Barratt et al, "Tips for learners of evidence-based medicine," CMAJ, August 17, 2004; 171 (4). doi:10.1503/cmaj.1021197 © 2004 accessible at Canadian Medical Also see dictionary. The Cochrane Collaboration has done an analysis covering scores of studies about understanding of statistics by both the public and medical practitioners.



Gail Garfield Schwartz

Gail is a breast cancer survivor and a SHARELeader.