Truth and things that sound like the truth are not the same, especially for health news moving at the speed of the Internet. It can be pretty easy to believe stories in major media publications where studies from reputable journals are cited.
While I was researching Low T: Separating Facts From Frenzy, I read many stories about the testosterone therapy news, like these at The New York Times, NPR, Los Angeles Times, Yahoo! Health and The Wall Street Journal.
The news about low T caught fire recently when two new studies suggested that millions of men may be risking a heart attack, stroke or premature death by using testosterone replacement therapy. Both studies were observational and retrospective, so I knew that any headlines or editorials suggesting a causal link were just click bait. Reputable outlets were careful to not overstep there.
Dr. Abraham Morgentaler, one of my story sources, took me through each study, confirming my issues and pointing out several more. He is Director of Men’s Health Boston and an Associate Clinical Professor of Urology at Harvard Medical School, Beth Israel Deaconess Medical Center. He lectures nationally and internationally, teaching physicians the latest information about the diagnosis and treatment of conditions affecting men’s sexual and reproductive health. He is also the men’s health expert who originally coined the term “low T” many years ago, long before it was co-opted by drug manufacturers.
It takes much longer than 90 days to develop atherosclerosis that leads to a heart attack. “Looking at events within three months of data is an unusually short period of time to evaluate cardiovascular risk. It takes many years to develop enough atherosclerosis in the coronary arteries to produce a heart attack,” says Dr. Morgentaler.
In the most recent study, published in the peer-reviewed journal PLOS ONE on January 29, 2014, researchers looked at prescription data and health records. The study compared heart attack rates in 55,593 middle-aged and older men with a total T level below 300 ng/dL in the 90 days following their first testosterone prescription with rates during the year before they received the first prescription. Researchers reported that within 90 days, men age 65 and older taking testosterone therapy showed more than double the incidence of heart attack compared to a comparison group taking erectile dysfunction drugs and state that the risk was nearly tripled for younger men with existing heart disease.
Here are the issues with the study:
-
- There is no way to know if the men actually took the drug or if their levels of testosterone changed or normalized over the time period evaluated. The study data came from insurance information, not clinical blood work. Researchers looked at rates of heart attack in men with low T levels, defined as less than 300 ng/dL at the beginning of the study period and then compared reported rates for heart attacks. To their credit, the study authors do state in the discussion section: “We were also unable to examine whether this excess (elevated heart attack rates) was related to indications such as level of serum testosterone or hypogonadism,” but you have to read carefully to find it.
-
- There is no control group, i.e. a group who did not take testosterone therapy. The study authors compared cardiovascular events for men who filled testosterone prescriptions with men who filled erectile dysfunction prescriptions. “I think this is nonsensical,” says Dr. Morgentaler. “This would be like taking men with diabetes and looking at the effect of one of the agents to lower blood sugar, and comparing it to men with COPD and treating them with a COPD respiratory medicine, and claiming that one represents an adequate comparison group to the other. You’re changing every variable with the two comparisons.” Indeed, a ‘comparison’ group is not the same as a ‘control’ group. Not by a long shot. Yet this post on in the Los Angeles Times says, “Men in both of those two groups tend to be of similar age, have similar health problems and complain of similar symptoms, and so are comparable.”
-
- It takes much longer than 90 days to develop atherosclerosis that leads to a heart attack. “Looking at events within three months of data is an unusually short period of time to evaluate cardiovascular risk. It takes many years to develop enough atherosclerosis in the coronary arteries to produce a heart attack,” says Dr. Morgentaler. The researchers tracked patient data longer but did not report findings for other follow-up time periods. Why not? What were the heart attack rates at a 6-month or 12-month interval? Why did they not compare a 12-month period to a 12-month period?
- The overall rate of increase in the heart attack rate is actually very small. Taking the figures in Table 1 of the study, the actual difference in heart attack rates between the testosterone prescription group and the no prescription group is 1.27 per 1,000 person years. If we assume the men live up to 85 years on average, that rate would mean there would be about one more heart attack per three hundred person years in the prescription group. The absolute numbers make for a far less compelling story than reporting double and triple the risk. “So when The New York Times editorial claims that this is a major public health issue, I don’t know what they’re talking about,” says Dr. Abraham Morgentaler, “I think that editorial was irresponsible.”
As with the PLOS ONE study, there is no way to know if the men actually took the drug or if their levels of testosterone changed or normalized over the time period evaluated.
The second study, originally published in the peer-reviewed journal JAMA on November 6, 2013, looked at data for more than 8,000 veterans with low testosterone (under 300 ng/dL). For a subgroup of 1,223 men who had coronary angiography, a heart test, they tracked data about heart attacks, strokes and death over a three-year period.
Here are the issues with this study:
-
- The absolute rate of heart attacks was incorrectly reported in the original paper and has since been revised. The original paper published on November 6, 2013 said “the absolute rate of events was 19.9% in the no testosterone therapy group vs. 25.7% in the testosterone therapy group, with an absolute risk difference of 5.8% (95% CI, −1.4% to 13.1%) at 3 years. But those numbers were actually rates after a complicated, high-level statistical analysis that adjusted for 50 variables, not absolute rates. (Eureka! I had been trying to calculate those percentages from the raw data provided to no avail!) The paper was revised on January 15, 2014 to state that those figures were “Kaplan-Meier estimated cumulative percentages with events.” Many media outlets are not aware of this correction.
-
- As with the PLOS ONE study, there is no way to know if the men actually took the drug or if their levels of testosterone changed or normalized over the time period evaluated.
-
- The study authors do not explain how they calculate their findings, which were based on a high-level statistical manipulation of over 50 variables. A plain language explanation would help, because if you add up the raw numbers provided, you end up with the complete opposite result — a lower rate of heart attacks, strokes and death for the testosterone prescription group compared to the no prescription group.
- 1,132 men who had testosterone therapy prescribed after a heart attack or stroke were excluded from the study data. Why were they excluded? How would their results have changed the study findings?
“One of the dangers for the average educated reader, medical or otherwise, is that these studies have now become so technical and statistical that we’ve lost contact with whether something makes sense or not,” says Dr. Morgentaler.
I’m looking forward to see the results of the National Institute of Aging’s study, The Testosterone Trial in Older Men. As a randomized, placebo-controlled prospective trial among 800 older men, this study promises to deliver the quality of evidence we need to make informed medical decisions. The study is expected to be completed by July 2015.
