From Abracadabra to Zombies

• a
• b
• c
• d
• e
• f
• g
• h
• i
• j
• k
• l
• m
• n
• o
• p
• q
• r
• s
• t
• u
• v
• w
• x
• y
• z

Skeptimedia

Skeptimedia is a commentary on mass media treatment of issues concerning science, the paranormal, and the supernatural.

»Skeptimedia archives

Skeptimedia replaces  Mass Media Funk and Mass Media Bunk. Those blogs are now archived.

fraud and bias in medical research

March 11, 2009. Last November I wrote about some central problems with medical studies and how such studies can be misinterpreted or misapplied. The problems I focused on concerned such things as statistics, sample sizes, replication, control groups, randomization, and what to do about studies that produce contradictory results: the usual suspects. Two recent media reports remind us that there are serious problems that can arise, and probably should be considered, before we begin analyzing and evaluating a new medical study: bias and fakery.

bias

Medical journals take it on faith that the researchers who submit their work have actually done what they say they've done. The journals rely on the honesty of the scientists, rather than reports of investigators, when it comes to accepting the data presented to them. This is really no different from the way journalists handle much of the information that is handed to them by interested parties. In most cases, whether the government report or the corporation press release is true is left up to the reader to determine. If newspapers, for example, had to investigate every government report to determine that it wasn't faked, they'd never publish anything. On the other hand, I take it for granted (though maybe I shouldn't) that most journalists are aware that any information given them by an interested party is going to be biased. Medical journals have assumed that studies that are industry-funded are more likely to be biased, in a positive way, than government-funded studies. Because of the massive quantity of material published each month in medical journals, readers of those journals are likely to focus on reading the prestigious ones. Readers are likely to assume that these prestigious journals would be less likely to have an industry bias in their selection of articles for publication. That assumption is questionable, at least in the UK, according to a new study published in the British Medical Journal (12 Feb 2009).

Dr. Ben Goldacre writes:

The British Medical Journal this week publishes a complex study that is quietly one of the most subversive pieces of research ever printed. It analyses every study ever done on the influenza vaccine - although it's reasonable to assume that its results might hold for other subject areas - looking at whether funding source affected the quality of a study, the accuracy of its summary, and the eminence of the journal in which it was published....

We already know that industry-funded studies are more likely to give a positive result for the sponsors' drug, and in this case too, government-funded studies were less likely to have conclusions favouring the vaccines. We already know that poorer quality studies are more likely to produce positive results - for drugs, for homeopathy, for anything - and 70% of the studies they reviewed were of poor quality. And it has also been shown that industry-funded studies are more likely to overstate their results in their conclusions.

But Tom Jefferson and colleagues looked, for the first time, at where studies are published. Academics measure the eminence of a journal by its "impact factor": an indicator of how commonly, on average, research papers in that journal go on to be referred to by other research papers. The average journal impact factor for the 92 government-funded studies was 3.74; for the 52 studies wholly or partly funded by industry, the average impact factor was 8.78. Studies funded by the pharmaceutical industry are massively more likely to get into the bigger, more respected journals.

Jefferson et al. found that there's no difference between industry-funded and government-funded studies with respect to rigor, quality, size of samples, or where they submit their articles, yet drug-company funded articles are more than twice as likely to be accepted by the high prestige journals. I don't know why this is the case, but two suggested reasons are: 1. Editors are influenced by potential advertisers; and 2. Editors are influenced by authors who are top-tier researchers and drug companies hire the big names whenever possible. There must be an "impact factor" for authors based on how many times an author is cited by other authors. So, it should be easy to determine whether studies submitted by industry-funded authors are significantly more eminent than those submitted by government-funded scientists.

fakery

Baystate Medical Center in Springfield, Massachusetts, has asked several anesthesiology journals to retract 21 drug studies published between 1996 and 2008 by anesthesiologist Scott S. Reuben, M.D., a pioneer in the area of multimodal analgesia.* The studies were funded by Pfizer, Merck, and Wyeth. Reuben was also paid by Pfizer as a speaker to promote its products.* Raymond F. Kerins Jr., a Pfizer spokesman, said: "It is very disappointing to learn about Dr. Scott Reuben's alleged actions."

Reuben is suspected of falsifying some of his data, including the names of patients and co-authors. Evan Ekman, MD, an orthopedic surgeon in Columbia, S.C., told Anesthesiology News that his name was forged on two of the retracted papers. Dr. Hal Jenson, Baystate's chief academic officer, said that the investigation of Rebuen's work found that in many cases "there was no clinical trial because there were no patients."

 "This would be the largest research fraud in anesthesia," said Dr. Steven Shafer, editor of the journal Anesthesia and Analgesia. "Doctors have been using [his] findings very widely. His findings had a huge impact on the field. The act of fabricating data is so difficult for me to comprehend. It's beyond my ability to imagine."*

Reuben is on "indefinite leave" from the hospital.

Dr. Reuben has been an extremely active and visible figure in multimodal analgesia, particularly as an advocate for its use in minimally invasive orthopedic and spine procedures. His research has provided support for several mainstays of current anesthetic practice, such as the use of nonsteroidal anti-inflammatory drugs and neuropathic agents instead of opioids and preemptive analgesia. Dr. Reuben has also published and presented data suggesting that multimodal analgesia can significantly improve long-term outcomes for patients.*

Dr. Shafer said: “We are left with a large hole in our understanding of this field. There are substantial tendrils from this body of work that reach throughout the discipline of postoperative pain management. Those tendrils mean that almost every aspect will need to be carefully thought through.”

Reuben's misconduct was discovered during an internal review at Bayview that was triggered during a routine review of research summaries. Reuben was going to present a summary of work that had not been approved by an internal hospital review board that oversees research on patients. That led to a full-scale investigation that uncovered more than a decade of medical malfeasance by Dr. Reuben.

Fakery, of course, occurs much less frequently than bias and has much more impact, but both can be related to industry-funded medical research. And, just as medical journal editors rely on the integrity of those who submit articles to them, the public relies on the integrity of the journals, the researchers, and the media. When the faith of the journal editors is shattered by news of bias or fraud, it is not only their readers who are affected. Public trust in science-based medicine is also shattered.

further reading

Scumbag by Ben Goldacre