What does it take to put an ugly fact through the heart of a beautiful hypothesis?

NPR yesterday did a piece on treating fractured vertebra and so did the NYTimes This is hot news because two papers appeared in the New England Journal of Medicine that compared vertebroplasty to placebo and found them comparable.

The great problem is that the treatment is perceived by both doctors and patients as effective. This treatment provides opportunity for discussion about a huge problem for health care reform. When common knowledge suggests a treatment is effective but the data show, in a compelling way, that the treatment (which has risks and financial costs) is no better than placebo should insurance pay for the treatment?

The fear of some is that because a treatment does not appear to be clinically effective "big brother" will say, "we won't pay." I understand the fear that some all powerful insurance company will get to decide what is and is not effective. But is this bad, per se? I do not think so. Both doctors and patients have for years mistakenly believed that various treatments that aren't effective actually are. I've blogged on this before, see Treatments That Don't Work.

I recently read a great paper whose title I love: What does it take to put an ugly fact through the heart of a beautiful hypothesis? (1) The title is from a quote by Thomas Huxley who lamented "The great tragedy of Science - the slaying of a beautiful hypothesis by an ugly fact." The point in this article is that our modern history of health care is full of beautiful hypotheses, that some treatment is effective, slayed by a ugly fact, research showing the treatment to be ineffective. The problem is that although the hypothesis, that the treatment is effective is dead, the belief in the effectiveness of the treatment isn't dead. Both doctors and patients alike have resurrected these treatments that ugly facts have slayed like the zombies in the classic Night of the Living Dead.

Another quote from Haynes (1) paper from Max Planck, renowned physicist

‘‘A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it.’’

However, it seems that while opponents of the new scientific truth die they unfortunately are capable of infecting a new generation that because of truthiness will accept these slain hypotheses.

Truthiness, is defined by Merriam-Webster as:

Truth that comes from the gut, not books
The quality of preferring concepts or facts one wishes to be true, rather than concepts or facts known to be true.

For more on truthiness and the danger it poses for patients see my article in Dynamic Chiropractic, The Dangers of Truthiness

As Prof. Dov Cooperman of the University of Maryland wrote in a letter to the editor of Newsweek: "...our society is more than happy to accept spin and cant because we have come to believe that all expertise is bias, that all knowledge is opinion, that every judgment is relative. I see this daily in my university classroom. Many of even my best students seem to have lost the ability to think critically about the world. They do not believe in the transformative power of knowledge because they do not believe in knowledge itself"

Unfortunately I see it everyday in my classroom and hear it from many of my professional colleagues too.

My final quote from Haynes (1): Samuel Johnson, the 18th century poet and critic

‘‘The chains of habit are too weak to be felt until they are too strong to be broken.’’

The habits that are so problematic are a lack of critical thinking skills and reliance upon the "wisdom" of others (i.e. dogma), unsystematic and uncontrolled observation, and just truthiness.

For more information on critical thinking I recommend the Foundation for Critical Thinking.

SMP

1. Haynes RB, Haynes GA. What does it take to put an ugly fact through the heart of a beautiful hypothesis? Evid Based Med. 2009 Jun;14(3):68-9. Pubmed link

Treatments That Don't Work

If there is a fundamental question in biomedical research it is does a treatment "work". By "working" most people mean is the treatment clinically effective. The first time this question was asked in a rigorous scientific way is often cited as the work of the British Naval physician James Lind, MD (1716 - 1794). Lind, a Scotsman, one might say is personally responsible for the British being nicknamed, Limeys. His research showed that citrus juice "worked" as a treatment for scurvy, which of course we now know is due to a Vitamin C deficiency that the citrus provide a good supply of.

Lind was ignorant of vitamins, which weren't discovered until early part of the 20th century. It is interesting to note that he thought that the cause of survey was putrefied food. So his experiment centered on acidic "foods" to prevent the food one ate from putrefying. He tried having sailors consume the following in six experimental groups : cider, sulfuric acid, vinegar, seawater, oranges and lemon, and a spicy paste plus barley water. In 1753 he published his A Treatise of the Scurvy. Despite his work the addition of citrus to the diets of sailors of the British Royal Navy was not universally accepted. Three lessons we can learn from Lind's work.

1. Rigorous scientific experimentation can discover effective treatments (ones that "work")
2. An effective treatment does not validate ones theory about why the treatment is effective.
3. People will often continue to believe that ineffective treatments are actually effective, even after compelling research has been published.
   

Dr. David Newman blogs for the NY Times (you've probably guessed I like The Times) writes about believing in treatments that don't work. There is an allure to treatments that one knows or is good at providing. When one starts along the path of evidence based practice one is constantly amazed by both patients and doctors of all types (medical, chiropractic, osteopathic etc.) who continue to use or advocate treatments that compelling research shows are ineffective and sometimes even more harmful than beneficial.

There is a great anecdote about Dr. Paul Glasziou. Dr. Glasziou had published a systematic review (SR) on otitis media (inner ear pain).(1) The most common treatment for this is to prescribe antibiotics. However, the SR found that "watch and wait" is the best approach. So what does Glasziou do when his 2 year old develops acute otitis media? The experience of so many parents and pediatricians with the antibiotics is that the children get better when they take them. Well Glasziou's review suggests that this is because the drug is often given when the pain is at its worst and that the condition will improve shortly on its own. However, doctors and patients ascribe the improving clinical situation to the antibiotics. Thus if we do not know the natural history of the disorder (what happens without treatment) we are all deluded into believing the treatment is what helped the patient. So what did Dr. Glasziou do? He gave his son analgesics only and he got better on his own.(2)

I think what Dr. Glasziou did is a good example of faith. It is faith that the best evidence will in the majority of cases result in the best outcome. Will it every time. No but it is more likely to result in a better outcome than using a treatment that the best evidenced shows is unlikely to help even if personal anecdotes suggest otherwise.

In this blog, Dr. Newman, notes: "More than a half million Americans per year undergo arthroscopic surgery to correct osteoarthritis of the knee, at a cost of $3 billion." This is despite a rare type of research study. The study was a placebo controlled clinical trial of surgery. This means that people who were not in the treatment group still had surgery but surgery that wasn't effective. The results showed a comparable outcome in both groups. Thus, the surgery that so many Americans are having is worthless.

What is interesting about this study using placebo surgery is that the last time placebo surgery was used the outcome was the same that the surgery was no better than placebo.(3) This was in the 1950s with a procedure that at the time was thought to be the best treatment for angina pectoris, internal mammary artery ligation. Some people say that the beginning of evidence based practice was with the publication of two placebo controlled studies comparing internal mammary artery ligation and sham operation for angina pectoris. After the publication of these two studies the use of this well loved procedure went away.(4, 5)

The Obama administration proposes that in health care reform that we place evidence ahead of belief, anecdote, profit, etc. Unfortunately, I fear that the First Amendment will get in the way. The right "... to petition the Government for a redress of grievances." I fear that lobbing will trump evidence.

SMP

1. Glasziou PP, Del Mar CB, Sanders SL, Hayem M. Antibiotics for acute otitis media in children. Cochrane Database Syst Rev. 2004(1):CD000219.
2. Tovey D. Evidence gets personal. BMJ Clinical Evidence. 2007;September 17.
3. Moseley JB, O'Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall DH, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med. 2002 Jul 11;347(2):81-8.
4. Cobb LA, Thomas GI, Dillard DH, Merendino KA, Bruce RA. An evaluation of internal-mammary-artery ligation by a double-blind technic. N Engl J Med. 1959;260(22):1115-8.
5. Dimond EG, Kittle CE, Crockett JE. Comparison of internal mammary artery ligation and sham operation for angina pectoris. Am J Cardio. 1960;5:483-6.

Pot calling the kettle black

When I started chiropractic college chiropractors were, according to the AMA, an "unscientific cult". Heck, some essentially still say that (see my Blog entry on Katz and Dogma). Nevertheless, Sharon Begley's column in Newsweek points out that standard medical practice is often unscientific. Looks like the pot calling the kettle black.

But I figure a double standard is the norm, and just get over it.

SMP