Thursday, August 6, 2009

What does it take to put an ugly fact through the heart of a beautiful hypothesis?

NPR yesterday did a piece on treating fractured vertebra and so did the NYTimes This is hot news because two papers appeared in the New England Journal of Medicine that compared vertebroplasty to placebo and found them comparable.

The great problem is that the treatment is perceived by both doctors and patients as effective. This treatment provides opportunity for discussion about a huge problem for health care reform. When common knowledge suggests a treatment is effective but the data show, in a compelling way, that the treatment (which has risks and financial costs) is no better than placebo should insurance pay for the treatment?

The fear of some is that because a treatment does not appear to be clinically effective "big brother" will say, "we won't pay." I understand the fear that some all powerful insurance company will get to decide what is and is not effective. But is this bad, per se? I do not think so. Both doctors and patients have for years mistakenly believed that various treatments that aren't effective actually are. I've blogged on this before, see Treatments That Don't Work.

I recently read a great paper whose title I love: What does it take to put an ugly fact through the heart of a beautiful hypothesis? (1) The title is from a quote by Thomas Huxley who lamented "The great tragedy of Science - the slaying of a beautiful hypothesis by an ugly fact." The point in this article is that our modern history of health care is full of beautiful hypotheses, that some treatment is effective, slayed by a ugly fact, research showing the treatment to be ineffective. The problem is that although the hypothesis, that the treatment is effective is dead, the belief in the effectiveness of the treatment isn't dead. Both doctors and patients alike have resurrected these treatments that ugly facts have slayed like the zombies in the classic Night of the Living Dead.

Another quote from Haynes (1) paper from Max Planck, renowned physicist
‘‘A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it.’’
However, it seems that while opponents of the new scientific truth die they unfortunately are capable of infecting a new generation that because of truthiness will accept these slain hypotheses.

Truthiness, is defined by Merriam-Webster as:
Truth that comes from the gut, not books
The quality of preferring concepts or facts one wishes to be true, rather than concepts or facts known to be true.
For more on truthiness and the danger it poses for patients see my article in Dynamic Chiropractic, The Dangers of Truthiness

As Prof. Dov Cooperman of the University of Maryland wrote in a letter to the editor of Newsweek: "...our society is more than happy to accept spin and cant because we have come to believe that all expertise is bias, that all knowledge is opinion, that every judgment is relative. I see this daily in my university classroom. Many of even my best students seem to have lost the ability to think critically about the world. They do not believe in the transformative power of knowledge because they do not believe in knowledge itself"

Unfortunately I see it everyday in my classroom and hear it from many of my professional colleagues too.

My final quote from Haynes (1): Samuel Johnson, the 18th century poet and critic
‘‘The chains of habit are too weak to be felt until they are too strong to be broken.’’
The habits that are so problematic are a lack of critical thinking skills and reliance upon the "wisdom" of others (i.e. dogma), unsystematic and uncontrolled observation, and just truthiness.

For more information on critical thinking I recommend the Foundation for Critical Thinking.

SMP

1. Haynes RB, Haynes GA. What does it take to put an ugly fact through the heart of a beautiful hypothesis? Evid Based Med. 2009 Jun;14(3):68-9. Pubmed link

10 comments:

  1. Excellent commentary. Health care reform without a willingness to let go of our 'sacred cows' is doomed to failure.
    But this talk of changing the behavior of providers calls to mind the quotation by Upton Sinclair (and I'm paraphrasing): "it is difficult to change someone's behavior when their income is dependent upon their current belief system"

    ReplyDelete
  2. I guess for health professionals there are two potential responses to Upton Sinclair.

    One the ethical idealist is that professionalism means putting the interests of the patient ahead of all the doctor's interests, even pecuniary. The problem with this is if one needs to make a living it may be hard to put the patient first (hasn't been for me but it appears to be for some).

    This then leads to the other response to Mr. Sinclair. We need to teach all types of doctors that they can still make money by doing what is best for the patient. I believe that as time goes on patients eventually will be able to know what does work and what doesn't and they'll start going to the doctors who do what works and avoid those that do what doesn't but makes them more money. This change in patient behavior might just force the change in doctor behavior by making it better from an economic stand point to do what is best, i.e. most effective or brings the greatest value to the patient.

    ReplyDelete
  3. I just read an article about an orthopedic surgeon who is promoting a "Refuse to Fuse" campaign based on the poor evidence for spinal fusions. I hope the vertebroplasty data is assimilated more quickly.

    ReplyDelete
  4. Stephen,

    Great article. I just tried to post to your blog and it would not take my password. I have reset the password so hopefully this one will go throug. I am not sure I will repeat exactly what I said the first time but I have 2 thoughts.

    1) First, I agree with John's comments and especially the paraphrase by Sinclaire. When a clinician does a procedure/treatment based upon his/her bank account, I feel the patient nor the carrier should be responsible for the clinician who provides a service under these circumstances, especially if they know the procedure/treatment has no validity.

    2) On the other hand, when a clinician has performed a particular procedure/treatment that he/she either strongly feels or knows it has been successful in the past, why not allow them to do that AND be reimbursed. We all know there is not enough evidence available on all forms of treatment, and thus at times we must rely on anecdotal care and the placebo effect.

    The question then arrises, if one treats a patient with a procedure/treament that he/she feels is the best available to him/her, should the patient or insurance company pay for that procedure/treatment?

    Insurance companies are concerned about the bottom line. They have a Board of directors and shareholders who want to see profits. How does an insurance company decide which procedures/treatments they will reimburse for? Is this by evidence available through RCT's or other studies? Is this the opinion of their medical advisors? If evidence for or against a procedure/treatment does not exist, then what? Is the use of this procedure/treatment up to the clinician to decide if it is best for his/her patient and if so, should the insurance company pay when rendered in good faith?

    Tom

    ReplyDelete
  5. To piggyback on Tom's post:

    If a physician is going to be reimbursed for a procedure for which there is no clear evidence, it should at least be incumbent upon that physician to demonstrate in a relaible and valid manner whether the treatment was effective for that patient. With most spine-related disorders there are simple, reliable, valid and clinically relevant measures that allow for this. The rest of the world should not have to rely on the physician's opinion that "it worked". This should be measured and the physician held accountable for the result.

    ReplyDelete
  6. Stephen,

    How's this story as an example of hard facts not being accepted by
    medicine. It's a great add-on to your blog!

    Mike Schneider (Scroll down for story)
    ------------------------------------------------------------


    Published 28 July 2009, doi:10.1136/bmj.b3049

    Cite this as: BMJ 2009;339:b3049 News

    President of British Pain Society is forced from office after refusing to
    denounce NICE guidance on low back pain


    Zosia Kmietowicz


    1 London


    The president of the British Pain Society was forced to resign on 21 July
    after a campaign from members who were unhappy with guidelines on the
    management of low back pain from the National Institute for Health and
    Clinical Excellence (NICE), which he helped develop.


    In a letter to the BMJ this week Michael Rawlins, chairman of NICE,
    and Peter Littlejohns, the institute's clinical and public health
    director, called the move to oust Paul Watson as "shameful" and
    "professional victimisation of the worst kind" ( BMJ 2009:339:b3028,
    doi: 10.1136/bmj.b3028 ).



    Professor Watson, who has been president of the society since April, said
    he was "very disappointed" about what had happened. "It gives the wrong
    impression of how the British Pain Society conducts its business, and that
    distresses me quite a lot," he said.



    Some members of the society called an emergency general meeting on 21 July
    because they were not satisfied with a statement the society put out about
    its opposition to NICE guidelines. In addition, they wanted Professor
    Watson to repudiate the guidelines or resign.



    The main sticking point was the recommendation in the guidelines that
    injections of therapeutic substances into the back for non-specific low
    back pain should not be offered, as the evidence of effectiveness is
    lacking.



    But the society said that NICE's guideline development group was
    "misguided" for not considering evidence from cohort studies and clinical
    case series in its deliberations on this and other treatments. "Although
    NICE has recommended further research into these procedures, we are most
    concerned that a significant number of patients will be denied this choice
    oftreatment in the interim," says the statement.



    When Professor Watson refused to denounce the guidelines, members of the
    society voted for his resignation, by 186 to 179.



    Professor Rawlins called the situation "absurd." He said, "I would have
    thought that a profession would want their clinical practice supported by
    good evidence."



    He said he was surprised that members of the society themselves had not
    conducted some randomised controlled trials on the effectiveness of
    injections into the spine. "They [RCTs] would be inexpensive and easy to
    organise," he said.



    In their letter Sir Michael and Professor Littlejohns say that the British
    Pain Society's action conflicts with a recommendation of the judicial
    review of NICE's guidelines on chronic fatigue syndrome in March ( BMJ
    2009;338:b1110, doi: 10.1136/bmj.b1110 ). The High Court judge who
    reviewed the case dismissed all challenges of the guidelines and said that
    "health experts must be able to express their opinions without fear of
    retribution," saythe professors.



    The Council of the British Pain Society said that it was saddened at the
    loss of an excellent president. It a statement the society said that as a
    limited company it had acted within the law when it considered that its
    president had a conflict of interest in drawing up the NICE guidance. "One
    could argue at length about the guidelines on such a difficult subject as
    low back pain. Council of The Society stands by its beliefs that the
    guidelines are inadequate. However, this in no way is a reflection of
    Professor Watson, but of the conclusions reached by the NICE guideline
    committee and the dogmatic way in which they were presented," it said.



    Cite this as: BMJ 2009;339:b3049

    ReplyDelete
  7. I think this post certainly point out the tendency in western medicine (and society) about the unwillingness to change a cherished belief in the face of facts. I can certainly recall arguments with my wife (that I invariably lose) where I refused to make such a necessary shift. Unfortunately, I see that all too often in the chiropractic profession as well and my concern with this for both chiros and MDs is the unwillingness to change is often motivated by money rather than what is best for the patient. Too often as a new DC I have been told to practice a certain way or schedule 'x' visits when my the evidence does not support that path. I choose to follow my conscience and the patient's interests. The fact DCs both newer and older docs still decide treatment without trying to use evidence or at least patient outcome measures will be an albatross on this profession for the future.

    ReplyDelete
  8. Certainly there are many doctors who fail in their professional duty to put the patient's needs ahead of the doctor's own needs, pecuniary or otherwise.

    I think often the issue is that anecdotes have a powerful impact on ones beliefs. Thus, seeing a patient in pain and then not in pain leads one to ignore potential confounders and ascribe the change to the intervention we've administered. Plus we tend to forget our failures and remember our successes - selection bias.

    In Groopman's book, How doctor's think, he talks about a colleague who kept a notebook of failures so he could learn from them.

    ReplyDelete
  9. Here are some people who, at first blush, ARE making changes in their care based on evidence from their own records.
    =====
    From an article at: http://www.healthtransformation.net/cs/MerrittIntermountain

    Intermountain Healthcare in Idaho and Utah is using EMR to gather information on treatments and disseminate best-practice procedures throughout their system, resulting in better quality care at 1/3 the cost.

    The Center for Healthcare Transformation wants to take "best practices from innovative institutions across the country that are delivering the highest quality care at the lowest cost and designing policy to encourage their widespread adoption."
    =====
    Could we not do something similar? I know we are not all part of one network, and certainly not non-profit, but the concept seems sound. Frankly, I had hoped some of the lists like chiro-sci would assist in putting us on a more information sharing basis. Alas, it seemed to fall toward some non-scientific and rancorous bickering.

    ReplyDelete
  10. Russel,

    Intermountain has demonstrated that following guidelines in the treatment of back pain results in better outcomes. Who knew? :)

    Fritz JM, Cleland JA, Brennan GP. Does adherence to the guideline recommendation for active treatments improve the quality of care for patients with acute low back pain delivered by physical therapists? Med Care. 2007 Oct;45(10):973-80.

    http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=17890995

    Stephen

    ReplyDelete