PACE Trial Gets Most Devastating Critique Yet

[Cort]

I'm unsure as to how all of this works, but I have a concern... These erroneous articles that have been generated and published are 'out there'. Is there any way to remove/delete/retract them so that as medical professionals gain interest and begin to try to educate themselves, they won't be misinformed by this mess? I am deeply grateful for all of those who have been so diligent in trying to right this grievous wrong to the ME/CFS/SEID patient community.

They can be retracted by the publishing journal. The retraction essentially means the journal recognizes that their findings cannot be trusted.

If that happened with this huge trial it would make quite a splash.

 

[Cort]

Honestly, given the weird problems with this study that were not picked up by Lancet you've got to think that the Lancet reviewers just passed the thing through. Look at what Julie Rehmeyer wrote - she's a scientific journalist - and she's obviously seen a lot and she couldn't believe that that study got through. The fact that it did really disillusioned her. That says a lot. It's a stain on Lancet's record.

Well, we attracted a lot of attention with XMRV, both as a possible cause and then for scandal, retraction, when XMRV turned out to be a contamination in several laboratories and not a virus at all. That is how virologists came to know us so well. The second open letter to the Lancet included more than one person whose specialty was proper study design, prominent in that field, and thus statisticians. I am glad to hear from one more.

I have always been glad for Tom Kinldon's work. The PACE study was a mistake from the start, with a poor design, and following a similar study that had no useful results. I never designed medical studies, but I did design studies for my own field, and the flaws in PACE were there from the start, a real waste of money. That the Lancet let itself be roped in, well, that led me to think that there is an in-group for UK medicine, where it is who you know and not what brilliant work you do, that gets your work published. Cynical I suppose, but what is a person to think? Especially when the authors act like spoiled children when criticisms arrive, valid criticisms.

 

[Dee4dogs]

Hmm, I'm trying to understand their intention with carrying out such a sloppy & damaging trial. An $8M trial to boot. And to think how $8M could benefit those who are dedicated and that work with integrity. Not to mention the huge smack in the face to the CFS Community. Perhaps the study should be extracted & those who caused the damage be held financially responsible ( at least in part). I highly doubt those involved would ever receive further financial support with their credibility shot.

 

[Cort]

Hey. It's the biggest study ever done in ME/CFS - by far! It was going to prove that CBT/GET were cost-effective answers to ME/CFS. There was a lot riding on this trial. I imagine that was one reason they start tweaking it. They really couldn't afford to fail. Even before the problems with the study design came out the results were pretty darn pitiful for anyone who really wanted to take a look.

I can't imagine the British government ever trying something like this again...

Hmm, I'm trying to understand their intention with carrying out such a sloppy & damaging trial. An $8M trial to boot. And to think how $8M could benefit those who are dedicated and that work with integrity. Not to mention the huge smack in the face to the CFS Community. Perhaps the study should be extracted & those who caused the damage be held financially responsible ( at least in part). I highly doubt those involved would ever receive further financial support with their credibility shot.

 

[Dee4dogs]

They starting tweaking it because their ego's got in the way.

 

[sharonklb]

Cort I think in the past - and not that long ago - you have written things suggesting that PACE wasn't as bad as we Brits have made i out to be and that Wessely wasn't as bad as we perceived him to be. The problem is that people with ME have been perceived as mentally ill and therefore it's very difficult to get anyone to listen. Most people with ME dont have the energy to go on and on presenting the same case. All credit to those who have managed to repeat the criticisms over and over and to do so with politeness and great patience in the face of extreme provocation.

However the latest criticisms are still from what will be seen in the UK as a minor university. It needs something like the UK's Royal Statistical Society to have much weight with the Lancet.

Hi Tatt
I have had Bipolar1 for a long time and in 2014, needed to see a new Psych Consultant. Within 10 mins of professionally assessed M.E people conducting a long assessment, this guy informed me that it was psychosomatic and therefore all in my imagination! He also sent a letter to a Neuro Consultant informing him of his thoughts and unfortunately, the exam I received for severe cognitive problems, was, without doubt unfeeling, dismissive and led the person I attended with to write him a letter and take it in personally the next day...I came out of that appointment with everything discounted and feeling like a leper.
I then made complaints to the trust...as per, they closed ranks, but even though it has been hard, decided that they were not getting away with this and am now at the Parlimentary Health Complaints stage....the initial consultant even implying on a letter for PIP that I was just trying to claim benefits (thank the lord for the freedom of info act!).
In the end I have cried and sat with mountains of papers in front of me wondering how I can get through it BUT, for all of us and the effects it has on our lives, I feel that I need to carry on shouting from the roof tops! One positive was the PIP tribunal recognised it and although they did not give me the criteria I needed, the fact that they took all the info and made an award was not a monetary victory to me, but a moral one.))) x

 

[LondonPots]

How did the Brits pass review on this? How did a Journal like Lancet (and other Journals) give it a pass?

Gosh, don't know - a government's desire to save welfare money and get a lot of 'feckless' people back to work, plus a massive old-boy network couldn't possibly play a part...

 

[Tony L]

Thank you so much Cort for using the energy you have to help me follow how this injustice is being revealed!

seems like there has been as obsession with winning the argument in the PACE trial at the expense of a commitment to truth/honesty.

As I have said elsewhere our devolved Welsh National assembly has instructed out Local Health Boards to develop services for ME/CFS/FM on the basis of the apparent success of these trials. Health policy based upon lies, distortion, manipulation. I will now challenge my local assembly member on the matter.

 

[choca]

Thanks, Cort, for an excellent article.

I may be able to shed some light, albeit cynical, on what went on with Lancet and the study. Being familiar with epidemiological studies and academic situations and research funding, it is not unknown for some researchers to tweak or falsify their findings in order to justify their funding and/or in hopes of getting more funding.

Although I do not read many research studies thoroughly, lacking the stamina, I'm amazed at how many I have read that make claims that are not justified by the study. For example, claims that an herbal remedy is not effective, with no discussion of dosage when they used very small amounts in the study.

Or confusing correlation with causation, when in fact, they may have it backwards, or there is no relationship to causation at all. These appear to be common in published studies that are reported in media. Additionally, remember the "publish or perish" concept which rules in academia and perhaps elsewhere. People are desperate to publish.

Now, as to how Lancet or others failed to see the errors and deceptions of the PACE study, there could be lots of factors: sloppy scholarship, being impressed by the amount of money behind the study, bias, the wrong reviewers (e.g. not statisticians?), and the common error in the US, having those who know nothing about ME/CFS review research grants, and presumably articles also. Or those who do not know how to critique statistical studies.

Getting published in Lancet is the brass ring, and think about all the incentives to get there. Some people will do anything to "succeed", and lots of people lack scientific discipline and morality.

I can't say what happened here, but these are all too common problems in our world today. My impression, based on an admittedly small sample, is that there are many studies published that are invalid and should not have been. And that there is widespread misunderstanding of statistical analysis, among researchers and journal editors and screening panels.

This is most unfortunate because it leads to general mistrust of science and the media's reporting of it, and many "truths" being overturned after many years of firm belief among medical practitioners.

Looking at how difficult it was for h.pylori to be proven as the cause of stomach ulcers, and that multiple sclerosis was once thought to be "all in one's head", we need to be particularly wary of those who would want to prove that diseases are psychological when there are plenty of data indicating otherwise.

 

[Kim Garrett]

I don't think Mayo really digs into the minutiae of studies - I'll bet they did just what everyone except for patients and some ME/CFS researchers did - they relied on the study findings - and trusted Lancet.

The CDC is another story. I don't know that their take on the PACE trials is...I'll bet most major medical sites were espousing CBT/GET long before the PACE trial though. I think he's a bit off on that.

You would hope in the future before the CDC adopts a new treatment protocol . They will take them time to review all of it before excepting it at face value. In other business organizations this would be called " cooking the books" and whom ever was responsible with have fraud charges brought on them . It is FRAUD no matter how you look at it and who it involves. Just my opinion. .

 

[Katie]

Can't wait until it's totally refuted in a major medical news report. Far too many (of my) doctors push CBT and GET.
My mother is 89 and is far healthier than me. I cannot keep up to her even on my best days. Yes, this is unusual for an 89 year old but even if I was as healthy as a 75 year old I'd be happy!! So does being happy mean I'm cured!!!!! Hilarious

 

[Cort]

Thanks, Cort, for an excellent article.
I may be able to shed some light, albeit cynical, on what went on with Lancet and the study. Being familiar with epidemiological studies and academic situations and research funding, it is not unknown for some researchers to tweak or falsify their findings in order to justify their funding and/or in hopes of getting more funding. Although I do not read many research studies thoroughly, lacking the stamina, I'm amazed at how many I have read that make claims that are not justified by the study.

For example, claims that an herbal remedy is not effective, with no discussion of dosage when they used very small amounts in the study. Or confusing correlation with causation, when in fact, they may have it backwards, or there is no relationship to causation at all. These appear to be common in published studies that are reported in media. Additionally, remember the "publish or perish" concept which rules in academia and perhaps elsewhere. People are desperate to publish.

Now, as to how Lancet or others failed to see the errors and deceptions of the PACE study, there could be lots of factors: sloppy scholarship, being impressed by the amount of money behind the study, bias, the wrong reviewers (e.g. not statisticians?), and the common error in the US, having those who know nothing about ME/CFS review research grants, and presumably articles also. Or those who do not know how to critique statistical studies.

Getting published in Lancet is the brass ring, and think about all the incentives to get there. Some people will do anything to "succeed", and lots of people lack scientific discipline and morality. I can't say what happened here, but these are all too common problems in our world today. My impression, based on an admittedly small sample, is that there are many studies published that are invalid and should not have been.

And that there is widespread misunderstanding of statistical analysis, among researchers and journal editors and screening panels. This is most unfortunate because it leads to general mistrust of science and the media's reporting of it, and many "truths" being overturned after many years of firm belief among medical practitioners.

Looking at how difficult it was for h.pylori to be proven as the cause of stomach ulcers, and that multiple sclerosis was once thought to be "all in one's head", we need to be particularly wary of those who would want to prove that diseases are psychological when there are plenty of data indicating otherwise.

Thank Choca - it could have just been a review panel without a good knowledge of statistics. It could be a simple and as innocuous as that. The journal gathers together a review panel to review each study....If the review panel is not particularly strong then it will miss stuff. (I tend to prefer innocuous reasons to conspiracies...)

 

[PamJ]

Major research effort doing everything they can -including things they shouldn't have done - to improve their results...and getting away with it for about five years.

The problems are so obvious, in retrospect, that they also seem to be that gang that couldn't shoot straight.

How did the Brits pass review on this? How did a Journal like Lancet (and other Journals) give it a pass?

My question now is, do you believe The Lancet will now publish an article on or print a retraction of the PACE study? Has The Lancet ever been in such a situation before?

 

[PamJ]

My question now is, do you believe The Lancet will now publish an article on or print a retraction of the PACE study? Has The Lancet ever been in such a situation before?

Oops, Cort I see that you already answered asimilar question to mine.

 

[Carole]

Cort I think in the past - and not that long ago - you have written things suggesting that PACE wasn't as bad as we Brits have made i out to be and that Wessely wasn't as bad as we perceived him to be. The problem is that people with ME have been perceived as mentally ill and therefore it's very difficult to get anyone to listen. Most people with ME dont have the energy to go on and on presenting the same case. All credit to those who have managed to repeat the criticisms over and over and to do so with politeness and great patience in the face of extreme provocation.

However the latest criticisms are still from what will be seen in the UK as a minor university. It needs something like the UK's Royal Statistical Society to have much weight with the Lancet.[
Tatt- The only way this study could say it is valid is if the" Patient were lying on a couch , reclining and napping during the Therapy session". How disheartening all this is.

Nothing wears me out any more than to go to a Physician and them tell me "What does this have to do with what I am treating you for"?? Yes it happens.

 

[Seven]

USA get Ready for the up coming fight from Insurance try to make this Psy desease( so there IS a 2 yet cap on benefits)

But my question and the next step is as follow: what is the known procedure once you discover a paper is flawed? It is just bizare to me that after all we have done this paper keeps being cited. Do we need to call for a special hearing somewhere to present evidence? I am sure this is not the first time and there is a stablished procedure somewhere.

 

[tatt]

I can't imagine the British government ever trying something like this again...

Oh I wish - but our government are only in power because they get way with worse things than this.

The Lancet have published poor science before (see below) , took a long time before that got retracted and did a hell of a lot of damage first. They clearly dont have good statistical advice on what they publish.

http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(97)11096-0/abstract

And in addition to this post "Gosh, don't know - a government's desire to save welfare money and get a lot of 'feckless' people back to work, plus a massive old-boy network couldn't possibly play a part..." you can add a psychiatrist's desire to have more patients to treat and hence a bigger empire and more status.

To be fair it probably started as a desire to help but the results didn't quite prove what they wanted and they then had too much invested in the study. Other doctors wanted ME to be psychiatric because they hate having to say I dont know what is wrong and I cant help you. They want to be able to push the heartsink patients at the psychiatrists so it will take a lot more than this to convince many doctors. Look how long it took doctors to accept hand-washing https://en.wikipedia.org/wiki/Ignaz_Semmelweis

 

[Forbin]

@Cort, where you say...

The PACE authors justified their criteria but in doing so exposed a basic mathematical error they made when they confused mean values for average values.

...did you intend to say that they confused the median and the mean (which is what the article at stats.org says) ?

The PACE researchers appear to have made a very basic mathematical error: confusing the median and the mean.
http://www.stats.org/pace-research-sparked-patient-rebellion-challenged-medicine/

[Average and mean typically mean the same thing.]

​

 

[Tony L]

USA get Ready for the up coming fight from Insurance try to make this Psy desease( so there IS a 2 yet cap on benefits)

But my question and the next step is as follow: what is the known procedure once you discover a paper is flawed? It is just bizare to me that after all we have done this paper keeps being cited. Do we need to call for a special hearing somewhere to present evidence? I am sure this is not the first time and there is a stablished procedure somewhere.

Unfortunately I don't see the people who produced this dross rolling over easily. They have staked their reputations on the presentation of this data this and will do all they can to preserve that. This is the peer review system at its worst. Science self-regulates. I'm pretty sure that the UK medical establishment would be none too pleased for the Lancet to do anything about this. But lets see whether pressure can tell in the end.

This dodgy data and the ideas that it is used to support will be debunked by good, objective science. When we are able to investigate the detailed physiological status of those with ME/CFS/FM then we will be in a position to get honest answers as to whether these so called therapies help anyone significantly or are in fact potentially harmful to some.

We may have the answer to that question in mind but it has to be proven. That's the way science works.

 

[Seven]

Unfortunately I don't see the people who produced this dross rolling over easily.

They will not, to be honest If I were in UK and be a ME organization, I would start paying for pressure, put paid articles about release the data or questioning the PACE trial. Few words keep it cheap and short, hit it hard in every week and as much outlets as possible. Create pressure. Then we go to the comments of articles and link to the very long detail explanation articles.
Pay Internet agency to push all the Articles critiquing PACE (imagine company) coming up in searches vs the bad old PACE data which is still ranquing on top when you search PACE.

Is time to play with the same tools., If we passively wait for change this will never happen.