Investigative reporters prod at an issue hoping to get a response. David Tuller’s piece on the problems with the PACE trial has definitely done that. Citing “numerous inaccuracies”, “reputational damage” and a social media flareup the study authors are definitely feeling the heat. They demanded equal time on the Virology blog to present their case and they got it.
Did the PACE trial authors attempts to put out a raging fire succeed?
Whether that’s going to help them is another question. They probably didn’t help their case by stating that Tuller made no attempt to contact them after Tuller reported that he offered to fly to the UK in person to meet with them and his attempts to interact with them via email failed.
So they got off to a bad start. But what about the main issues?
- Did they actually create criteria that produced the possibility that patients could be labeled as disabled and recovered at the same time?
- Did they really in the middle of the trial suddenly and significantly relax the criteria for recovery?
- Did they actually change the criteria for improvement in midstream to those a recently published trial suggested might improve their results?
- Did they pump up the participants in the trial by providing them mid-trial glowing accounts of how helpful it was?
Did they, in short, appear from the outside like panicked researchers desperately trying to salvage a failed trial?
Tuller asserted that the PACE authors attempts to correct his “inaccuracies” lacked substance and skated around the main issues raised. Let’s see what how they dealt with several of the issues Tuller raised.
13% of Patients in the Trial Meet a Threshold for Recovery On Entry into the Trial
How do you get around the fact that a significant percentage of the patients meet one of the thresholds for recovery upon entry into the trial? Lacking the ability to challenge it the PACE authors simply changed the subject and asserted that the patients needed to meet other markers to be labeled as “recovered”.
But really – so what? What about the fact that 13% of the patients – you know the sick people the trial was attempting to get well – were already halfway home to “wellness” (according to the PACE criteria) before the trial even started? How can you trust a trials results when they were to some extent guaranteed? (If 13% were halfway home I’ll bet 25% were knocking on the door; i.e. they just needed a little boost to meet one of the two factors for recovery – before the trial had even started.) Talking about gilding your lily.
If the authors could have gotten around this issue they would have – they’ve been dealing with it for years. They couldn’t….
- Retraction Watch – Score I
PACE Biased Results By Cheering the Participants On
This is another toughie for the PACE crew. Since they couldn’t deny they pumped the participants up mid-trial they could only argue that doing so was OK. That assertion was completely at odds with the study experts Tuller contacted.
Their newsletter did get the OK from their Independent Review Board…. suggesting that maybe things are done differently in the UK than in the U.S. However it happened Tuller asserted they were simply wrong:
“They introduced an uncontrolled, unpredictable co-intervention into their study, and they have no idea what the impact might have been on any of the four arms.”
- Retraction Watch – Score II
Bias Was Introduced by Changing the Two Primary Outcome Measures and How They were Analyzed
The PACE authors simply stated that they found better outcome measures and decided to employ them. Why they were better no one knows. Why the PACE authors didn’t do the sensitivity analyses Tuller believes such changes demand is unclear as well. The only thing we do know is that a similar, recently published trial got better results when they implemented the same changes the PACE trial did.
Hey, if it looks and walks like a duck – it’s probably a duck. This looks from the outside like an attempt for the PACE trial authors to pad their results.
- Retraction Watch – Score III
PACE Authors Revised the Criteria Downward For Most of the Recovery Measures
The PACE authors relaxed one of the measures of recovery so much that there was little difference between recovery and being disabled.
Again, the PACE authors admitted they relaxed most of the measures of recovery but beyond stating that they felt the new measures better reflected recovery they said nothing more. They didn’t just “relax” the requirements for recovery, however, they practically obliterated some of them. One measure that fell from 85 to 60 left the PACE trial with a “recovery” score of 60 and a “serious disability” score of 65. They also decided later in the trial patients who stated they were “much better” had, probably unbeknownst to them, actually recovered. One wonders how a trial could fail with such a headwind behind it…
- Retraction Watch – Score IV
Check out more issues the PACE authors raised and how Tuller reacted to them in the full blog here.
Tuller’s Full Pace Trial by Error PACE Series
- Trial By Error : the Troubling Case of the PACE Chronic Fatigue Syndrome Study I- the PACE Trial Deconstructed
- Trial By Error : the Troubling Case of the PACE Chronic Fatigue Syndrome Study II – the PACE Trial is Published
- Trial By Error: the Troubling Case of the PACE Chronic Fatigue Syndrome Study III: After Publication
- PACE Authors Respond
- David Tuller Responds
Conclusion
The PACE trial authors appear to have done themselves few favors by presenting their side of the argument.
Their responses demonstrated two things:
- Tuller’s critique and the response to it by the patient community has hit home and
- they don’t have answers to most of his questions.
If it looks like a duck and walks like a duck – is it a duck?
Notice that every change they made it more possible for the trial to get positive results. Notice how moderate the results were even with these added boosts.
Consider the fact that the researchers were never blinded to the results flooding in. Consider the possibility that they gloamed early on to the fact that things were not going their way and that changes needed to be made.
While the study findings were initially lauded (and misinterpreted) by the press (apparently with Chalder’s help) the controversy over the PACE trial continues to muddy it’s results.
Remember we’re talking about what is surely the most expensive study ever done on ME/CFS. The PACE authors are right. Reputations are at stake. Until they can satisfactorily explain – if that’s possible – the many questions surrounding the trial – it doesn’t appear that the controversy is going to go away.
Petition
Troubled by the PACE trial? Sign the petition to compel Lancet and other journals retract numerous claims in the PACE study papers including that 30% of the trial participants recovered.
it’s just sad….to much suffering already
One very bizarre fact about the incredibly flawed PACE trial is that it is the only medical trial ever to be funded by the UK Department of Work and Pensions.
This department are responsible for paying benefit money to chronically ill and disabled people.
I wish I knew what social media the PACE trial researchers use so I could check out their claims to be receiving “abuse”. From the way they react generally I think they regard any disagreement with their opinions, however politely it is expressed, as abuse.
As far as I know no=one has ever been charged with any offence in relation to this so called abuse, although the UK has laws against such behaviour http://www.bbc.co.uk/news/uk-29678989
Setting a normalcy and recovery bar at SF36PF of 60 would mean that its possible for a formerly super fit teen to regain the capacity of an 80 year old and be said to be recovered. How the heck can they justify this? Not a super fit and health 80 year old, but a typical one with all the health issues risks normally seen. Nursing home material. Some recovery.
Clearly, someone got their hands caught in the CFIDS Cookie Jar and are desperately trying to back peddle. The only thing Tuller can do now is cry abuse so they look like the victims instead of the CFIDS patience. Very Sociopathic behavior. The lies and innaccuracies are so blatant to those who actually understand this disease, the research and trials so infair and miniscule in terms of the true population of those afflicted, that the results would be laughable, had they not been abusive in nature and intention. Let us make sure this Dog and Pony show isn’t taken seriously and all the whining they are doing isn’t taken as sincerity.
I have noticed that most MD’s aren’t very good with statistics or analyzing study design. This seems to be the case with the PACE study authors. Can you imagine being them, having been given funding for the biggest ME study ever, winging it, not getting the results they wanted, tweaking it another way to make it work, massaging the results, presenting them to the world, then being publicly eviscerated for their lack of scientific skill? Does anyone expect them to say, “We’re sorry, we didn’t know what we were doing, we shouldn’t have pretended CBT and GET worked, we were wrong?” It’s never going to happen. We are lucky that both Tuller and Coyne have taken up the cause. I hope they can set things straight in the medical world.
Your dog pictures cracked me up, Cort. Thank you. The new web site format looks great!
Small point, Cort, but the boxes with an outline of a person cover some of the text posted by that person.
This is really great, to have all the links to the Virology Blog posts in one place. I did not know of the instant response by the PACE trial authors and more from Tuller. I have read it all now. Your own score card was helpful, too. The PACE trial always looked like a waste before it was done — the design seemed unlikely to reveal anything useful in my view. Now it is done and it revealed nothing useful, but we have the clear explanation why it failed, I need not rely on my intuitive sense of the design of a useful study.
It has been interesting to read how the three pout, while not responding to the points raised in criticism. People say hurtful things about us — and so what? I will not pout as they have done, as they get paid to do their harms to patients.
Thanks for pointing that out. I think that was a social media thingie. I removed it – is the problem gone now?
Yes, the problem is still there, Cort.
The pictures still cover part of text.
Working on it Helle – thanks for the reminder
The PACE authors and their study was funded, in part, by the Ministry of Work and Pensions in England.
So weird….
Not so weird when they were declaring UNUM among their vested interests after the trial finished.
New journal article on same PACE data discussed with author on BBC Radio 4 @ 6.55am on Thursday. More of the same…
The #MEAction petition that you link to, that calls for the misleading claims in PACE of “recovery” to be retracted and for the real results to be published, is going great guns!
It already has over 4,500 signatures in just over two days.
A big petition gets results and this one is getting HUGE!
Please, everyone, make it huger and sign the petition and pass it on to your friends and relatives – it’s on the internet like wildfire.
Pressure is mounting on the PACE authors and on the journals that published their work.
Let’s keep it up!
Here’s the link again:
http://my.meaction.net/petitions/pace-trial-needs-review-now
Wow- amazing response!
Cort, contrary to what the PACE authors say in their own paper, they did in fact lower all four of their “recovery” thresholds to make it easier for patients to appear to recover.
Here’s an extract from a post I made, addressing the authors, on the Virology Blog discussion:
The key issue is that you added – after the protocol and statistical analysis plan were written, apparently – fatigue/physical function trial-eligibility criteria to the case criteria (your paper, p. 2229) […] So to be diagnosed with CFS at the end of the trial, a patient had to fit the case criteria AND have a fatigue score better than trial entry AND have a physical function score better than trial entry.
All a patient would have to do to fail these supercharged criteria would be to scrape over just one of those trial entry thresholds, even if they still fell below the other.
And you used a post-hoc (according to you yourselves), nonsensical “trial recovery” analysis, in which patients could still be sick according to the CDC and London criteria, but classed as recovered as long as they no longer fit the Oxford. That’s the analysis that you broadcast in your abstract. I really can’t understand your claim not to have altered this fourth recovery criterion.
Here is an excellent summary of the issues (I have checked it against both your “recovery” paper and the study protocol and analysis plan and it is accurate):
http://www.meaction.net/whats-wrong-in-psychological-medicine/
They changed the scale to a likert scale which is only ordinal data and as such only limited mathematical analysis is possible. Slightly agree for example and strongly agree if used would not imply that strongly agree is twice as much as slightly agree. There is much academic debate on whether it is acceptable to treat a like et scale as interval data in order to be able to use statistical tests that can test hypotheses. I do think it will also be helpful here to post the responses from the trial main authors given yesterday. I cannot understand why anonymised data cannot be made available which it should be in any study. I wonder if anyone has seen sight of the ethics clearance documentation and participant information sheets that will have been issued as part of the ethics process? If their sample included people who were TATT and did not suffer from post exertional malaise then they could well have benefitted from CBT and GET.
You put in the link to the response, thanks. Another point worth making is when considering findings to a study academics are expected to consider those findings in the light of other published findings from other studies. Hence where there is irrefutable evidence of biophysical research ingicating changes eg brain scans and cell enzyme biology shortages as published by Prof Newton of Newcastle university we would need to see evidence on how CBT or GET improved the enzyme cell biology or changed the brain pattern as this would then be an objective physical measure of the improve to gained from the therapies suggested. The likelihood anyway is that there was a mixed sample as we know that ME patients were excluded and we also know that different academics write different percentages on those people who were referred with ME / CFS in fact were later found to have other illnesses. I recall Julia Newton suggesting a figure of around 40% and the Belgian clinic was higher.
Quite right Linda. Thank you
Clearly the researchers on the PACE study had no understanding of the disease ME/CFS.
How could walking and talking improve our immune function? Walking and talking has never raised out natural killer cells.
How could walking and talking correct our circadian rhythm and insomnia?
How could walking and talking correct all our endocrine system?
Walking and talking could stop our joint and muscle pain!
Calm our swollen lymph nodes?
And have a great refreshing night of sleep?
Really, could walking and talking return our herpes titers back to normal?
If only walking and talking could return our memory or ability to concentrate we’d be singing Hallelujah in the streets.
Good one Lois. You are so right. I saw a psychiatrist initially to help me get off narcotics as that was his speciality. I started seeing him regularly and did so for five years. I also followed his and all my other doctors advise on GET. I got worse as the years have gone by. I had to stop all exercise as I tend to get severe diplopia, vertigo and very high heart rate.
So did I fail at therapy and exercise. Absolutely not. I worked on my issues and feel emotionally strong (as can be with ME) and I’m not going to kill myself over doctors instructions to make exercise my job who really are very ignorant of our disabling problems with ME.
Katie, sorry that you spent 5 years on “treatment” that only made you worse. It is tragic that not one treatment is available for us. A black mark on the state of medicine.
I have four different recipes for fruit cakes. Cake 1 is my normal fruit cake recipe. Cake 2 recipe I have got from a celebrity chef. Cake 3 recipe I have got from a TV baking competition. Cake 4 recipe I have got from a professional chef in a 5 star hotel. I make all cakes following the recipes taking advice from helpful sources in some of my cake making. At the end I compare my cakes and see that all the cakes vary in texture, flavour, smell and such like. Cake D is better than cake C and cakes A and B are about the same. I rank them accordingly in terms of which recipes I think work best. Unfortunately when I enter them into a baking competition they are still far from perfect and do not win. They are as good as many others that are entered but still are not perfect cakes. Multiply this scenario by 600 people doing the same. I can analyse that data set until I am blue in the face but perfect cakes will only result if I do something different and I want a perfect cake! Why could all of the cakes not been perfect?
this is a typically rational peice . but in the Uk the press was is always full of stories about how we need exercise to get better after each pace trial story is published- doctors and everyone is influenced by this
“What about the fact that 13% of the patients – you know the sick people the trial was attempting to get well – were already halfway home to wellness before the trial even started? How can you trust a trials results when they were to some extent guaranteed? (If 13% were halfway home I’ll bet 25% were knocking on the door; i.e. they just needed a little boost to meet one of the two factors for recovery – before the trial had even started.) Talking about gilding your lily.”
This may be a fine point but my understanding is that patients in the 13% group weren’t actually halfway on their way to wellness at all. They were severely disabled. They didn’t stack the decks at enrollment, but rather manipulated their measurement outcomes.
At the beginning of the trial, disabled was 65 and under and recovery was 85 or above. The issue is that in the middle of the trial, they abandoned the original threshold for recovery and changed it to 60, five points below the threshold for entering the trial in the first place. 13% is the proportion of patients who were below 65 (threshold for entry) but above 60 (new threshold for recovery) when they entered the trial. This 13% could thus be classified as “recovered” before undergoing any of the four treatment conditions. But this is due to manipulation of the outcome measure, not the enrollment criteria.
Either way, anyone in that range of function is seriously sick. This is why the petition focuses so much on “congestive heart failure” sick isn’t a reasonable standard for recovery or wellness.
Explained in greater detail:
http://www.meaction.net/whats-wrong-in-the-lancet/
http://www.meaction.net/whats-wrong-in-psychological-medicine/
You missed my point Jen. The point was not that the patients were actually on halfway on the wellness but that according to the PACE trial criteria they were. That’s why I said the results were guaranteed. I will put wellness in quotes so that others don’t make that mistake.
Excellent analysis and summary, Cort.
The aspect of the PACE trial that gets to me the most is that the authors don’t just promote GET (they would not have been the only ones doing that), but moreover, they explicitly try to discredit pacing. A golden rule with this illness. That is injury added to injury…
Ouch!
Hi. After reading the book, Screaming To Be Heard, I read stories of others with fibromyalgia that describe themselves feeling just like. The book was written by a doctor. She said fibromyalgia is caused by a traumatic injury to your spine and triggered by rapid horomone changes.
Here’s yet another link to the Retract PACE petition. https://t.co/ZLHibkuQ8T
Also,a request for anyone who can translate the petition to Spanish, Italian, Portugese, Dutch, Danish, Norwegian, or any other non-English language (otheG than german or French–we already have those) to please contact MEAction.net.
Thanks.
The #MEAction petition that you link to, that calls for the misleading claims in PACE of “recovery” to be retracted and for the real results to be published, is now at an amazing 5,300 signatures from patients all over the world!
This is amazing! And it’s going to have a HUGE impact. It has Jen Brea and #MEAction behind it and they know how to make a splash.
Please sign it and tweet it and Facebook it and pass it on to your friends and relatives – let’s pile the pressure on the PACE authors and on the journals that published their work.
Here’s the link again:
http://my.meaction.net/petitions/pace-trial-needs-review-now
Wow – this must be the fastest-moving ME/CFS petition I’ve ever seen. One thing I like about it is that it is so well done. It doesn’t just ask the paper to be retracted – it specifics exactly which parts should be retracted. Unfortunately for the PACE authors it would cut out core findings in the paper.
I was offered CBT and pacing alongside patients doing the PACE trial in London around 10 years ago, though I’m not sure that my results were included in the trial for some reason. Patients had to fill out surveys about our progress at the end of sessions. With the specialist sitting opposite me as I filled it in – and then I had to hand it back to them to read – there was a lot of pressure to indicate that you were making progress. I am sure that other patients felt the same pressure to provide positive feedback and this probably skewed the findings. My specialists acknowledged the CBT was a waste of time for me (as I didn’t have any ‘issues’), though the pacing helped a little. My biggest improvement came from plenty of rest, though I still have mild to moderate ME/CFS.
I thought PACE was supppsed to be 5 year study. What happened to the rest of it? It’s more bizzare to me than anything else that they published only one year result, drop the trial and then claim that the gains were sustained after 2 years. 6MWT data after several years of trial would’ve surely shown that there was no significant gain in any of the groups. How do I know? I’ve been walking religiously for 8 years and my walk speed/distance hasn’t budged.
There is a very simple and obvious explanation for these shenanigans: The PACE trial is a complete fraud perpetrated to protect the profits of their insurance industry paymasters.
Last week NIH tossed Sir Simon overboard with their announcement of an intramural biomedical research program and a promise to increase extramural research funding. Even if NIH fails to follow through the damage is done. David Tuller and James Coyne have torn these studies to shreds, and we can expect other academics will soon pile on.
When Sir Simon’s academic institutions and the insurers walk away he may start to understand what it is like to be abandoned by society and kicked to the curb.
We owe it to the families of dead patients to see the “Wessely School” held to account for their actions in a court of law. In spite of the daily misery of this illness, I intend to stay alive long enough to see this happen.
@jimells
Not just the insurance industry; as people have pointed out, it was also funded by the Department of Work and Pensions (DWP). There have been rumours of mandatory CBT/GET for benefit claimants, with sanctionns (money stopped) if refused. So far this has not happened – perhaps because the trial’s positive outcomes were not robust enough. Personally, I have no doubt that the DWP expected positive outcomes, a huge bias.