The Pace  Trial

In March 2011, the outcome of the rather infamous PACE trial was published in Lancet. The trial sought to prove that behavioral therapies were both effective  and cost-effective in treating chronic fatigue syndrome (ME/CFS).  At over 8 million dollars in cost and years in development it was easily the most expensive trial ever for chronic fatigue syndrome. This was the study to rule them all; a successful trial would set UK treatment priorities on a CBT/GET course for the forseeable future.

The results were underwhelming to say the least but while the study authors were careful to note the ‘moderate’ benefits achieved, some of the media were less cautious, stating that an effective treatment had been found.  A closer look  at the study suggested that even the more moderate statements by the study authors may have overstated the results. The overly positive media reception and the authors statements prompted a group of ME/CFS patients with math and statistical backgrounds (plus some fantastic html skills (see below)) to take a closer look.

They labored over the PACE study for over a year to produce their  report; a standard setter in rigor, scope and creativity from the patient community.

The PACE Trial lives on. The 2011 report was just the first analysis; in 2012  a study asserting the PACE trial proved CBT/GET was both effective and cost-effective was published, and a 2013 study asserting a 22% recovery rate for CBT/GET participants in the trial lead the authors to state

This study confirms that recovery from CFS is possible, and that CBT and GET are the therapies most likely to lead to recovery

A Tour de Force

The patients analysis of the study suggested that positive results were, intentionally or not,  baked into the  study design. If that’s so, it’s a bad sign for PACE, as its results were underwhelming to begin with. Even worse some of the same problems appear to be present in some of the standard tools used to assess patients well-being. If this is true then the assessments of many studies may be up for grabs.

One of the most troubling aspects of the PACE report was its lack of objective measures of functioning. The original trial proposal included actometers to measure physical activity levels but after they were dropped the authors relied on a 6 minute walk test. Be sure to check the often hilarious animation as the healthy controls and CFS patients race each other to see who walks farthest…..

As you read the blog be sure to check out  the entertaining and educational animations – they’re a blast. The first one takes a look at the exercise component of the trial.

Setting the Pace….For Futility (?)

The authors came up with 10 conclusions…among them

• some scales designed for CFS and widely used in CBT/GET and other CFS studies have fundamental faults
• some of the statistical measures the PACE trial used erred on the side of showing improvement
• patients assigned to Specialist Medical Care actually showed a greater degree of improvement than patients given CBT/GET
•  the statement that all ME/CFS patients can receive at least moderate benefits from these therapies is not borne out by the study results
• CBT/GET does appear to help a subset of patients but we don’t know which type that is
• The positive effects CBT/GET had on CFS in the PACE trial were about the same as these therapies do in disorders like heart disease and multiple sclerosis.

Their reasons for these conclusions are explained in a wonderful, down to earth manner. If at some points you get stuck (statistics are involved in some areas) simply go to the animations, which often have soundtracks and sit back and enjoy because the animations are simply astounding; using the latest html 5 technology, the authors are able to get across complex factors in often disarming fashion..

This analysis suggested the CBT/GET field itself has a ways to go, something a Cochrane Report alluded to in its overview of the field.

A Talk With Graham

One of the leaders of the PACE analysis effort, I asked Graham how it all got started and about what they found..

Can you talk about how and why this effort got started and how many people it involved?

Originally Bob and I were working on a project together analysing the PACE results and we published a blog that received a number of positive comments. While this was going on, a group of people in Phoenix Rising started what turned out to be an enormous, high-quality thread, looking in detail at the PACE study. Bob persuaded me to join Phoenix Rising, and I started to read through the PACE thread.

Many people there were discussing what to do with all the issues that they had found. Several people wrote papers or submitted letters for publication (e.g. Tom’s “harms” paper): there was a lot of discussion on how to put all these ideas together. I raised the subject of perhaps trying to take a different, more accessible approach alongside the conventional analyses. A number of people liked this idea, so we discussed it further and so formed a team to produce this response.

Not all the contributors have been from PR, the numbers have fluctuated according to energies, and health and other projects have also dictated how much each person could contribute, but overall I would say that around a dozen or more people have been involved.

There seemed to be lots of problems with the PACE study: you found that their use of averages instead of medians skewed the results upwards; the control group was not adequate; there was no attempt to assess functionality; the study design was poor. Have you looked at other CBT/GET studies and, if so, do you find similar problems?

I haven’t spent the same amount of time on other studies (this project has taken nearly a year), but I have looked at hundreds of different studies. Most have major problems with the sample size being small. Almost all of them seem to be similar in their reliance on subjective assessments, and completely fail to impress me.

“60- the New 75” or How to Change the Design of a Study Twice to Achieve the Results You Want (refrain from eating prior to viewing)

There are studies which compared the use of actometers/pedometers with subjective assessments and concluded that actometers were more reliable indicators of activity levels, but expensive. One trial, using actometers and questionnaires to assess the value of GET, found that the questionnaires suggested activity levels had improved, but the hard evidence from the actometers showed that activity levels had not changed. PACE was funded to include actometers, but decided to drop them. Apparently wearing a wristwatch type of fitting around the ankle was considered to be too great a demand on the patients.

It is pretty clear that the questionnaires can lead to falsely optimistic conclusions.

Your statistical analysis of a group of patients who tracked their wellness over a year suggested that some of the CFS patients are too sick for the scale to handle ie; since they’re already at the bottom of it, if they worsen it wont show up in the statistics. Again, given that these scales are used in other studies, is this kind of a field-wide problem?

So many people with ME/CFS score 3 (the maximum) on most of the questions. This clumping of patients’ scores at the most severe end of the scale must be a major problem, and clearly prevents many patients from recording when they feel even worse. That is probably why there is such a conflict between studies claiming that GET does no harm and surveys of patients which claim it does. The Chalder fatigue scale in particular is hardly used outside studies of ME/CFS, so it is difficult to make many comparisons with other illnesses, but the few that we can make puts the ME/CFS scores at the worst end.

Can you briefly explain how it is that someone could improve enough for the authors of the PACE trial to say they’ve recovered but they’re still ill enough to meet  the criteria for CFS. (Did I get this right?).

Yes, you got it right. One can simultaneously qualify as severely fatigued and able to enter the study, and as “back to normal”. The basic reason is very simple – they changed the way that they measured the answers to the fatigue questionnaire. They also have a rather peculiar standard for “normal”. But it is very hard to explain properly in words. That is why we have so many graphics in the project (it’s much easier if you follow the graphics in the extra layers to conclusions 3 and 6 in particular).

How Turn A Small Gain into An ‘Effective Treatment’

The authors changed both the way that they scored the answers on the fatigue scale, and changed the criterion for “normal function”. Originally a patient was enrolled on the trial if they indicated that six or more of the eleven items on the questionnaire gave them difficulties. The target was to halve the number of problems that they had noted on the questionnaire, which seems a valid aim (though not necessarily what you or I would call a return to “normal”). After the trial had started, the authors changed the assessment to a scoring system, where each item was scored 0 to 3, and they decided upon a target of 18 (which itself is quite controversial – six out of eleven items could still be scoring 3 points each – maximum difficulties). According to our survey, a more accurate match to halving the original number of problem areas would have been a target of 12 on the new marking scheme. (Remember that the aim is to reduce the fatigue score).

According to your analysis, patients actually improved more at the Specialist Medical Clinics than they did with the CBT/GET practitioners. To put it another way, if the authors’ original focus had been to prove that Specialist Medical Clinics were more effective at treating ME/CFS, they would have had ample evidence to do that. In fact, the improvement with CBT/GET was so small that it could have been explained by the natural swings of the illness that most people experience. I don’t remember reading anything about this. How could this have been missed?

I don’t think it has been missed at all, but most of the submissions in reply to the PACE trial have been very lengthy and complex. It is easy to show what is happening with the right graphics (although it takes a while to get that right!), but much harder to explain clearly in words.

It would be wrong to believe that the Specialist Medical Clinics were effective at improving patients’ health. The improvements were tiny: it’s just that the improvements that GET or CBT added on were even smaller. But the fact that 40% of the people sent to the Newcastle ME centre were found not to have ME/CFS hammers home the need for people to be assessed by specialists, particularly the severely ill who are lucky to even have their GP visit them.

Also, I don’t believe that there is any evidence out there about the extent of the natural variation in fatigue levels for people with ME. We could be a first!

You believe that CBT/GET should not be labeled as a treatment or therapy but more as a support system that provides help with sleep, pain control and other areas. Why do you believe this?

It is more than that. A couple of years ago I fell over and broke my hip, and I hadn’t even had a drink! While it was healing, I followed all the proper advice about exercises etc. and when it finally healed, I had no need for physiotherapy. But of course there are some people who would be unsettled by breaking their hip, and would be reluctant to engage in the full range of stretching and moving while the hip was healing. They may also suffer from a lack of confidence afterwards in repeating whatever it was that they were doing when they broke their hip in the first place. But a broken hip is purely a physical problem. Physiotherapy may be needed to help people come to terms with it and regain their confidence.

Cha, Cha, Changes Galore…Plus A Missing Bottom: Conclusions and Questions

I see no reason whatsoever to think that ME/CFS is any different from a broken hip, and what people need is time to adjust to the illness, support in learning to make the most of things, and education and experience in recognizing the warning signs before doing too much. One study found that people who were members of an ME/CFS support group showed less improvement when they attended an ME clinic than those who did not. The study decided that ME/CFS support groups have a negative influence. But of course, the sensible conclusion would have been that ME/CFS support groups start patients down that path of adjusting and learning.

In fact, what I would like to see is a trial that applies CBT and GET to a random group of healthy adults. I have the feeling that they would show greater average increases in stamina and fitness than shown by the patients in the ME trial. What would that tell us?

• Check out the Complete Paper Here

________________________________________________

 

Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial.White PD, Goldsmith KA, Johnson AL, Potts L, Walwyn R, DeCesare JC, Baber HL, Burgess M, Clark LV, Cox DL, Bavinton J, Angus BJ, Murphy G, Murphy M, O’Dowd H, Wilks D, McCrone P, Chalder T, Sharpe M; PACE trial management group. Lancet. 2011 Mar 5;377(9768):823-36. doi: 10.1016/S0140-6736(11)60096-2. Epub 2011 Feb 18.

PLoS One. 2012;7(8):e40808. doi: 10.1371/journal.pone.0040808. Epub 2012 Aug 1.Adaptive pacing, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome: a cost-effectiveness analysis. McCrone P, Sharpe M, Chalder T, Knapp M, Johnson AL, Goldsmith KA, White PD.

Psychol Med. 2013 Jan 31:1-9. [Epub ahead of print] Recovery from chronic fatigue syndrome after treatments given in the PACE trial. White PD, Goldsmith K, Johnson AL, Chalder T, Sharpe M; PACE Trial Management Group.