Many people know that Julie Rehmeyer has chronic fatigue syndrome (ME/CFS), but not many people probably know that she's also an award winning journalist with a speciality in statistics. Rehmeyer has written op eds on ME/CFS for the New York Times and several pieces on the PACE trial.
[fright]
[/fright]MEAction reported that Rehmeyer recently gave a talk at the biggest statistical conference in North America titled "Bad Statistics, Bad Reporting, Bad Impact on Patients: The Story of the PACE trial”. U.K. statisticians attended the conference as well.
The American Statistical Association thought enough of her presentation to urge its delegates to attend the talk to hear how “how bad statistics harm patients and our profession”.
Multiple Breakdowns
In her talk Rehmeyer, a specialist in statistics journalism, stated that the PACE trial provided "one of the most damaging cases of bad statistical practice that I have personally encountered in my career as a journalist"
The trial has been hugely influential, and for many doctors is the one study they know about ME/CFS. Rehmeyer, however, stated that the data was so "highly flawed" as to be "uninterpretable."
Multiple breakdowns, she believes, needed to occur in the realm of statistics, science, journalism and in public health in order for the study to get published in its present form. She asserted that ME/CFS patients are being hurt by it every day.
Even the New York Times, which gave one of the more "nuanced" accounts of the trail, got key aspects of the trial wrong. It sreported that "About 30 per cent of patients given cognitive behavioural therapy or graded exercise made a full recovery to normal levels of activity...". The Time didn't realize that the criteria for recovery had been doctored early in the study so that some people could both meet the criteria for ME/CFS, and be labeled as recovered at the same time.
Rehmeyer also noted that an ME/CFS patient's fatigue score could be somewhat better (due to taking sleeping medication) but have her physical functioning score drop - and not be able to do something like vacuum her living room - and still be listed as not having ME/CFS at the trials completion.
The PACE trial also concluded that CBT/GET were more effective than ordinary care despite the fact that the improvements were not "clinically significant". After the tiny improvement seen on the walking test the participants were still in worse shape "than patients waiting for a lung transplant."
Encouraging Response
MEAction reported that Rehmeyer was encouraged by the response to her talk:
Lancet Continues to Avoid Issue
Rehmeyer's talk was particularly important because statisticians may be able to apply the most pressure on Lancet to release and reanalyze the study data.
One of the most influential medical journals in the world, Lancet, now has the distinct honor of having published a study that a talk at a major statistical conference stated was doing harm to the statistics profession; i.e. it's study is a lesson on how not to do statistics.
The ball, right now, is in Lancet's court. In 2011, Lancet asserted "Your complaint and (the authors’) response were discussed at the highest management level and this group of executive editors was fully satisfied that there were no grounds whatsoever on which to take further action…From an editorial perspective, the case is now closed.", but promised to respond to the recent request for a re-analysis of the data by 42 scientists earlier this year.
Several statisticians have joined the fray. Bruce Levin, a biostatistician at Columbia University, told Julie Rehmeyer that “The Lancet needs to stop circling the wagons and be open. One of the tenets of good science is transparency.”
Eight months later Lancet still has not responded. Richard Horton, it's editor, has called the complaints about the PACE trial "vexatious". Lancet, it appears, is playing a waiting game, hoping that the controversy will go away. Hopefully Rehmeyer's talk will help to keep the pressure on.
[fright]
The American Statistical Association thought enough of her presentation to urge its delegates to attend the talk to hear how “how bad statistics harm patients and our profession”.
Multiple Breakdowns
In her talk Rehmeyer, a specialist in statistics journalism, stated that the PACE trial provided "one of the most damaging cases of bad statistical practice that I have personally encountered in my career as a journalist"
The trial has been hugely influential, and for many doctors is the one study they know about ME/CFS. Rehmeyer, however, stated that the data was so "highly flawed" as to be "uninterpretable."
Multiple breakdowns, she believes, needed to occur in the realm of statistics, science, journalism and in public health in order for the study to get published in its present form. She asserted that ME/CFS patients are being hurt by it every day.
Even the New York Times, which gave one of the more "nuanced" accounts of the trail, got key aspects of the trial wrong. It sreported that "About 30 per cent of patients given cognitive behavioural therapy or graded exercise made a full recovery to normal levels of activity...". The Time didn't realize that the criteria for recovery had been doctored early in the study so that some people could both meet the criteria for ME/CFS, and be labeled as recovered at the same time.
Rehmeyer also noted that an ME/CFS patient's fatigue score could be somewhat better (due to taking sleeping medication) but have her physical functioning score drop - and not be able to do something like vacuum her living room - and still be listed as not having ME/CFS at the trials completion.
The PACE trial also concluded that CBT/GET were more effective than ordinary care despite the fact that the improvements were not "clinically significant". After the tiny improvement seen on the walking test the participants were still in worse shape "than patients waiting for a lung transplant."
Encouraging Response
MEAction reported that Rehmeyer was encouraged by the response to her talk:
Lancet Continues to Avoid Issue
Rehmeyer's talk was particularly important because statisticians may be able to apply the most pressure on Lancet to release and reanalyze the study data.
One of the most influential medical journals in the world, Lancet, now has the distinct honor of having published a study that a talk at a major statistical conference stated was doing harm to the statistics profession; i.e. it's study is a lesson on how not to do statistics.
The ball, right now, is in Lancet's court. In 2011, Lancet asserted "Your complaint and (the authors’) response were discussed at the highest management level and this group of executive editors was fully satisfied that there were no grounds whatsoever on which to take further action…From an editorial perspective, the case is now closed.", but promised to respond to the recent request for a re-analysis of the data by 42 scientists earlier this year.
Several statisticians have joined the fray. Bruce Levin, a biostatistician at Columbia University, told Julie Rehmeyer that “The Lancet needs to stop circling the wagons and be open. One of the tenets of good science is transparency.”
Eight months later Lancet still has not responded. Richard Horton, it's editor, has called the complaints about the PACE trial "vexatious". Lancet, it appears, is playing a waiting game, hoping that the controversy will go away. Hopefully Rehmeyer's talk will help to keep the pressure on.
- Check out MEAction's post here
Last edited:
![ME/CFS lecture | Dr Nancy Klimas | 2011 [Chronic Fatigue Syndrome / SEID / New Zealand] - YouTube](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
