Recently, Nature published an article about data fraud in medical research. The conclusion was, in essence: it’s really bad and it’s a genuine problem.
Gasp. Who would’ve thought?
Well, me, for one.
That’s mainly because of the work of my former research mentors, Professor Ben Mol and Doctor Wentao Li. They were some of the original researchers to speak out about data fraud, back when it was very uncool to call out other researchers and suggest they had conducted fraudulent studies. For the most part, it still is – but these data detectives (fraud finders?) do it anyway.
As such, they are global experts in research fraud, and the people to go to if you’re worried about how a study has been conducted – or if it even was conducted.
Now, all of this is very far from my usual purview of glitter pens, cute cat videos and vintage designer goods. So how did a pint-size 20-something with a sparkle in her eye get involved in the deep, dark underworld of scientific fraud?
Why thank you for not asking. Now let me tell you a story.
Back in my medical student days, when I was umming and ahhing over whether I actually wanted to be a practising doctor (spoiler alert: I did not), I added an honours research year to my degree to explore the world of academia. Think of it as a peak nerdom gap year.
Prof Mol and Dr Li were my project supervisors; under their leadership, I collected, cleaned and reanalysed the original data from randomised controlled trials that compared specific methods of labour induction. This style of research is called an individual participant data meta-analysis (IPD-MA), which is the gold standard. If there’s an IPD-MA on your chosen topic, that’s the research you should be using.
And if there’s not one for your chosen topic, it might be worthwhile asking why.
One day, while cleaning a dataset, I realised that some of the numbers appeared to be repeating themselves in a pattern.
I panicked. And it wasn’t just a little panic either, but a full 48-hour meltdown. What had I done? How could this be? Had I made an error? Had I corrupted the data? I spent the next two days checking, double-checking and triple-checking every little thing to work out what I had done wrong.
But the data I was looking at was correct, and I hadn’t made a mistake.
This realisation led me to a second (meta?) panic: what would Prof Mol and Dr Li think?
Finally (and sheepishly), I met with my supervisors and explained my situation. They quickly confirmed my fears – this data was most definitely not randomised. And it looked like it was a copy-paste job of a few original data points.
It was then, sitting in their office, that I was introduced to the nefarious reality of data fraud analysis. I felt like I’d been given the map to a murky underworld.
From there, we developed additional checks and analyses of all submitted datasets to verify their authenticity, we reported our findings to the journal that published the concerning dataset I detected, and the article was retracted. This was only the beginning of the fraud concerns we came across though.
At the completion of my honours year, I contacted authors across four different IPD-MAs to ask about original data sharing. The email response rate was over 90%, but the data share rates were only around 50%.
In other words, most authors replied and indicated an interest in collaborating with us – but after we explained what the collaboration would involve (i.e. that they’d need to share their original data), many authors stopped replying. Nearly half of them, in fact.
Nature’s article on untrustworthy trials helps to raise the alarm about the authenticity of some studies, but it only focuses on developing data integrity checklists for published materials. And like seeing a dating profile versus meeting someone in real life, published materials can be deceiving. There is no substitute for the original data.
All of which begs the question: should journals require researchers to submit their original data with their manuscript? After all, what have they got to hide?
At Wellmark, we take healthcare research seriously. A lot of our work involves a rigorous examination of the scientific literature, to ensure that our clients’ communications are both compelling and supported by the strongest possible evidence.
PS. Prof Mol and Dr Li continue to do fantastic work in data fraud detection – if you’re interested, check out this article about Prof Mol’s experience in the field.
Dr Lith Peel,
Healthcare writer at Wellmark.
Connect with me on LinkedIn
