Cremieux writes about what he calls Cargo Cults in science: things that attempt to work without knowing the principle behind it. He chooses experiments on semagutide extending lifespans in mice as a pertinent recent example.
The new semaglutide paper does not meet this standard: the researchers behind it failed to set up an experiment that could inform us about semaglutide’s effects on extension of healthy, normal lifespan...
The semaglutide study attempted to anticipate this objection, but did so in a self-defeating way. They referenced six studies on B6 mouse lifespans to note that the control group had a lifespan that was close to typical. But the numbers in the citations don’t support their case. For example, the Ogiso et al. paper found mean lifespans of 798 ± 161 days for B6J females and 763 ± 163 days for B6N females, which includes early deaths—meaning it’s almost-certainly mechanically brought down relative to the numbers in the semaglutide study.
I get the principle. I am seldom interested enough in medical research these days to put in the effort. But for those who still try and follow these things, it's a good caution to be aware of with all the new miracles being offered at the candy store. Researchers are still chasing cool and sexy results and can fool themselves, as they always have.
1 comment:
Richard Feynman said: “The first principle is that you must not fool yourself—and you are the easiest person to fool." It isn't always nefarious, sometimes people really do fool themselves. I spent my career working in R&D, and I saw this. People would design experiments that subtly favored the results they expected. Or they would throw out outliers because "there must have been something wrong with that experiment."
Or, they would do just a handful of experiments, and jump to a conclusion that wasn't really justified, because they were just SURE it had to be the right answer. I once had a new guy tell me he had solved a longstanding problem in just three experiments. He was very proud of himself, and couldn't understand why nobody had solved this simple problem years ago. I told him to go back and repeat the experiments. A few days later he walked into my office, exclaimed "You son of a bitch!", turned on his heel and walked out.
Post a Comment