Fact-Checking and Reviewing

With Tim King having asked his sister (well-known to us) to press hard on his research for Addiction Nation, including an assessment of the Controversial or Disputed Research, I am a little more alert for problems of misinterpreting research at the moment.  Or perhaps there is just more bad publishing these days, as publishing houses leave the fact-checking to the authors. There was a recent furor when Naomi Wolf had a key statistic to her work exploded while she was being interviewed. She thought the phrase "death recorded" meant gay men had been executed, when it actually meant "Judge is required to order death sentence but doesn't want to, so we're calling this guy dead and letting him go." You can see where she might assume what she did, but when one is publishing, we expect better.

There's more.  Greg Cochran noticed that historian Allen Guelzo didn't understand the physics of bullet trajectories, in his book about Gettysburg.  Reading the questionable paragraph, it may be that Guelzo knows but just didn't explain it clearly, so that others came to the wrong conclusion. Gray Kimbrough was suspicious of Paul Dolan's use of the American Time-Use Survey and decided the author had an important point completely wrong. Dolan's claim was that married women say they are happy when their husbands are present, but confess the truth of how unhappy they are when the husband leaves the room.  He got this from misinterpreting the phrase "spouse absent." I grant that I have at times gotten a quick misinterpretation stuck in my head - I see how it happens.  On the other hand, that seems amazingly stupid for a researcher to not pick up on a more likely meaning of the phrase. And I didn't draw a major conclusion for my book from that misunderstanding either.

The Vox article hits on two other incidents I had not heard of: Jill Abramson's book Merchant of Truth, and Clive Thompson's upcoming book Coders both have serious factual questions. The NYTimes shares a problem with Thompson, quoting a study that apparently doesn't exist. There is also the current replication crisis in several fields, especially psychology, which Bethany has highlighted for us.

Is it worse now than in years past, or did we just not catch them them before, but are now? As the replication crisis is proving to extend back decades, it suggests the latter. We should all hold facts we believe because we read them somewhere lightly at this point.