Glass Half Full

There has been a lot written lately about the lack of replicability of many studies, of things we thought we knew but now have to hold at arm’s length. The social sciences have long been notorious for this, but medical claims are also looking fragile.  It is hardly surprising that entities with something to gain or lose, such as pharmaceutical companies, might bury results that showed nonsignificant or even contradictory outcomes. Not honest, but not surprising – and pharma is not necessarily exceptional.  Researchers vying only for honor, or perhaps publication to keep their jobs, do it as well.

We can always blame the journalists for oversimplifying and raising our expectations unrealistically.  That’s fair. We can also blame researchers for not having enough courage, objectivity, humility, sense of honor – pick a missing virtue, really – and that’s fair too.  Yes it’s difficult and I might be worse at it, but it IS their job, no? Yes, it is problematic that those classic experiments in every freshman psychology book are not all proving out, undermining theories and further research that was based on those truthies.  But the point is to get them right.  Right?

Yet I do share some of the concerns of the overdefensive researchers.  In the presence of doubt, it is true that much of the general public will conclude that we know nothing, and therefore taking Vitamin C for cancer is just as good as all those fancy-schmancy chemicals the medical establishment is pushing. Lord knows knocking down the idea that there is something magical about things that are “natural” is often on my mind. Similarly, in my own field, sometimes diagnosis is inaccurate and/or treatment does little good. Yet some things can be shown to work often, and that has value.  More important, we have things that we know don’t work, and can at least not waste our time on them.

40% replicability means that 40% did turn out to be true.  We thought it was 90-100% and it isn’t, but we still do know some things we didn’t a century ago.  Real life doesn’t tie up as neatly as it looks at the science fair. Periodic overthrow of 30% of what we were sure was true might be the normal course of events.  All this upheaval, chicanery, and accusation might be the only way forward among human beings. Particularly in the social sciences, where all theories rather obviously have societal implications, we come up against a difficulty. Because the possible invisible confounding variables are so many, all experiments must of necessity show only narrow, tentative results. Ooh, except everyone who goes into that field wants things that are much grander.  They want large theories of everything, explaining why boys do what they do, or whether criminals played particular video games.