Sunday, February 12, 2023

Failure To Revise

 One of the main lessons of the Tetlock and Gardner book on Forecasting, and their "Good Judgement" evaluation and training since then was that Superforecasters have an ability to make themselves evaluate coldly and revise accordingly. If you read much military history, you will find the same theme repeated, that the successful generals were those who could revise strategy depending on what they saw in front of them and evaluate coldly whether they had the resources to attempt a change. When I was in emergency psych, I joked that the value of projective testing is that it measured the psychologist's ability to consider different diagnoses. Joke, but also quite true. in a case puzzling everyone that is not a small thing. If you read the crime and justice literature from any perspective, you keep coming up against the idea that an innocent (at least of that crime) person got railroaded or a case went cold because the detectives were so convinced of one theory of what happened that they were unable to look at evidence right in front of them.* It's the whole point of the Bill James books about true crime, Popular Crime and The Man From The Train,that detectives, newspapers, and the general public believing false narratives interfered with true narratives being considered. Such as the crazy idea that Jon Benet Ramsey's parents were guilty of the murder despite the complete lack of evidence. (The evidence was basically that people found it creepy that the little girl was in pageants and therefore her parents must be creepy, and therefore could have murdered their daughter. Which says something creepy about us.) Those of us who followed James as a baseball statistician could have seen that coming. He hammered home the point repeatedly throughout the 80s and 90s that we would overvalue both individual players and types of players on the basis of bad evidence, but be unable to reevaluate them objectively.

It is the principle behind Bayesian reasoning, to treat every new bit of information as an opportunity to run the numbers again. 

So it really isn't surprising that Philippe Lemoine thought our main failure during Covid was our inability to revise. No need to appeal to explanations of bad motives or paranoia, or need to control - other than to note that there will be mild but constant pressure in a single direction for all those things - the natural arrogance of liking the answer you got the first time explains a lot, all by itself.

*It's a common device in murder mysteries, letting the reader know that one of the suspects is guilty of something else that they are lying about, such as covering up an affair or cheating others out of an inheritance, causing us to regard him as a guiltyish sort of person in general, and therefore the likely murderer. It may be this human tendency that causes us to be too ready to speak to the police right away without legal advice, intuitively wanting to be seen as not a guiltyish person.  It has probably been a solid strategy for most of human existence.

No comments: