Sunday, June 10, 2018

How Many Levels? Superforecasting.

There was a program at work which encouraged people to recognise co-workers for good deeds done: acts of kindness or grace under pressure, projects completed. After about two years of this someone had the idea "Let's Recognise the Recognisers!" because it is of course a prosocial behavior to be encouraged to look for good deeds in others and mention it.

And yet...I wondered if this were not just a bit too much, so I appended a typed "Let's Recognise People Who Recognise The Recognisers!" We do have actual work we are supposed to be focusing on, after all. It is like the new Christian praying in The Screwtape Letters who has been praying for humility and discovers that by Jove, he actually has shown some - which he is immediately proud of.  And then, sadly, concludes that this requires its own prayer. It could go on endlessly,*  unless the man were able to step back and laugh at himself a bit.

I thought of this in reading Tetlock and Gardner's Superforecasting. He lays out very quickly that one of the first necessary skills is to question assumptions. I think about 10% (or 40%, or 23.9%) of the population does that - not always with skill, but well enough to get the idea.  These are the people who keep telling us to "think outside the box." My own view is that we have plenty of people who think outside the box, and they cause a good deal of trouble, believing themselves smarter than the others.  I am not merely claiming that the wisdom of crowds suggests that the conventional wisdom  is likely to be right after all.  I am noting that those who pride themselves on questioning the conventional wisdom seem to immediately eliminate the possibility that the CW is one of the possibilities.  They need to reject the conventional wisdom and find something else to be true. It shows how smart they are, how unconventional, how much better than all the rubes. In the early 2000's there were politicians who bragged that they were capable of nuance, not like those other troglodytes. I have found this attitude common among those who reject the faith when young.

That 10% of questioners gravitate to each other and create a new conventional wisdom. They reinforce these ideas to each other, to show that they have learned the language of the smart set. In all my writing and thinking, I may have provided very little of value beyond my ability to question the questioners.  Let's look at all these brilliant new ideas the 10% has been coming up with and apply the same questioning to that.  Let us regard the New Conventional Wisdom as the dreaded box that we must think outside of.

I was challenged by Tetlock's experiments to wonder if "Whoa, what if being part of the 1% that questions the 10% is not enough? What if there is a wiser 0.1% who question the questioners of questions? How far down the rabbit hole is it good to go?" And "What if I have the same blind spot, in needing the New Conventional Wisdom to be un-true? It makes a fellow think, Jeeves." My previous experience with CS Lewis, and the handbook, and the recognisers allowed me to anticipate the answer: treat all answers as provisional and keep updating. Protect none of them. Even the best of them might be only 90% correct.

Unfortunately, I'm not sure I do that despite understanding the principle.  Learning to go down one level, questioning the smartasses and remembering that sometimes the hoi polloi got it right is pretty simple, and automatic for me at this point. I may also have some ability to notice assumptions that others breeze by. Added note on the book: in describing the attitude toward fate that many of the best forecasters show - that things did not have to turn out the way they did and were not bound to happen - Tetlock and Gardner don't really get the doctrines of free will versus predetermination quite right. As Christians don't get them all that right either, perhaps that's understandable.  The description I read this week is that determinism and free will are still tied in the fourth quarter. Which is humorous, but captures things well.

Book recommended.  I may have a go at trying this.  I have a suspicion I might be only moderately good at it, but perhaps with experience...

*Like the cover of the Boy Scout Handbook, with the scout carrying a handbook which in turn has a picture of a scout carrying a handbook.  This bothered rather than amused me as a boy, because it was going to be impossible to draw and print that on a visible level and so would have to be approximated which just seemed wrong. Children can be OCD at times.  This child, anyway.

2 comments:

james said...

"Thinking outside the box" isn't always all it is cracked up to be.

I attended a seminar on "how to attract and retain talent" in IT. They didn't get around to my comments until in the hall afterwards, unfortunately. I suggested that a university, nominally brimming over with knowledge and education, might want to offer free courses as a benefit to staff. Apparently the other Big 10 universities do, but not us. Maybe the box is invisible?

Texan99 said...

I like questioning assumptions, but of course the great thing is to get in the habit of doing it unobtrusively and acknowledging that, most of the time, the assumptions are good enough to get by on. Now and then, it will be a terrific idea to pipe up and say, "Hey, we're only using this shortcut because our assumptions in the past led us to believe it was reasonably accurate, but look--things are completely different now." It used to be pretty safe to ignore certain kinds of lifestyle risks before using public blood-bank stock, but now maybe we should rethink, even though it will be expensive and inconvenient. It used to be quite dangerous to eat undercooked pork, but the reason behind the assumption disappeared some time back, and it's no longer necessary to cook a chop into shoe-leather. But it's still a good idea not to cut raw vegetables on the same cutting board as the raw chicken.