Friday, May 04, 2012

Jim Manzi's "Uncontrolled"

A new book by Jim Manzi discusses the lack of evidence that most of our planned interventions have going for them before they are put into place. Uncontrolled: the Surprising Payoff of Trial-and-Error for Business, Politics, and Society.
I would add churches to that list. We love problem-solving approaches that sound like they should work. Because they were designed by pastors - an entirely different type of training, usually - who have really thought about this, and been a few places, and talked to people. So it only stands to reason that their ideas should work, right?*
Almost all programs fail when tested. That is, the vast majority of new "education, safety, and health" programs that appear to work based on non-experimental analyses cannot demonstrate statistically significant benefits in excess of costs in replicated, controlled experiments.
There is additionally some reference to a favorite topic over at Bad Data, Bad: Failing to distinguish between correlation and causation.  The full interview is here.

*All church growth numbers should be first examined in light of the population growth in the area, for example. Churches tell themselves that it's their music ministry, or their preaching, or their Sunday School, or their caring attitude that brings 'em in. 

HT: Bird Dog at Maggie's Farm.

1 comment:

james said...

Chesterton: Napoleon of Notting Hill

The human race, to which so many of my readers belong, has been playing at children’s games since the beginning, and will probably do it till the end, which is a nuisance for the few people who grow up. And one of the games to which it is most attached is called, “Keep tomorrow dark,” and which is also named…“Cheat the Prophet.” The players listen very carefully and respectfully to all that the clever men have to say about what is to happen in the next generation. The players then wait until all the clever men are dead, and bury them nicely. They then go and do something else. That is all. For a race of simple tastes, however, it is great fun.

For human beings, being children, have the childish wilfulness and the childish secrecy. And they never have from the beginning of the world done what the wise men have seen to be inevitable. They stoned the false prophets, it is said; but they could have stoned true prophets with a greater and juster enjoyment.