In my rather obsessive questioning of my own positions, especially those which are controversial, I wonder especially about those which appear on the surface to be wrongheaded, to go against the common wisdom, the obvious. However much I love the Chestertonian (and rabbinic) approach of asking immediately Perhaps the opposite is (also) true, I am aware that this can sometimes go wrong; the cliche is in place for a good reason, and "what everyone thinks" is true, while I am simply being obstinate and contrary.
This arises especially in discussion of The Elites. Shouldn't it be the most natural thing in the world that those who have devoted their life to the study of a subject and earned their daily bread by considering it from a variety of angles in consultation with like-minded others actually be the experts, the people we turn to for answers. Why should we think that some jibroney who's read a couple of articles online is more likely to get economics, or Venezuela, or climate change right? At a minimum, we should see why people who have been listening to those experts find it strange that alternative voices should take priority.
Because I am often on the side of the jibroneys these days I think it wise to consider whether this makes any sense at all or I am just being difficult. In my own field of mental health, for example, I can often see where many of the experts, both historical and current, have gotten things dangerously wrong, but also where the general public is equally stupid or worse. What it often comes down to is listening impatiently to someone lecturing me about my own knowledge and thinking to myself the downfall of your particular point-of-view is that it does not - and sometimes rather spectacularly does not - actually work. James may know to his great frustration the blind spots of modern academic physicists, or JMSmith of geographers, or my half-dozen lawyers, doctors, and engineers the same in their fields. But that doesn't mean they therefore sign on to Worlds In Collision, or Scientology, or QAnon, just because. Those groups may have gone down ridiculous paths with arrogant assurance, but they are seldom dolts. There are things they do know, which you and I do not, and emphatically, those populists who are certain that the Common Man just somehow knows more about these things for no apparent reason also does not know.
If I were to (again) engage in an extended discussion of how it is that the people who know a great deal can go so deeply awry I would start with the assumptions passed down among them. When they are wrong we often have to go very far back into their training, the formation of their bubbles, and even their personalities to figure out how that came to be.
So I doubt, and I wonder. And then I encounter writings entirely by surprise, when reading up on other subjects entirely, which illuminates the issue from another direction. In reading CS Lewis "Lilies That Fester," written in 1955, which could be written today, with only slight changes of vocabulary. He writes from start to finish about how the Refined might be wrong, might not be reliable in matters of literature, but in the midst of that broadens the topic to the influence and even control that such people have over many aspects of British society in his day.
Mr. Forster feels anxious because he dreads Theocracy. Now if he expects to see a Theocracy set up in modern England, I myself believe his expectation to be wholly chimerical. (We now know that Lewis was enbtirely and obviously right in this and Mr. Forster ridiculously wrong, AVI) But I wish to make it very clear that, if I thought the thing in the least probable, I should feel about it exactly as he does. I fully embrace the maxim (which he borrows from a Christian) that ‘all power corrupts.’ I would go further. The loftier the pretensions of the power, the more meddlesome, inhuman, and oppressive it will be. Theocracy is the worst of all possible governments. All political power is at best a necessary evil: but it is least evil when its sanctions are most modest and commonplace, when it claims no more than to be useful or convenient and sets itself strictly limited objectives. Anything transcendental or spiritual, or even anything very strongly ethical, in its pretensions is dangerous and encourages it to meddle with our private lives. Let the shoemaker stick to his last. Thus the Renaissance doctrine of Divine Right is for me a corruption of monarchy; Rousseau’s General Will, of democracy; racial mysticisms, of nationality. And Theocracy, I admit and even insist, is the worst corruption of all.
But then I don't think we are in any danger of it. What I think we are really in danger of is something that would be only one degree less intolerable, and intolerable in almost the same way. I would call it Charientocracy; not the rule of the saints but the rule of the χαρίεντες, the venustiores, the Hotel de Rambouillet, the Wits, the Polite, the "Souls," the "Apostles” the Sensitive, the Cultured, the Integrated, or whatever the latest password may be*. I will explain how I think it could come about.
The old social classes have broken up. Two results follow. On the one hand, since most men, as Aristotle observed, do not like to be merely equal with all other men, we find all sorts of people building themselves into groups within which they can feel superior to the mass; little unofficial, self-appointed aristocracies. The Cultured increasingly form such a group. Notice their tendency to use the social term vulgar of those who disagree with them. Notice that Mr. Allen spoke of rebels against, or deserters from, this group, as denying not that they are "intellectual" but that they are "intellectuals” not hiding a quality but deprecating inclusion in a class. On the other hand, inevitably, there is coming into existence a new, real, ruling class: what has been called the Managerial Class. The coalescence of these two groups, the unofficial, self-appointed aristocracy of the Cultured and the actual Managerial rulers, will bring us to Charientocracy.
But the two groups are already coalescing, because education is increasingly the means of access to the Managerial Class...
I do recommend it all, as it does suddenly expand into issues we are discussing everywhere for the last twenty years and even in 2021. Lewis has foreseen much of it.
*We might now say “The Elites.”
Let me open this discussion you are inviting with a proposal: the "elites are wrong" position does not imply that the jabronis are right. It may be that no one is right.
ReplyDeleteVery often they are wrong because they are reasoning on the basis of some complex model they have developed, e.g., of economics, of climate, of disease vectors, etc. Now what I have noticed about these models is that we don't actually ever develop one that works. It's a relatively easy problem to put a rocket on Mars, which we do well (and I never doubt the 'elites' in that field when they say they think they can do something like that, though occasionally experiments fail). When the models extend into greatly complex fields like fluid dynamics, economies, managing climate change, and the sort, I notice these models always fail.
That doesn't mean jabroni can build a better model; but it does mean that we should reject the elites as leaders. Jabronis will at least only screw their own lives up locally. Putting an elite in charge of big things will screw everything up for everybody. And they'll proceed confidently down the road of screwing things up because they think they know what they don't in fact know.
Where I find myself disagreeing with elites it is also often because I see them as pursuing their own class interests while giving the false impression (maybe even to themselves) that they are pursuing Justice and Right. The jabronis are probably not better at distinguishing Justice and Right from what is good for them and those like them; but there are a lot more of them. If we let them have their way, the net benefit likely will be bigger because the class being benefitted is much bigger.
"It may be that no one is right." This. Yes.
ReplyDeleteI also agree on the amount of damage that can be caused. CS Lewis wrote that a bad cow can only do so much damage, a bad farmer only a bit more, but a bad genius can cause huge damage. Not for nothing do our movies have the stock character of the Mad Scientist.
It is true that they can also do more good that less-powerful ones such as you or me. But that is a snare and a danger. Gandalf saw that. Saruman did not.
Assistant Village Idiot: Shouldn't it be the most natural thing in the world that those who have devoted their life to the study of a subject and earned their daily bread by considering it from a variety of angles in consultation with like-minded others actually be the experts, the people we turn to for answers.
ReplyDeleteElite is a somewhat broader term than expert. Someone born rich and well-connected is certainly part of the elite, though not necessarily an expert in anything. Nor do we typically consider a Hollywood actor an expert, but they are considered part of the elite.
On the other hand, expertise necessarily has an increasingly important role in modern society. Scientific advances have affected virtually all aspects of modern society.
An appeal to authority is an inductive argument. An expert in a valid field of study speaking to a consensus in their field is more likely to be right than a non-expert. That doesn't mean the expert is always right, and the non-expert is always wrong, just more likely so. Hence, you generally get medical opinions from your medical doctor, not your bartender.
The wisdom of crowds on the other hand doesn't refer to the individual, but the collective wisdom of the group. And when the group is most affected by decisions, then certainly the group should have a say on those decisions, for good or ill.
Grim: Very often they are wrong because they are reasoning on the basis of some complex model they have developed, e.g., of economics, of climate, of disease vectors, etc. Now what I have noticed about these models is that we don't actually ever develop one that works.
ReplyDeleteAll models are wrong, but some are useful. — George E. P. Box
Grim: When the models extend into greatly complex fields like fluid dynamics, economies, managing climate change, and the sort, I notice these models always fail.
That's clearly not the case. Weather is a highly chaotic phenomenon, but modern weather models are much more accurate than previous models, or not having a model at all. Weather models are useful.
As for climate models:
https://climate.nasa.gov/system/internal_resources/details/original/2299_Updated_CMIP3_Model_Comparisons_Hindcast_Forecast_20210122.JPG
"On the other hand, expertise necessarily has an increasingly important role in modern society. Scientific advances have affected virtually all aspects of modern society."
ReplyDeleteYet sometimes, scientific advances have *reduced* the amount of expertise necessary in many aspects of modern society, by encapsulating the relevant expertise in devices and procedures.
Navigating a ship using a moving-map GPS today equires much less expertise than did navigating a ship via celestial navigation in 1840. Managing the inventory in a chain store, in which a centralized computer at headquarters tells you what you need to order, requires less expertise than managing the inventory in an individual store in 1950.
Often, when a system with embedded expertise is implemented, it will fail (or at least be less-successful than it could have been) because insufficient attention is given to the environment and incentives affecting the front-line workers who are the interface between the system and the world. In her book The Good Jobs Strategy, Zeynep Top describes the problems afflicting the sophisticated (and no doubt expensive) inventory control systems at Target and another company:
ReplyDeleteA former Target cashier said she was under so much pressure to ring up sales as quickly as possible, so if a customer bought 10 bottles of Gatorade–in two flavors–she would scan the first one and then hit the quantity key for ten. The inventory system thought the store had sold 10 lime-flavored Gatorades and no cherry-flavored Gatorades, rather than the mix that had actually just been sold. And the cashier, who had received only 8 hours of training before starting work, probably wasn’t even aware of the problem she was creating via this shortcut.
She cites a study of another company ($10 billion in sales) which found that the system had the right information for only 35% of the products…for the other 65%, the discrepancies between the system inventory balances and the actual quantities available averaged 5 units…a third of the target stocking levels. In one case, a certain item was continually out of stock, to the frustration of a regular customer. It turned out that the inventory system thought there were 42 of these on hand, whereas there were actually none. AND, since this particular store hadn’t sold any units in several weeks (because they didn’t have any to sell), the system automatically reduced the target stocking level for that item!
https://chicagoboyz.net/archives/60771.html
At a more directly-physical level: the centralized fire-control system in the B-29 bomber was an electromechanical marvel, automatically computing the necessary leads for the guns with the human gunners needing only to accurately track the target. BUT, a GE engineer who worked on the system noticed that under the stress of combat, gunners tended to tightly grasp a support, in a manner that could deflect the sight enough to cause a miss...potentially fatal in a fighter-attack situation. He tried to get Boeing to reinforce the necessary aircraft structure, but was not successful during the course of the war.
David Foster: Yet sometimes, scientific advances have *reduced* the amount of expertise necessary in many aspects of modern society, by encapsulating the relevant expertise in devices and procedures.
ReplyDeleteThat's intrinsic to increased specialization. If your neighbor is a blacksmith, you don't have to be. You pay for horseshoes with the money you make from selling your apples. If your horse throws a shoe, though, you might not have the skills necessary to fix the problem. You may have to walk your horse to your neighbor smithy for want of a nail. Nevertheless, the specialization is a net positive for society.
David Foster: BUT, a GE engineer who worked on the system noticed that under the stress of combat, gunners tended to tightly grasp a support, in a manner that could deflect the sight enough to cause a miss...potentially fatal in a fighter-attack situation.
Sure, but even with all the inherent problems of new technology, the modern military would probably be able to successfully take on a Roman legion in the field of battle.
The first point I'm making above is different from your point about specialization. Most likely, horseshoes today aren't made by an individual smith with forge and hammer, but are made in a factory by machines, operated by individuals who probably do not know all that much about the production process as a whole. The amount of horse-shoe-making knowledge required in the economy becomes much less for a given quantity of horseshoes produced.
ReplyDeletere the second point, my point is not that new technology is generally inferior to older ways, or even that the B-29 gunnery system was necessarily inferior to manual sighting at individual turrets (opinions are mixed, but one Japanese interceptor pilot, who we might consider a genuine expert in this context, gave it a very good review!)...rather, that creators of such systems often fail to properly understand and deal with the conditions under which they will actually be used.
David Foster: The first point I'm making above is different from your point about specialization.
ReplyDeleteWhat the apple farmer knows about horseshoes is that he goes to the smithy for a new shoe. He doesn't need to know more than that. What the smithy knows about apples trees is that he goes to the apple farmer for apples.
David Foster: The amount of horse-shoe-making knowledge required in the economy becomes much less for a given quantity of horseshoes produced.
That's right. There is a nail specialist. The smithy no longer needs to know how to make crude nails — just where to buy manufactured nails. The point being that increased specialization means that more and more knowledge is incorporated in the product. Really, how many people know how a pencil is made?
https://www.youtube.com/watch?v=IYO3tOqDISE
David Foster: rather, that creators of such systems often fail to properly understand and deal with the conditions under which they will actually be used.
Never rely on 1.0 of anything. Then again, where would technology be without early adopters?!
Z: "That's clearly not the case. Weather is a highly chaotic phenomenon, but modern weather models... As for climate models..."
ReplyDeleteYes, we've all been impressed by the accuracy of the weatherman.
David Foster:
I think you raise a good point. The weather reports are terrible, but I do have an app on my phone that shows satellite imagery of fronts moving through. I can't rely on the experts, but I can see for myself in a way that I couldn't before whether or not I'm going to get wet.
But that's back to the way that rocketry admits of genuine expertise, while economics apparently does not.
Grim: Yes, we've all been impressed by the accuracy of the weatherman.
ReplyDeleteWeather reports are far more accurate nowadays than they were a century ago (about 90% accuracy over five days in most climates). In particular, countless lives have been saved from hurricanes. Not sure why you would argue otherwise. Did someone rain on your parade?
You should try riding a motorcycle and see how it affects your outlook (on this and many other things). I'm not talking about what the weather will be like in five days, about which the weather service is perfectly useless around here. I'm talking about whether or not it will rain in the next twenty minutes.
ReplyDeleteI'm definitely not an expert in meteorology, but I have a reasonably good ability to look at the sky and tell the answer to that question. I have an even better ability to look at the satellite imagery on my phone and tell. The weather service is terrible at it. Last week they were off on the start of the snowstorm by nine hours. If you'd done nothing other than look at the radar, you'd have known it was coming in many hours earlier than predicted. Yet they confidently continued to assert that snow would not begin before midnight.
Even then, you get non-Bayesian percentages: it's already raining across the Zip code, but they tell me there's a 70% chance of rain.
This is a problem as old as Socrates. Shoemakers have techne knowledge that is demonstrable; roketeers are like that now. Other people claim different kinds of knowledge, but don't really have it. Then as now, it's the ones who think of themselves as the Great and the Wise who are most likely to make false claims to the latter, and believe in them devoutly -- even enough to kill over, as they killed Socrates.
This discussion is excellent, but I was deeply impressed as I continued with that essay "Lilies That Fester" about how prescient Lewis had been about the nature of education and the growth of the Managerial Class which has received a type of education that as widely recognised as a Real Education, suitable for those ascending to authority, but is not only useless but positively ruinous in leadership, simply because it is unable to question itself. I liked the quote as far as I had read so much that I posted prematurely. Please continue from there to see what Lewis has to say about education. Barack Obama mentioning "offhand" that he had read Urdu poetry is a sickening illustration of this. In one sense, he is not to be blamed - though we might expect more independence of thought from one who wishes to rule over us - because he is illustrating that he he discerns exactly what the trainers of the Elites required of him. He says as much in his many autobiographies, that he learned early to ell people what they want to hear.
ReplyDeleteAs for the definition of the Elites versus the Experts, I don't think the distinction falls along the lines Zachriel describes. I think it perhaps should fall along those lines and it would make it easier for us all if it did so. But I was not thinking of the mere rich or the celebrities when I used the term "elites," though I think Z's definition is one of the valid modern usages. I was thinking of the people I went to William and Mary with who are quite smart in their way but excelled at the worst parts of the educational attainment Lewis describes later in the essay, and then went on to run the country in Washington. The children of the wealthy (federal gov't-heavy) suburbs of DC in VA and MD often sent their children to W&M, Bucknell, Georgetown, UVA, Duke, or Johns Hopkins, who happily went back to staff State, Defense, Commerce, etc. We rail about the Ivies, but those schools go into government in a big way.
I went to school with them and was impressed with few of them, but they did know how to drop references to Urdu poetry and have the right crease in their pants once they graduated from denim. Even in the early 70's when education in an older sense was still recognised if not practiced they were beginning to be ascendant. I was very good at it myself and have a deep intuitive sense of how much intelligence versus cleverness vs wisdom vs bullshit it requires.
Experts are indeed something else. I know real experts in my own field and I cannot touch them for knowledge nor would I lightly contradict them. However, I have noticed two things: once the roads to the credentials and the positions of authority are established, people can hack into them with cleverness and social intuition substituting for at least some of the real ability their predecessors showed; second, even those with undeniable abilities conform without even noticing it to the groupthink. This is gradual and the brightest of them retain an ability to draw back for years. But it is there, even among those I admire most who have displayed considerable independence. It just happens.
"a type of education that as widely recognised as a Real Education, suitable for those ascending to authority, but is not only useless but positively ruinous in leadership, simply because it is unable to question itself."
ReplyDeleteReminded of an old Neptunus Lex post, in which he reflected on a verbal interaction with a subordinate: "He had abused his familiarity, but I had abused my authority. Mine was the greater crime."
https://thelexicans.wordpress.com/2016/07/06/old-ghosts/
There are too few people in positions of authority who are capable of that kind of self-analysis and self-criticism. Education, as currently conducted, doesn't help much, and may hurt.
Grim: I'm talking about whether or not it will rain in the next twenty minutes.
ReplyDeleteNor do weather projections tell you when and where the very next raindrop will fall. Or the price of tea in China, for that matter.
Rather, models explicitly project only certain aspects of a phenomenon. In the case of weather, a projection of scattered showers may mean you will or will not get rain, but that rain will fall here and there in your geographic area. The prediction isn't wrong if you don't get rain. It's only wrong is it doesn't rain in the geographic area or rains everywhere. (The coverage is typically expressed as a percentage.)
Modern weather prediction is much better than guessing and significantly better than in decades past.
Grim: I'm definitely not an expert in meteorology, but I have a reasonably good ability to look at the sky and tell the answer to that question.
Which is why, of course, the military always checks with Grim before launching any major aerial operations, when lives are at stake.
A movie producer was making a movie in the wilderness and had a dire need to know the weather for the next day. He had heard about how the native people had a sense for the weather, so he asked a local. The old native looked at the sky and pondered for a moment before telling the movie producer it would be clear the following day. Day after day, the Indian accurately predicted the weather, rain or shine, wind or calm. Finally, the movie producer asked the old native what signs he saw, how could he tell. The old native answered, "I heard it on the radio."
Grim: I have an even better ability to look at the satellite imagery on my phone and tell.
Which is why, of course, Florida always checks with Grim when there is a tropical storm developing in the Atlantic, when lives are at stake.
Modern models run on supercomputers have advanced weather prediction considerably. They don't predict with absolute precision, but they do predict with some accuracy. When it matters, people rely on weather models. When it doesn't matter, people complain about the weather — but no one ever does anything about it.
Right. "We can model the climate accurately enough to justify complete seizure of power over your economy and way of life; we can model the weather well enough to be accurate except when we're not, which we ourselves estimate only to be a minority of the time. Will it rain later this afternoon? Well, now, that's just not what we do here!"
ReplyDeleteThis is a straw man of yours that you're busily beating up. At no point have I argued that hurricane forecasting isn't better than it was before we invented satellites and airplanes. Of course it is; but it's not the supercomputers that are making it so. It's the ability to see what the things are doing. Those hurricane computer models are constantly updated with new data about the actual facts observed by the technology I've been praising from my first comment on the subject. And they're still wrong; I've been in hurricane evacuations where the thing missed the evacuated city entirely, while they were still predicting a strike less than a day out from landfall. It was still worth doing just in case, because the risk of death and damage was high; but the model wasn't close to right about what really happened.
It’s important to start with facts.
ReplyDeleteGrim: Of course it is; but it's not the supercomputers that are making it so. It's the ability to see what the things are doing.
Prediction of tropical storm tracking has been improving for decades due to improvements in computer modeling.
https://s.w-x.co//util/image/w/NelGraph1.jpg
Prediction of tropical storm intensity didn’t significantly improve until the development of the supercomputer-based Hurricane Weather Research and Forecasting model starting in 2007, which has led to a rapid gain in accuracy.
https://s.w-x.co//util/image/w/NelGraph2.jpg
This is an excellent example of someone thinking he knows more than experts.
I see you're very fond of this strawman, but I was never defending the proposition that hurricane models aren't better than a hundred years ago, or even ten. (I certainly never said I could do it better; just the opposite, I opened with the proposition that just because experts are wrong doesn't mean ordinary guys are right.) I also didn't even propose that we shouldn't evacuate in cases where they might be wrong, given the cost/benefit analysis.
ReplyDeleteThat they do whatever they do as well as they do it because they're being given constant updates from those with eyes on what the storm is actually doing strikes me as completely noncontroversial. If you think you can model a hurricane accurately without such inputs, go do it. It's a better use of your time than talking with me.
The original idea of numerical weather modeling was developed by a guy named Lewis Fry Richardson, in the 1920s..it involved, and still involved, the division of the atmosphere into cells, and numerically solving the differential equations that describe the interaction among those cells.
ReplyDeleteThere being no computers in the 1920s, the approach that Richardson envisaged for solving the problem was a huge hall, like a theater, with hundreds of human computers solving the equations for particular cells and passing the results to adjacent people. There was to be a conductor, orchestrating the whole process.
The potential accuracy of numerical forecasting gets better the smaller the cells are, and the size is limited on the downside both by computing capacity and by the problem of knowing the initial conditions in all those cells.
What I observe anecdotally is that forecasts a day or so out are a lot more accurate concerning temperature, precipitation, and ceiling/visibility than they are with surface and near-surface winds. Probably due to the effects of variable terrain and building/other structures.
The error graphs are interesting..the improvements in track error since 1970 look pretty linear. I wonder how much of this is due to modeling per se (computer sppeds and model math) versus how much is due to better & more frequent input data, per Grim's point.
ReplyDeleteDavid Foster: I wonder how much of this is due to modeling per se (computer sppeds and model math) versus how much is due to better & more frequent input data
ReplyDeleteBoth are required, of course. However, massive data came first. Computers did not have sufficient power until more recently to churn through terabytes of data with sufficient speed as to be useful.
David Foster: per Grim’s point.
Grim’s original point about complex models was that “we don't actually ever develop one that works.” Weather models are very useful, and have improved significantly. With regards to tropical storms, current models can predict three days out what used to only be predictable one day out.
What I observe anecdotally is that forecasts a day or so out are a lot more accurate concerning temperature, precipitation, and ceiling/visibility
ReplyDeleteBecause, as Grim pointed out, we can fairly readily predict that the temperature, cloud cover, and precipitation over *here* is going to become the temperature, cloud cover, and precipitation over *there* with a fair degree of accuracy over a twelve to twenty-four hour period. We have better observation tools, data collection, and means of disseminating the information to those interested. As he pointed out, that's not nothing but it's also not a validation of modeling.
Get much beyond projection from actual conditions headed your direction, and the 5 or 10 day forecast regresses to the average seasonal weather in your location.
Zachriel is tripping over molehills here. That weather models are now much better - which no one is disputing - does not mean that they are mechanistic and never fail. They sometimes do. The last three inches of snow for yesterday's storm, predicted as last as 11AM to still be on the way, never arrived. Just missed us. So one could take the view that the model "mostly worked," or that it failed. Fussing that Grim put his left foot in rather than his right foot before doing the Hokey-Pokey is not the sort of discussion that gets us anywhere.
ReplyDeleteChristopher B: As he pointed out, that's not nothing but it's also not a validation of modeling.
ReplyDeleteIndeed, being able to predict the course of a tropical storm three days out what used to only be predictable one day out results in lives saved. Perhaps you don't find that significant, but we are rather fond of human beings. Call it a peccadillo, if you will.
Assistant Village Idiot: That weather models are now much better - which no one is disputing
(We appreciate that you are trying to create an environment conducive to conversation with your regulars. We don't intend to antagonize you or Grim. However, that doesn't change the underlying facts.)
With regards to models of complex phenomena, Grim said, “we don't actually ever develop one that works.” We disputed this statement and provided the demonstrable example of tropical storm prediction.
Assistant Village Idiot: So one could take the view that the model "mostly worked," or that it failed.
When a model makes a probabilistic prediction, then that is how it is to be judged. We can then note whether the model is better than random guessing, better than someone eyeballing satellite maps on their smart phone, or better than previous models. With regards to current models of tropical storms, we can answer yes to all three measures.
Nor is this discussion a diversion from the original post. To loop around, Grim pointed to models of complex phenomenon as an example of the "elites" being wrong. We showed, rather, that it is an example of wrongly discounting the validated findings of experts. In other words, it was directly relevant to the topic.
--
Everybody complains about the weather, but nobody does anything about it. — Charles Dudley Warner
This month is the 75th anniversary of the ENIAC, the first electronic computer with generalized programming capabilities. I'm working on a post re this milestone, but some ENIAC applications interesting from a modeling standpoint were:
ReplyDelete--artillery and bomb trajectory calculations..the reason for the Army's original sponsorship of the machine
--thermonuclear calculations for H-bomb initiator design
--critical-mass calculations for fission
--numerical weather prediction
The trajectory work was pretty straightforward and deterministic (if you ignore unpredictable wind effects.) For the thermonuclear modeling, the equations had to be greatly simplified to fit on the machine, and there was a dispute between Teller and Ulam as to whether the results actually meant that the initiator design would work or not...Teller asserted that they indeed did prove feasibility, Ulam thought otherwise. Ulam was right, and a different approach had to be used.
The fission work was interesting in that it used a 'Monte Carlo' method in which thousands of individual neutrons followed random paths, as they apparently do in the physical world. And the weather prediction work, with modeling by John von Neumann and some of the programming by his wife, Klara, was purely experimental at that stage.
When a model makes a probabilistic prediction, then that is how it is to be judged.
ReplyDelete"Even then, you get non-Bayesian percentages: it's already raining across the Zip code, but they tell me there's a 70% chance of rain."
You know, as I reflect on this discussion, I realize that the whole introduction of tropical storms was another aspect of the straw man. I never asserted that tropical storm models were among the kind of models I don't think are within human potential. Tropical storms are relatively simple cases of weather prediction, because they are (a) highly observable, (b) perduring, and (c) powerfully stable on their own terms. It's much easier to predict that an existing tropical storm will push this way or that way than to say if it'll rain here at 2 or at 12.
Now in fairness to the weatherman, I do live in a difficult place to model. Maybe it works better in Indiana. This terrain is both mountainous and an Alpine rain forest. You can get a long way to perfection in accuracy by asserting that it will probably rain on any warm day; but if you want to say just when and why, there's a lot to deal with.
Still, I think AVI has said more than he is getting credit for saying here: "So one could take the view that the model 'mostly worked,' or that it failed."
All the evidence being presented is that the errors are smaller than once; but this is still also evidence that the models are generally always in error. If 'being in error' is failure, then the very evidence for their improving accuracy also proves that failure is the normal condition.
Meanwhile probability can't be used to say that a 70% probable prediction is 'accurate' if it rains or if it doesn't rain. "We said it would probably rain, and it did, so we were right," doesn't capture the fact that you also said that it might not rain, and it did, so you were wrong. "We said it probably would rain, but might not, and it didn't, so we were right," likewise doesn't actually show that your model was accurate at all.
"We said it was 70% likely," is even worse when it was already raining, because the actual probability was demonstrably 100%. That model was just wrong, simpliciter.
Likewise if it was allegedly 70% likely and it didn't rain. The model was wrong; it turned out to be 100% likely that it wouldn't rain.
One might say that the probabilistic model proves the point that models are always wrong; because the real probability will finally prove to be 0 or 1, and the models never return that.
But I think the real thing to point out is that the argument is that they should be accepted as wisdom even though they can't be falsified. If they say it was 70% and it turns out to be 1, then they were right because they predicted in line with results. If they say 70% and it turns out to be 0, well, they predicted a non-zero chance of 0. So they must be right!
The more faith one places in these things, in light of these facts, the more I think it's really an article of faith. I don't have a problem with articles of faith; but I'd like to place my faith where faith belongs. That is not in men, however wise, but in higher things.
Grim: I never asserted that tropical storm models were among the kind of models I don't think are within human potential.
ReplyDeleteYou made a very broad claim about complex models. Tracking tropical storms is one of the most complex problems in meteorology. Tropical storms are very sensitive to initial conditions, chaotic by definition.
Grim: All the evidence being presented is that the errors are smaller than once; but this is still also evidence that the models are generally always in error.
All models are wrong, but some are useful. — George E. P. Box
Grim: if it was allegedly 70% likely and it didn't rain. The model was wrong; it turned out to be 100% likely that it wouldn't rain.
You don't seem to understand probability. If our model says that rolling an eight with two dice is 14%, and we roll boxcars, we don't exclaim the "model was just wrong." Probabilistic models are judged through multiple trials. If there is a 70% chance of rain, well, by golly, sometimes it's going to rain!
Z: Tropical storms are very sensitive to initial conditions, chaotic by definition.
ReplyDeleteAlso, to predict tropical storms you have to be able to predict the general weather conditions. That's because not only do tropical storms meander like a spinning top, but they are pushed about by meandering pressure zones and jet streams as they interact with Earth's topography.
What I understand about probability is that people like gambling models because you can replicate the throw. It’s possible to say what the probability is because you can repeat the exercise. Thus, you can judge whether the accuracy claimed held up.
ReplyDeleteNature doesn’t repeat in the same way.
In Bayesian probability, however, once a thing is happening the probability of it happening drifts to one and stays there for the duration. Thus, I can judge that these models are simply wrong probability wise insofar as they predict otherwise.
However!
“All models are wrong (but some are useful)” is, in the dominant clause, exactly equal to the claim that “models are always wrong.” Thus, we actually agree. These models are always wrong.
Grim: It’s possible to say what the probability is because you can repeat the exercise.
ReplyDeleteRepeatable experiments are quite possible when predicting chaotic systems. The question is whether the model can reasonably predict the behavior of the chaotic system. Consider whether it will rain tomorrow. The simplest model is to flip a coin, that is, the probability of rain is 50%. A more complex model might predict 90% chance of rain on some days and 10% on others. After a large number of tomorrows come and go, if the more complex model more frequently predicts the weather, then we can say the more complex model is more accurate than the simplest model. More refined models may be even more accurate, though all have a margin of error, which can also be determined.
Keep in mind that even well-established models, such as Newtonian mechanics, are "wrong" in the sense of only being approximations. And even deterministic gravitational systems can be chaotic over sufficiently long periods of time. Everything in science is approximation, but not all "wrongs" are equal.
Your claim was that such models don't "work", but that is not correct. They do work, that is, they approximate reality sufficiently as to be useful. Meteorological models work at saving lives, and are certainly more useful than saying “we don't actually ever develop one that works.” It's important to understand this point, because all scientific knowledge depends on the concept.
Looping back to the original topic, not knowing everything doesn't mean not knowing anything.
This comment has been removed by the author.
ReplyDeleteLook, this actually has been a useful discussion because it uncovered something about where exactly we disagree. I understand what you think you're doing; I also understand that, pragmatically, I get better results not using the expert "will it rain today?" model, but my own ability to judge. I would tend to say the model is wrong; you want to describe the same facts as the model being less useful than it might be, while still noticing that the degree of error has been decreasing over time. I don't dispute that the models might be less inaccurate than once, but that in many areas they're still too inaccurate to be -- as you would say -- useful.
ReplyDeleteThis also uncovers that there are models that might someday be accurate; perhaps we'll get there with the weather, just as we got there with the rocketry (which I agree works fine). There's a difference in kind with models of economics, psychology, etc., where there's a necessary aspect of imputing the effects of things like psyche, etc., that are not directly observable. We have models for guessing what they'll do, but we can't actually even see what they are, not in any direct way.
So: good talk, after all.
We still end up with the second problem, which is that there are class interests at work that experts are bad at accounting for; very often "we don't know, but my model is the most useful thing we've got" turns out to mean "everyone should agree to be governed by me," which leads to outcomes that unsurprisingly favor the interests of the class of experts. That's a psychological problem and a philosophical one; psychologically because we have a lot of confirmation bias that humanity seems bad at getting away from (so: given that model, we should tend to avoid concentrating power in any class, even 'the class of experts'); and the philosophical question of when non-experts should be consulted in matters that affect their lives, even if their decisions might 'be worse for them' according to someone else's model of what right looks like.
Those are harder problems, but it is important to have made progress on the easier one.
One thing about models...weather models, in particular...is that the forecasts, at least those accessible to the public, tend to cover fairly large areas and to be updated intermittently. If I look at the weather being reported at a major airport about 30 miles to the west, and the winds are from that direction, I can often forecast what is going to happen right here better than can the official forecasts, which are not specifically concerned with these few square miles.
ReplyDeletere Grim's point.."We still end up with the second problem, which is that there are class interests at work that experts are bad at accounting for; very often "we don't know, but my model is the most useful thing we've got" turns out to mean "everyone should agree to be governed by me"...sometimes it's not so much class interests as *individual* interests. In my ENIAC thermonuclear example, both Teller and Ulam were members of the class of Los Alamos scientists, but Teller had a strong ego interest, and maybe a strong career interest as well, in having his detonator approach turn out to be the right one.
Grim: I get better results not using the expert "will it rain today?" model, but my own ability to judge.
ReplyDeleteWhich is why air flight planners and FEMA go to you for the weather prediction.
You ignored our comment about how all science is an approximation. You claim you are more accurate than meteorologists, but without quantification, your claim can't even be properly evaluated. What you would have to do is predict the weather three days out over a period of a year or so, then compare it to model projections. So far, the only prediction we have seen is that the weather service predicted 70% chance of rain, and it was raining 100%, like rolling boxcars.
This illustrates the problem above. You denigrate the "elites" but have presented no objective basis to do so.
Grim: the philosophical question of when non-experts should be consulted in matters that affect their lives
Of course people should be consulted, but if the weather service says a hurricane is going to hit your area in two days, you may want to take actions to protect your family and property. If several independent doctors say you have cancer, you may want to consider an appropriate plan of action.
Which is why air flight planners and FEMA go to you for the weather prediction.
ReplyDeleteNow, now. We have an opportunity to end on a positive note. Don't return to insults and straw men. I never claimed to be an expert; in fact, I specifically disavowed expertise in meteorology. No one would ask me; but if they ask the weatherman, they'll find him wrong, and if I listen to the weatherman instead of myself, I'll end up wet a substantial percentage of the time. That's just true.
You ignored our comment about how all science is an approximation.
Just because I didn't respond to it doesn't mean I ignored it; perhaps I acknowledged it internally and saw no reason to dispute it. In fact, I made a similar point about Kant just this morning. We may not understand the implications of that fact in anything like the same way, but it's true as far as it goes.
This illustrates the problem above. You denigrate the "elites" but have presented no objective basis to do so.
I don't think that I have denigrated the elites so much as disputed that they should be put in charge -- even where they actually are experts, which claims (esp. in economics, etc) are sometimes dubious since Socrates. You are their representative here, self-appointed but I will assume you have reasons to consider yourself a member of that class. It is to you that we should look for evidence of the denigrations that elites are sometimes charged with: whether, for example, they hold those they consider non-experts in contempt and treat them contemptuously. If you want to defend them from such charges, you can best show by example that the charges are false.
Mr. Foster:
ReplyDelete"Teller had a strong ego interest, and maybe a strong career interest as well, in having his detonator approach turn out to be the right one."
Quite right. Class interests are something I've been thinking about a lot lately; I think we have misunderstood many of our social conflicts as racial, for example, when they are really better analyzed as class conflicts. So too political conflicts, which are about class interests more than political factions.
But you are correct. The sin of pride -- or as you say, 'ego' -- is a very important factor as well. It always has been.
But that doesn't mean they therefore sign on to Worlds In Collision, or Scientology, or QAnon, just because.
ReplyDeleteOr feminist theories about Newton’s Rape Manual;
Or homo- + trans- theories of human sexuality;
Or that the US government was almost overthrown on 1/6;
Or that systemic racism exists;
Or...
And yet all those positions, unlike Velikovsky, Hubbard, or Q are mainstream, academic, and supported by the state.
I’m just one step away from believing everyone is nuts.
Grim: Don't return to insults ...
ReplyDeleteWe're not trying to insult you when we point out that when planning a flight path, people don't call Grim, but the weather service. There's a reason for that.
Grim: I never claimed to be an expert
But you did claim that you "get better results not using the expert 'will it rain today?' model, but my own ability to judge." However, you provided no objective evidence to support your claim, while mangling the notion of probability and verification.
Grim: I don't think that I have denigrated the elites so much as disputed that they should be put in charge
Sure, and that would be a worthy discussion, but it's hard to advance such a discussion when demonstrable facts are in dispute.
People have to balance costs and benefits. Should the family still have a picnic when there is a 50% chance of rain? 30%? Should the governor order an evacuation when there is a 75% chance that a hurricane will hit the coast in three days? 90%? Or should the governor eyeball it on his iPhone before reaching a decision?
"...while mangling the notion of probability and verification..."
ReplyDeleteI would have phrased it that I identified through discussion that you're not using Bayesian probability, which I think is superior in this sort of approach where the problem is knowing or not knowing something. You are using a simpler approach of running a simulation that approximates the conditions in nature a certain number of times, and then imputing the results to the day. That contains a big problem, which is that you can't know the initial conditions of the day the way you can know the conditions in the box; but I understand now why you think it's fine that you work with a model that is comfortable continuing to assign a 70% chance of rain when it's already raining.
That, I think, is one of the places where the discussion was useful. I'm not accusing you of ignorance in not having understood that difference, even when I named it; even the best expert isn't an expert in everything. Indeed, one of the problems of expertise is that the more of an expert you are in anything, the more your attention will have been focused away from everything else. It's likely we will all have blind spots and need to learn from each other.
As for objective proof of my own experience, there is none; you'll have to take me at my word. As far as I know, it's true.
"Should the family still have a picnic when there is a 50% chance of rain? 30%? Should the governor order an evacuation..."
You may have misunderstood my basic claim. Notice in both of these examples, experts are not making the decisions. I never said you shouldn't be willing to consider expert advice. I said they shouldn't be in charge.
Grim: I understand now why you think it's fine that you work with a model that is comfortable continuing to assign a 70% chance of rain when it's already raining.
ReplyDeleteThe same reason we can assign a 14% chance of rolling an eight with two fair dice after having just rolled boxcars. Contrariwise, it's odd to claim that when the weather forecast is for 70% rain, and it rains, that this is somehow a refutation of weather forecasting. If the forecast is 70% rain, that means it will rain sometimes. More particularly, probability of precipitation = confidence * coverage area.
Grim: even the best expert isn't an expert in everything.
True. Experts often speak outside their field of study with an unearned air of authority.
Grim: You may have misunderstood my basic claim. Notice in both of these examples, experts are not making the decisions. I never said you shouldn't be willing to consider expert advice.
Actually, you indicated that meteorologists are not actually experts in weather prediction. That means you can safely ignore them when planning a flight path or preparing for a hurricane.
You can't have a discussion about the relationship between policy and science when you claim the science is faulty somehow. That is a different argument and requires addressing the scientific support. Modern weather prediction can now predict as accurately five days out as was possible one day out as recently as the 1980s. See Alley et al., Advances in weather prediction, Science 2019.
You are probably wise in many things, and know many things about which we are ignorant; but you originally said, "Now what I have noticed about these {complex} models is that we don't actually ever develop one that works." That was a false statement. We provided an example and support.
I accept that you think it is false according to your own standards for what the claim should mean; however, I have learned that you mean something different by the claim that 'the model works' than I do. You and I agree that the models aren't strictly accurate, you offering a proof that they're less inaccurate than once and improving; it suffices for you if they're "useful," meaning accurate enough on their own terms to use. You think they're more accurate than ever, even if they're not capable of producing the kind of accuracy I want from a model (as, for example, rocketry models do: the Curiosity probe had to land itself on Mars because the communication lag was too great for any new inputs from Earth to guide it down).
ReplyDeleteHurricane predictions may well be getting better, and may even get there someday. A governor tasked with making a decision about a hurricane might look at them, although he or she should probably also consult the observations and not just the models. (In the hurricane I was thinking about, the observations were that it was as big as Texas and headed straight for us; the models suggested it might hit. Evacuation seemed like a good decision to me, even though in fact it turned north, weakened, and didn't cause serious damage.)
Now, to return to your claim that “All models are wrong (but some are useful)," my complaint about the weather models is that they're not even useful. Maybe they will be someday; maybe they are in other parts of the world than in this particularly difficult one. Here, though, I strap my leathers to my bike in the summer regardless of whether or not the weather reports say the chance of rain is zero percent (it snowed last week when it wasn't supposed to for nine more hours) or 80 percent (sometimes we end up with nothing). I can usually dodge the rain by watching the sky, but not by heeding the reports. I do look at them, but I make my decisions based on inexpert calculations like where the big clouds are, how hot it's getting, and so forth. Usually I stay dry, sometimes barely. If not, I have the leathers.
I remember a hike I took up Blood Mountain one year with some friends. I checked the weather report that day. The chance of rain was said to be 30%; but it rained so hard that by the time I got there I was soaked through. We climbed the mountain anyway, though my gear was so wet that it was many pounds heavier. At the top there's an old CCC shelter, and some through-hikers on the Appalachian Trail had built a fire inside it. Sitting inside, between the heat of the fire and the body heat of having climbed the mountain, my clothes began to steam so thickly that I looked like a wizard.
That's a good memory, though. No harm, no foul.
Grim: You and I agree that the models aren't strictly accurate
ReplyDeleteNor is Newtonian Mechanics. However, Newtonian Mechanics is much more accurate than the science that came before it, and it is still accurate enough for many applications.
Grim: it suffices for you if they're "useful," meaning accurate enough on their own terms to use.
Knowing there is a high probability that a hurricane will hit a particular place on the coast three days in advance is very useful information.
Grim: You think they're more accurate than ever, even if they're not capable of producing the kind of accuracy I want from a model
Setting an arbitrary goal is inimical to science. Science progresses by providing limited knowledge in a vast universe of ignorance. Knowing there is a 90% probability that a hurricane will hit your town in three days is scientific information that has only come about through the use of models run on supercomputers.
Grim: Now, to return to your claim that “All models are wrong (but some are useful)," my complaint about the weather models is that they're not even useful.
Of course they're useful. For instance, weather predictions are used to set flight paths and for disaster preparedness.