Data Republican believes EA should be banned from AI safety, policy-making, and governance. That's a pretty extreme position, but she makes a solid case for both the mild and extreme versions.
Fundamentally, EA is a maximizing philosophy: it always tries to price future value. And extinction is the “negative infinity” number in the ledger. Once you are determining outcome by probabilities, just having even a tiny chance of extinction dominates every other calculation.
One such scenario that many conservatives might find sympathetic is: do you do gain-of-function research for a 70% chance that you will prevent the next swine flu, or is the 1% risk that the virus escapes containment in Wuhan and kills 10 million people acceptable? Many of us, having lived through COVID-19, would put our foot down on allowing any kind of gain-of-function research again within our lifetimes. That’s one scenario where conservatives can potentially reason their way to the same answer as EAs.
EAs run that same logic in every calculation. What this means, inside their heads: an extinction level event scenario is the greatest threat vector.
Scott Alexander, like others, is obsessed with AI, alignment, and its effects. He is very much a supporter of Effective Altruism in general, though not slavishly so. Still, I imagine he would object to such a blanket disallowing of EA from the discussion (or at least, from the decision-making). There are so many discussions and counter discussions over at Astral Codex Ten that I won't even attempt to choose for you. As his commenters are also quite brilliant - I consider it one of the few sites to compete with this one for average IQ - you can either dabble or go hard against them, trying to steelman your own arguments. This would be the major leagues. And yet...
I have felt uncomfortable with Effective Altruism since I first heard of it, though it took me a while to articulate why. Frankly, it probably took reading others who had thought about it more rigorously to be able to articulate it. One of the consequences of becoming lazier, and thus less intelligent, is not being able to provide some of the service I was designed for. I have excuses. Not good excuses, but at least some things I can say out loud with some earnestness.
The best arguments against EA are in Data Republican's essay. Pricing future value is a great example. It makes entire sense, but somehow leads to insane conclusions. If a future life is worth the same as a life now, then the sheer volume of future humans and their needs with forever dwarf what is good for us. More interestingly, perhaps, is how that completely devalues our ancestors. There is the (bogus in reality but worth valuing as a myth) Native American belief in Seven Generations, that we owe something to the three generations before us and the three after us. GK Chesterton wrote similarly that tradition is the only way of giving a vote to the dead. CS Lewis, describing the Tao in The Abolition of Man shows that the wise in every tradition teach that we owe something to both our heritage and our posterity. But EA very quickly reveals that it owes nothing to heritage and everything to posterity. The dead have made their contribution and their suffering is now a sunk cost.
There is also comparative suffering among species, leading to calculations that the horrible suffering of three chickens or thirty-two shrimp are equivalent to the suffering of one human. The actual numbers are immaterial. Thirty or three-hundred chickens would be the same. The principle of comparing it to any human suffering is something new in the world. It may be a logically consistent and defensible philosophical and moral position.
But it is not a human position, despite being favored by "humanists." It gives away the game, recognising that humanists are in fact subhumanists, valuing some bare minimum of the species as fungible with any other human. Dismiss it as a mere prejudice on my part if you will, but the idea of perpetuating humans who are only biologically and not culturally related to me is not just disquieting, it's silly.
Humanists are actually Subhumanists. I should probably think about that and post on it. Yet now that I have said that, I recall that CS Lewis already covered that in the last debate section of Out of the Silent Planet.







