Bethany has a new piece this morning about conspiracy theories using some true facts but leaving out others. Down The Rabbit Hole, One True Fact at a Time. A favorite psychiatrist of mine used to refer to "the truths people use to lie with," and it is a useful concept. We have better evolved to detect lies - which are there, right in front of us - than to detect truths that have been omitted. Makes sense.
In this case, the user was aware the bot might just be agreeing with them, but the bot was constrained to factual information only. It appears that constraining the bot to factual information basically negated the effect of warning people that the bot might just be agreeing with you. The paper’s authors suggest that this is because the “traces of sycophancy are harder to detect among selectively-presented factual data than fully hallucinated data.” You don’t say. In other words: you can absolutely cause catastrophic delusional spiraling by always telling the truth, as long as the truth you’re telling is carefully selected.
Contrary to my science friends telling me this is all due to poor science training in the schools, I don't find that even advanced degrees are a sure defense against such theories. It is worth noting that even non-paranoid reasoners respond in the same way. Look at this terrible thing that Trump did. And that one. Well, not necessarily untrue...but compared to Biden, Harris, Obama...? No, all that started under Bush and Cheney...Whoa, Clinton was there first. As Wally in Dilbert once said "We ate that banana years ago." Which of us would bring up the facts that are against us in an argument? We stay stuck in our wrong beliefs not because of lies, but because of clinging to some truths and hiding others in the closet where we don't have to look at them.
For reference, my children largely went to those same schools and are also up on the conspiracy theories that circulated in the evangelical community in the 90s and beyond. The two Romanians arrived still wondering if we were going to sell their body parts. Kyle, a Child of the Sixties even though he was born in 1996, went down the rabbit hole of Paul McCartney being replaced by an imposter years ago. Fortunately, none of them married into families that have conspiracy theorists.
Oh wait...
It is interesting in online comment sections if you challenge someone that their idea is paranoid they will often respond by saying "Show me one thing on this list that isn't true." That seems like informal support for Bethany's theory here.
Update: In reading an article today cautioning (but not opposing) the use of Welbutrin, Thomas Cook writes "In fact if you really want to mislead people, there's no better way than with obscure science."
6 comments:
How was Galileo recharacterized from the alienating abrasive type he was to a noble martyr? Carefully curated half-truths.
Her post reminded me of the often repeated line that the best propaganda includes some truthful statements.
A twisted Bayesianism
Interesting. We all have a sort of natural Bayesian system, though often not very precise, e.g recency bias. But this is a way of hacking into a Bayesian system, or gaming it.
Love your last statement here. When I made my comment about TikTok in the piece I was actually thinking of one I saw on a topic I know a lot about where a guy rattles off about two dozen questions in all starting with “is it a conspiracy theory that…”
I actually went through them one by one once, and as I recall only about 25% of what he said was fully made up. The rest would have fallen in the context free truth bucket. I’ll note that’s one scenario the authors didn’t test: what happens when you combine a small amount of hallucination with a lot of selective facts?
There is an old evangelical principle, that I think...no, let me state it first and then put in the qualifiers: "The devil will use a lake of truth to wash down a pint of poison." I believe that is the point we are making here and it is deeply true, but in the evangelical context I think it can be applied in reverse. Fundamentalists can (metaphor-switch alert) nit-pick a gnat of inaccuracy in order to ignore a lake of truth.
Whenever things come down to "balance" or "perspective" I suddenly remember "Oh right! We're supposed to ask the Holy Spirit about this. Duh! Shoulda thought of that first off. Lord, is it too late to put in a bid for discernment now?"
Post a Comment