Do Not Trust Your Arguments

Image source

How wonderfully constituted is the human mind! How it resists, as long as it can, all efforts made to reclaim it from error!

In last month’s post on Chinese attitudes towards Hong Kong I had cause to mention Dan Sperber and Hugo Mercier’s book The Enigma of Reason. At some point I probably ought to devote an entire post to a review of the book; its ideas are subtler and logic more involved than one paragraph glosses (or even most of its newspaper reviews) can give justice to. A running theme throughout the book, is that in experimental settings and in “real life” human capacity for reason is not optimized for the pursuit of abstract truth. Mercier and Sperber suggest that this is because reason did not evolve for that end. Reasoning’s role is essentially a social one—at the level of the individual it is not about deciding what to do, but about justifying what we do.

What we decide to do is for the most part entirely intuitive. As Mercier and Sperber see it, our decisions are the products of mental subsystems as opaque to us as the mysterious mechanisms that classify what we see as “beautiful” or “ugly,” determine what we are doing as “boring” or “fun,” and judge what others are doing as “admirable” or “disgusting.” Although his brain will supply the child with reasons for why he likes to watch Star Wars, the teenager with reasons for why she favors purple eye-liner, and the lover with reasons for doting upon his beloved, these thoughts are not the actual cause of the behavior in question. They are justifications. They are seized upon and articulated by the brain not to make us aware of why we make our decisions, but to make it possible for us to justify and explain our behavior to others.

Sperber and Mercier confirm this general thesis through dozens of experiments and lots of clever thinking. They explore the implications of these ideas further (“wait,” you might ask, “what is happening then when someone else’s ideas changes our mind?”), but as they have limited relevance to the point I am going to make here, I will reserve their discussion for a different post. The main idea we are to run with here is this: to reason is to justify. Arguments, especially arguments over policy, are not about finding the truth. Both sides come with the truth already decided. In this context, reasoning and arguments are tools of persuasion. We reason to convince third parties that we are right.[1]

That is the evolutionary utility of reasoning. It makes you sound heck of a lot more convincing than the guy who (more honestly) says, “I believe we should do this because it feels right.” The jerks who wander across the internet yelling “facts don’t care about your feelings” have missed the mark. We assemble facts and data precisely because of our feelings. Facts are byproducts of a quest to justify our feelings.

None of this means that arguments and reasoning are vain deceptions. At the individual level reasoning is mostly about justifying propositions already held. At the level of the group this is not true. Groups must decide which chain of reasoning (of the many presented) is best. They tend to do this more skillfully than any individual can. The individual is always biased towards her own case. She has a much easier time assessing the merits of an argument someone else has made.

Political psychologist Philip Tetlock came to a complimentary conclusion to Sperber and Mercier, though through different means. Tetlock made his name with experiments designed to judge the effectiveness of political expertise. He gathered together experts of all stripes (economists, political scientists, intelligence analysts, think tanker, financial gurus, and more) and political persuasions. He then asked them to make predictions about the future. He tracked these predictions over time, finding that most experts did worse than chance, and none better than simple algorithms that follow easy rules. That is all interesting. For our purposes, however, what is most interesting is not that the experts were wrong, but their inability to recognize it.

Tetlock followed up with his cadre of experts and asked them to assess the strength of their own predictions. What did he find? Most of the experts believed they had predicted events correctly, even when they had not. When presented evidence that they had not done as well as they imagined, these same experts did not abandon the tools that delivered them into error. Instead they justified. They argued. They reasoned. With brilliant ingenuity they showed how the world would have tilted just as their methods said it must, had just this or just that been different.[2]

I do not blame the experts for getting the future wrong. Humans are not very good at this. I also cannot blame them for thinking they got it right. Humans are extremely good at that. Getting the future wrong while thinking we got it right is a human specialty. To hate this is to hate humanity. We cannot get rid of such a human set of tendencies. The best we can do is find ways of limiting their ill effects.

Tetlock’s experiments suggest one way to do so: have experts commit in writing what they predict will happen. If you placed fine grained predictions somewhere everyone can see, you will have provided the world—and more importantly, yourself—with an objective way to measure the utility of your reasoning. Others have suggested a more costly twist on this same logic, asking forecasters to put money down on all of their predictions. In both cases there is now an easily identified yardstick by which to judge a conviction.

But what of convictions that are not predictive? Not all arguments, after all, are about the future. Arguments over the past can be quite violent. No prediction market will ever be able to solve those arguments. No are predictions especially useful for solving debates over questions like “How much of Trump’s support comes from racist sentiment?” or “Is it wise to continue a trade war with China?”

What to do in face of such thorny problems?

Here is my personal solution: Identify a major political or ethical conviction you have. Then write out what you would need to know or have proven to you in order for you to change your position. Your position may be wrong or it may be right. But if you are not willing to sketch out what would make it wrong, you will not recognize when it has been proven so! Instead you will shift your arguments. You will shift the goal posts. You will move your target. You will do all of that—and barely be aware that you are doing so.

I am not dogmatically against moving targets. Occasionally, new aspects of a debate will arise that you had never considered before. Perhaps your argument about whether to go into Iraq was focused entirely on how easy it might be to control or contain a nuclear armed Saddam. Perhaps it had never occurred to you to think over the problems of reconstruction in a land violently divided. Perhaps that is actually the better thing to focus on. Shifting goal posts is sometimes necessary. But if you do not bother sketching out where your current posts are moored, you will not know when you have moved them.

I have found this entire exercise very useful for deciding what arguments to engage in and which to avoid. I recently had an acquaintance contact me, a very intelligent fellow who works Australian defense issues. While he is not quite Hugh White, this man believes America is on its way out of the Asia-Pacific. He makes the case that Australia must prepare for a world without American support. Knowing me to be the hawkish China type he asked for an assessment of his argument. I replied with a question:

“You believe that America is not committed enough to its allies in the Pacific, and will abandon them within fifteen years?”

“Yes.”

“What would America need to do change your mind? What could America do to prove to you that it was here for the long haul? What could it do to demonstrate the sort of commitment you think it lacks.”

His answer: “Nothing.”

That was my signal to politely excuse myself from the argument. Perhaps my friend is right. Perhaps America is on its way out of the region; perhaps America can do nothing to demonstrate her commitment to Australia and the other democracies of the Western Pacific. But if nothing could convince him, what exactly is the point of continuing this argument?

It is too easy to be attached to a conviction. I am not more intelligent than most of those who disagree with me. Nothing about my biography privileges my beliefs. The only thing that privileges my arguments is my own biases. I think it wise to fight against them. That means acknowledging from the beginning what knowledge or event should force me to change what I believe. It makes it easier to actually change your beliefs when that moment comes.

Increasingly, I ask others to do the same thing. This is an easy test to separate those of good faith from those of bad. The person willing to acknowledge at the beginning that they may be wrong, and then lay out in the conditions in which they would be wrong, is a person worth your time. A person unwilling to admit the possibility of error is not. That person simply is not interested in truth. They are interested in winning. They more they squirm to avoid explaining what might make them change their mind, the less you should trust them. Propagandists do not confess possibility of blunder.

That is my take. The first step in any serious argument, any argument with real consequences, should be this. Each side should lay out their case—and then explain, in equal detail, what would make them determine that their case is wrong. When that moment comes they still might cling to their original conviction. They might change the grounds of argument to protect their beliefs. But in that day, everyone, including themselves, will know that this is exactly what they are doing.

Edit 5 September 2019: Slightly edited the exchange with my Aussie friend for the sake of contextual clarity.

[1] Dan Sperber and Hugo Mercier. The Enigma Of Reason (Harvard: Harvard University Press, 2019).

[2] Philip Tetlock. Expert Political Judgement: How Good is it? How Can We Know? (Princeton: Princeton University Press, 2005).