Reading Notes #9
“That which can be destroyed by the truth should be.”
WE DON’T KNOW, OPRAH, THAT’S WHAT WE’RE TRYING TO FIGURE OUT.
Map and Territory
by Eliezer Yudkowsky
Map and Territory is actually the first book of a larger ebook called Rationality: from AI to Zombies, and is a collection of essays about rational thinking. You can access all of these essays at LessWrong.com.
Here’s a direct link: click the gray “Start Reading” button.
It’s an approachable and nicely organized look at rational thought: what it is, how to do it, and what stops it from happening for most of us.
Let’s start there, maybe.
Thinking rationally is really about comparing what you expect to happen with what does happen—noticing where things line up, and where they don’t, and adjusting your expectations and trying again.
In other words, rational thinking isn’t a set of beliefs or logical proofs or rules. It’s a way of thinking, or trying to think, and the goal is to think in a way that matches up with reality, as much as possible.
Rational thinking can help us form more accurate expectations, and then use them to navigate reality better:
“Instrumental rationality, on the other hand, is about steering reality—sending the future where you want it to go. It’s the art of choosing actions that lead to outcomes ranked higher in your preferences. I sometimes call this “winning.””
Thinking rationally does not mean being emotionless. This is not a thinking versus feeling dichotomy, which is good.
“When people think of “emotion” and “rationality” as opposed, I suspect that they are really thinking of System 1 and System 2—fast perceptual judgments versus slow deliberative judgments. System 2’s deliberative judgments aren’t always true, and System 1’s perceptual judgments aren’t always false; so it is very important to distinguish that dichotomy from “rationality.” Both systems can serve the goal of truth, or defeat it, depending on how they are used.”
Our fast perceptual judgments can be based on a long history of rational thinking, knowledge collecting, experimenting, and noticing. So they may turn out to be true, most of the time. And our feelings may encourage us to make quick decisions that “feel right”—those hunches, intuitions, that gut feeling—and then they turn out to be correct.
“Becoming more rational—arriving at better estimates of how-the-world-is—can diminish feelings or intensify them.”
We might use rational thinking to find out that something is true and we wish it weren’t. When rational thinking destroys a closely held little bit of magical thinking, for example, you might feel sad and disappointed.
When rational thinking supports a decision you already want to make, or removes an obstacle you’ve feared, you might feel better.
Either way, feelings don’t make a thought rational or irrational. And feelings—strong feelings, lots of feelings—do not mean you’re bad at thinking. That’s kind of a relief, honestly.
“But I know, now, that there’s nothing wrong with feeling strongly. Ever since I adopted the rule of “That which can be destroyed by the truth should be,” I’ve also come to realize “That which the truth nourishes should thrive.” When something good happens, I am happy, and there is no confusion in my mind about whether it is rational for me to be happy.”
There are a few terms, or descriptions, which popped up and explained things that I’ve felt or done or experienced but didn’t have a way to describe.
For example, the illusion of transparency:
“Closely related is the illusion of transparency : We always know what we mean by our words, and so we expect others to know it too.
…Be not too quick to blame those who misunderstand your perfectly clear sentences, spoken or written. Chances are, your words are more ambiguous than you think.”
I don’t go back and read my own writing very often, and whenever I do I always hit a point when I’m like “What the fuck was I talking about here?” Yeah. I get confused by my own writing. Editing helps, of course, but that’s too bad for you poor suckers on my email list because these babies get only the most cursory glance before I send them out. Fresh out of my brain in all its confusion and inconsistency, straight to you. Sorry about that.
Anyway, we also have a tendency to focus on the details so much that we completely lose sight of the big picture:
“A society well-protected against minor hazards takes no action against major risks, building on flood plains once the regular minor floods are eliminated.”
A great example of this is when someone wrecks their car because they were trying not to spill their coffee.
Or when we put all our energy into getting the external details lined up—better job, safer car, 401k, new curtains, getting that dental plan added to our health insurance—and then wake up one day and realize: “Fuck, I’m totally miserable and I hate my life.”
That sort of thing.
Focusing on the minors and ignoring the majors is a common problem.
It’s easier to fix details than to deal with big shit. And, when you have some big shit to deal with that, and you’re not, it puts you on edge. Then you really can’t handle the details falling apart, because you’re on edge, you’re feeling unstable or unhappy—and you need some way to feel better, like everything is okay, like you’ve got it handled.
I am convinced that this is why many women, at some point in their lives, become super controlling about many very minor household and family details. I know that this was certainly the situation for me. It still is, at times. If I’m upset about some bigger issue, but either can’t do anything about it or am ignoring it because I don’t know what to do, then I have a low tolerance for minor stuff going wrong.
On the other hand, when I feel good about the majors—secure, okay, peaceful—then the minor stuff is no big deal, even if if it does go wrong.
It turns out that when you are able to deal with the big shit, to make some headway on the majors, you don’t have to care as much about the details.
Let’s talk about ignorance:
“But ignorance exists in the map, not in the territory. If I am ignorant about a phenomenon, that is a fact about my own state of mind, not a fact about the phenomenon itself. A phenomenon can seem mysterious to some particular person. There are no phenomena which are mysterious of themselves. To worship a phenomenon because it seems so wonderfully mysterious is to worship your own ignorance.”
That’s a good one.
Last quote, which made me think a lot about writing, and why I do it, and what I hope to accomplish, and what I’m looking for when I read or study or research or discuss things I care about with people I trust:
“Rational thought produces beliefs which are themselves evidence. If your tongue speaks truly, your rational beliefs, which are themselves evidence, can act as evidence for someone else. Therefore rational beliefs are contagious, among honest folk who believe each other to be honest.”