generated from rstudio/bookdown-demo
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path01-judgment.Rmd
176 lines (121 loc) · 11 KB
/
01-judgment.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
---
output:
pdf_document: default
html_document: default
---
\mainmatter
# Systems of human judgment
An understanding of the mental traps to which we are prone can help us avoid them. Kahneman's book [-@kahneman_2013] *Thinking Fast and Slow* is a good starting point for thinking about the strengths and limitations of human thinking processes. There is no good substitute for the use of "educating gossip", as Kahneman describes it, for training in effective judgment and in decision making.
Important themes that Kahneman notes are
- We have "an excessive confidence in what we think we know".
- We too readily judge decisions by outcome, rather than by the
strength of the arguments that support them.
- We have two selves --- an experiencing self and a remembering self
- These do not always have the same interests.
- Automatic memory formation has its own rules
- This is exploited to improve the memory of a bad episode
- "We easily think associatively, ... metaphorically, casually, but statistics requires thinking about many things at once ..."
## System 1 and System 2 --- further comments
Humans have been conditioned to respond quickly to immediate risks and challenges, without stopping to consider too carefully whether what we heard was a false alarm. They also have the ability, when the occasion seems to demand it, to stop to ponder. This is the basis for Kahneman's categorization of human thought processes as of two types --- System 1 which jumps rapidly to make a judgment, and System 2 which takes time for careful consideration.[^01-judgment-1]
[^01-judgment-1]: See https://suebehaviouraldesign.com/kahneman-fast-slow-thinking/ for further comment.
System 1 Features are
- It may answer an easier question in place of a harder.
- It responds to irrelevancies --- priming, framing, affect, memory illusions, illusions of truth, ...
- Priming by one stimulus can affect the response to a second
stimulus that occurs shortly aferwards
- The portrayal of logically equivalent alternatives in different
ways or 'frames' can affect the response
- The emotion or 'affect' generated by a question can affect the
response
- It has little understanding of logic and statistics
- It cannot be turned off, but it can be trained
- When flummoxed, it calls on System 2
System 2 features are
- It keeps you polite when angry, alert when driving in a severe
rainstorm
- In its world, gorillas do not cross basket-ball courts ...[^01-judgment-2]
- Problems that put 1 & 2 in conflict may require large mental effort & self-control to overcome the impulses and intuitions of System 1.
- Its effectiveness depends, in important areas, on training.
[^01-judgment-2]: See http://theinvisiblegorilla.com/gorilla_experiment.html
Both systems are amenable to training. A well-trained System 2 helps
greatly in creating a better System 1. Further points are:
- Healthy living is a compromise
- Recognize situations where mistakes are likely
- Aim to avoid significant mistakes when the stakes are high
- Untrained humans are poor intuitive statisticians
- Judgments about statistical issues may require us to think about more than one, even many, things at once.
- Too often, we jump to conclusions, without careful assessment.
- We may not be equipped to make an informed and carefully thought through decision.
## The Intuition of Professionals
Effective professional training is designed to ensure that at least some of the results of well-tuned System 2 expert judgment operate at a System 1 level.
The professional will, if the training is doing its job, build up a
repetoire of System 2 judgments that will later, when the circumstances
seem to demand it, be available at a System 1 level.
> The situation has provided a cue; this cue has given the expert access to information stored in memory, and the information provides the answer. Intuition is nothing more and nothing less than recognition.\
> [@simon1992explanation, "What is an Explanation of Behavior?"]
### Obstacles to effective judgment {.unnumbered}
Even those who are experts in their field can be similarly prone to judgments that have no foundation in fact. The following comment appeared in a discussion of
the response to a U.S. Preventive Services assessment that prostate screening,
when used in accordance with then current treatment practices, was doing more
harm than good.[^01-judgment-3]
[^01-judgment-3]: Association of Professional Psychologists, web post on @arkes2012psychological
> Even faced with ... evidence ... from a ten-year study of around 250,000
men that showed the test didn’t save lives, many activists and medical professionals are clamoring for men to continue receiving their annual PSA test.
New evidence emerges as time proceeds, and there are advances in the approach to treatment. At least part of the problem has been a rush to treatments that themselves risk increasing damage and the risk of death. Note the comment in @brawley2018prostate that
> Over the past few years, the benefit‐to‐harm ratio has improved in favor of benefit if the man understands that active surveillance may be a reasonable path if diagnosed.
## A demand for discipline & careful thought
- We make judgments based on evidence that is too limited
- We are easily fooled by irrelevancies
- Kahneman has brought together evidence on what & how.
- Even when data are there for the taking, someone has to notice, to collate the data, and to understand its uses and limitations
- Randomized controlled experiments (RCTs) are often the ideal, but require meticulous planning. If effects of interest are small, the numbers required may be very large. See further, Subsection \@ref(ss:rct)
- A limitation is that results apply only to the population from which
trial participants were taken. Any wider generalization has to be justified
- Observational data does not easily, if at all, substitute for the use of
RCTs. It is in general impossible to be sure that all sources of bias have
been properly accounted for
- Keep in mind Yule-Simpson "paradox", which we will encounter later in Subsection \@ref(sec:yule1)
- The paradox lies in the failure of human intuition to accommodate straightforward arithmetic!
## Further examples
### The "conjunction fallacy" {.unnumbered}
This has also, as a result of the example given in
@tversky1983extensional come to be known as "the Linda problem".
The name 'Linda' comes from the question and usual response
that are given by way of example.
> Linda is a 31-year old philosophy graduate, single, outspoken, and bright. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which of the following is more probable?
- Linda is a bank teller.
- Linda is a bank teller, and active in the feminist movement.
Adding the further descriptor "active in the feminist movement" can only lower the probability, or just possibly leave it unchanged. Instead of assessing the balance of probabilities, we are tempted to ask which description best meshes with what we have been already told about Linda.
"Linda is active in the feminist movement" is the single descriptor" that respondents see as best fitting Linda. While that was not what was asked, one has to pay close attention to prevent System 1 from substituting that for the question that was asked. Note that the correct answer will be "a bank teller", irrespective of the way that Linda was characterized before the question was asked.
In part, the issue is one of use of language. The "correct" answer is asking us to use the word "probable" in a strict technical sense.
### Even careful critics sometimes get it wrong {.unnumbered}
An irony is that Kahneman was, as he has acknowledged, himself fooled into taking at face values papers that claimed to show that verbal concepts could have the effect of altering behaviour. Thus
- Being asked to write down stories about unethical deeds made people more likely to want to buy soap;
- Subtly drawing attention to money, e.g., leave banknotes lying around, made people feel more self-sufficient, and care less about others;
- Priming people with old age related words leads people to walk more slowly away from the lab as research assistants armed with stopwatches timed their movements.
As @ritchie2020science notes (p.28) Kahneman was not alone in being fooled --- the study about priming with old age related words has been extensively cited in psychology textbooks. None of these claims have stood up in attempts at replication, with larger numbers and with greater care to avoid unconscious sources of bias. Thus, in the replication of the study relating to age-related words, infra-red beams were used to measure time taken to walk between two points in a hallway, rather than research assistants who knew the group to which participants had been assigned.
### Think again --- a very simple example {.unnumbered}
Is symptom X associated with disease A?
| | Has Disease | No Disease | |
|-------------------|-------------|------------|-----|
| Symptom X Present | 20 | 10 | |
| Symptom X Absent | 80 | 40 | |
The symptom occurs with the same relative frequency, whether or not a person has the disease. Nisbett comments that most people, including nurses and doctors, interpret such evidence wrongly [@nisbett, pp.129-130].
### A test to check understanding of risk {.unnumbered}
See the 2-minute testm "Do you understand risk?".[^01-judgment-4]
[^01-judgment-4]: http://www.riskliteracy.org/
## Misbehaving humans!
The discipline of "behavioural economics" largely took shape as a result of the work of Richard Thaler. Kahneman was one of two mentors who strongly influenced Thaler --- the other was Amos Tversky, who had worked closely with Kahneman.
@thaler2015misbehaving explores the extent to which humans do not behave like the rational agents of classical economics, agents to whom Thaler gives the name "econs". Added to the irrationality with which we often act is that our personal priorities are unlikely to align precisely with those of econs.
Note also comments in Part 4 of @kahneman_2013, titled "A conversation with the discipline of economics". In a discussion on "The Prospect theory model of choice", Kahneman comments on
- "... unfortunate tendency to treat problems in isolation"
- Framing effects --- inconsequential features shape choices
Hence, "a challenge to the assumptions of standard economics".
## Negotiating Life in an Uncertain World[^01-judgment-5]
[^01-judgment-5]: These questions came originally from the Harding Center web site <https://www.hardingcenter.de/en>.
Questions that it can be helpful to ask include
1. Risk of what? (Showing a symptom, ..., Death)
2. What is the time frame? (next 10 years, or lifetime)
3. How big is the risk? (Look at risk in absolute terms)
4. Does the risk apply to me? (Age, sex, health, ...)
5. What are the harms of "finding out"? (False alarms, invasive diagnostic procedures, unnecessary or dangerous treatments.)