Skip to content

Cognitive Biases

April 17, 2008

Eliezer Yudkowsky’s article on heuristics and biases Cognitive biases potentially affecting judgment of global risks is one of my favorite on the topic.  I have read it a few times now and find it valuable every time.

Why is this topic interesting? Partly, because it is one of a handful of research areas in which sociology/psychology seems to contribute to fairly direct knowledge of how the brain works. More practically, H & B are interesting because one cannot avoid them.

Even when we are thinking hard and focusing narrowly on self awareness, it is practically impossible to avoid making these errors.  The best we can hope for is to leave notes for ourselves (sometimes it might be wise to leave actual physical notes!) instructing us to distrust our convictions. But knowing you are wrong because you trust your passed-self’s authority isn’t really avoiding thinking with H & Bs, it is merely switching the train onto a different set of rails. 

It seems true that, while many have explained the patterns of failure that can result from trusting these heuristics and biases, there are also important and powerful benefits from our brains having evolved to work this way.

In the table below, I outline simplistic thoughts trying to illustrate that every bias identified in Yudkowsky’s paper as a potential failure pattern also accounts for a great deal of human success.  Together, they not only represent potential pitfalls in perception and reasoning, but constitute a great deal of what we consider the basic, innate human ability to act, adapt and get on in the World.

Cognitive Bias

Heuristic Strength

Availability—Information at-hand is adequate for risk

Reasonable good caching algorithm in that recent work is
most likely to be accessed next; Coherence of action (focus) is maintained

Hindsight—Story explaining past accounts for outcome

Learning! Emotional conviction of the connection between
cause and effect. Coherent stories stick in our memories very well.

Black Swans—Assume unlikely events are more unlikely than
they often are (This is like Availability in that current or average are
assumed to continue)

Allows action in an uncertain world. Assume continuity
because planning for all unlikely events uses up all your resources.


Great first guess because local action is most likely
(action-at-a-distance is a bad first guess—and spooky)

Probability Increases with Detail (A mixing of Conjunction
and Availability)

Personal experience works this way in that we always have
most detail about personal experience—and personal experience is taken to be
most trustworthy.

Confirmation—Confirming Facts are Selected from Sea of Facts

Intentions and Synchronicity

Anchoring, Adjustment and Contamination (Availability
& Conjunction)

Data we experience in succession are often related


Action in uncertain situations is better than being
frozen. Rules available when little info is available but you an answer is
still required.

Scope Neglect

Rules available when little info is available but you an
answer is still required.

Calibration and Over Confidence

Rules available when little info is available but you an
answer is still required.

Bystander Apathy

Optimization and queuing heuristic.


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: