Optimism is an inherently good thing, right?
Be honest with yourself for a moment. How many times have you compensated, consciously or otherwise, for a lousy plan with the phrases "I'll figure something out" or "It'll all work out"? Statistically, you won't and it won't. But our minds are great at convincing us that these phrases are not merely a rapid escape hatch from a conversation we don't want to have; we're hard-wired to believe it.
Optimism bias is one of the oldest and most well-established facts in psychology and cognitive science. Armor & Taylor (2002) run down a list of experimental evidence. But if this seems like a simple phenomenon, it isn't. The problem is not that we underestimate the odds of negative outcomes even when we have full information (contrarily, in fact, we tend to wildly overestimate the odds of unlikely negative consequences – plane crashes, being hit by lightning, satanic ritual abuse, etc.). An example of that kind of reasoning might be, "I know 90% of smokers develop lung cancer, but I'll be one of the 10%." There's no need to give that a fancy name; denial and stupidity work just fine. No, optimism bias is our ability to convince ourselves that we're not in denial of the odds – instead, we tell ourselves we've found a way to change them in our favor.
We know that 75% of credit card holders make a late payment at some point, allowing issuers to impose punitive fees and APRs, but you won't be one of those people because you're very organized and you always pay on time.
Students beginning law or business school (and sinking $100k in the process) wildly overestimate their odds of getting one of the high-paying jobs one gets by finishing in the top 10. It's obvious that only 10% of any class can have that outcome, but the odds are not 9 to 1 against me because I'll study harder than everyone else and I'm smarter anyway.
Smokers routinely convince themselves that the grim statistics about tobacco and mortality are attenuated by some other behavior, i.e. your odds of getting lung cancer aren't really 80% because you smoke but you also jog three times per week and eat lots of organic stuff (or even more hilariously, "because I smoke lights / brand X / etc.")
Confronted with the cold reality that 50% of marriages fail, newlyweds inevitably conclude that their special bond with one another makes their odds of avoiding divorce much better than 50-50.
Gamblers understand that the odds are always on the house, but you have some kind of "system" – counting cards or whatever – that means the odds don't apply to you.
I can't help but think about optimism bias when I look at the daunting statistics about foreclosures and delinquent mortgages. Lazy minds assume that the problem is as simple as poor people taking out loans they couldn't afford, but when we look at the geographic distribution we see the fastest-growing states are leading the way – Nevada, Arizona, California, and Florida. In other words, the places where the upper-middle class buys its vacation properties and eventually moves to retire. Behind every bad loan, "investment property," and second mortgage is a series of powerful rationalizations: I can afford it as long as it keeps going up in value indefinitely (and it will!), I can't afford it but I'll be making more money soon, my 401(k) will never lose value so I'll be OK, or the generic "It'll all work out."
In other words, it's another one of those cognitive biases upon which our entire economic system is based. We invest because we think we're smart enough to do it without real risk and we buy under the assumption that we'll come into the means to pay in the future.
FMguru says:
"It's obvious that only 10% of any class can have that outcome, but the odds are not 9 to 1 against me because I'll study harder than everyone else and I'm smarter anyway."
A classic. The engineering program at the school I went to was famous for it's brutal weed-out odds. During orientation, the speaker asked every new student to look to right, now look to the left, now look at yourself – two of the three of you won't be graduating from here with a degree in engineering. And every single person in that room looked at their neighbors and though "heh, sucks to be you guys."
And it wasn't just dumb optimism – they'd acheived at every level. They'd aced their SATs and racked up a ton of AP credits and science fairs and science olympiad and academic decathalon and 3.91 GPAs, and managed to land a spot in a very competitive engineering program. Why wouldn't they overestimate their chances?
It's like how even the scrubbiest of pro atheletes was probably the best player in the history of their high school, won all-state, and received national recognition in college. It takes a lot of talent to bounce between AAA and the major leagues, or to be a backup player on the Cleveland Browns. Their talent just wasn't quite enough to take them all the way to the big time.
As for card counting – hey, that does work. It's really hard to pull off and it's grinding work and if the casinos even so much as sniff that you're doing it you're out on the street with a lifetime ban. Compare the treatment of card counters to the people who look for "patterns" in roullette wheels. Note that casinos happily post electronic screens showing the last 30 results on each roullette table, and compare that to their treatment of people who try collect similar records of blackjack cards played.
Vegas casinos, in general, are wonderful places to go for spotting cognitive biases in the wild. Every feature down to the tiniest detail is precision engineering to exploit biases, blind spots, and flaws in cognitive processing. The free (if watery) booze, the lack of sight lines, the blinking lights, the lack of clocks, the lack of windows, the way all traffic in the hotel has to pass through the casino at some point, the way machines make a big deal out of paying out winnings but quickly move on when you lose – they're like 30-story tall neon-bedecked Skinner boxes.
nicolien says:
There's a 'professor in optimism' at the KUN (university in Nijmegen, the Netherlands), who is (obviously, hence the name) specialized in the optimism bias. Asked in an interview about the same issue you describe (odds are 9 against 1 to come out top student / get the well-paying job), she said the optimism bias is necessary for people; if they had a more realistic view of the odds and of their own skills and abilities, nobody would ever even try anything. Found it an interesting way of looking at it.
grendelkhan says:
The half-of-marriages-fail statistic is a bit misleading. The truth is more muddled, as it's hard to get statistics on this. (It's still a lot, of course, but it's likely less than half.)
Grumpygradstudent says:
One might add, "it makes sense to spend 7 years in grad school even though there's a real chance that I'll never get a teaching job that pays a living wage. Something will work out!"
I see this phenomenon a lot with hippie-types and health concerns. "Don't drink diet soda, cause it might give you cancer! Excuse me while I light up this J before I go rock climbing in the snow."
AG says:
Well written, Ed.
I am sure you've seen this TED talk but if you have not, please do: http://www.ted.com/talks/dan_gilbert_researches_happiness.html — I think it will resonate with your post. You can extend the optimism bias to other areas too: "The out-of-control free market will do better than the government because ___". Or perhaps that is a different bias or just plain retardation.
Chuck says:
Optimism bias is the bedrock upon which capitalism / corporatism stands.
Andrew says:
I do always make my credit card payments on time and am never charged interest or a late fee. When 1/4 of people are in the "good" group, it's not that hard to make sure you're one of them. Being smart, financially well-off, and well-organized do have their advantages.
As for the housing bubble, most people's income _does_ trend upwards over the course of their career, so assuming yours will is not all that unreasonable. Also, people will sacrifice vacations, restaurant meals, even health insurance to keep up with their mortgage payments. That having been said, a lot of people who bought houses during the boom truly did not understand the risks.
I try to live my life according to probabilities, but being in the top 2% of IQ, income, etc., it's not at all unreasonable to assume that my personal odds are somewhat better than the aggregate. How much better, who knows? But if I weren't optimistic to a degree, I'd be stagnant. Besides, if option A doesn't work out, maybe option B will.
JohnR says:
shorter version: PT Barnum was right.
Pan Sapiens says:
"…but being in the top 2% of IQ, income, etc…."
I really am amazed at how many genius millionaires I meet on the Internet. What are the odds? And you'd never even know it from just reading their comments. Good thing they identify themselves.
Oh! Oh! Wait. Sarcasm! I get it now! Ah-ha-ha.
Prudence says:
Ha! Thank you, Pan Sapiens for that titter.
Can't say with certainty I'm in the top 2% of any category (blethering?) but I have a job which exposes me to different parts of the developing world on a very regular basis– and I always come away with a deep sense of humility and gratitude– which doesn't last nearly long enough, thanks to my own character failings. Most of these people have next to nothing, but are more than willing to share it with a stranger.
Hemingway, that self-obsessed old sot, was wrong about a lot of stuff, but I agree wholeheartedly that "the world is a fine place, and worth fighting for".
Desargues says:
I wonder if the adherence to irrationally optimistic worldviews like Christianity and the persistence in kind of cognitive bias aren't a bit related. Comparative studies in self-professed optimism across Western nations may shed some light on this. A certain kind of American prides himself on the "sunny" optimism he projects (and excoriates as America-haters fellow citizens who refuse to bask in the same light) — an attitude that's quite baffling to many Western Europeans. Is it entirely coincidental that America is also the last Western nation to take biblical superstitions so seriously? E.g., the injunction to "take no thought for your life, what ye shall eat, or what ye shall drink; nor yet for your body, what ye shall put on."
Desargues says:
Optimism is an inherently good thing, right? Have you tried that on a German?
Prudence says:
Frankly, I'm going hell-for-leather on the optimism front– I've just booked my trip from London back to DC on British Airways…
Andrew says:
I never said I was a millionaire. Being a millionaire requires wealth, not income.
Pan Sapiens says:
So your pissing your income away? Good for you!
Sour Kraut says:
So your pissing your income away? Good for you!
Heh. So what's it like running the Bellagio?