09 — Emotional Drivers

Overconfidence#

Statistical1 min read
◆ The PatternBelieving you know more than you do.

Overconfidence — systematically overestimating one's ability to predict, analyze, and control outcomes.

Three types:
1. Overestimation: "I'll beat the market" (in one U.S. student sample, 93% rated themselves above-median drivers — Svenson 1981)
2. Overprecision: Confidence intervals too narrow ("I'm 95% sure it'll hit $50" — it won't)
3. Overplacement: "I'm a better trader than most people"

Barber & Odean (2000): the most active fifth of households earned 7.1 percentage points a year less, after costs, than the least active fifth. Overconfidence → overtrading → underperformance.

Worked example

Ten price ranges, each one given with “90% confidence”. Only 5 of 10 contained the actual value. A calibrated forecaster would hit about nine; hitting half means the ranges should have been much wider. This is overprecision, the most robust form of overconfidence, and you only see it by scoring a batch of forecasts.

Where it misleads
  • Barber & Odean (2000) found the 20% of households that traded most earned 11.4% a year net, against 17.9% for the market — but trading costs, not bad picks alone, did much of the damage.
  • “93% think they are above-average drivers” comes from a small U.S. student sample (Svenson 1981); the Swedish sample was 69%. The effect is real, the number is not universal.
  • Confidence is useful when it is earned and calibrated; the fix is scoring, not self-doubt.
In code
fc = pd.DataFrame({                              # ten ranges you were "90% sure" of, made up
    'low':    [95, 40, 1.8, 210, 70, 12, 3.1, 150, 88, 25],
    'high':   [105, 48, 2.2, 240, 80, 15, 3.5, 170, 96, 30],
    'actual': [108, 44, 2.5, 236, 64, 13, 3.4, 181, 90, 31]})
hit_rate = fc.actual.between(fc.low, fc.high).mean()
Python, in your browser — edit the code and run it again

Assumes import numpy as np and import pandas as pd. Each snippet carries its own example numbers; the comments say which are made up.

Sources
  • B. M. Barber & T. Odean, “Trading Is Hazardous to Your Wealth: The Common Stock Investment Performance of Individual Investors”, Journal of Finance 55(2), 2000
  • B. M. Barber & T. Odean, “Boys Will Be Boys: Gender, Overconfidence, and Common Stock Investment”, Quarterly Journal of Economics 116(1), 2001
  • O. Svenson, “Are We All Less Risky and More Skillful Than Our Fellow Drivers?”, Acta Psychologica 47(2), 1981
  • D. A. Moore & P. J. Healy, “The Trouble with Overconfidence”, Psychological Review 115(2), 2008

Last reviewed October 2026 · How this site checks its content · Report a mistake

Check yourself

1. Barber and Odean (2000) sorted households by how much they traded. What did the most active traders earn?

Show answer

B. Their paper is titled “Trading Is Hazardous to Your Wealth”: costs ate the gains that confident trading was supposed to bring.

2. People asked for 90% confidence intervals on quantities they do not know contain the truth…

Show answer

C. Intervals are too narrow: people are more sure than their knowledge justifies.

3. Which habit helps against overconfidence?

Show answer

A. A decision journal turns vague memories of being right into a calibration record.

Pattern bridge: Too-narrow confidence intervals from too much certainty. In statistics, this is literally miscalibrated confidence intervals. In ML, low bias, high variance — the overfit model that’s sure it’s right.
Linked from 3
← Previous
Regret Aversion
Open in the full reader, with the topic sidebar