MIMR
The Science of Improvement
Measure4 min read

The Horse Race Bettors Who Knew Too Much

Expert horse handicappers were given more and more information. Their confidence soared. Their accuracy didn't move.

Related articles

Imagine you're about to bet on a horse race.

You know five things about each horse: its recent finishes, its jockey, the weight it's carrying, and a couple of other basics. Would you feel better with ten facts? Twenty? Forty?

Of course you would. More information should mean better decisions.

In the early 1970s, psychologist Paul Slovic tested that assumption with people who bet on horses for a living. The results are still quoted today, because almost nobody expects them.

The experiment

Slovic worked with a group of experienced horse race handicappers, people whose job was predicting race results.

He gave them a list of 88 variables that could be known about a horse, from past performance to the jockey's record, and asked each handicapper to rank which ones mattered most.

Then came the races. For each race, the handicappers first saw only the five pieces of information they'd rated most important, and predicted the winner. Then they got more: ten pieces, then twenty, then forty. After each round, they could update their prediction and say how confident they were.

Remember, these were experts choosing the information they themselves believed was most useful.

What happened

With each race featuring 10 horses, random guessing would pick the winner about 10% of the time.

With five pieces of information, the handicappers picked the winner about 17% of the time. Better than chance, and their confidence roughly matched their accuracy.

With forty pieces of information, they still picked the winner about 17% of the time.

Their accuracy didn't improve at all. But their confidence climbed from about 19% to more than 30%.

Eight times more information. The same results. Much more certainty.

Why more data can backfire

The handicappers weren't foolish. They were human. Three things seem to be going on.

The best information comes first. The handicappers' first five variables were the ones they believed mattered most. Everything after that was less useful, but it still felt like progress.

More information feels like more knowledge. Every new fact gives you a sense of understanding, even when it doesn't change the answer. Confidence tracks how much you know, not how much it helps.

Extra data mostly confirms what you already think. When people take in a flood of information, they tend to notice the pieces that support their first instinct.

It's not just horses. In a 1965 study, psychologists were given more and more background about a patient. Their accuracy leveled off early, while their confidence kept rising. A later study of people predicting college football games found the same pattern: more statistics, more confidence, no better predictions.

(A caveat worth knowing: Slovic's horse-racing study is best known through secondhand accounts, especially a widely read CIA analysis manual by Richards Heuer, rather than a major journal paper. But the pattern has held up in the studies that followed.)

Your health is the new racetrack

Today, most of us carry a handicapper's worth of data on our wrists. Steps, sleep stages, heart rate, calories, readiness scores, stress scores, notifications about all of the above.

It feels like we understand our health better than ever. Sometimes we do. But Slovic's handicappers are a warning: more numbers can make you feel certain without making you more right, and they can make it harder to see the few signals that actually matter.

The MIMR connection

MIMR is a company built on measurement. That's exactly why we take this study seriously.

The goal isn't to measure everything. It's to measure the few things that matter, measure them well, and measure them again.

A short list. We focus on a handful of meaningful domains, like cardiovascular health, strength, recovery, and cognition, instead of dozens of metrics competing for attention.

One clear score. Measurements roll up into a single improvement index, so people and employers can see whether things are actually getting better.

Re-measurement over more measurement. Measuring the same important things again at the end of each 90-day cycle tells you more than adding thirty new numbers ever could.

The handicappers didn't need forty facts. They needed the right five, and an honest way to check whether their predictions came true.

Try this

Pick your five. If you track your health, choose the few numbers that matter most to your goals, and ignore the rest for a month.

Check your confidence. Before a big decision, ask: has the last piece of information actually changed my answer, or has it just made me feel surer?

Keep score. The handicappers only knew they weren't improving because someone measured the results. Track whether your decisions work, not just how sure you felt making them.

The takeaway

More information feels like a better bet.

Sometimes it's just a more confident one.

Sources

  1. Slovic, P. (1973). Behavioral problems of adhering to a decision policy. Paper presented at the Institute for Quantitative Research in Finance. https://hdl.handle.net/1794/23607
  2. Heuer, R. J. (1999). Psychology of Intelligence Analysis, Chapter 5: "Do You Really Need More Information?" Center for the Study of Intelligence, Central Intelligence Agency.
  3. Oskamp, S. (1965). Overconfidence in case-study judgments. Journal of Consulting Psychology, 29(3), 261–265.
  4. Tsai, C. I., Klayman, J., & Hastie, R. (2008). Effects of amount of information on judgment accuracy and confidence. Organizational Behavior and Human Decision Processes, 107(2), 97–105.

Measure. Improve. Measure. Reward.

That loop is the whole idea. The Science of Improvement is just us showing our work.

Share this

Read next

Keep exploring