[sei]

[the genius filter]

How To Know If You’re Getting Better, or Just Getting Comfortable

Experience feels like a surefire way to wisdom.

Ten years on the job, thousands of decisions made, patterns seen time and again: these should sharpen judgment, right?

Not always.

A famous New York physician once developed a reputation for diagnosing typhoid fever before symptoms appeared. His method: feeling patients' tongues with his bare hands. He was consistently correct. His colleagues marveled at his skill. But his success masked a darker truth. He wasn't just diagnosing typhoid, he was spreading it. Each accurate prediction came from the infection he'd just transmitted.

This is what happens when experience teaches the wrong lesson.

We assume time spent builds expertise. More reps, better decisions. More exposure, sharper intuition. In some environments, this holds. Tennis players improve through thousands of serves. Chess masters refine their play across countless games. Their feedback arrives quickly and accurately, and the rules stay consistent.

But life rarely resembles chess.

Psychologist Robin Hogarth spent decades studying when experience actually improves judgment. His framework centers on a simple distinction: some environments reward experience with growth, while others corrupt it. The difference lies in the quality of feedback itself, whether the world tells you the truth about your actions or quietly misleads you.

Understanding this distinction changes how we learn, how we trust our instincts, and how we evaluate the expertise of others. Because in many of the decisions that matter most, experience alone guarantees nothing.

[the spark]

Kind and Wicked Learning Environments

Why does experience make some people experts and others just confidently wrong?

That's the question that Hogarth, a behavioral scientist who spent his career studying judgment and decision-making, sought to answer.

His conclusion came down to the structure of the environment itself.

In kind environments, the world plays fair. Cause and effect are visible. Feedback arrives quickly and accurately, and the rules hold steady over time. Chess players see the consequences of moves immediately. Skilled tradespeople watch their work succeed or fail in real time. In these settings, repetition builds genuine skill.

Wicked environments operate differently. Feedback is delayed, absent, or misleading. Outcomes are messy, confusing, and often hard to replicate. The patterns that seem to emerge may not be patterns at all. This happens in spaces like investing, macroeconomic forecasting, and complex organizational decisions. These domains generate intuition, too, of course. But that intuition often points in the wrong direction.

The physician spreading typhoid learned something from each accurate diagnosis; he just learned the wrong thing. Hogarth's insight flips our common assumption by pointing out that experience doesn’t guarantee learning. It can reinforce error just as easily as it builds skill.

The question isn’t how many years someone has spent in a field, but what kind of feedback that field provides.

[the science]

Some feedback throws you off course.

Hogarth and his colleagues formalized the problem in a 2015 paper that mapped six distinct ways learning and application can fall out of sync.

They called this the "two-settings framework." One setting is where information is acquired. The other is where it gets used. Kind environments produce a match between the two. Wicked environments produce systematic mismatches.

The research identified specific mechanisms that create these mismatches:

Survivorship bias filters out failures before they can be observed. Entrepreneurs who failed disappear from the data set. Mutual funds that underperformed get dropped from portfolios. What remains looks like success, but the sample is incomplete. Conclusions drawn from these “survivors” don’t generalize to the full population.

Censorship bias hides information beyond a cutoff point. A manager sees when employees fall short but rarely observes how much more they could have done. Basically, performance above a threshold stays invisible. Learning from what you can see teaches you nothing about what you’re blind to.

Selection bias narrows the sample before analysis begins. Investors judge future performance by looking only at top-performing funds. Analysts study successful businesses to find what made them succeed without examining the failures that did the same things. The logic feels sound, but there’s a lot hedged on inference.

The hot stove effect shows how early negative results can shut down exploration entirely. One bad outcome makes people avoid an option forever. The system never self-corrects because the rejected path is never revisited. A manager abandons a new process after one failure, never learning whether it would have worked with time.

In each case, people form confident beliefs. They really feel they have learned something true. But the structure of the environment has filtered, delayed, or distorted the information before it reached them.

The takeaway isn’t that experience is unreliable, just that reliability depends on structure. Without a match between where knowledge is gained and where it’s applied, feedback tends to reinforce whatever pattern happened to appear first.

[the takeaways]

1) Diagnose Your Environment First​
​Before trusting your instincts, assess the feedback structure around you. Ask whether signals arrive quickly or with a delay, and whether outcomes clearly trace back to decisions.

2) Separate Outcome from Decision Quality​
​Good outcomes can follow bad decisions. Bad outcomes can follow good ones. Judge the method, not just what happened to work once.

3) Be Skeptical of Experience​
​Time spent doesn't guarantee skill gained. In many environments, feedback is delayed and distorted. Recognize when your field doesn't play fair.

4) Shorten Feedback Loops​
​Create faster signals by testing ideas in smaller cycles. The tighter the loop between action and outcome, the cleaner the lesson. Speed matters when feedback is your only teacher.

5) Use External Validation​
​Introduce metrics, peer review, or independent checks to counter distorted internal learning. Outside signals reveal what experience alone cannot.