Michael J. Maubossin: The Success Equation; Untangling Skill and Luck in Business, Sports, and Investing

Skill or Luck

Kahneman and Tversky argue that three types of information are relevant to statistical prediction. The first is prior information, or base rate. The second type of information is the specific evidence about an individual case. The third type of information is the expected accuracy of the prediction, or how precise you expect it to be given the information you have. The key to statistical prediction is to figure out how much weight you should assign to the base rate and specific case.

When skill plays the prime role in determining what happens, you can rely on specific evidence. In activities where luck is more important, the base rate should guide your prediction.

Skill tends to follow an arc; it improves for some time, peaks, and the glides lower.

Useful statistics are persistent (the past correlates highly with the present) and predictive (doing well or poorly correlates strongly with the desired goal).

Skill, luck, and three easy lessons

The first step in untangling skill and luck is to define the terms. We just have to be clear on what elements of performance we are considering. Next, we want to agree on the measures of performance. The benefit of measurement is that it allows us to assign specific values to skill and luck. Now we can turn to definitions.

Luck is a chance occurrence that affects a person or a group. Luck can be good or bad. If it is reasonable to assume that another outcome was possible, then a certain amount of luck is involved. Luck is out of one’s control and unpredictable.

Randomness and luck are related, but there is a useful distinction between the two. You can think of randomness as operating at the level of a system and luck operating at the level of the individual.

Luck is residual; it’s what is left over after you’ve subtracted skill from an outcome.

Preparation and hard work are essential elements of skill. Patience, persistence, and resilience are all elements of skill.

The best-known advocate for the idea that you can create your own luck is Richard Wiseman. He identified the “four principles” of luck: “The principles include maximizing your chance opportunities, listening to your lucky hunches, expecting good fortune, and turning bad luck into good.”

The dictionary defines skill as the “ability to use one’s knowledge effectively and readily in execution or performance.”

When luck has little influence, a good process will always have a good outcome. When a measure of luck is involved, a good process will have a good outcome but only over time. When skill exerts the greater influence, cause and effect are intimately connected.

A quick and easy way to test whether an activity involves skill: ask whether you can lose on purpose.

The process of acquiring a skill follows three stages:

In the cognitive stage, you try to understand the activity and you make a lot of errors.
Next comes the associative stage.
Finally, there is the autonomous stage, where the skill becomes habitual and fluid.

What distinguishes elite performers, or experts, from the rest of us is that they advance beyond their natural plateaus through deliberate practice.

In considering skill, it is also important to distinguish between experience and expertise. There is an unspoken assumption that someone doing something for a long time is an expert. In activities that depend largely on skill, though, expertise comes only through deliberate practice, and very few individuals are willing to commit the time and effort to go beyond a plateau of performance that’s good enough.

Gregory Northcraft differentiates experts and people with expertise in a way, that experts have predictive models, and people who have experience have models that aren’t necessarily predictive.

When explaining why bigger samples are needed if luck plays a bigger role, we can think about Moivre’s equation from Abraham de Moivre, that states that the variation of the mean (average) is inversely proportional to the size of the sample.

If what happens is mostly the result of skill then reversion toward the mean is scant and slow. If luck play a role, reversion is much higher and quicker.

What happened in the past does influence the future, a process known as path dependence.

Nassim Taleb offers a useful way to figure out where statistical tools are likely to work and where they fail. He introduces a 2×2 matrix, where the rows distinguish between activities that can have extreme variation and those that have a narrower range of possibilities. The columns of the matrix are the payoffs, and distinguish between the simple and the complex. Black swan domain is based on extreme outcome and complex payoffs. Statistical methods work in all other quadrants except “black swan”.

Why we’re so bad at distinguishing skill from luck

Our mind loves story to explain the world around us. The lessons from the past are often wrong. When we see an effect, we naturally seek the cause.

The conscious mind – the self or soul – is a spin doctor, not the commander in chief. Once we know the ending, we stand ready to create a narrative to explain how and why events unfolded as they did.

When we look at winning companies, we rarely sample the failures. The performance of a company always depends on both skill and luck. We should ask our self, how many of the companies that tried that strategy actually succeeded?

Attributing success to an individual makes for good narrative, but it fails to take into account how much of the skill is unique to the star and is therefore portable. Boris Groysberg, a professor of organizational behavior at Harvard Business School, has studied this topic in depth. He find out that a lot of stars suffered when they changed the environment (company). His explanation is based on them leaving behind a good fit between their skills and the resources of their employer.

The luck-skill continuum

We’re naturally inclined to believe that a small sample is representative of a larger sample. In other words, we expect to see what we’ve already seen.

The paradox of skill. As skill improves, performance becomes more consistent, and therefore luck becomes more important. When everyone in business, sports, and investing copies the best practices of others, luck plays a greater role in how well they do.

Luck can overwhelm skill in the short term if the variance of the distribution of luck is larger than the variance of the distribution of skill.

Luck is boring as the driving force in a story. So, when talking about success, we tend to place too much emphasis on skill and not enough on luck.

Gladwell puts it, that outliers reached their lofty status through a combination of ability, opportunity, and utter arbitrary advantage.

We can use formal measurement in specifying the rate of reversion to the mean by introducing the James-Stein estimator with a focus on what is called the shrinking factor.

Estimated true average = Grand average + shrinking factor (observed average – grand average)

Grand average is average of all players and observed average is the average of the player we are estimating.

For activities that are all luck, the shrinking factor is 0, which means that the expected value of the next outcome is the mean of the distribution of luck. If skill and luck play an equal role, then the shrinking factor is 0.5, halfway between the two. The closer the activity is to all skill, the closer the factor is to 1. The James-Stein estimator can be useful in predicting the outcome of any activity that combines skill and luck.

Placing activities on the luck-skill continuum

How can you assign an activity to skill part of continuum. Ask if you can easily assign a cause to the effect you see. If you can repeat the behavior and get the same result. These are activities that are generally stable and linear. Stable means that the basic structure of the activity doesn’t change over time, and linear means that a particular action leads to the same reaction every time.

Another question that can help us in placing activities on continuum is what is the rate of reversion to the mean. Slow reversion is skill, fast is connected with luck.

And the third question is can we predict well. When the predictions of experts tend to be uniform and accurate skill is the driving force.

One method to estimate the activity place in continuum is to get the distribution if it would be all luck and then a distribution if it would be all skill. To blend the distribution, we need to sort out how much skill and luck contribute to the actual results. Let’s assign a value to luck as a percentage of actual results. Call it p. if the value of luck is p and the actual results combine skill and luck, we know that the value of skill is 1-p. We are looking for the value of p that gives us a distribution that most closely resembles the actual results.

The final method for placing activities on the continuum is based on what is known as true score theory. The equation for true score theory is as follows:

Variance (observed) = Variance (skill) + Variance (luck)

Performance distributions in business are statistically indistinguishable from distributions in non-business domains.

The paradox of skill is an effective way to explain why markets are so hard to beat consistently. The more everyone’s level of skill looks the same, the more you’d expect the range of excess returns for money managers to shrink.

It is essential to emphasize that it is not where activities lie per se that is important but rather what that position means for helping us to make decisions. A common mistake is to use a process for making a decision that is appropriate for activities that are nearly all skill and then apply it to an activity that is mostly luck.

The arc of skill

With skill, the age is not you friend.

When it comes to cognitive tasks, skill is closely related to being competent at making decisions. There are a couple of ways of conceptualizing how good someone is at making decisions. The first way suggests that competence is the result of developing expert knowledge that allows you to choose a course of action automatically. This type of competence is intuitive. Intuition works when the environment is stable and an individual has the opportunity to spend a great deal of time learning about it.

When we age, we tend to avoid exerting too much cognitive effort. This means that we make poorer choices in environments that are complex and unstable.

Fluid intelligence refers to the ability to solve problems that you’ve never seen before. Crystallized intelligence is the ability to use the knowledge accumulated through learning. Fluid intelligence peaks around the age of twenty and declines consistently and steadily throughout life. The ability to reason with numbers also tends to erode with age. The good news is that crystallized intelligence tends to improve with age.

Keith Stanovich distinguishes between an individual’s intelligence quotient (IQ) and rationality quotient (RQ). Objectively intelligent people make poor decisions. The attributes of RQ, as Stanovich lists them, include “adaptive behavioral acts, judicious decision making, efficient behavioral regulation, sensible goal prioritization, reflectivity, (and) the proper calibration of evidence.” Plenty of people have adequate intelligence, but an inability to think and behave rationally. The gap between IQ and RQ is the result of trouble with mental processing and limits to what we know.

People lose skill with age, but so do organizations. One of the reasons that teams are so difficult to run is that managers must constantly replace older players with younger players.

Exploring known markets requires optimizing processes and executing effectively, and leads to reliable, near-term success. Exploring unknown markets requires search and experimentation and offers none of the immediate benefits of exploitation.

The many shapes of luck

One way to learn about the distribution of luck in the real world is by asking whether events are dependent on or independent of one another.

Statisticians have a name for the normal ups and downs that you should expect when the distribution of luck is known: common-cause variation. In economics, common-cause variation is akin to risk. Frank Knight defined risk in its economic sense as a case where “the distribution of the outcome in a group of instances is known.”

Whenever people can judge the quality of an item by several different criteria and are allowed to influence one another’s choices, luck will play a huge role in determining success or failure. Skill does play a role in success and failure, but it can be overwhelmed by the influence of luck.

The process of social influence and cumulative advantage frequently generates a distribution that is best described by a power law. One of the key features of distribution that follows a power law is that there are very few large values and lots of small values. As a result, the idea of an “average” has no meaning.

Since skill alone clearly cannot explain power laws, we need to turn to the mechanisms that generate those lopsided outcomes. The distinction between independent and dependent outcomes is crucial. A path-dependent process is one in which what happens next depends on what happened before. The processes are very sensitive to initial conditions and lead to phenomena such as the rich getting richer and the poor getting poorer. Robert K. Merton called this the Matthew effect, after a verse in the Gospel of Matthew.

Critical points and phase transitions are also crucial for the Matthew effect. A phase transition occurs when a small incremental change leads to a large-scale effect. This is known colloquially as a tipping point.

We have seen that path dependence and social interaction lead to inequality. Technology and competition also contribute to this phenomenon.

But no matter how we assess someone’s skill, luck will also help to shape our opinion through social influence. So luck is not only behind the inequality of outcomes, it determines what we perceive to be skill.

We are very good at fooling ourselves about our own success, a phenomenon that psychologists call the self-serving attribution bias. It is common for us to attribute success to our own terrific skill, even in endeavors that are determined mostly by luck.

When we observe the success of others, we fall victims to the fundamental attribution error. In this context, the error is the tendency to base our explanation of what happens on an individual’s skill rather than the situation.

You should be very skeptical of anyone who claims to be able to predict results whenever social influence is a factor.

What makes for a useful statistic

Useful statistics have two features. First, they are persistent, which means what happens in the present is similar to what happened in the past. If the job you do is predominantly a matter of skill, you can expect to be able to repeat your performance reliably. In statistics, this persistence is called reliability. Good statistics are also predictive of the goal you seek. Statisticians call this validity. Statisticians assess persistence and predictive value by examining the coefficient of correlation.

The process of determining which statistics are useful begins with a definition of your objective. Next, you have to determine what factors contribute to achieving your objective. This allows you to assess how skill, measured as high persistence, translates into your objective, measured as high predictive value. Because high skill is associated with high correlations for persistence and predictability, correlations allow us to infer a great deal about the nature of the activity.

Too many companies select statistics for their ubiquity rather than their utility.

There is a statistic in investing, called active share, that is worth considering. Developed by two economists, Martijn Cremers and Antti Petajisto, active share measures the fraction of a portfolio that is different from the benchmark index.

Statistics in sports are often more persistent and predictive than those in business and investing.

Building skill

Daniel Kahneman and Gary Klein wrote a paper Conditions for Expertise. They agreed that expertise is valid under relatively narrow conditions: you can become an expert if cause and effect are clear and consistent in what you do (the condition of validity) and if you practice intensely and are guided by accurate feedback.

In most cases, checklists don’t contain anything that you don’t already know. They just ensure that you actually perform all the tasks you’re supposed to perform. It’s surprising how many fields there are in which checklists could help but are not used.

The main problem remains that people use their intuition in situations where they shouldn’t.

You can become an expert by using a deliberate practice to train your System 1. Deliberate practice and the concept of expertise apply only near the skill side of the luck-skill continuum.

Deliberate practice is powerful in domains where it applies. But acknowledging its limits is crucial. Deliberate practice begins with a coach or teacher who designs the curriculum specifically to improve performance. Done correctly, deliberate practice demands performance that is just outside of your comfort zone so that there’s a constant sense of being challenged without feeling overwhelmed. Once you have an appropriate routine, deliberate practice requires an enormous amount of time and effort. Feedback is the glue that holds together the elements of deliberate practice.

In activities that are largely a matter of skill, basic ability counts.

Most jobs combine tasks that are procedural with tasks or situations that are novel. Where cause and effect can be clearly established, checklists have been widely embraced. Checklists are highly effective but underutilized in jobs that combine probalistic tasks with tasks that follow a set of rules or set procedures.

Atul Gawande wrote a book The Checklists Manifesto. Daniel Boorman, an engineer at Boeing thinks that checklists should be short. The rule of thumb is that it should be five to nine items and fit to one page.

Boorman describes two types of checklists: DO-CONFIRM and READ-DO. With DO-CONFIRM checklists, pilots do their jobs from memory but pause from time to time to ensure that everything is complete and has been done properly. READ-DO checklists typically deal with an emergency or an abnormal situation.

Whether you’re managing a sports team, running a business, or investing in stocks, a skillful process will tend to have three parts: analysis, psychology, and the influences exerted by your organization.

The analytical part. Value is what you think it’s worth. Price is what people are paying (or receiving) today.

The second part of skillful process is psychological. This part deals with Kahneman and Tversky’s work on biases. Because good decisions can have bad outcomes, not everyone has a temperament that is well suited to making decisions about activities that involve luck. When most people come to believe the same thing, large gaps open up between price and value.

The third part of the process of skill addresses organizational and institutional constraints. The most important job is to manage agency costs, or the costs that arise because the one acting (agent) can have different interests than an owner.

In careers where skill involves developing a process and then following it, professionals will often strive to be different enough to succeed but not so different as to be considered unconventional.

For activities near the luck side of the continuum, a good process is the surest path to success in the long run.

Dealing with luck

One approach deals with reducing the advantage of a skilled opponent if you are the underdog and improving your advantage if you are a favorite. The second approach involves reducing the influence of luck by more effectively tying cause and effect. And it is also important to understand the limits of your understanding. Especially important is to find a way to cope with events with small and incomputable probabilities, but very large consequences.

When competing one-on-one, follow two simple rules. If you are the favorite, simplify the game. If you are the underdog, make it more complicated. The game called Colonel Blotto is a useful model for quantifying and understanding these rules.

Randomness and luck are the result of insufficient information – an inability to pinpoint cause and effect. Controlled experiments can be a quick and effective way to improve our understanding of cause and effect.

Moral hazards refer to a person or organization taking an action on behalf of others without suffering the consequences if the outcome is bad.

Taleb recommends that we avoid optimization and allow for redundancy if we are working in »the black swan« quadrant. Because optimization works well in some systems, there is a temptation to optimize in the fourth quadrant during times to relative stability.

In some cases, it is helpful for you to consider luck as equivalent to a lack of knowledge.

Reversion to mean

Francis Galton was behind idea of reversion to the mean in lates 1800s. reversion to the mean says that an event that is not average will be followed by an event that is closer to the average.

Reversion to the mean is an idea that most people believe they understand. Yet the concept is actually very hard to grasp and even harder to employ in making decisions. Specifically, reversion to the mean creates three illusions. The first is the illusion of cause and effect. There is also illusion of feedback and the illusion of declining variance.

The illusion of cause and effect arises when people try to assign significance to an instance of reversion to the mean, when in reality they are simply seeing a lack of correlation.

Francis Galton figured out that correlation and reversion to the mean are two takes on the same concept. The coefficient of correlation between two variables determines the rate of reversion to the mean and provides valuable guidance for forecasting.

Combining the ideas of weighting the information (How much does it count?) and persistence (Will you see the same results again?) gives you specific guidance in judging what the next results is likely.

Kahneman and Tversky said that there are three types of information that are relevant for a statistical prediction: the prior information, or base rate; the specific evidence about the individual case; and the expected accuracy of the prediction. The trick is determining how to weight the information.

For reversion to the mean to be relevant, there has to be some sense of the mean, or average.