The Illusion of Stock-Picking Skill

In 1984, Amos and I and our friend Richard Thaler visited a Wall Street firm. Our host, a
senior investment manager, had invited us to discuss the role of judgment biases in
investing. I knew so little about finance that I did not even know what to ask him, but I
remember one exchange. “When you sell a stock,” d n􀀀I asked, “who buys it?” He
answered with a wave in the vague direction of the window, indicating that he expected
the buyer to be someone else very much like him. That was odd: What made one person
buy and the other sell? What did the sellers think they knew that the buyers did not?
Since then, my questions about the stock market have hardened into a larger puzzle: a
major industry appears to be built largely on an illusion of skill. Billions of shares are
traded every day, with many people buying each stock and others selling it to them. It is
not unusual for more than 100 million shares of a single stock to change hands in one day.
Most of the buyers and sellers know that they have the same information; they exchange
the stocks primarily because they have different opinions. The buyers think the price is too
low and likely to rise, while the sellers think the price is high and likely to drop. The
puzzle is why buyers and sellers alike think that the current price is wrong. What makes
them believe they know more about what the price should be than the market does? For
most of them, that belief is an illusion.
In its broad outlines, the standard theory of how the stock market works is accepted
by all the participants in the industry. Everybody in the investment business has read
Burton Malkiel’s wonderful book A Random Walk Down Wall Street. Malkiel’s central
idea is that a stock’s price incorporates all the available knowledge about the value of the
company and the best predictions about the future of the stock. If some people believe that
the price of a stock will be higher tomorrow, they will buy more of it today. This, in turn,
will cause its price to rise. If all assets in a market are correctly priced, no one can expect
either to gain or to lose by trading. Perfect prices leave no scope for cleverness, but they
also protect fools from their own folly. We now know, however, that the theory is not quite
right. Many individual investors lose consistently by trading, an achievement that a dartthrowing
chimp could not match. The first demonstration of this startling conclusion was
collected by Terry Odean, a finance professor at UC Berkeley who was once my student.
Odean began by studying the trading records of 10,000 brokerage accounts of
individual investors spanning a seven-year period. He was able to analyze every
transaction the investors executed through that firm, nearly 163,000 trades. This rich set of
data allowed Odean to identify all instances in which an investor sold some of his holdings
in one stock and soon afterward bought another stock. By these actions the investor
revealed that he (most of the investors were men) had a definite idea about the future of
the two stocks: he expected the stock that he chose to buy to do better than the stock he
chose to sell.
To determine whether those ideas were well founded, Odean compared the returns of
the stock the investor had sold and the stock he had bought in its place, over the course of
one year after the transaction. The results were unequivocally bad. On average, the shares
that individual traders sold did better than those they bought, by a very substantial margin:
3.2 percentage points per year, above and beyond the significant costs of executing the
two trades.
It is important to remember that this is a statement about averages: some individuals
did much better, others did much worse. However, it is clear that for the large majority of
individual investors, taking a shower and doing nothing would have been a better policy
than implementing the ideas that came to their minds. Later research by Odean and his
colleague Brad Barber supported this conclusion. In a paper titled “Trading Is Hazardous
to Yourt-t􀀀 Wealth,” they showed that, on average, the most active traders had the poorest
results, while the investors who traded the least earned the highest returns. In another
paper, titled “Boys Will Be Boys,” they showed that men acted on their useless ideas
significantly more often than women, and that as a result women achieved better
investment results than men.
Of course, there is always someone on the other side of each transaction; in general,
these are financial institutions and professional investors, who are ready to take advantage
of the mistakes that individual traders make in choosing a stock to sell and another stock
to buy. Further research by Barber and Odean has shed light on these mistakes. Individual
investors like to lock in their gains by selling “winners,” stocks that have appreciated since
they were purchased, and they hang on to their losers. Unfortunately for them, recent
winners tend to do better than recent losers in the short run, so individuals sell the wrong
stocks. They also buy the wrong stocks. Individual investors predictably flock to
companies that draw their attention because they are in the news. Professional investors
are more selective in responding to news. These findings provide some justification for the
label of “smart money” that finance professionals apply to themselves.
Although professionals are able to extract a considerable amount of wealth from
amateurs, few stock pickers, if any, have the skill needed to beat the market consistently,
year after year. Professional investors, including fund managers, fail a basic test of skill:
persistent achievement. The diagnostic for the existence of any skill is the consistency of
individual differences in achievement. The logic is simple: if individual differences in any
one year are due entirely to luck, the ranking of investors and funds will vary erratically
and the year-to-year correlation will be zero. Where there is skill, however, the rankings
will be more stable. The persistence of individual differences is the measure by which we
confirm the existence of skill among golfers, car salespeople, orthodontists, or speedy toll
collectors on the turnpike.
Mutual funds are run by highly experienced and hardworking professionals who buy
and sell stocks to achieve the best possible results for their clients. Nevertheless, the
evidence from more than fifty years of research is conclusive: for a large majority of fund
managers, the selection of stocks is more like rolling dice than like playing poker.
Typically at least two out of every three mutual funds underperform the overall market in
any given year.
More important, the year-to-year correlation between the outcomes of mutual funds is
very small, barely higher than zero. The successful funds in any given year are mostly
lucky; they have a good roll of the dice. There is general agreement among researchers
that nearly all stock pickers, whether they know it or not—and few of them do—are
playing a game of chance. The subjective experience of traders is that they are making
sensible educated guesses in a situation of great uncertainty. In highly efficient markets,
however, educated guesses are no more accurate than blind guesses.
Some years ago I had an unusual opportunity to examine the illusion of financial skill up
close. I had been invited to speak to a group of investment advisers in a firm that provided
financial advice and other services to very wealthy clients. I asked for some data to
prepare my presentation and was granted a small treasure: a spreadsheet summarizing the
investment outcomes of some twenty-five anonymous wealth advisers, for each of eight
consecutive years. Each adviser’s scoof 􀀀re for each year was his (most of them were
men) main determinant of his year-end bonus. It was a simple matter to rank the advisers
by their performance in each year and to determine whether there were persistent
differences in skill among them and whether the same advisers consistently achieved
better returns for their clients year after year.
To answer the question, I computed correlation coefficients between the rankings in
each pair of years: year 1 with year 2, year 1 with year 3, and so on up through year 7 with
year 8. That yielded 28 correlation coefficients, one for each pair of years. I knew the
theory and was prepared to find weak evidence of persistence of skill. Still, I was
surprised to find that the average of the 28 correlations was .01. In other words, zero. The
consistent correlations that would indicate differences in skill were not to be found. The
results resembled what you would expect from a dice-rolling contest, not a game of skill.
No one in the firm seemed to be aware of the nature of the game that its stock pickers
were playing. The advisers themselves felt they were competent professionals doing a
serious job, and their superiors agreed. On the evening before the seminar, Richard Thaler
and I had dinner with some of the top executives of the firm, the people who decide on the
size of bonuses. We asked them to guess the year-to-year correlation in the rankings of
individual advisers. They thought they knew what was coming and smiled as they said
“not very high” or “performance certainly fluctuates.” It quickly became clear, however,
that no one expected the average correlation to be zero.
Our message to the executives was that, at least when it came to building portfolios,
the firm was rewarding luck as if it were skill. This should have been shocking news to
them, but it was not. There was no sign that they disbelieved us. How could they? After
all, we had analyzed their own results, and they were sophisticated enough to see the
implications, which we politely refrained from spelling out. We all went on calmly with
our dinner, and I have no doubt that both our findings and their implications were quickly
swept under the rug and that life in the firm went on just as before. The illusion of skill is
not only an individual aberration; it is deeply ingrained in the culture of the industry. Facts
that challenge such basic assumptions—and thereby threaten people’s livelihood and selfesteem—
are simply not absorbed. The mind does not digest them. This is particularly true
of statistical studies of performance, which provide base-rate information that people
generally ignore when it clashes with their personal impressions from experience.
The next morning, we reported the findings to the advisers, and their response was
equally bland. Their own experience of exercising careful judgment on complex problems
was far more compelling to them than an obscure statistical fact. When we were done, one
of the executives I had dined with the previous evening drove me to the airport. He told
me, with a trace of defensiveness, “I have done very well for the firm and no one can take
that away from me.” I smiled and said nothing. But I thought, “Well, I took it away from
you this morning. If your success was due mostly to chance, how much credit are you
entitled to take for it?”