The $4 Trillion Question

How to Judge Mutual Fund Managers

Research by Lubos Pástor

New research suggests that mutual fund managers are best evaluated by looking at the company they keep.


"Suppose an individual chooses a mutual fund manager with a stellar three-year history and invests all his retirement money with that manager," says Lubos Pástor, an assistant professor at the University of Chicago Graduate School of Business. "It's not clear that the manager will perform well in the future, and the investor may lose a large sum of money. Figuring out which managers perform well is really a $4 trillion question that the investment community wants to understand."

Together with Randolph Cohen and Joshua Coval of Harvard Business School, Pástor suggests a new approach to answering this question in the study "Judging Fund Managers by the Company They Keep."

When people compete, their success can often be predicted by their techniques, and the track records of others who use the same techniques.

Take a group of basketball players, some of whom shoot with both hands and some with only one hand, who have been taking ten shots each. So far two-handed shooters have on average made eight out of ten shots, while one-handed shooters have made four out of ten. Two players, one one-hander and one two-hander, have completed only half of their shots, with both sinking four out of five shots. Although the track records of
both players are identical, it seems sensible to forecast that the two-handed shooter will achieve a higher score, because previous scores have shown that two-handed shooters tend to score higher. The one-handed shooter, using what appears to be an inferior technique, is more likely to have just been lucky in his first five shots.

Mutual fund managers also rely on a variety of techniques when trying to outperform their peers, and many of these techniques are common to groups of managers. They may, for example, collect information from the same sources or use the same valuation methods.

"Managers using similar techniques should be expected to deliver similar future performance," argues Pástor.

Since managers' techniques are difficult to observe, the study focuses directly on managers' investment decisions, which can be observed from periodic portfolio disclosures. Managers using similar techniques are likely to make similar investment decisions, so a fund manager's ability to select outperforming stocks can be judged by the extent to which his investment decisions resemble those of other successful managers.

Traditional methods of judging mutual fund managers rely solely on a manager's own historical returns, typically taking his track record and then calculating average returns from that track record. Since the return histories of many existing funds are short, these textbook measures are often imprecise, and it is difficult to separate skill from luck.

Pástor, Cohen, and Coval propose a new way of ranking mutual fund managers that should predict performance more accurately. Their new measures look at the returns of all managers whose holdings or trades overlap with those of an individual manager. The most significant detail is the individual manager's degree of similarity with managers who have performed well, and difference from those who have performed poorly. The authors' results indicate that their new skill measures are approximately four to eight times more precise than traditional return-based measures.

If two managers hold similar stocks or buy and sell the same stocks at the same time, then they would be said to be following similar techniques. For each stock at each quarter-end, the authors calculate the average track record of all managers who recently bought this stock and subtract the average track record of all managers who recently sold this stock. The result is referred to as "stock quality." The authors next define a manager's skill as the average quality of all stocks that the manager recently bought minus the average quality of all stocks that he recently sold.

"In evaluating the performance of manager A, we take account of manager A's historical track record, as well as the historical track records of managers B and C," says Pástor. "If manager A's investment decisions are more similar to manager B's than manager C's, we pay more attention to manager B's track record than to manager C's track record in judging manager A."

Effectiveness

Are these new measures useful in evaluating true managerial skill? Using simulated data, the authors find that the answer is yes. Their measures prove to be increasingly accurate as the number of managers increases.

Pástor, Cohen, and Coval also analyzed the performance of a large sample of U.S. equity funds. For all 3,281 funds that had existed for two or more years at the end of their sample (September 30, 2000), they estimated all performance measures and their precision using the complete return history for each fund. The authors find that their new system is more precise than the traditional measure for over 90 percent of all funds.

This result is not surprising because a typical fund holds dozens of stocks, and a typical stock is held by dozens of funds. Each additional holder improves the precision of the new estimate of stock quality, and each additional stock makes this fund performance estimate more precise.

More Positive Than Expected

According to these new measures, are there mutual fund managers who demonstrate true skill?

Using traditional returns-based measures of performance, one would find as many bad managers as good managers (even before accounting for transaction costs and fees charged to investors). Good managers, with "positive skill," are defined as those who can beat their passive benchmarks. Bad managers, with "negative skill," are those who underperform their passive benchmarks.

Pástor argues that those results do not make much sense.

"For a rating system to judge a manager's skill to be negative would require the manager to be less skilled than a monkey managing a portfolio by throwing darts," says Pástor. "Our results are more plausible, because we find that there are quite a few managers who are able to beat their benchmarks before costs and fees, but very few who can't."

According to the study, fund managers with significantly positive skill before costs and fees vastly outnumber fund managers with significantly negative skill. The authors find only three funds in their sample that show significantly negative skill, while 1,076 funds show significantly positive skill.

Nonetheless, managers' superior skill does not necessarily persist over time. Mutual fund returns remain almost unpredictable, as should be expected in a competitive market.

Lubos Pástor is assistant professor of finance at the University of Chicago Graduate School of Business.

>> Request a copy of the research paper featured in this article