Morningstar ratings: Useful or useless?

The financial industry generally places more emphasis on style than substance. Because of this, when their work is actually evaluated, results tend to be disappointing. Wall Street's earnings forecasts? Overly optimistic. Performance of mutual fund managers? Quite embarrassing. You may be wondering: Do Morningstar ratings also belong in the same category?

You're probably familiar with Morningstar and their one- to five-star mutual fund ratings. Many investors rely on Morningstar for stock and mutual fund research, and mutual fund companies love using Morningstar ratings in their marketing materials. But is there any value in a five-star Morningstar rating? (Disclosure: I use Morningstar software sold to investment advisors almost everyday.)

The Morningstar ratings for one of J.D.'s mutual funds.
The Morningstar ratings for one of the funds J.D. owns.

Fortunately for us, researchers recently looked into these ratings and published their results. They compared Morningstar ratings to fund expense ratios as a predictor of future performance.

The expense ratio is the annual fee for investing in a fund. This fee is charged by the mutual fund manager, and it's one of my favorite metrics. If you assume that mutual fund managers have no value — which I find to be a very good approximation — you would expect lower costs to predict better performance. And the report found just that:

Expense ratios are strong predictors of performance. In every asset class over every time period, the cheapest quintile produced higher total returns than the most expensive quintile.

What about Morningstar ratings? Five-star ratings predicted better performance than one-star ratings in 13 of 20 observations — a success rate of just 65%. That sounds pretty good on its own, but it's still worse than a metric that anyone can look up in seconds.

Since Morningstar uses prior performance (after fees) to calculate its ratings, the ratings already include information about expense ratios indirectly. So what is Morningstar adding with its fancy algorithm? Let's use a little high-school algebra to find out. (Geek Alert!)

Morningstar Rating = Expense Ratio + Morningstar's Additional Analytics

And we just found out that:

Expense Ratio > Morningstar Rating

Finally, using my graduate degree in math, I get this:

Morningstar's Additional Analytics < 0

Yes, Morningstar's algorithm is horrible. And that's not all.

Morningstar reserves its five- and one-star ratings for the top and bottom 10% of funds. However, the researchers conducting this study divided expense ratios into quintiles — or, as normal people would say, 20% buckets. The expense ratios were handicapped by using 20% buckets instead of 10%, and still beat Morningstar ratings. Ouch!

Well, there's one thing I forgot to tell you. People have performed this evaluation many times with similar results, so it isn't news to serious students of investing. The interesting part of the report I quoted is the publisher: Morningstar. If you read its report [PDF], it sounds like a politician answering a tough question — uncomfortable. Independent thinkers can go directly to the results here [PDF].

Note: After writing this, I noticed that Morningstar clarified that ratings are indicators of past performance, and should not be used to predict future performance. If Morningstar were concerned about substance, it would tailor its ratings to how investors actually use them — as an indicator of a good investment. If it did that, most five-star rated funds would just be index funds. Unfortunately, Morningstar emphasizes style (and money), so it ends up with an imperfect rating system that benefits one of its biggest clients: mutual funds.

Another Note: Morningstar responded to a version of this article that was published on Mariposa Capital Management's blog in August. You can read the comments here.

More about...Investing

Become A Money Boss And Join 15,000 Others

Subscribe to the GRS Insider (FREE) and we’ll give you a copy of the Money Boss Manifesto (also FREE)

Yes! Sign up and get your free gift
Become A Money Boss And Join 15,000 Others
guest
22 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Elysia
Elysia
9 years ago

I read stuff like this and the article Sierra posted about 529s and I just get frustrated. I don’t know how to pick the right thing to do. I have a financial advisor through my credit union, she seems less than useful to me (doesn’t even offer options on 529s when the internal management fees for individual funds is 2.06% and the annual fee is 25$). Research indicated to me that I wanted low mgmt & portfolio expenses (under 1.25% combined), no maintenance fees and no sales charges. Morningstar apparently lists this info on their site. When I indicated that… Read more »

Nicole
Nicole
9 years ago

I always wondered what those ratings measured (not enough to look them up though). They didn’t seem to match expenses so I couldn’t figure it out. I guess the market always does about a “3”? Elysia– If Ing isn’t your only retirement option, I’d look around for other ones. At least with our plan they’re taking a full .7% off the top of whatever the original expenses are. On top of that, their index funds are very expensive AND they try to push you into even more expensive non-index funds. You might be lucky and able to buy Vanguard funds… Read more »

David
David
9 years ago

I suppose I’m not terribly surprised that a financial institution fails to see the forest for the trees.

Starshard0
Starshard0
9 years ago

I’ve never really paid attention to the Morningstar ratings of my funds, I made it a point to look at the actual numbers and compare to other similar funds. I completely agree that index funds are going to be the winners every time since the fees are so much lower.

Roger Wohlner
Roger Wohlner
9 years ago

Like Mr. Choi I am also na financial advisor who uses a version of Morningstar’s online software geared to financial advisors. I don’t frankly even look at the star rankings. Morningstar is a great provider of data for mutual funds, ETFs, Closed-end funds, stocks, etc. To me this is the value in their service and why I’ve used Morningstar since I started in the business in the mid 90s. I think the star system is used by many fund companies to promote their funds which is terribly misleading to the investing public.

Luke
Luke
9 years ago

Interesting article, very thought provoking.

As a new investor, it’s always useful to have a few pointers when it comes to fund performance. Still, I’ve always taken ‘star’ ratings/letter ratings etc. with a pinch of salt.

While past performance is no indication of future gains etc. etc. I tend to try and pick the funds that meet my investing style (large funds, growth, fairly agressive) and are also performing well vs. their sector/peers.

Still, it’s a minefield for regular investors who don’t have a financial background!

Andrew
Andrew
9 years ago

I am a licensed broker and manage a few client accounts, although as I’ve stated many times on here before I believe everyone would be better served by increasing their own knowledge and managing their own portfolio. I am very much opposed to mutual funds in general, but that is nothing compared to my distrust for Morningstar’s rating system. It is simply a crutch to novice investors and a tool to transfer the blame if a fund does not perform well. It allows a losing investor to think “it was a 5 star fund when I bought it; it’s not… Read more »

Kevin M
Kevin M
9 years ago

Very interesting article, Edwin. It’s a shame there are so few comments.

I can’t say I’ve ever cared what Morningstar or any other ratings company said about any funds I’ve owned. I use the expense ratio as a first criteria whether to consider a fund or not. Looks like I’m on the right track.

Andrew
Andrew
9 years ago

Morningstar gives Vanguard Institutional Index only 3 stars?
http://www.google.com/finance?client=ig&q=VINIX

It’s expense ratio is just 0.05%

Anyway I shifted most of my 401(k) funds into it 🙂

Mike
Mike
9 years ago

I’m an advisor and manage accounts. I do not use actively managed mutual funds in my practice. In my opinion they’re too expensive and usually underperform the index that they are trying to beat. A very broad based low cost index strategy that invests in US, Developed and emerging market Mutual Funds for growth and short term maturity AAA and AA rated bond funds for safety might be the way to go. Size also matters, usually funds that invest in smaller value oriented stocks do better than large growth stocks over time. But they can also me more risky.

Janette
Janette
9 years ago

Investing takes work. Sorry, I have found few financial advisors in cheap suits. They are out there to make money- Off YOUR money. You can follow them- giving them a cut as you go along- or do it yourself. Doing it yourself saves money and complaining (since you can only blame yourself). I work at it every day. Reading, listening, working the numbers and growing it. Got out of mutual funds a long time ago. Why should I pay an advisor to put me in a company to take another take? I guess if I didn’t have the time I… Read more »

Edwin Choi
Edwin Choi
9 years ago

Thanks everyone for the great comments.

@Luke (#6)

“I tend to try and pick the funds that meet my investing style (large funds, growth, fairly agressive) and are also performing well vs. their sector/peers.”

Although investing style/asset class is important to use, relative performance is essentially what Morningstar uses in its ratings. You should be careful about using recent performance in your evaluation.

Pirate Jo
Pirate Jo
9 years ago

It does seem that if a stock or fund has been skyrocketing (thus earning it a five-star Morningstar rating), it might be a good time to sell!

Rob
Rob
9 years ago

The logic in this article is completely faulty, comparing two different metrics.

The author has applied a comparison of the averages of two sample sets (performance of the cheapest and most-expensive quintiles), to the particular results of Morningstar ratings (65% results outperform).

Instead, analysis should be applied to either of two questions:

1. How many funds (%) in the cheapest quintile outperform funds in the most-expensive quintile?
2. Does the average performance of five-star rated funds out-perform, ON AVERAGE, the average performance of one-star rated funds?

Luke
Luke
9 years ago

Thanks for the comment Edwin.

I also consider things such as the TER, the age of the fund and how it has performed historically.

Again, I don’t assume future performance to be the same as past, but isn’t it logical to have more faith in a fund that has produced decent returns for 20 years than a brand new one that hasn’t yet proven the management skills of the fund managers?

Edwin Choi
Edwin Choi
9 years ago

Hi Rob (#14), thanks for commenting. Here are my thoughts on the concerns you bring up. 1. Unfortunately the research report did not look at % of cheap funds that outperform expensive funds (or % of 5-star funds that outperform 1-star funds). However, they did look at % of funds that survived and outperformed all their peers, calling it the success ratio. Not exactly the question you had in mind, but fairly close. Expense ratios did better than ratings in predicting both the success ratio of the top bucket of funds (lowest expense and 5-star rating respectively) and the difference… Read more »

Kevin Cimring
Kevin Cimring
9 years ago

Hi J.D. – this comment comes a long time after the original article was written but the issue of Morningstar and other ratings is a very interesting topic. My colleagues recently completed some research on the Morningstar Fund Managers of 2010, and I thought readers might find the resultant article very interesting: http://www.jemstep.com/blog/2011/01/are-the-morningstar-fund-managers-of-the-year-2010-the-best-for-you/
Kind regards,
Kevin

Phil
Phil
7 years ago

Morningstar is a great data resource. The star ratings can be helpful but is limited. The info on expenses, portfolio content, and past performance is more useful than the stars. Having said that, a portfolio of all five star funds has done me well over the years and keeping to such fund companies as Vanguard, T Rowe Price, Fidelity, and some of the better smaller no load shops gives the investor a price advantage as well as a performance advantage.

Xoche
Xoche
5 years ago

Morningstar is a dishonest service that practices sales tactics that I wouldn’t expect to see in the financial world. They will bait you with a $23.99 cost of a monthly membership, and bury the part about it being “recurring” in the terms and services, refusing to refund the amount they misled you into paying.

I’ve worked with much lesser known companies who, after complaining about this type of bait and switch tactic, will refund your money. Not morningstar. It seems like they need to depend on misleading their customers to make any money.

Lee Rhodes
Lee Rhodes
4 years ago
Reply to  Xoche

It is fully disclosed.

Gguppy Gghyu
Gguppy Gghyu
5 years ago

Ive noticed similar phenomenon with ratings system. 1. Analysis issues buy signal…after price goes up. 2. Sell signal ….after price goes down Historical accuracy….releativly poor. Ratings systems seem to hold no value with regard to the future. They seem to only give good scores to companies that…did good. Past tense Basically they have you buy high and sell low unless you get lucky. Also, bogle head research shows the previous top performers often do poorly lateron. Do you blame them, after a few years of outperforming markets a sector or business gets tired or has bad luck. Nothing goes straight… Read more »

Bob DesRochers
Bob DesRochers
4 years ago

I’m not sure that financial advisors offering a critique of another financial advisor is very helpful. I don’t subscribe to Morningstar and never have BUT this article seems self-serving.

shares