Morningstar - Q3 2022 - 8

Dispatches
Rating Our Ratings
On balance, the
Morningstar fund
ratings do their jobs.
Phillips Curve
12 No Laughing Matter
Ivory Towers
14
Direct Indexing for All?
Policy
17 With Crypto, Advisors Need to
Tread Carefully
Sustainability Matters
18
Sustainability or ESG?
Global Briefs
21
25
Americas
Greater Europe
26 Australia & New Zealand
29
Asia
LETTER FROM THE BOARD
Jeffrey Ptak
We recently analyzed the performance of
Morningstar's fund rating systems and found that,
on balance, they have done a good job of
sorting funds based on future performance.
Higher-rated funds were more likely to survive
and outperform than lower-rated funds.
While this is encouraging, we also found some
opportunities for improvement.
An Overview of the Fund Ratings
Morningstar has four fund ratings that are similar,
but with notable differences. The Morningstar
Rating for funds (more commonly known as
the star rating) is backward-looking, whereas
the Morningstar Analyst Rating and Morningstar
Quantitative Rating are forward-looking.
The star rating and the Quantitative Rating are
quantitative in nature, while the Analyst
Rating and Morningstar Sustainability Rating
turn on analysts' qualitative judgments. Finally,
the Sustainability Rating's primary goal is to
denote how much material unmanaged risk these
funds court from environmental, social,
and governance factors, not to predict their
performance.
Nevertheless, we know investors tend to use
these systems to achieve better financial
outcomes. Accordingly, we assessed whether the
ratings tended to sort funds based on their
future performance.
How We Analyzed Performance
We measured ratings' performance in a few ways.
We examined the trailing returns of U.S. funds
and exchange-traded funds in each ratings cohort
8
Morningstar Q3 2022
through Feb. 28, 2022. We also ran an
" event study " in which we took monthly snapshots
of the ratings since their inception, tracked
performance of the rated funds over a subsequent
time period, and then averaged those
measurements.
To control for stylistic differences, we compared
rated funds' returns to a relevant peer group
average and to the benchmark index that
corresponds to the category to which the funds
had been assigned. In addition, we evaluated
risk-adjusted returns by calculating rated funds'
capital asset pricing model alphas versus
the relevant Morningstar Category average or
the assigned category index.
The study used rated funds' net-of-fee returns,
adjusted to control for multiple share classes
(so that multi-share-class funds didn't have undue
weight in the calculations) and included dead
funds whenever possible.
Here's what we found for each rating.
Morningstar Rating for Funds
Though the star rating struggled recently, it
did a good job of sorting funds over the trailing
three-, five-, and 10-year periods based on
their future performance. Higher-rated funds were
far likelier to survive and outperform their
average peer than lower-rated funds (EXHIBIT 1 ).
The same pattern held when we measured rated
funds against their benchmark indexes (EXHIBIT 2 ).
True, not even the highest-rated cohorts were able
to generate positive average annual excess
returns versus their assigned indexes, reflecting
the challenge of topping a costless benchmark.
All the same, high-rated funds lagged their
indexes by significantly smaller margins than
low-rated funds.
But these average annual excess returns do not
incorporate the results of funds that started
but didn't survive to the end of the trailing periods
shown. To derive a sense of how well the
star rating did in predicting the odds of success-
which we defined as surviving to the end of
a trailing period and outperforming a peer-group
https://www.morningstar.com/authors/489/jeffrey-ptak
Morningstar - Q3 2022

Table of Contents for the Digital Edition of Morningstar - Q3 2022
