Washington Monthly - September/October 2019 - 105

A NOTE ON METHODOLOGY:
4-YEAR COLLEGES
AND UNIVERSITIES
o establish the set of colleges included in the rankings, we
started with the 1,732 colleges in the fifty states that are listed in the U.S. Department of Education's Integrated Postsecondary Education Data System (IPEDS) and have a 2018 Carnegie basic classification of research, master's, baccalaureate, and
baccalaureate/associate's colleges, are not exclusively graduate
colleges, participate in federal financial aid programs, and plan
to be open in fall 2019. We then excluded 212 baccalaureate and
baccalaureate/associate's-level colleges which reported that at
least half of the undergraduate degrees awarded were below the
bachelor's-degree level, as well as twenty-seven colleges with
fewer than 100 undergraduate students in any year they were
open between fall 2015 and fall 2017 and an additional four colleges with fewer than twenty-five students in the federal graduation rate cohort in 2016.
Next, we decided to exclude the five federal military academies (Air Force, Army, Coast Guard, Merchant Marine, and Navy)
because their unique missions make them difficult to evaluate
using our methodology. Our rankings are based in part on the
percentage of students receiving Pell Grants and the percentage of students enrolled in the Reserve Officers' Training Corps
(ROTC), whereas the service academies provide all students with
free tuition (and thus no Pell Grants or student loans) and commission graduates as officers in the armed services (and thus not
the ROTC program). Finally, we dropped an additional fifty-one
colleges for not having data on at least one of our key social mobility outcomes (percent Pell, graduation rate, net price, or the
number of Pell recipients earning bachelor's degrees). This resulted in a final sample of 1,431 colleges and includes public, private
nonprofit, and for-profit colleges.
Our rankings consist of three equally weighted portions:
social mobility, research, and community and national service.
This means that top-ranked colleges needed to be excellent
across the full breadth of our measures, rather than excelling in
just one measure. In order to ensure that each measurement contributed equally to a college's score within any given category, we
standardized each data element so that each had a mean of zero
and a standard deviation of one (unless noted). Missing social mobility data (affecting less than 1 percent of all observations) were
imputed and noted with "N/A" in the rankings tables. We adjusted data to account for statistical outliers by allowing no college's
performance in any single area to exceed five standard deviations
from the mean of the data set. All measures use an average of the
three most recent years of data in an effort to get a better picture
of a college's performance rather than statistical noise. Thanks to
rounding, some colleges have the same overall score. We have
ranked them according to their pre-rounding results.
The social mobility portion of the ranking also doubles as
our Best Bang for the Buck rankings, with the exception that

T

the main rankings are by Carnegie classification while the Best
Bang for the Buck rankings are by region (while predicted rates
are calculated by Carnegie classification). We again used a college's graduation rate over eight years for all students instead
of the first-time, full-time graduation rate that is typically used
but presents an incomplete picture of a college's success. This
graduation rate counted for 16.66 percent of the social mobility
score. Half of that score was determined by the reported graduation rate and the other half came from comparing the reported
graduation rate to a predicted graduation rate based on the percentage of Pell recipients and first-generation students, the percentage of students receiving student loans, the admit rate, the
racial/ethnic and gender makeup of the student body, the number of students (overall and full-time), and whether a college is
primarily residential. We estimated this predicted graduation
rate measure in a regression model separately for each classification using average data from the last three years, imputing
for missing data when necessary. Colleges with graduation rates
that are higher than the "average" college with similar stats score
better than colleges that match or, worse, undershoot the mark.
A few colleges had predicted graduation rates over 100 percent,
which we then trimmed back to 100 percent.
We used IPEDS data comparing graduation rates of Pell and
non-Pell students to develop a Pell graduation gap measure.
Colleges that had higher Pell than non-Pell graduation rates received a positive score on this measure, which was based on
just the two years of available data and counted for 16.66 percent of a college's score. We included a new measure this year of
the number of Pell recipients earning bachelor's degrees, which
is designed to reward colleges that successfully serve large numbers of students from lower-income families. This measure, from
IPEDS, counts for 5.56 percent of the social mobility score and replaces the percent Pell measure from last year.
We also used IPEDS data for the percentage of a college's
students receiving Pell Grants and College Scorecard data on
the percentage of first-generation students to get at colleges'
commitments to educating a diverse group of students. Our
measure compared actual shares of Pell and first-generation
students to the predicted share after controlling for ACT/SAT
scores and the share of families in a state with incomes below
$35,000 and between $35,001 and $75,000 per year. This change
is informed by recent research by Caroline Hoxby of Stanford
and Sarah Turner of the University of Virginia showing that the
percentage of Pell-eligible students varies considerably across
states on account of different household income distributions.
The Pell enrollment performance measure counted for 5.56 percent of the social mobility score, while the first-generation enrollment performance measure counted for 2.77 percent. We
also allocated another 2.77 percent to the unadjusted share of
first-generation students.
We measured a college's affordability using data from IPEDS
for the average net prices paid by first-time, full-time, in-state students with family incomes below $75,000 per year over the last
three years. We focused on these income categories because of
our interest in affordability for students from lower- to middleWashington Monthly

105
Washington Monthly - September/October 2019

Table of Contents for the Digital Edition of Washington Monthly - September/October 2019
