By: Caroline Koppelman
In 1995, Reed College, one of the best liberal arts schools in the country, announced it would no longer submit data for consideration in the US News College Rankings. They made this decision based on the arguments that the ratings were deeply flawed and not representative of a school’s best qualities. In researching the system they found that just 2% of college counselors believed that US News’ methods were effective, and that 91% of colleges were manipulating their submissions to artificially raise their ranking.
The US News and World Report rankings are the most discussed and cited college ranking in the world. We find the rankings come into a family’s college discussion early on, and are being utilized when students design their list of schools. Often, they have too much of an effect when students are choosing among the schools to which they have been accepted. Although we know that it can often seem impossible not to fixate on a college’s position, we strongly urge families to not focus on the rankings, as they are not demonstrative of a college’s fit for any particular student. We understand the difficulty of this. The college process is so multi-faceted and complicated, how could you not want to reduce each school to a digestible number? The idea of taking a living institution and quantifying it seems inherently problematic, and it is. When you examine exactly how US News and World determines these numbers, we think that you will agree with us that they’re hardly worth considering.
The US News and World ranking is broken down into seven categories:
1. Undergraduate academic reputation (22.5%): This includes the opinion of those “in a position to judge a school’s undergraduate academic excellence,” which includes presidents, provosts, and deans of admissions. This allows them to account for “intangibles” like “faculty dedication to teaching.” 2,200 counselors at public high schools and 400 counselors from the largest independent schools are also surveyed. But these counselors are only from the high schools that received gold, silver, or bronze medals in the recent edition of the U.S. News Best High School rankings. These counselors had to rank each school on a 5 point scale—1 is marginal and 5 is distinguished—or mark any schools they didn’t know as “don’t know.”
2. Retention (22.5%): This is the number of first-year students who return to campus for their sophomore year and eventually graduate. The two components of retention are the six-year graduation rate (80%) and first-year retention rate (20%).
3. Faculty resources (20%): 30% of the faculty resource score is the proportion of classes with less than 20 students; 10% is the proportion with 50 students or more. Faculty salary accounts for 35%, the proportion of professors with the highest degree in their fields accounts for 15%, the student to faculty ratio for 5%, and the proportion of full time faculty for 5%.
4. Student selectivity (12.5%): 65% of this is SAT or ACT score. Students who graduated in the top 10% of their high schools for national Universities and Liberal Arts Schools or the top 25% for Regional Universities and Regional colleges is 25%, and acceptance rate is only 10%.
5. Financial resources (10%): average spending per student on instruction, research, student services and related educational expenditures.
6. Graduation rate (7.5%): the difference between the school’s actual six-year graduation rate and the rate US News predicted.
7. Alumni giving rate (5%): living alumni with bachelor’s degrees who gave money
US News first calculates the weighted sum of the standardized scores and then factors all of the other things in. The top ranked school in each category is then bumped up to 100 and every other school’s scores are calculated in proportion to that top score. The final scores are rounded to the nearest whole number.
If this sounds like a lot of complicated percentages and formulas, it's because it is. And this is because of one simple fact. There’s really no good way to quantitatively rank schools against one another.
Just from the breakdown above we can see how flawed US News’ use of data is.
The abstract concept of “reputation” accounts for approximately a quarter of its analysis. This predominantly favors the larger, older, brand name schools.
Additionally, although there are 33,000 high schools in America, only 2,200 counselors are surveyed. These counselors are chosen from US News “Best High Schools” ranking, which is, as could be expected, also biased and flawed in much the same way. This part of the data is extremely skewed and realistically serves no real purpose.
The faculty resources portion of the equation is nonsensical, as 35% of it, or 7% of the entire ranking, is devoted to the professor’s salary. A professor’s ability to teach and inspire is not necessarily represented by their salary. Schools with largest endowments (the larger and older schools with the most brand recognition) are going to have more full time professors who they pay more. Like most of the parts of the ranking, this statistic is in no way indicative of how an individual student will do at the school.
The emphasis on test scores is perhaps the most questionable component of the rankings. They account for 8% of the ranking, only 1% more than teacher’s salaries! Many schools are moving towards being test-optional, so US News uses very limited data for them, and schools that do not require tests to be submitted are on a completely separate list. There is also a potential risk for schools to underreport scores if they become test optional.
One of the few good metrics US News uses is the retention and graduation rates. If a school has a low six-year graduation rate or sees many students leaving after their first year, it is most likely failing the students. However, this data is often available elsewhere and is much better looked at in isolation, as opposed to as a small part of the rankings.
It is easy to understand how this faulty methodology can cause more harm to prospective students and parents. The rankings are detrimental at every step of the process, from picking which schools to apply to, to weighing which school you want to attend. Sure, the rankings may seem like an easy way to save time throughout the college process. If you’re admitted to the #18 school and the #29 school, wouldn’t it be logical to go to the higher ranked one? Sadly, no, and we really caution against using the ranking system at all. You should narrow down your choices based off of real, tangible things, such as location, price, and the feeling you have when you visit the campus. You should not feel bad if the schools you’re applying to or have gotten into do not rank within the top 20, and you should not make any decisions based off of the rankings. So take a note from Reed College and ignore the rankings. In the end it will help you make the best decision for yourself.