|
General rankings have limited use. The value of data depends on the interests of the reader, which for college 'rankings' vary widely. For example, if you are interested only in the status of your college degree among the public, then the ranking itself over time and the credibility of US News are probably most valuable, regardless of how they are calculated. But if you are interested in the status of your degree within academia then the 'peer assessment survey' seems most important. Your interests and thus the value of different data will vary by what you want to do (study? work? donate? evaluate research? hire researchers or teachers?), what field(s) you are interested in and at what level, how you learn/work, etc. For many people, the 'ranking' might rely almost completely on one department or just a few specific faculty members. The US News & World Report's ranking system apparently depends (at least in part) on class size (8% of ranking), proportion of faculty with terminal degrees (3% of ranking), proportion of faculty who are full-time (1% of ranking), student-faculty ratio (1%), financial resources per student (10%), retention and graduation rates (35%), student debt (5%), and "'peer assessment survey' [20%] in which college presidents, provosts, and admissions deans are asked to rate other institutions." All that data can be valuable but says little directly about the quality of education received by students or the quality of research, though the latter might be outside USN's scope. Sometimes, objectivity is a distraction because the objective data is too limited to inform us. For example, objective data about a play - number of words, etc. - doesn't tell you much about it. In those cases, expert judgment is an excellent tool and I think USN's 'peer assessment survey' is the right approach for many purposes, but USN surveys people with narrow knowlede - administrators, including admissions officers. I might look at Times Higher Education, which performs a 'reputation survey' that asks (IIRC) thousands of published, tenured faculty about the quality of other schools' departments in their field of expertise. These are people with expertise and significant access, though of course their understanding of the student's perspective will be limited. (Columbia rates around 10th, IIRC.) |
I've been obsessed with computer science rankings for a while, and worked with others to compare US News Computer Science rankings with other computer science rankings: https://drafty.cs.brown.edu/csopenrankings/
Some rankings are naturally harder to game, like that your bachelors or doctoral graduates get hired as professors at other research universities. Or that professors at your universities win best paper awards at leading conferences. So I've been looking at whether various rankings are "biased".
There's some clear biases with US News, like it ranks CalTech as 11, but CalTech is listed as rank 39 in the aggregate because it does poorly on faculty publications and best paper awards. Yale CS is another example of a highly ranked department by US News (rank 20), that has an aggregated ranking of 35. Harvard CS does amazing with placement (ranked 6 for their undergraduate and doctoral students becoming professors), but has an aggregate ranking of 23.