President's Blog

Archive for the Assessment Category

Couldn’t Have Said It Better Myself

The topic of college rankings has come up on this blog from time to time, usually with some cautionary words from me about understanding rankings for what they do and don’t mean, thinking critically about how rankings are defined and compiled, and so on.

This week the topic of rankings, and specifically of the U.S. News and World Report rankings, is much in the news. The unfortunate occasion is the revelation that an official at Claremont-McKenna College, a fine institution with outstanding students and a strong reputation, misrepresented its students’ SAT scores in pursuit of higher rankings. (They are not alone; similar approaches to “gaming the rankings” have been reported at other colleges.) Several interesting and insightful commentaries on this situation have been published this week; some I’d recommend to you are here, here, and here.

Sweet Briar, like many of the liberal arts colleges in the Annapolis Group, does not participate in the “reputational survey” aspect of the USN&WR rankings — that is, we do not respond to surveys asking us to give our subjective impression of the quality of other schools. Nor do we promote our own U. S. News rankings in admissions materials. We take this position because we do not believe the reputational survey collects meaningful information and that, because of its weight in the methodology, the overall scoring is significantly flawed. Besides, it is inherently unreasonable to think that all the complex factors that make an educational institution effective can be distilled into one single number on which meaningful comparisons can be based.

What thoughtful people in higher education are talking about this week is a larger issue. They are talking about the skewed system of incentives created by the dominance of rankings such as those published by USN&WR. Here’s an example, from a piece by William D. Adams, president of Colby College:

“Perhaps the time has come for us to admit that we are trying to work within a system that encourages us to skew our work toward improving our performance in the rankings rather than focusing on our core educational missions; deploy teams of people to squeeze each ranking data point until it screams, rather than take all possible measures to find students who seek – and will truly benefit from – the distinctive experience we each have to offer.”

I couldn’t have said that better myself. When I was a candidate for the presidency at Sweet Briar, I remember being asked by a faculty member what I would do to improve Sweet Briar’s standing in the U.S. News rankings. And I remember my reply: that I would do very little for that reason, as I believe it is a serious mistake to chase rankings. The thing to chase, in my view, is excellence. If institutions keep focus on educational excellence for students, the world — and any truly useful rankings systems — will recognize it. If we chase rankings, we may succeed in moving an institution from, say, #84 to #81, or from #25 to #20, or from #7 to #3.  But there is no reason to believe that we would be better serving students because of it.

At Sweet Briar, we spend much more time thinking about how we assess and improve student learning than about where we stand in the eyes of U. S. News. And that’s as it should be.



From the Association of American Colleges and Universities

I’ve spent much of this week at the annual meeting of the Association of American Colleges and Universities, along with Dean Amy Jessen-Marshall (who is doing two presentations on the program) and a team of six Sweet Briar faculty members.

In my last post I mentioned the recent faculty retreat and our ongoing discussions of curriculum review. Bringing a campus team to this conference was another aspect of that process: here there are sessions highlighting research and best practices from around the country on a wide variety of topics. One member of our team is following discussions of the Bologna Accords and the development of European learning standards: another is attending sessions on business curriculum: several are going to sessions on how to integrate research, assessment, and learning outcomes into campus planning; and all of us are going to the occasional session that just looks interesting! (For me, yesterday, that was a session about how students at elite liberal arts colleges define success. . . for a couple of members of our group yesterday afternoon, it was a session on digital humanities that is featured in this article.)

For my part, I’ve been participating in the President’s Forum. At lunch today, a representative from the Department of Education spoke with us about the education proposals just, and I mean just, announced by President Obama. As you might imagine, having the opportunity to respond to some of these proposals within a day of their announcement was valuable; the presidents who are here posed a number of important and fundamental questions.

It’s both sobering and encouraging to sit with a group of college and university presidents these days. Each and every one of us is aware of the serious problems facing higher education. College needs to be more affordable, more Americans need to complete degrees, the learning that takes place in college needs to be more clearly and persuasively demonstrated, and state and federal investments in education and research need to be commensurate with the importance of higher education to the economic future of the nation. No institution is immune from these considerable pressures — no community college, no elite research university, no liberal arts college, no comprehensive state university. And no one knows what the answers should be or will be. (That’s the sobering part.)

But every single person at this conference is deeply committed to being part of crafting the best possible answers. Every dean and faculty member here is deeply committed to making higher education the best and most meaningful experience it can be for every student on their campuses.  And finally, at the end of the day, whatever the answers turn out to be, we all recognize that they will need to be centered on student learning. As long as student access and learning goals are kept at the center of our thinking, our research, and our planning, we will get to the right answers. That’s the encouraging part!



Interesting Research

In my Thanksgiving post, I mentioned attending a recent meeting of the Annapolis Group presidents.

One of the ways the Annapolis Group supports liberal arts colleges is by sponsoring research. Recently, a survey of graduates showed that alumnae of residential liberal arts colleges report consistently higher levels of satisfaction with their undergraduate educations than do alumnae of either public or private universities.

To quote a few points from the piece at the link above:

  • Seventy-six percent of liberal arts college graduates rated their college experience highly for preparing them for their first job, compared to 66 percent who attended public flagship universities;
  • Eighty-nine percent of liberal arts college graduates reported finding a mentor while in college, compared to 66 percent for public flagship universities;
  • Sixty percent of liberal arts college graduates said they felt “better prepared” for life after college than students who attended other colleges, compared to 34 percent who attended public flagship universities.
  • Liberal arts college graduates are more likely to graduate in four years or fewer, giving them a head start on their careers.

This research has sparked considerable discussion. True, it doesn’t shed light on what might be “selection effect” and what might be “treatment effect,” by which I mean simply that it doesn’t analyze the impact of possible differences in the characteristics of students who choose to attend residential liberal arts colleges. And it relies on self-reported data; it doesn’t prove that liberal-arts college graduates are demonstrably better prepared than others, for example,  just that they report that they believe they are. In other words, it’s more of a J.D. Power “consumer satisfaction” survey than a Consumer Reports study.

But to my mind it’s important to know that students who invest in themselves by choosing a residential liberal arts education end up certain that they’ve received excellent value. I can, and do, tell families confidently that if they choose Sweet Briar they will, in decades to come, feel they made an excellent choice. And really, if our graduates aren’t the best authorities on the value of their experience, who is?




Rating the Rankings

I have posted here occasionally about how Sweet Briar fares in various college rankings. And I will certainly continue to do so in the future.  (Recently I’ve posted on how we did in the Forbes “America’s Best Colleges” list, a “Great Colleges To Work For” survey, and the Princeton Review rankings.)

As I’ve noted before, though, it’s very important to be aware of how any ranking system collects and analyzes data, defines criteria, and weights categories.  Today there is a very interesting — although I warn you, rather long — article called “Reputation Without Rigor” on Inside Higher Ed about one aspect of the U. S. News and World Report college rankings. That highly influential ranking scheme gives significant weight to a “reputational survey.” U. S. News polls academic leaders for their impressions of the quality of other colleges and universities and uses the results to rank institutions by “reputation.”

Sweet Briar has, since 2007, taken a position shared by most members of the Annapolis Group, to which America’s most selective private colleges belong.  That position is that the reputational survey component of the U. S. News rankings is impressionistic and lacks methodological rigor. Sweet Briar therefore does not participate in the reputational survey — which means that Dean Jonathan Green and I do not respond to the surveys inviting us to rate other institutions by our impressions of their quality. And because the reputational survey results are heavily weighted in determining the ultimate rankings, Sweet Briar (like many of its Annapolis Group colleagues) does not advertise or promote its standing in the annual U. S. News rankings.

Critical thinking and a close review of the evidence are fundamental to Sweet Briar’s educational values. I am neither pro- nor anti-rankings, as a matter of principle. But I am firmly in favor of understanding how various rankings are created and what they really do — and don’t — mean.

Facts We’re Proud To Share

Sweet Briar professor Steve Bragaw pointed out this article on “facts colleges hate to share” yesterday.

It’s the kind of article that appears with some frequency.   Indeed, a few of its points echo comments I heard at a recent meeting of corporate leaders.

It’s also the kind of article that educators find, well,  frustrating.  Campuses dedicate a significant amount of time and effort to collecting and reporting precisely the information that we’re accused of trying to hide.  Take a look, for example, at the Department of Education’s College Navigator, or at U-CAN (the University and College Accountability Network.)  Many commercial sources, including U. S. News and Peterson’s and others, publish similar information.  Which leads to the question — why, as more and better information is actually available about higher education generally and about individual colleges, is there so strong a public perception that students and families are unable to learn what they need to know to assess their options?

If you’re interested in how Sweet Briar performs on some of the specific points mentioned in the article, check out our page at U-CAN.

I take the larger issue very seriously.  Sweet Briar, like all colleges and universities, needs to understand the questions and concerns of the public and to address them in meaningful ways.  In forwarding the article to me Professor Bragaw noted that,  as the College begins a new cycle of assessment and planning, we need to be aware of and responsive to the public interest in clear, specific, and accessible information.  I couldn’t agree more; we are educators, after all, and educating the public with regard to our institutions and our successes is essential to our mission.