US: The 'black box' of peer review

Countless decisions in academe are based on the quest for excellence. Which professors to hire and promote, writes Scott Jaschik for Inside Higher Ed.

Which grants to fund. Which projects to pursue. Everyone wants to promote excellence. But what if academe actually doesn't know what excellence is? Michèle Lamont decided to explore excellence by studying one of the primary mechanisms used by higher education to, in theory, reward excellence: scholarly peer review. The result is How Professors Think: Inside the curious world of academic judgment, just published by Harvard University Press, which aims to expose what goes on behind the closed doors where funds are allocated and careers can be made.

Applying sociological and other disciplinary approaches to her study, Lamont won the right to observe peer review panels that are normally closed to all outsiders. And she was able to interview peer review panellists before and after their meetings, examine notes of reviewers before and after decision-making meetings, and gain access to information on the outcomes of these decisions.

For those who have always wondered why they missed out on that grant or fellowship, the book may or may not provide comfort, comments Inside Higher Ed. Lamont describes processes in which most peer reviewers take their responsibilities seriously, and devote considerable time and attention to getting it right. She also finds plenty of flaws - professors whose judgment on proposals is clouded by their own personal interests, deal making among panellists to make sure decisions are made in time for panellists to catch their planes, and an uneven and somewhat unpredictable efforts by panellists to reward personal drive and determination over qualities that a grant programme says are the actual criteria.
Full review on the Inside Higher Ed site