Quantifying the damage of the rush to quantify
It was unsurprising to hear, as we did Tuesday, that Claremont McKenna College had lied about its students’ SAT scores to boost its position in the U.S. News & World Report annual ranking of colleges. University officials are famously obsessed with these rankings, and this is not the first time that a school has admitted fudging data. Just last year, Villanova Law School said that it had given false information to U.S. News.
Today we quantify and rank the performance of people and institutions as never before – all in pursuit, supposedly, of better outcomes and greater efficiency. Yet this obsession with metrics, a hallmark of the free-market ideology, invariably creates more incentives to cheat.
University presidents fret endlessly about the U.S. News rankings because they can have dramatic effects on everything from the quality of student applicants to the ability of schools to attract faculty and raise money. In an earlier era, one free of U.S. News, schools would not have had much reason to lie about SAT scores or admission rates. But now, with these numbers seen as hugely important, you can understand the temptation to monkey around with the reported data.
It’s not just colleges. Metrics have also taken over our K-12 schools, and they’re worse off for it. The education world has been roiled by major scandals recently in which administrators and teachers have been found to have manipulated student scores on standardized tests. These have not been isolated incidents, with revelations of false reporting in Atlanta, Chicago, Washington, Birmingham, New York City and Los Angeles.
It’s no secret why so many teachers and administrators are fudging test numbers: The pressure surrounding high-stakes testing is evident enough in the phrase itself. Tests now determine how schools are funded and, in some places, which teachers get bonuses. Incentives to cheat are baked into this system, and those incentives will only grow if more localities link teacher performance to compensation.
Similar temptations are elsewhere, with more disastrous consequences. The ever-greater focus on quarterly earnings by public companies has partially led to a rising tempo of corporate scandals since the 1990s. Executives, fearing shareholder backlash, have cooked the books to prop up stock values – most sensationally in the gigantic Enron and WorldCom frauds. (The Japanese manufacturing company Olympus is now embroiled in a major accounting scandal of its own.)
Executives didn’t used to obsess much about quarterly earnings, according to former CEOs I interviewed for a book I wrote about the Harvard Business School class of 1949. Now these numbers dominate corporate culture and pervert both business decisions and the ethics of executives.
[Counterpoint]: In favour of metrics: if you can set them up so that you can cross-check different people’s figures, and set up a system of incentivisation that works against collusion; you can sometimes get accurate metrics reported. In this case, metrics can be VERY useful, and you can do all kinds of useful analysis on them… Otherwise, my previous comments apply, and metrics should be regarded very skeptically.