Quantifying the damage of the rush to quantify
It was unsurprising to hear, as we did Tuesday, that Claremont McKenna College had lied about its students’ SAT scores to boost its position in the U.S. News & World Report annual ranking of colleges. University officials are famously obsessed with these rankings, and this is not the first time that a school has admitted fudging data. Just last year, Villanova Law School said that it had given false information to U.S. News.
Today we quantify and rank the performance of people and institutions as never before – all in pursuit, supposedly, of better outcomes and greater efficiency. Yet this obsession with metrics, a hallmark of the free-market ideology, invariably creates more incentives to cheat.
University presidents fret endlessly about the U.S. News rankings because they can have dramatic effects on everything from the quality of student applicants to the ability of schools to attract faculty and raise money. In an earlier era, one free of U.S. News, schools would not have had much reason to lie about SAT scores or admission rates. But now, with these numbers seen as hugely important, you can understand the temptation to monkey around with the reported data.
It’s not just colleges. Metrics have also taken over our K-12 schools, and they’re worse off for it. The education world has been roiled by major scandals recently in which administrators and teachers have been found to have manipulated student scores on standardized tests. These have not been isolated incidents, with revelations of false reporting in Atlanta, Chicago, Washington, Birmingham, New York City and Los Angeles.
It’s no secret why so many teachers and administrators are fudging test numbers: The pressure surrounding high-stakes testing is evident enough in the phrase itself. Tests now determine how schools are funded and, in some places, which teachers get bonuses. Incentives to cheat are baked into this system, and those incentives will only grow if more localities link teacher performance to compensation.
Similar temptations are elsewhere, with more disastrous consequences. The ever-greater focus on quarterly earnings by public companies has partially led to a rising tempo of corporate scandals since the 1990s. Executives, fearing shareholder backlash, have cooked the books to prop up stock values – most sensationally in the gigantic Enron and WorldCom frauds. (The Japanese manufacturing company Olympus is now embroiled in a major accounting scandal of its own.)
Executives didn’t used to obsess much about quarterly earnings, according to former CEOs I interviewed for a book I wrote about the Harvard Business School class of 1949. Now these numbers dominate corporate culture and pervert both business decisions and the ethics of executives.
Other examples abound of how a growing focus on bottom-line results can bring out the worst in people. Perhaps no police department in the U.S. has relied more heavily on metrics than New York’s, with its famed Compstat program. And sure enough, testimony last year by a former New York City narcotics detective revealed that it was common practice among NYPD officers to fabricate drug charges against innocent people to meet arrest quotas.
The number-crunching crowd argues that stronger metrics lead to better outcomes, and certainly there are places where this is true. The Compstat program did help make New York a safer city by using data in new ways and holding police precincts more accountable for their performance. Better data on school performance has empowered parents and elected leaders seeking to choose the best schools and improve public education.
But, as many critics have pointed out, trying to quantify everything is questionable given the subtleties of the human experience. And the unethical things that happen when numbers rule should reinforce these concerns. Rankings and quotas aren’t the only way to motivate people to do better or measure what is happening. Softer qualitative indicators can also tell a lot about performance, and there is growing movement in various spheres to embrace these indicators as a complement to hard numbers.
Colleges have already begun to migrate away from the quantified life. More than 850 schools no longer require SAT or ACT scores as part of admission applications, and that number is growing. There have also been recurrent efforts to boycott or undercut the U.S. News & World Report rankings – most recently with a group of law professors urging deans at law schools to withhold LSAT data from U.S. News.
Meanwhile, there is greater awareness of how the tyranny of the bottom line leads corporations to behave badly. Reformers are working to change the laws around corporate charters to allow public companies to pay heed to a wider array of stakeholders and move beyond a fixation with short-term profits at any cost.
Finally, the Obama Administration is moving to dismantle the No Child Left Behind regime – which some have joked should be called “No Teacher Left Honest” – and create a more flexible way to measure schools that hinges less on standardized tests. While the administration’s “Race to the Top” program does stress rigorous quantitative assessments, it blends an emphasis on data with other factors like better preparation for teachers. Obama himself has pointedly criticized too much standardized testing.
The obsession with metrics rolled into U.S. society amid the triumphalism of free-market economics starting in the 1980s. It won’t go away easily, and, to be sure, there are still areas where better measurements are needed – like gauging happiness and quality of life.
But it seems as if more leaders finally understand that measuring everything is not such a hot idea. Let’s hope that the scandal at Claremont McKenna serves to reinforce this point.