Comments on: Quantifying the damage of the rush to quantify Thu, 21 Jul 2016 07:57:19 +0000 hourly 1 By: matthewslyman Fri, 03 Feb 2012 20:04:33 +0000 [Counterpoint]: In favour of metrics: if you can set them up so that you can cross-check different people’s figures, and set up a system of incentivisation that works against collusion; you can sometimes get accurate metrics reported. In this case, metrics can be VERY useful, and you can do all kinds of useful analysis on them… Otherwise, my previous comments apply, and metrics should be regarded very skeptically.

By: FredF99 Fri, 03 Feb 2012 18:33:34 +0000 Mr Callahan, it’s not a matter of too much metrics in schools, but TOO LITTLE ETHICS, which flows all the way down from the various “leaders” in this country. Other countries use metrics to gauge the success of their government programs without widespread cheating and self-dealing. We need to fire and prosecute all the corrupt individuals in this country starting at the top.

By: excrement Fri, 03 Feb 2012 17:57:23 +0000 its not the measuring, its the method that matters.
metrics are unavoidable. any institution that selects people must select based on some criteria. the criteria can be debated, but the fact that a selection based on some critera must occur cant be.

By: OneOfTheSheep Fri, 03 Feb 2012 04:38:58 +0000 I have seldom seen a conclusion more at odds with presented facts. In American philosophy, who is more revered than Horatio Alger?

Success is the Holy Grail of capitalism. It is deemed so desirable, like our Medal of Honor, that people incapable of achieving it will claim or steal it if they think they can get away with it. While it is impossible to change fundamental human nature, it is not only possible but necessary and desirable to study them so as reduce their effects upon our society.

In the 1980s America had the inspiration of Ronald Reagan after the despair of Jimmy Carter. Once more “we, the people” bought into the idea that we were exceptional and could do anything. You can’t be “exceptional” without comparison. You can’t be “successful” without comparison. Comparison is the inseparable companion of achievement. Comparison is also the inseparable companion of accountability.

Unfortunately America is a country run by bureaucracy. There is the Academic, or Educational bureaucracy, largely unionized. There are government bureaucracies at local, state and the federal level, largely unionized. Accountants are “private bureaucrats” of considerable influence. Executives “run” companies administered, by and large, according to bureaucratic principles.

The “tyranny of the bottom line” is but a euphemism for accountability. Notice how quickly the subject went from a negative to a positive? Words are important, but proper understanding and comprehension are even more so.

The last thing in the world the average bureaucrat wants is to be accountable in any manner. No average institution, no average professor, no average teacher, no average union worker, no average executive, no average police chief, no average military leader and no average politician will willingly subject themselves to the bright light of accountability so long as there is an alternative. But the excellent in any line of endeavor always welcome comparison.

The single weakness of comparison is in how what is compared and what is the goal are diddled. Too often the dull start with the answer they desire and contrive a “results-oriented” process backward from there. Lazy judges often pervert our legal process this way.

Conversely, it is devilishly difficulty for incompetents to muddy the water when a method is agreed in advance with which to equitably and accurately measure progress once the process begins towards a clearly defined goal. It’s like spray painting, where the most important and time consuming work takes place before you fill and pick up the spray gun.

So yes, each of us is solely responsible for “doing our homework” before reaching a decision or venturing an opinion. We each must be intellectually “accountable” or no one will take us seriously. Is not being “taken seriously” the pinnacle of credibility AND accountability? A good “bottom line” is a reward for good performance. It is “tyranny” only for those who would pass off bad performance for good.

A society that would be a meritocracy must think, speculate, innovate, compare, evaluate, decide, implement, observe, and repeat, ad infinitum. We need not more “metrics” but more understanding. We are no different economically than we are politically. Too many forget that the very survival of democracy is utterly dependent on “informed” voters. And no, I’m not about to try to measure THAT!

By: RMoS Fri, 03 Feb 2012 02:45:42 +0000 Metrics are much of what we have. What we lack is the ability to evaluate these measures. We live in a culture that’s disinclined to question the numbers: how were they produced, what precisely do they reveal?

Rational folks have always been extremely suspicious of anecdotal evidence. This same level of inquiry should be applied to statistics, and the methods used to produce same.

Please forgive me, for stating the obvious.

By: matthewslyman Thu, 02 Feb 2012 23:05:30 +0000 Remember the old management mantra:
“Performance only improves where it is measured”?
An entire school of management has been built around this purported panacea. If only I had a dollar for every meaningless statistic I’ve been cajoled into reporting to supervisors when the time would have been better spent discussing observations and recommendations instead.

It’s only a few years since Léo Apotheker wrote a Reuters blog article explaining how SAP could help big business keep track of carbon consumption, for reporting under the new carbon credit markets. I pointed out at the time that no software would accomplish that goal effectively if businesses could populate their databases with any data they liked (or deliberately focus on aspects of their local carbon cycle most likely to be favourable) so as to “prove” the predetermined point they wanted to make. There’s also not much point if it just turns into an exercise in diving down a rabbit-warren of ever-increasing detail regarding geological/ biological aspects of the carbon cycle (the only people who would benefit from these approaches are those at SAP).

Sometimes we’re better off with an inexact approach, like IR spectroscopy via satellite digital imagery to detect and quantify large-scale carbon emissions. That would take a broad measurement that would be extremely difficult to falsify. My feeling is that the best possible approach is to combine these sorts of aggregate statistics with personal impressions and fully personalised communication.

By: Araes Thu, 02 Feb 2012 19:08:23 +0000 The police department brings up another primary issue with metrics, which is if they will be used, significant thought needs to go into creating a metric that incentivizes the behavior you’re actually aiming for.

IE: a metric that purely rewards arrests is useless, as it can easily be gamed, and doesn’t actually achieve what you’re trying to reach – less crime.

Corporate culture is rife with this problem as well, particularly in areas where groups can create their own metrics to try and illustrate how wonderful they are. Most are fluff, like widgets produced, or quarterly earnings, that don’t actually achieve the important goal – a healthy, profitable company or a satisfied customer.