The microeconomics of restaurant letter grades
Many congratulations to the WSJ for putting this chart together; it says much more than the thousand-word accompanying article does. New York City restaurants are periodically graded by inspectors, and then given a grade based on how many violation points they have. And the chart demonstrates, more than any set of anecdotes ever could, that inspectors are trying very hard to give the restaurants the highest letter grade they can.
Now that’s not what the restaurants think, of course. And indeed the fiscal incentives run the other way: restaurants which get B or C grades get fined, and “the amount collected in fines has skyrocketed,” the WSJ reports, “to $42.4 million in the fiscal year that ended last month, from $32.7 million in the previous fiscal year.” (Does a 30% increase really count as “skyrocketed”? I’m not sure about that.)
One theory in the story doesn’t ring true to me at all: Columbia University statistician Roger Vaughan speculates “that restaurant owners learn, after the inspection, about the criteria they are graded on and what the thresholds of the grading system are, and do enough improvements so that at the next inspection they are able to move their scores below 14 violation points or below 28.” But restaurant inspections are much less predictable than that — it’s simply not possible to fine-tune the cleanliness of your kitchen to within a point or two.
Another Vaughan theory doesn’t compel me much either — that restaurant inspectors just don’t want the extra work involved in re-grading restaurants, something which happens to everybody getting a B or a C. The inspectors are salaried, and inspect a fixed number of restaurants: the grades they give out don’t affect that.
Which leaves the hypothesis that “pressure from restaurant groups” is driving the inspectors to give higher grades where it matters. That wouldn’t be the first time, of course, that a regulator was captured by the industry it was meant to be regulating. But of course neither restaurants nor inspectors would ever admit to such a thing, so it’s hard to nail down — until you generate a chart like the one above.
My feeling is that there could be a couple of other factors in play. Firstly, NYC restaurant inspections are harsh, and the inspectors know it. And I remember from my schooldays that the toughest and strictest teachers — the ones we were all most afraid of — were often the ones who ended up giving out the highest grades at the end of the year; something similar might be going on here.
And on top of that is the fact that a simple add-up-the-points heuristic is not always going to be the best way of judging the final grade a restaurant should get. The chart says to me that a restaurant inspection is a somewhat iterative process: you add up the points, you see where the letter grade is, you then compare that letter grade to the letter grade you feel that the restaurant as a whole deserves, and if there’s a difference, you shave off some points here or there. And the inspectors nearly always err on the side of generosity: they’ll shave off points to get you an A or a B, but they won’t be extra-tough to get you a B or a C.
This is no great scandal. Restaurants lose business if they don’t have an A rating, and getting a B or a C costs real money in fines, as well. Given what the public will think when they see a B or a C rating, it’s good that the restaurant inspector takes a minute to ask if that rating is really deserved.
Bad ratings are serious punishment, which shouldn’t just be handed out by an unfeeling computer. Judging, perforce, involves the exercise of judgment. What this chart really shows is that restaurant inspectors are human. Which is something I’m frankly quite grateful for.