The microeconomics of restaurant letter grades

By Felix Salmon
July 28, 2011
accompanying article does.

" data-share-img="" data-share="twitter,facebook,linkedin,reddit,google" data-share-count="true">

NY-BC175_NYREST_NS_20110727194503.jpg

Many congratulations to the WSJ for putting this chart together; it says much more than the thousand-word accompanying article does. New York City restaurants are periodically graded by inspectors, and then given a grade based on how many violation points they have. And the chart demonstrates, more than any set of anecdotes ever could, that inspectors are trying very hard to give the restaurants the highest letter grade they can.

Now that’s not what the restaurants think, of course. And indeed the fiscal incentives run the other way: restaurants which get B or C grades get fined, and “the amount collected in fines has skyrocketed,” the WSJ reports, “to $42.4 million in the fiscal year that ended last month, from $32.7 million in the previous fiscal year.” (Does a 30% increase really count as “skyrocketed”? I’m not sure about that.)

One theory in the story doesn’t ring true to me at all: Columbia University statistician Roger Vaughan speculates “that restaurant owners learn, after the inspection, about the criteria they are graded on and what the thresholds of the grading system are, and do enough improvements so that at the next inspection they are able to move their scores below 14 violation points or below 28.” But restaurant inspections are much less predictable than that — it’s simply not possible to fine-tune the cleanliness of your kitchen to within a point or two.

Another Vaughan theory doesn’t compel me much either — that restaurant inspectors just don’t want the extra work involved in re-grading restaurants, something which happens to everybody getting a B or a C. The inspectors are salaried, and inspect a fixed number of restaurants: the grades they give out don’t affect that.

Which leaves the hypothesis that “pressure from restaurant groups” is driving the inspectors to give higher grades where it matters. That wouldn’t be the first time, of course, that a regulator was captured by the industry it was meant to be regulating. But of course neither restaurants nor inspectors would ever admit to such a thing, so it’s hard to nail down — until you generate a chart like the one above.

My feeling is that there could be a couple of other factors in play. Firstly, NYC restaurant inspections are harsh, and the inspectors know it. And I remember from my schooldays that the toughest and strictest teachers — the ones we were all most afraid of — were often the ones who ended up giving out the highest grades at the end of the year; something similar might be going on here.

And on top of that is the fact that a simple add-up-the-points heuristic is not always going to be the best way of judging the final grade a restaurant should get. The chart says to me that a restaurant inspection is a somewhat iterative process: you add up the points, you see where the letter grade is, you then compare that letter grade to the letter grade you feel that the restaurant as a whole deserves, and if there’s a difference, you shave off some points here or there. And the inspectors nearly always err on the side of generosity: they’ll shave off points to get you an A or a B, but they won’t be extra-tough to get you a B or a C.

This is no great scandal. Restaurants lose business if they don’t have an A rating, and getting a B or a C costs real money in fines, as well. Given what the public will think when they see a B or a C rating, it’s good that the restaurant inspector takes a minute to ask if that rating is really deserved.

Bad ratings are serious punishment, which shouldn’t just be handed out by an unfeeling computer. Judging, perforce, involves the exercise of judgment. What this chart really shows is that restaurant inspectors are human. Which is something I’m frankly quite grateful for.

10 comments

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/

I may be cynical, but this screams of graft to me. Taking a little money on the side to tweak the grades seems just as likely as the inspectors systematically improving grades out of the kindness of their hearts.

Posted by Dan_K | Report as abusive

There are definitely ways to game the system. It’s not even a matter of the cleanliness violations and shaving off a point or two.

A local cafe that I frequent got a B on their first inspection, and learned that they could shave off about 10-15 points if more employees got some sort of food handling license, so they had several employees go down and take the exam. This was a significant enough jump to put them squarely in the A category on reinspection.

Posted by very-simple | Report as abusive

Dan_K, you are right to be cynical. On the East Coast, restaurants that pass inspections usually do so after the bar, freezer, and cash till have been properly inspected. On the West Coast (LA was the municipality that came up with letter grades) it’s contingent on hiring a “consultant,” who will pass on his findings to the inspection group.

Posted by OnkelBob | Report as abusive

Well, the chart looks like it could be a Poisson distribution, with a peak a few points before the first down grade.

If that be the case, then the best explanation I can see is human nature. An inspector doesn’t want to be a [expletive deleted] by giving a restaurant a down grade when they are just a few points from passing, and even those points would be fine-point regulatory items, not rats in the sausage kind of offenses.

So, cut everything above n=1,500 off the blue area and distribute them in descending fashion at the first 2 points of the B grade.

Then it looks like a duck, and Vaughan’s threshold hypothesis prevails, overcoming your objection due to the two facts that people stop doing things that are difficult when they’ve done enough to get by, and that the people grading the restaurants know that a lot of violations are silly and they are not willing to be sadists.

Posted by Anonymous | Report as abusive

maybe because I’m chinese and I frequent many establishments in various bits of china town but I daresay that the number of restaurants that I go to feature a B or a C much more often than an A. And the joke always goes, “don’t trust a chinese restaurant that’s too clean.”

Hell, even Felix himself poses that same question (in a video that I can’t find anymore) about the Four Seasons Hotel restaurant v. Starbucks/McDonalds.

Posted by GregHao | Report as abusive

Hahaha, I don’t know why people do love Chinese foods. Sorry, no offense meant.

http://iamdavie.com/sams-club-credit-car d/

Posted by airabaxter | Report as abusive

Having been inspected over two dozen times I can tell you there is very little, if any, graft. What this really highlights is how stupid and meaningless the whole system is. With over 1,000 points in the Health Code, an inspector can go into ANY restaurant at ANY time and award whatever points he/she wants to the place. They are giving just enough B’s to make sure the revenue stream is in line with the City’s fiscal needs. Of course there’s always the inspector who had a bad day and you’ve got yourself a C, or vice-a-versa. But by and large, the letter grades are just a by-product of an overzealous Mayor, bureaucratic power-seekers and some powerful consumer groups. The only losers are the restaurants who now face the Scylla of fines and the Charybdis of stigmatization.

Posted by 6091joshua | Report as abusive

An “A” grade doesn’t necessarily mean a cleaner restaurant, just as this A-graded restaurant has been found to have live mice. See it to believe: http://dopaminejunkie.blogspot.com/2012/ 03/evidence-of-live-mice-in-a-graded-nyc .html.

Posted by DopamineJunkie | Report as abusive

If you want to see inspection results on your iPhone/iPad/iTouch, check out Grade Pending in the App Store. Free, and works even when you don’t have Internet access (in the subway, for example). http://tinyurl.com/gpend

Posted by DanCostin | Report as abusive

I’m very happy to read this. This is the kind of manual that needs to be given and not the random misinformation that is at the other blogs. Appreciate your sharing this greatest doc.