PrintIn Search of the Perfect Food Review Rating System
Every time I've reviewed restaurants in print, on television or radio, or, now, online, I've always wrestled with the question of what rating system to use. There are so many ratings systems already in use it's hard to make sense of them all. The New York Times uses a four-star rating system for its upscale restaurant review, which makes some sense, but I've often found that the stars don't match the prose. Plus, where does that leave the places that appear in the "$25 and Under" column? Nowhere in fact, as those are done with no rating system at all. New York magazine has adopted a five-star system to allow more latitude, and it differentiates between fancy and nonfancy restaurants by using hollow stars for the cheaper restaurant rating. The red Michelin Guide uses a three-star system, but I don't find the Michelin guides to be all that relevant anymore.
I've been searching for a rating system that is easily recognizable that will enable serious eaters to know what to expect when they go to any kind of eatery, from a hot dog stand like Gray's Papaya to a fancy-pants restaurant extravaganza like Daniel or Per Se.
I would like our rating system to place each experience and each food in context. So yesterday someone here at the Serious Eats office suggested a report card–like rating system, A to F, with every plus and minus gradation included.
I thought about it and decided I really liked the idea. Doesn't a perfect natural-casing all-beef hot dog served on a toasted bun deserve an A just as much as a ten-course tasting menu at Daniel or Per Se? I think it does, though maybe not if there's no place to sit to eat that hot dog.
So that's what we're going to try starting tonight in about an hour or so. Thoughts, serious eaters?
Add a comment:
Previewing your comment:
HTML Hints
Some HTML is OK: <a href="URL">link</a>, <strong>strong</strong>, <em>em</em>
Comment Guidelines
Post whatever you want, just keep it pleasant. We reserve the right to delete off-topic or inflammatory comments. Learn more at our Comment Policy page.
If you see something not so nice, please, report an inappropriate comment.
8 Comments:
Oh, great topic!
I'm covering a different world, but I spent a long time considering this same exact thing. I eventually narrowed it down to a 5-star system or an A-F system.
I felt 5 stars gave much more flexibility than only having 3 or 4. Even when you add the half (.5) star, it still seems too limiting to have so few stars. And the stars are a recognized symbol that people understand. They are also easy to make rating graphics for.
The A-F system with plus and minus would provide the most flexibility of all. It's a recognized format, and one of my favorite gaming sites recently switched from a score system (1-100%) to a A-F system and I liked it.
In the end, I chose the 5-stars because... (and this is the only reason)... I couldn't figure out how to make a decent looking graphic or text effect representing the Alphabet ratings, and the little stars were easy to make and fit nicely on my page. Hehe. :)
I look forward to seeing what you guys do. As with almost everything S.E. related, I'm sure it will be great!
FastFoodCritic at 9:26PM on 05/06/08
Hmmm... some interesting food for thought (pun, of course, intended.)
Here are a few disparate thoughts:
-- As mentioned in the comment above, a gradation of stars allows for a list of ratings to be scanned easily. Yes, a list of letter grades can be scanned as well, but not as easily.
-- You talk about docking the perfect hot dog a perfect grade because there's no theoretical seat. The Ny Times and NY Mag do this as well. NY Mag even says as much at the end of the review -- "3 star food minus one star for the loud room." Hmmm... maybe this is where Zagat actually got it right? They break out the rating of food from service from setting. For those who REALLY care about the food -- those, dare I say it, who are SERIOUS about food, the pure food rating is what matters, no?
Peter at 9:31PM on 05/06/08
I think the key to any rating system is to make certain that the reader/audience understands that the rating is within a certain class or against a preconceived expectation. In other words, people need to understand that an A (or 4 stars) for Katz's Deli and the same grade for Le Bernardin don't mean that you'll have the same experience at both. But it also doesn't mean that you won't enjoy them equally -- just differently.
Is there a way to codify that in a rating system? I think you could say that Katz's is an A-rated Deli and Le Bernardin is an A-rated Fine Dining Establishment. Not perfect or particularly eloquent, but it communicates the essence of the rating.
Dominic
the zen kitchen
dvchurch at 11:20PM on 05/06/08
I like the flexibility that A through F with all the + and - involved gives you, but be wary... its hard to stay consistent with so many options. A review you do 3 months from now may get a B+ even though a similar experience garnered an A- today... and even though they're very close, A- feels a LOT better than a B+.
In any case, i think the biggest problem with ratings systems is not so much how many starts you can give or how wide the range is, but clearly defining what each of the ratings MEAN. Maybe a C+ in your book is actually quite good, and you never/rarely plan to give any A's. It may all be codified in your head but its important to get it down on paper (or screen as it were) and let everyone else know what you're thinking. Plus, it will help avoid that gradual grade inflation/deflation i mentioned above.
Good luck!
mh330 at 9:36AM on 05/07/08
i am a numbers person, so those appeal more to me. i like the idea of breaking out a dining experience into categories, such as: atmosphere (10), service (10), value (10), food (20), giving food more of a weight noting it's importance. this way, the readers have an idea as to why a restaurant received the rating it did.
mbbeamer32 at 10:29AM on 05/07/08
Ed - I like your alpha-rating system for one simple reason - NYC is "starred" out - just completely starred out - TONY, NYM, NYT, RG, etc ad naseum. If you had chosen to go with stars I think you would have created dis-interest from the outset.
I also think that the alpha-system allows you to go into more areas of the eating/dining experience...
Food A
Bun C
Burger A
Seating D
Decor B
Service A
etc, etc
Good luck - I look forward to your reviews - have always loved to read your take.
Livetotravel at 11:38AM on 05/07/08
Why not Casual 5-0 stars, and unqualified 5-0 stars?
HeartofGlass at 7:42PM on 05/07/08
I don't have a strong preference for letter grades vs. stars vs. numeric scores... as long as you understand the value of the rating, it's pretty much same-same to me. Being able to use plus or minus does allow for nuance, which is helpful. But the 14 possible ratings aren't much different than Zagat's 30 point scale (which in reality is more like a 14-29 scale).
But I do very strongly agree with the idea of "context" -- i.e., comparing an A+ dive to a B- haute cuisine? Their scores are relative.
Perfect real-life example: Out here in So California, the Zagat for In-N-Out chain is a 24 for food, while Morton's and Ruth's Chris are both rated 25 for food. Clearly, comparing an above-average fine-dining-type steak house to even the country's best drive-thru fast food joint, is like comparing saltines to artisanal bread. (Similarly, in NYC, Gray's Papaya at 20 vs. Nick & Stef's at 22 may be accurate, but in wholly different contexts.)
I've always wanted to see at least two levels of categorization, and ideally three or four... it's alway been my one huge peeve with Zagat, especially when you're in an unfamiliar area, and have no real way of being sure what kind of eatery you're in for.
LoCo at 8:56PM on 05/07/08