Something's wrong with the rating system
For some reason, steepster’s rating system is messed up. Check out http://steepster.com/teas/upton-tea-imports/17270-zg41-china-green-sencha I’m the only one who put in a rating (25), but the steepster score is 64. Same thing with http://steepster.com/teas/upton-tea-imports/23588-tc47-oliphant-estate-opa-green I rated it 60, but the steepster score is 72.
Steepster uses a weighted rating algorithm. It’s supposed to be that way. And there’s pretty good reasoning behind it. You can read about it here:
http://blog.steepster.com/post/247562587/brewing-a-better-rating-system-the-2nd-steep
Another simplified discussion of Bayesian averages:
http://newbreedofgeek.blogspot.com/2011/02/using-bayesian-average-to-rank-content.html
I was playing with the math the other day, and it looks like the formula currently assumes an average rating of around 75, and an average number of ratings/votes per tea to be around 3. This of course gets updated and modified slightly with every rating that gets logged, but it’s fun to know if you ever wanted to check the math of it all.
Tea Rating =
(Average Number of Votes across all teas * Average Rating of all teas) + (Total People Who Rated TEA-X * TEA-X’s Total Rating) / (Average Number of Votes across all teas + Total People Who Rated TEA-X)
Slack, I completely understand your confusion. Bayesian algorithms aside, I still think it’s kind of a silly rating system. It’s like grading on a curve so everyone gets a C. What’s the point?
It’s more so that the # of ratings a tea has and the company factor into the actual score. That prevents a tea that has a single rating of 100 to be higher than a tea that has dozens of 99 ratings. It keeps the outliers from skewing the scores as much.
I don’t understand bell-curved grades, either, and may refer my university to this discussion.
@Jason, I don’t really see a problem with a tea that has a single rating of 100 being higher than a tea with dozens of 99 ratings. It clearly says how many ratings there are under the average, so you can deduce for yourself how accurate the rating is.
In my opinion a straight average rating system would only be accurate if everybody scored their teas in EXACTLY the same way. Not everybody scores across the entire scale, some people are harsher and more demanding in their scoring than others and some people prefer to abstain from scoring teas they didn’t like. I think the weighted system compensates for this.
I think I understand the system, but i can’t help but feel that this penalizes the smaller shops.
If a particular place only has a few customers and great tea, the ratings won’t be any different than a mega tea store with tons of reviewing customers and average tea.
I guess it can work the other way too; one bad review won’t kill a particular tea’s reputation.
I agree about penalizing the smaller shops. That’s a big reason why it doesn’t sit right with me.
Actually, from what I’ve seen, it doesn’t penalize the smaller shops in this way. Look at the top rated teas right now, and you’ll see a lot of Verdant Tea. I live in Minneapolis and have seen David’s operation first hand, it is still quite small. Verdant has only been in business for less than a year. So many of its teas have risen to stand among the highest rated precisely because they are teas of very great quality. Being around at the beginning, I watched David start with zero exposure. And even now, when his business and teas have generated a lot of buzz here, he still proceeds with a marketing budget of zero. If the Steepster rating system penalizes anything, I’d say it penalizes uninteresting and low-quality tea. In my honest opinion, this is as it should be.
Add a Reply
Please keep things friendly and report anything inappropriate.