Fair Tea Ratings

hello fellow steepsters!

I am a very objective and often annoyingly pragmatic person. While sitting and mulling over my tea-log, I realised that many teas on my log were likely unfairly rated, not on the quality of the tea, but my personal preference.

It is with this revelation in mind that I became determined to create a fair objective rubric for which to actively rate my teas, allowing my to rant freely, yet not lead people astray as far as the quality of teas. 2 hours later, I realize that this very unnecessarily detailed rubric has gotten massively long as I have detailed every nuance from A+ to F-. I later realized that for the most part each score is really just a matter of how much the tea deviates from an A+ or a supertea. So I think I will make some sort of point system based off of that…

With all of this in mind, I would like to show you all my example of an A+ tea and would like your input on whether this is a fair objective rubric for a high quality tea.

Thanks for your time everyone!
<3=

I strive to follow the proceeding guidelines and rubric:

1. I shall not provide a grade based upon an improperly brewed tea. This includes: experimental blending, scalded leaves, oversteeping/understeeping and insufficient amounts of leaves provided.

2. All rated teas must initially be steeped in accordance to the guidelines provided by the tea company. Addendum/reevaluation may occur if properly documented.

3. I shall try my very best to leave personal preference out of the scope of grading. However, I shall endeavor to be very clear of my opinion in my tealog of the teas that I rate.

Grading Rubric

A+(97-100): This tea
- has a clear explanation of ingredients, allergens and preparation instructions. (5 points?)
- Requires little to no adjustment to preparation (sans accounting for regional differences in altitude, airpressure, etc…) for a full flavored steeping (10 points)
- Clearly and concisely conveys the flavors of the artist’s intent with no supplemental explanation if the flavors are within the taster’s experience (20 points)
- Requires menial ingredient adjustment (i.e.: milk, sugar, stevia, etc.) to achieve the flavors intended by the artist. (15 points)
- has a very high quality tea base that compliments the blend (note: does NOT require whole leaves)(10 points)
- Re-steeps at least two to three times strongly if able. (if bagged, re-steeps twice well) (10 points)
- Stores very well (No flavor diminishment when stored in a dark, cool and airtight space for short periods of time). If bagged, is packaged promoting optimal infusion, ease of use and freshness. (10 points)
- Is completely consistent in flavor and quality from batch to batch. (15 points)
- Is very well priced in relation to the quality of ingredients and quantity provided. (5 points)

11 Replies

I wanted to toss out feedback, but it’s all like 5am and I can’t sleep, so I hope I’m not sounding weird.

Resteeping – Some teas, like a good pu erh, resteeping is huge part of the experience, other teas aren’t. So I think putting weight on resteeps biases the rating system for certain teas.

Wondering how to rate stores well, as optimally, to judge this you’d need to test the tea when first purchased, then again months later to achieve this scoring. However, some blended teas (ie, with fruit) don’t hold flavor as long due to the nature of the product and shouldn’t be rated poorly, thus those points bias towards unflavored teas. Matcha is another sketchy point too as their shelf life can be different.

consistency is huge, imo – but to be optimal, hard to rate. I mean, I’ve gotten bags of tea where I’d get very little of X item and thats obviously not good. And there is huge human / environmental errors as parts of teas floating to the top. And of course, some teas change taste with age, ya?

In the end, I think the magic of tea is how there are variations, cup to cup.
Or how times theres that one night someone oversteeps and come across something magical (cough, Gui Fei oolong, cough)

Kittenna said

Mmmmm, Gui Fei. I can’t believe I was steeping it in boiling water the last few times. Such an idiot! I couldn’t for the life of me figure out why it tasted so wrong!

Login or sign up to post a message.

WtFGoD said

would really need to improve the sites rating system to correct this(doubtful), or for everyone to adopt a universal system of rating(doubtful). but cant hurt for us to personally try to improve our own rating systems.

i think that to an extent rating should be based on personal preferance, ie. if 80/100 people really are fond of a flavor/tea, it should be rated higher than a tea where only 20/100 people are really fond of the flavor/tea, even if both are the exact same quality baring the flavor profile.

i’d imagine some of the things you list are good for rating but others are too subjective, like “stores well” or “requires little adjustment” as the first can be very strongly effected by environment/storage procedure and the second really doenst have a ton to do with the tea and can also be effected by different factors, for example most companies use universal recommendations for black/green/puerh not accounting for differences(not the teas fault) also things like water could effect this.

i personally try not to rate teas im too biased against, i’d imagine the rating system for steepster would be strongly improved if you gave a very easily seen opt-out box (this is not my cup of tea) just to encourage people to not vote if theyre gonna vote a tea 0/100 for personal preferance. also you could scale the teas ratings better across individuals if you gave more clear indicators of which ratings correspond to which feelings about the tea(like the smiley faces but more explicit) to avoid people who rate teas either 0 or 100 with nothing in between(extreme example)

Login or sign up to post a message.

JC said

Although I agree that sometimes it can be confusing to use the same system when we all seem to give it different meaning, I still prefer to keep it as it is. Why? Well, that’s what the Notes writing part is for. To explain and expand on the reason for your rating.

It is impossible to be impartial with rating because you are judging based on your experience alone. Even if you learn from others, it is still you who chooses the words and final rating.

I do feel like there are some teas that would ‘deserve’ higher rating but the truth is that I usually explain that on my notes. “I really don’t like this tea, but I understand that it is what is supposed to be even if I don’t like it”.

The question would be, is the rating system for your ‘taste’ or for ‘quality’?. I feel ‘quality’ would be unfair anyway because it depends on preference anyway. Aaaaaand that was me being pointless.

I agree with this. My only experience with lapsang souchong was it gave me an asthma attack. I gave it a low rating – I couldn’t possibly rate it highly when it made me ill, and I could barely imagine what a positive experience with that tea would be. I explained my medical issues in my tasting note. Now, some might say that’s unfair, or that I shouldn’t have rated it. No way! I say:

1) All our ratings are subjective to some degree
2) When scouting the reviews of a tea I want to buy, I always scout the lowest-rated reviews to see WHY that person rated it so low – medical issues? Personal hatred of a particular taste in the tea? Something I should be warned about, something that might apply to me? Oh, this person knocked it down because it’s “too sweet,” I love sweet, maybe I should check it out.
3) Some people having a bad experience with the tea and rating it as such WILL result, with enough ratings, in an overall-pretty-fair rating of the tea. Lapsang souchong is strongly flavoured, and not for everyone. I think the ratings should reflect that, while LS lovers will know to scout other LS lovers’ notes for recs. We have people of all tastes and backgrounds on the site, after all. If this were a site for LS lovers, it would make no sense to go there and rate one badly because they make me ill, but on a site where people have widely varying experience with tea, it makes sense.

JC said

You make a good point. The way the ratings here work make it hard for only a few ‘unfair’ ratings affect the overall rating that much. And besides, I feel like people should avoid trying to use steepster as a catalog of tea. Use it as a catalog of possibilities. The difference? You learn from one, the other is just for shopping purposes.

I love reading the notes. Not because I think everyone is a 100% accurate description, but rather that I learn about perception of taste and scent as well as how other people enjoy their tea. Also, I see what people with similar taste are trying and ‘liking’ so I can give it a try. Also, people who hate things I love I try those as well. lol

Angrboda said

Daniel Scott, your third point. I AGREE SO MUCH! I wish more people would understand this. Only rating the stuff that gets a high rating is just as unfair as giving stuff a low rating.

Login or sign up to post a message.

I have stopped giving teas a numerical rating that I don’t like because of person preferences.

Fair enough! I’ve noticed a lot of people tend to do that. :)

Login or sign up to post a message.

These are all very interesting points!

@awkward soul: Point! Like rooibos, for example! A resteep probably shouldn’t hold so much weight, but it is definitely a feature that always scores brownie points with me. :)

@WTFGod: I’m not looking to change the way that others rate tea, I think everyone should be able to rate their teas however they want! But I feel like I give a sufficient amount of opnion in the actual log and should reserve some part of the experience for the quality of the tea. Even if it were a foot note, rather than my final rating, if I am hyping a low quality tea of which I love the flavor or bashing a high quality tea because I hate it, I think that I should make sure that potential flavor lovers can distinguish between the two, which is why I thought to use the sliding scale provided.

@JC: I somewhat agree with you; I do try to utilize my tea log to the fullest as far as my personal experiences with scent and flavor, but I think that quality for the most part can be evaluated. I just think a lot of people don’t take the time to question: Is this a good tea or do I just like the taste? This is why I am searching for the qualities that make a high quality tea, and I’m sure that the biggest ones involve the teaartist’s intent and the quality of the base and ingredients, but I know that there is more. I do the same with music, I don’t recommend music that I like, I recommend music where I know the musicians are talented and within the realm of genres/styles the person I am talking to is fond of.

@Daniel scott: I think we all generally have similar gradings, buts its more of a e part scale,there’s 1. Omg buy this, 2. Eh. It’s okay, and 3. OMG don’t buy this. For the most part I only look at ratings abover 89 and below 50 because everything in between gets very very subjective. My biggest concern is people that actively read my t-log, I don’t want someone to to make a decision based off of my log without being fully educated about the tea, and I feel kind of snooty commenting of the quality of the leave and such more than “omg I steeped this 20 times and it still tastes great!” I think when I started steepster, I was grading teas quite unfairly considering the experience that I have now. I think trying to maintain a subjective and objective outlet will help me keep my logs (and my personal journal I have at home) atlot better.

,y thought was not to replace detailed reviews, I love a good detailed reveiw, I just felt guilty for example, giving David Tea’s red velvet cake a fairly low rating when I knew it tasted exactly like it is supposed to, but I don’t like it. I will and encourage detailed logs.

Login or sign up to post a message.

TeaLady441 said

I had a hard time with the rating system when I first started here, but then I realised that the rating system is pretty forgiving (one low rating or high rating won’t skew it), and the tasting notes are what a lot of people look at for more info anyway.

With that in mind, I refined the rating scale mostly to be used as reminder of what I want to re-purchase, and how much (with the smiley faces as good reference points). If someone sees a tea that I rated poorly (because of hibiscus etc), they can just look through the notes really quickly to see why.

If I drink a tea that everyone else loves, but it’s not for me, I still rate it, although I rarely put anything under 50 even if I don’t care for it.

These are my guidelines:
1-50: Bleh. It’s either undrinkable, or not worth drinking.
51-64: A decent blend. There are some things I like about it, but it’s not a favourite. Unlikely that I’d re-stock.
65-75: I enjoy this cup, but it doesn’t blow me away. May or may not replace as there are teas I enjoy even more out there.
76-85: Oooh. Very good. Unless it’s super-expensive, I’d keep this around. (Get 20-30g+)
86-100: Amazing. This cup can never go wrong. (Get 100g+)

If Steepster allowed us to rate how much to buy (lots more, a little more, never again) in the cupboard, I might change the way I rate things, but for now it’s essential for me to use the rating system this way so I remember how much I like everything, and so that I’m consistent too!

I hope that this doesn’t retract from how everyone else uses Steepster, but as I said above, the system seems pretty forgiving so I doubt I’d be throwing the numbers off a lot or misleading people.

Login or sign up to post a message.

Login or sign up to leave a comment.