Uh, how was the "#1 rated pu'er in the world of over 54,000 teas" determined?

This annoys me.


61 Replies

Oh I see:

“In 2014, the online tea community on the world’s largest tea review website, Steepster, voted us the #1 Pu’er Tea in the world out of over 5,000 different Pu’er teas with over 10,000 voting people! We were also voted #1 Pu’er tea out of over 54,400 different teas. A tremendous milestone for us and for the tea community”

So here’s where the unscientific way in which teas are rated here starts to be problematic. I don’t think it’s appropriate for a site that sent us all free samples — meaning we’ve all tried that one particular pu’er — to suggest that that vote represents any kind of systematic, official rating system. Or that those of us who did reviews (I don’t think I have yet) are pu’er experts or official reviewers or anything but a bunch of random self-selected tea drinkers who differ quite a bit in terms of experience, expertise and personal taste.

Login or sign up to post a message.

There is also this mess of ratings, that is the 2013, 2014 and 2015 teas rating into one review. When I went to review the 2015 and found this my head exploded. All three of those years and seasons are very different teas.


Also, 10,000 people at Steepster voted on this one pu’er? Is this even possible? I’ve never seen more than 30 reviews of a single tea here, including those sent as free samples.

And also,among the more experienced pu’er drinkers… How many could say they’ve tried more than 500 of them? More than 1000? Anywhere close to the supposed 54,000 varieties out there?

Could the ownership of this site say something to Misty Peaks? This is just as big an issue of transparency as we saw with Verdant, IMO. And it really makes me mad. If I’d looked at this site 2 years ago, I would have believed this was some sort of official ranking and been conned.

Actually, I think this might be worse than Verdant. This is the whole sales strategy, repeated over and over across multiple parts of the Misty Peaks website.

Dexter said

From what I can see – if you sort all the teas by highest rated then choose just pu’erh – the top rated Misty Peaks Pu’erh is 18th with 42 ratings – that’s not quite what their web site says…..

I think at one point it was #1. Mandala’s Wild Monk gets the top spot on and off. But I think the weird consolidation of ratings effected it.

mrmopar said

Maybe a couple of us pu heads have had 500 or so but 54,000 is a BIG number to try.

@mrmopar Yep, an impossible number to try.

Dr Jim said

The difficult we do right away. The impossible takes a little longer. Set your goals for 2016 today!

Login or sign up to post a message.

Okay, but seriously… Is no one else put off by the…generous way in which Misty Peaks interpreted this?

Brian said

i dont dig it/them. no thanks. slim shady.

Login or sign up to post a message.

And where did the 10,000 number come from? Is that just a tally of all the pu’er reviews ever posted on steepster? Because those are hardly a referendum on Misty Peaks Tea.

Login or sign up to post a message.

AllanK said

Actually, while I don’t know about 2014. Misty Peaks today is not the number one rated puer on Steepster. I think the 2012 Wild Monk is actually rated number one among puers, but not among teas, that would be Golden Orchid by Whispering Pines. To put it bluntly what is said by them is not true for today. I didn’t look at the listing in 2014 to know what the number one rated tea was.

Exactly. It’s out of date, even if it was maybe true for a minute some time in 2014.

Login or sign up to post a message.

cookies said

Misty Peaks has had some controversy in the past. A shame, because I enjoyed the tea I received from them, but they don’t seem to be terribly upstanding.

I’ve sort of decided… Well, at least until I know more about pu’erh… Never to buy it from anyone but YS or White2tea.

Ive got a bunch of samples from other places to get through, but after that… I just don’t trust anyone.

jschergen said

Haha. Well those are two good ones. There’s more than that of course, but if you just ordered from those two you’d do pretty well.

I just posted a response to Oolong Owl’s review of their tea. Didn’t see this thread til just now.

@jschergen: I saw that and replied there, but maybe you want to move it over here? I sort of have the same concerns you mentioned.

cookies said

There really are a quite a few reliable places. I think they just don’t get as much press as the questionable ones. Teaurchin, pu-erh.sk, essence of tea… I’m sure pu-heads will have more suggestions. People here are very friendly and will be more than happy to help if you have questions about a certain vendor.

@cookies: Oh, I know. I’m more adventurous with other types of tea but am at a place with pu’erh in which it makes the most sense to keep it simple. Also, I’m limiting myself to 4-5 tea purchases this year, so that’s part of it.

@kristinalee: I would add Mandala Tea, Bana Tea Company, and Crimson Lotus Tea to that list. I’ve been very happy with their puer and they seem very reputable.

Login or sign up to post a message.

jschergen said

Originally posted on Oolong Owl’s reviews of their autumn sheng.

I find it interesting that Verdant got hammered so hard much moreso than Misty Peaks.

By now, it’s clear from the multitude of reviews that Misty Peaks has tea that’s plenty good enough for most people. I’ve had a couple of their teas and found them alright at best.. Like Oolong Owl, I think there’s better options in the market but it’s not as if the tea is obviously crap.. That being said, I cannot bring myself to recommend or support Misty Peaks. I find their marketing (&marketing copy) to be highly troubling and their claims about tea to be just as bad as Verdant. They’ve never even responded to the criticisms laid out, which more or less puts them in the same boat as Verdant as far as I’m concerned.

One addendum..

At the risk of calling out myself and others.. I’ll also note that Misty Peaks is one of the most active vendors at sending out free tea to people, especially those with a platform to talk about tea. I think this might subtly play a little into their more favorable treatment.

Agree 100%. And I couldn’t help but notice that some of Verdant’s most strident critics have recently given some of Misty Peaks’ teas overwhelmingly glowing reviews.

I am not defending Verdant. But I certainly find what Misty Peaks does AT LEAST as troubling as Verdant.

Oh they’ve been hammered on reddit, there’s plenty of snark in the original reddit verdant topic about MP too.

Steepster chat tone is much more friendly. People certainly wring their hands here if something is off and give benefit of the doubt as seen in the Verdant topic before Scott did the finishing blow.

Marketing wise, it’s likely to attract a certain crowd. For example, I’ve had some great matcha but the bloody matcha was marketed like some stupid miracle cure. There’s that point, (like i said in my comments thread thing) you make that “well if it tastes good that’s all that matters” and suspend disbelief/ethics or make that call that the ethics mean more.

MP sent a marketing email last month about giving away a freebie with purchase to give to your friends who “aren’t a size 0” as pu’er is some weight loss thing. I wasn’t happy about that. The pu’er is still decent nor would it ever make me a size 0.

In the end, Rain me with free tea everyone, I’m not a size 0!

Oolong owl, WHAT… Oh, that story makes me even more disgusted with them than I already was. Because apparently “healthy” is being a size 0! And women who aren’t that should be shamed into purchasing ridiculously expensive tea to “lose weight.”

When tea is marketed as a weight loss solution, I think that’s not only irresponsible, but potentially dangerous, especially in light of the tea cleanse fad. It’s advocating laxative abuse and disordered eating.

Edit: If I were a size 0, I would be extremely unhealthy. I look unwell if I go under a size 8, so uh… No. Human bodies are not meant to be all the same size.

A free gift for all your friends that are not a size 0

It’s that time of year ago: one more round of desserts and a long nap on the couch instead of that extra walk around the block.

Pu’er tea has been consumed for centuries as its benefits are many, most notably the benefits that counteract the negative impact of the foods we consume. Excess fat, high blood pressure, high cholesterol, and a whole host of ailments that are linked to dietary habits are literally curable with Pu’er tea.

“Let food by thy medicine” was not merely a cute thing to say by Hippocrates, it is a lesson for us to be conscious of what we put into our bodies, especially during this time of year where we are likely to pack on the pounds and stay indoors because of guests or poor weather.

There is no need to put your body through this. Pu’er is a tool, let’s use it!

As an extra special thank you, ALL orders, no matter how huge or semi-huge, receive a free box of our loose leaf Pu’er tea. You can keep it, but it is the season of giving so it is better to give it to a friend, maybe even a friend who wouldn’t mind canceling out some of that last slice of apple pie.

Free box of our Loose Leaf Green Pu’er Tea with all orders from now until this Sunday, December 13th. Just order, and we will include the box. No code needed :)

Seasons greetings and thank you all again for a fantastic year together.

Your friends at Misty Peak

I gagged.

btw, there is no such thing as a size 0 in men’s clothing, so it’s clearly directed at women.

Sil select said

wow. size zero, eh? not cool.

and the perfect time to link Cwyn’s post about this email http://deathbytea.blogspot.com/2015/12/waiting-for-rapture.html

Good post, thanks for linking it.

Dexter said

That email is not just questionable marketing or stretching truths – that is offensive I would never buy anything from any company that suggested someone who isn’t a size 0 needs to loose weight. That is just so wrong on so many levels….

I have been increasingly frustrated with some of MP tactics and such, but honestly that email was the nail in the coffin for me. I found it immensely offensive and won’t be shopping from them in the future. Luckily for me their 2015 autumn harvest did not impress me as much as the 2014, so I am not losing out on much.

I think I’m on their mailing list, but I rarely check anything that goes to the “promotions” folder in gmail, so I would never have read it. I find it offensive too. It would be enough reason for me to put them on my personal shit list even if I weren’t already annoyed by their marketing in general.

I’ve never had any MP tea but after seeing that email I will make sure to continue not having any MP tea.

AJ said

That made me choke on my breakfast. Especially as I’m severely underweight (technically a size zero), with a host of health-problems, and in a constant struggle to get my weight UP.

Brian said

but hey guys. they form their tea into mountain peaks and put it in a over the top presentation box. yippee

Anlina said

Yup, incredibly gross. I’m icked out in general by marketing tea as a weight loss product, but that bit is particularly fat- and body-shaming.

Login or sign up to post a message.

Just tried to change this to “a transparency and Misty Peaks thread” in the topic so people would know what it was, but it didn’t seem to work.

It tells me something about the way the site is coded didn’t allow for it.

Login or sign up to post a message.

AJ said

This actually ties into Bumble’s thread about what made Whispering Pines “top” Steepster’s tea list. Unfortunately, when I went back to look for it, I found it was apparently deleted for some reason… cough

I did find a cached version on google though, if one wants it.

At any rate, I feel like the Tea landing-page’s title should be changed to “Trending Teas”, not “Best Teas”. It probably won’t do much to dissuade tea companies from basing marketing claims, however.

Hey, that’s a much better way to put it! You have my vote. =)

Rasseru said

The rating system is pretty subjective at best though, right. I dont ever go by score, but I do like reviews and talking about a tea

@Rasseru Yeah, I agree, I don’t look at the ratings either, I just don’t like the idea of teas being marked as ‘best’ or ranked as such, as it’s such a very subjective thing. There’s no best tea!

I think AJ’s suggestion of changing it to ‘trending’ or something along those lines would be more accurate.
In the end, it’s not such a big deal, of course. =P

Rasseru said

yeah of course, its misleading in its wording, I agree.

The numbers help me personally track which teas I like better than others, but I agree — “Trending teas” would be much more accurate. I’d absolutely support this kind of change. I’d be interested in seeing the cached version of the thread you’re talking about if you know where it is.

I really like everything I’ve tried from Whispering Pines (except for this one shu that I wasn’t very fond of), but I just can’t justify the prices for any large buys.

Rasseru said

Yeah same, I just use my numbers as a way of remembering which I liked the best

cookies said

+1000 Trending teas makes far more sense!

I would be very interested to know why that WP thread was deleted.

cookies said

@kristinalee which WP teas did you enjoy? I might have some suggestions…

@cookies, I liked all the Chinese black teas, especially the ones with gold tips (Golden Snail).

cookies said

Okay, sending you a PM.

I, for one, wish that the top teas were actually trending teas rather than the ones that have amassed the most consistently high reviews. Unfortunately that is not the case. Golden Orchid has sat at #1 or #2 for two years straight now, and hasn’t even been sold for the last 8ish months. For my personal reference, I’d be pretty stoked to be able to see what teas are currently popular, as there definitely seems to be trends and I’d love to be able to formulate my offerings around what people are currently loving. I’d also be into it just because it’s cool to see how the trends change over time :-)

The rating system here is definitely strange and I have seen a good amount of retailers use the numbers here on their sites as a sales pitch, which is fine in my opinion as long as they remove that pitch when those numbers change, which happens pretty regularly.

Totally loving the idea of the change to “Trending Teas”.

What would it take to make this happen? Is anyone with the power to change things on the site reading this?

Probably not until tomorrow morning at least.

And you could post it here where it might help catch Jason’s attention: http://steepster.com/discuss/3-which-new-features-do-you-want-the-most

Yes, Trending Teas would be such a more useful title me thinks.

Also, I am pretty sure that the people with the power to change this are taking a nice extended holiday, at least the current trolling shenanigans have me thinking that. Steepster admins have always been a bit slow on the draw when compared to other forums, but this is slower than usual for a response.

AJ said

I didn’t think my suggestion would garner so much attention. ‘Trending’ does seem to better reflect the subjective nature of tea reviews and Steepster.

As for the current highs—Steepster’s system tends to take into account both the high rating, and the number of reviews (hence the drop off for the ratings, despite there probably being MANY more 91 and 90-rated teas, with barely half the number of reviews…). Granted, I’m just guessing from watching the boards so often, but I don’t think it really takes into account most RECENT reviews. WPs is known for free samples, which boosts the number of reviews significantly. Alternatively, Butiki was going out of business, and their last sale would have done the same.

But on that note, I think the Teas page has gotten stuck before in the past. Used to happen to Explore a lot too.

@kristinalee here:
Page 1: http://webcache.googleusercontent.com/search?q=cache:uXZ-ueD5394J:steepster.com/discuss/12143-whispering-pines-tea-question-why-does-this-one-company-have-so-many-highly-rated-teas%3Fpost_id%3D186436+&cd=1&hl=en&ct=clnk&gl=ca
Page 2: http://webcache.googleusercontent.com/search?q=cache:YPPd18wjuucJ:steepster.com/discuss/12143-whispering-pines-tea-question-why-does-this-one-company-have-so-many-highly-rated-teas%3Fpost_id%3D186571+&cd=2&hl=en&ct=clnk&gl=ca

I don’t know why it was deleted. It got a bit tense, but never uncivil. But Steepster does sometimes just eat threads.

Yeah, Jason and related co haven’t posted in a while. But sometimes it takes a while to shake off the holiday stupor.

Yes, ratings are subjective, and there are teas and tea companies that seem to be getting extraordinary amounts of love from time to time. When I first joined it was Samovar, and I probably wouldn’t have tried their teas if there hadn’t been such a love fest going on. But I’m glad I did, and I am still a huge Samovar fan.

With regard to ratings, I’m not sure I’m even consistent within my own ratings system. It’s often the case for me, though, that if I am looking to try a certain type of tea and want to get one I can feel comfortable is good (even if it ultimately doesn’t turn out to be one I like), looking at teas with many ratings that average high is often a good bet. It’s like looking at ratings on Amazon or really anywhere else. If it’s just a couple the data don’t mean a lot because one person could be a high grader and the other a low grader and cancel each other out, but large numbers of ratings that tend high also tend to cancel out some of the noise. I also pay particular attention to the ratings of people whose tastes I share when making a buying decision. There are a lot of folks here whose tea notes I love reading, but who I can be pretty sure that if they like something I won’t. ;-) There used to be a film critic, I’ve now forgotten who it was, whose reviews I always read because if he hated something I knew I’d like it and vice versa. So there’s a bit of that as well.

Login or sign up to post a message.

Okay, just added “trending teas” to the “Steepster changes” thread, so support up there would probably be helpful in terms of making that happen.

Login or sign up to post a message.

Login or sign up to leave a comment.