Some interesting tea rating logs I was able to uncover...

Jason select said

Update: Yesterday we were able to take a little time and look deeper into the situation. There were a number of factors playing into the recent ratings changes and I’ll explain them below.

1) Suspicious Accounts: There were a number of accounts that we decided to remove or clear the ratings from because we felt there was sufficient evidence of an effort to game the ratings system. I am in the process of notifying those users and communicating what we consider to be appropriate behavior on Steepster.

2) Ratings Bug: We discovered a bug in the ratings system that was counting ratings from tasting notes that had been edited or deleted, thus skewing the ratings calculation. This has been fixed.

3) Automatic Updates: During this process we discovered that our hosting company (the company that manages and keeps our files on their servers) had accidentally deleted the file that started all of our automated processes. The processes include things like updating the Steepster ratings for teas, updating the tasting note counts, and updating new places. Without these automated checks which run every few minutes to make sure the data stays up to date, the data had become old and, even though new tasting notes and ratings could be added, the numbers were not reflecting the new data. We have re-written these processes so things should be updated regularly now and the ratings should be accurate according to the rating system.

4) Full Inbox: Since the above processes weren’t running, a lot of errors were popping up and each time that happens an email is sent to our general email account. But since so many emails were sent it filled up our inbox and we didn’t receive our normal notices telling us that problems were occurring (like GetSatisfaction feedback). We’ve cleared out those emails so we’ll get updates now.

After all this, the changes we’ve made have cleared up a lot of the ratings issues. If you check out the Best Teas page (http://steepster.com/teas) you’ll see that everything is acting as you would expect.

As an added benefit of this work, we also re-indexed the search so new teas should show up in the search results now.

With all that said, I want to mention one thing. We understand that people were concerned about the system being abused. The people here have formed a welcoming, considerate environment for open discussion of all things tea. However, we feel this thread is not inline with the environment here and the activity we encourage. If anyone has concerns about users gaming the system, please email me directly ([email protected]). I will do better to respond to and address issues so no one feels like they need to handle situations on their own. But even if we are not able to address issues in a timely manner, we do not think it is appropriate to call out users in this public fashion. It creates an negative mob mentality and makes people think twice before interacting here. I this makes sense and our reasons are clear. If you have any questions, please feel free to contact me directly.

Update: Yesterday we were able to take a little time and look deeper into the situation. There were a number of factors playing into the recent ratings changes and I'll explain them below.

1) Suspicious Accounts: There were a number of accounts that we decided to remove or clear the ratings from because we felt there was sufficient evidence of an effort to game the ratings system.  I am in the process of notifying those users and communicating what we consider to be appropriate behavior on Steepster.

2) Ratings Bug: We discovered a bug in the ratings system that was counting ratings from tasting notes that had been edited or deleted, thus skewing the ratings calculation.  This has been fixed.

3) Automatic Updates: During this process we discovered that our hosting company (the company that manages and keeps our files on their servers) had accidentally deleted the file that started all of our automated processes.  The processes include things like updating the Steepster ratings for teas, updating the tasting note counts, and updating new places.  Without these automated checks which run every few minutes to make sure the data stays up to date, the data had become old and, even though new tasting notes and ratings could be added, the numbers were not reflecting the new data.  We have re-written these processes so things should be updated regularly now and the ratings should be accurate according to the rating system.

4) Full Inbox: Since the above processes weren't running, a lot of errors were popping up and each time that happens an email is sent to our general email account.  But since so many emails were sent it filled up our inbox and we didn't receive our normal notices telling us that problems were occurring (like GetSatisfaction feedback).  We've cleared out those emails so we'll get updates now.

After all this, the changes we've made have cleared up a lot of the ratings issues.  If you check out the Best Teas page (http://steepster.com/teas) you'll see that everything is acting as you would expect.

As an added benefit of this work, we also re-indexed the search so new teas should show up in the search results now.

With all that said, I want to mention one thing.  We understand that people were concerned about the system being abused.  The people here have formed a welcoming, considerate environment for open discussion of all things tea.  However, we feel this thread is not inline with the environment here and the activity we encourage.  If anyone has concerns about users gaming the system, please email me directly (jason@steepster.com).  I will do better to respond to and address issues so no one feels like they need to handle situations on their own.  But even if we are not able to address issues in a timely manner, we do not think it is appropriate to call out users in this public fashion.  It creates an negative mob mentality and makes people think twice before interacting here.  I this makes sense and our reasons are clear.  If you have any questions, please feel free to contact me directly.

or cancel Delete This Reply

Reply