tag:blogger.com,1999:blog-3993498847203183398.post3176104865474391511..comments2019-11-20T13:53:33.840+00:00Comments on RevK<sup>®</sup>'s rants: 5 + 5 = 7.7RevKhttp://www.blogger.com/profile/12369263214193333422noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-3993498847203183398.post-52454139367467981452013-01-30T13:34:53.196+00:002013-01-30T13:34:53.196+00:00"...the wisdom of the crowd..."
Now the..."...the wisdom of the crowd..."<br /><br />Now there's an oxymoron if ever I've heard one!Nicholashttps://www.blogger.com/profile/12140259189768633804noreply@blogger.comtag:blogger.com,1999:blog-3993498847203183398.post-43642606166351301172013-01-30T10:40:38.812+00:002013-01-30T10:40:38.812+00:00Thanks for raising this question - this is Joakim ...Thanks for raising this question - this is Joakim from Trustpilot. <br /><br />42jon is right. The scoring algorithm (also known as the TrustScore) is based on Bayesian statistics. But if a company has no reviews the score is 7.0 - not 5.0 as you may assume. <br /><br />The TrustScore is the wisdom of the crowd concentrated into a single number. We need enough data to soundly make bold claims about the customer satisfaction. A single angry customer should be able to make his/her voice heard on Trustpilot but not rip an entire business apart. <br /><br />If you are interested, there's more info on our blog: <br /><a href="http://blog.trustpilot.com/review-ranking-system/" rel="nofollow">Review ranking</a><br /><br />Hopes this solves the riddle. Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3993498847203183398.post-47909658518566185812013-01-29T20:56:29.615+00:002013-01-29T20:56:29.615+00:00If they know what they're doing, they're p...If they know what they're doing, they're probably using something called Bayesian statistics. This is a method where prior information (or a prior assumption) gets updated as more data gets added. With no reviews at all, the prior assumption might be that AAISP would have an average score of 5 out of 10. However, after the first person provides a 5 star review, you can't say the average would be five stars, but now based on this one data point, you can infer that the odds are skewed slightly in favour of getting higher scores. You can then update the prior assumption with real information, and your best guess at the mean score creeps up (you can't say it's 10/10, as this might be a one off). As more 5-star reviews come in, your confidence rises, and your best guess at the mean score goes up, but you wouldn't be fully confident of 10/10 being the mean score unless you'd had a huge number of reviews.<br /><br />I suspect the site isn't showing the bottom end of a possible range, it's showing the current best estimate with limited information. This is based on the starting assumption of a mediocre mean, but gradually being updated as real data comes in. Of course, with our knowledge of AAISP, our prior assumption might be a score of 9.9 out of ten, and not five out of ten, so we'd much more quickly hit a realistic rating.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-3993498847203183398.post-5583429660052265542013-01-29T16:31:47.818+00:002013-01-29T16:31:47.818+00:008.1 now.
If it's going up .2 per review we ne...8.1 now.<br /><br />If it's going up .2 per review we need another 9 people :p<br />Tony Hoylehttps://www.blogger.com/profile/06485210895681350152noreply@blogger.comtag:blogger.com,1999:blog-3993498847203183398.post-17294244142256204892013-01-29T12:13:40.759+00:002013-01-29T12:13:40.759+00:00Interesting.
I added another 5 star review, it...Interesting.<br /><br />I added another 5 star review, it's now showing 7.9 out of 10.<br />Josephhttps://www.blogger.com/profile/09814744420039144545noreply@blogger.comtag:blogger.com,1999:blog-3993498847203183398.post-66071536455386950022013-01-29T10:19:07.488+00:002013-01-29T10:19:07.488+00:00I wonder if they're displaying the lower bound...I wonder if they're displaying the lower bound of a confidence interval?<br /><br />http://www.evanmiller.org/how-not-to-sort-by-average-rating.html discusses why you'd want to do that, instead of some straight averaging. Basically, though, the point is to avoid the paradox where two 5 star ratings is considered "better" than ten thousand 5 star ratings, and one 4 star rating; you do this by using statistics to answer the question "if I treat the ratings I have as input to a statistical model, what is the lower bound on the 'real' rating they'd have if everyone rated them, given a 95% confidence interval?"<br /><br />If they are doing that, what they're really showing you is that their model suggests that AAISP's real rating is somewhere between 7.7 and 10, given two ratings so far, both of which are tens. As the number of ratings increases, the range will decrease - if you had a thousand 5-star ratings, the range might be 9.99 to 10, and displayed as 10/10.Simon Farnsworthhttps://www.blogger.com/profile/15190608047563530091noreply@blogger.comtag:blogger.com,1999:blog-3993498847203183398.post-92110643878271407332013-01-29T09:54:28.082+00:002013-01-29T09:54:28.082+00:00What I've seen with some sites is that you can...What I've seen with some sites is that you can give a star rating, but not write a review. Maybe it's counting these unseen ratings.Ianhttps://www.blogger.com/profile/14249692867869614591noreply@blogger.comtag:blogger.com,1999:blog-3993498847203183398.post-27634507945123685402013-01-29T09:50:42.856+00:002013-01-29T09:50:42.856+00:005+5 is now "up to" 10 :)5+5 is now "up to" 10 :)The Paper Boyhttps://www.blogger.com/profile/04561063207545406257noreply@blogger.comtag:blogger.com,1999:blog-3993498847203183398.post-9203680983214563162013-01-29T09:28:38.745+00:002013-01-29T09:28:38.745+00:00Maybe they are using one of BT's algorithmsMaybe they are using one of BT's algorithmspebshamhttps://www.blogger.com/profile/02373814486358099104noreply@blogger.com