Average Hub Rating in dashboard inaccurate?

wirlybird · 2017-03-22T13:25:25.000Z

So why then if my rating is 5.00 average rating in my hub that this shows me at 4.5? Doesn’t add up.

3DPrintandStuff · 2017-03-22T13:37:18.000Z

could be due to the amount the data is based upon

I have a bad metric not yet displayed upon the dashboard, which I have completed 33 orders, but number shown in the hubs listing in brackets only shows 20 (based upon 20 votes). Some customers haven’t left a review or feed back upon my prints even though the experience and service they received was good.
Showing stats should always taken with a pinch of salt and I’m sure it’s work in progress so as time goes by the shown figures will make more sense

PepCo_Parker · 2017-03-22T13:41:48.000Z

I’m not sure I understand your point on this one either. Are you suggesting that having less reviews by default lowers your average review score?

3DPrintandStuff · 2017-03-22T14:00:44.000Z

sort of depends what 3D Hubs means by votes. I don’t work for 3D Hubs so not able to say what these figures mean or how they are worked out. but in my areas listing

https://www.3dhubs.com/3dprint/Brighton-and-Hove--United-Kingdom#?place=Brighton%20and%20Hove&latitude=50.833330000000&longitude=-0.133330000000&shipsToCountry=GB&shipsToState=England

I have 20 in brackets (based upon 20 votes) but also a London based hub which has 434 (based upon 434 votes) but both hubs have the same rating 4.9 stars The only difference is the distance. There is a closer 5 star hub with the same number (20) in brackets and they have able to produce more consistent print and service as shown in their reviews.

Update

There is also a hub with 58 in brackets that has a 4.9 star rating. Which make me think that votes are orders. As I spoken to the hub owner and he has had many orders

adamhayek · 2017-03-22T16:10:45.000Z

Hi Parker,

You are right, there is an error on our side. We are in the middle of switching to a new service and there is some data discrepancy. We will fix it as soon as possible.

Don’t worry your ranking is not affected by this.

Adam, 3D Hubs

PepCo_Parker · 2017-03-22T16:17:41.000Z

Thanks for confirming this Adam! I think we were all just worried there was something else going on. Knowing it is just an error I’m sure will bring relief to many! Thanks for letting us know.

PepCo_Parker · 2017-03-22T16:20:00.000Z

The brackets are the number of actual reviews that hub has received. It’s not necessarily the number of orders, in most cases I find no matter how great a service is some customers just don’t take the time to write a review. That being said my hat goes off to anyone who has had a review for every customer they have had! I want to know what they’re doing that I’m not to get all those reviews Hope this clears things up for you

wirlybird · 2017-03-22T17:01:01.000Z

Appreciate the update.

HoosierDude · 2017-03-23T14:41:54.000Z

Hi all,
Just got my Marvin test print reviewed. Got 5 stars. But, the graph showing my average hub rating versus everyone else’s average rating shows up as 4.39%. So, yeah, how do you get a 5 Star rating, but an average of 4.39% from 1 review? Math was not my strongest subject, but that doesn’t seem to add up.

BoozeKashi · 2017-03-23T14:46:59.000Z

LOL, not sure about your Math skills, but Reading comprehension… well, um…

screenshot_2017-03-23_21.45.15.png2502×636 41.4 KB

ProtoArt · 2017-03-27T14:54:57.000Z

I wonder if these numbers work through up to hub recommendation on the order page?

When I make a dummy order, I see a list of hubs, where hubs who are closer to my place, have a higher rating, lower lead time and within the country, are lower in the recommended list than other hubs?!? Why?

adamhayek · 2017-03-27T15:46:00.000Z

Because we look at number of other metrics as well. For example accept rate or retention.

Adam, 3D Hubs

ProtoArt · 2017-03-27T17:00:41.000Z

Well, then this is an insight that is completely missing.

So how does it actually work?

Retention: customers returning to your hub
What if I have customers not returning:

- Because they are simply not interested in more 3D-prints at all?

- Because they need a total different process like FDM and rarely use SLA?

Does this affect my rating?

Accept rate: Orders accepted

- What if the customer does not respond to your requests and archives an order?

- What if they placed the same order at 5 hubs simultaneously and asked to archive the order from 4 of 5 hubs?

- What if they see a certain price, still place an order, then decide they don’t want to pay it?

Does this affect my rating?

There are so many factors that influence above two metrics and do not say _anything_ about the quality and work of the hub that it seems unfair if they affect your rating.

I honestly do not see why some nearby hubs with good ratings, reviews, response time and pictures are rated lower than one 400km away somewhere in another country. Or hubs with only 3 reviews which are almost in the top of the list(So I guess they have 5 star metrics on everything).

Being recommended is about the most important rating for a hub… But there is no way for us to understand how and why.

PepCo_Parker · 2017-03-27T17:03:39.000Z

Referring to your point about customers placing mass numbers of orders: I actually created a thread about this a long time ago when I started seeing it more. No resolution was ever given. I said that I felt customers need to be more informed about how to properly manage the system and to place orders.

ProtoArt · 2017-03-27T17:10:32.000Z

Can’t agree more. Same goes for active orders being archived by 3DHubs or the customer: you don’t receive any notification at all and there is never an explanation why it was archived. Customers are asked why they want to cancel an order, but we don’t get to see it.

It’s annoying to see a drop in customers without knowing or understanding why. Is the market quiet? Have a bad rating? Something else?

I hope these comments don’t affect my rating

adamhayek · 2017-03-27T17:34:52.000Z

Retention will be in the insights shortly, we are working on that this week.

I already answered this before but we made sure that a couple of customers that won’t come back will not influence your ranking. We only look at people with statistically low or high retention.

Declining orders affects your ranking, as it does for everyone else. It may seem a bit unfair at certain cases and we are working on making the system better, but everyone gets a bad order from time to time.

Hubs with few ratings may rank higher as we want to give them a chance to get their first orders. This is still hard for them as they are competing with Hubs that have potentially hundreds of reviews. In this case being in the first place doesn’t necessarily mean you get the order.

The Hub ranking may seem as a simple problem, but trust me it is very hard to get it right for thousands of Hubs. For example reviews are not the best indicator as they are public and people give more positive reviews to avoid potential conflict. There is a caveat like this for almost every metric.

We will continue to improve the insights in a way that they will give you an overview of all the metrics we use.

I hope I answered at least some of your question.

Adam, 3D Hubs

adamhayek · 2017-03-27T17:35:46.000Z

This is a good point. I will put this on our todo list.

Adam, 3D Hubs

PepCo_Parker · 2017-03-27T17:47:51.000Z

@adamhayek which part lol, hopefully not that his comments would affect the hub rankings

adamhayek · 2017-03-27T18:10:16.000Z

Haha The part that we should inform you about the reason why we canceled or archived the order.

YOUto3D · 2017-04-21T05:19:30.000Z

I was just looking at this. My average rating on my dashboard is 5/5 yet the graph says it’s 4.74 sooo yeah… I say there’s something wrong here.