B.

Did you know we’re changing the way guest reviews work?

Hi everyone,

Guest reviews are one of the most important factors of managing a property. We at Booking.com we appreciate the feedback on the fairness and accuracy you have provided of the guest review score. 

Thanks to your input, we have made some changes to the way the guest review score is calculated.

In this new system the overall score won’t be determined by the average of the 6 categories, you’ll find all the details in this dedicated article.

Let’s find out what’s new in the improved review system:

  • Now guests select an "overall" score themselves

  • Smiley faces are replaced by a new sliding scale of 1–10 for this score

  • The 1–10 sliding scale means more objective scoring based on guest feedback. 

Your feedback helps us improve! Please leave your thoughts in the comments below:

-Ilaria

 


8
pibomarco

Reviews older then 2 years are removed which give you abbilty to improve in future and get a better overall score. Again: Overall score is calculated as arithmetic mean (The SUM of ALL scores divided by the count of scores in the collection). "10 + 10 + 10 + 7 = 37 / 4 = 9,25"

1
Raphaël Amiri

Hello, 

This new notation system is inequitable.

Indeed we only have a few comments (only 37) because we start our activity. 

Our note with the old system is 9.7.

I think that with the new system customer will give us 8 or 9 / 10.

Our neighbour has a note of 9.2 but more than 800 comments.

I think they will have many 8 or 9 / 10 with the new system too.

But with only 37 comments our note will decrease quickly than with 800 comments.

Moreover the global note is calculated on a 2 years period. Then we will impact for a long time.

However the quality of our service will be the same :(.

This new notation system will be very bad for our business.

BR

 

8
pibomarco

We're all on the same boat, so if it is "bad" for your business it will also be "bad" for the others. 

1
Raphaël Amiri

Yes, you're right.

But you can understand that if you're new on the business and you have 40*10/10 the impact of a 8/10 is not the same that a people which have 1000*10/10.

 

I think this new system has sense if they reduce the period we keep the note in the calcul. I think a one year period should be better.

8
pibomarco

Of course It's not the same. This will always be the "problem" for small properties no matter what type of the review system is used. Also properties that have 1000+ reviews with an overall score above 9 is not that common. With one year period, the score would be swinging up/down even more which is not OK.  

8
fluff

Once the contradiction between smilys and 1 to 10 is ironed out, the scoring will then be in line with the other big players.

For us our lowest score is on BDC's old system! Therefore, we are hoping the new system will help us even out our scoring.

10
M Adamopoulou

I got my first review with the new system and it’s excellent!!!!

Hope It stays like this...

We are in the same boat...but some are in the VIP section...ha...ha...

pibomarco love your comments....

fluff wish you have better reviews with new system...

Cheers!!!

1
Guest house Vi…

Hello, 

This decision is really stupid and i have the proof. 

Today i had a review with all 10 and the final review 9.

It does not make sense at all. 

Please change this stupid decision. 

It is worsing our grade feom 9.6 to 9.0 at this point.

Best regards

8
pibomarco

I just received 7,5 and the final review 8.  In this case it does make sense right? :) 

4
Jennifer and A…

Here is the message I just sent to the customer service: We are upset about the change to the review system. Other platforms like Airbnb have been heavily criticized by hosts and have struggled because of changing to a review system like the one you're introducing. It leaves room for people to leave high reviews on categories and yet a low overall review because they had a cloudy weekend or the snow wasn't good. To make matters worse, the only review that counts is the overall, which isn't based on logical reasoning, but an emotional impression of a holiday. We urge you to go back to the previous system where the final review is a calculation based on the category reviews.  Thank you.

8
fluff

This mix of smilys and score is definitely causing confusion.

We just had a rather obnoxious guest drop a minimum rating on all smilys BUT erroneously gave us an overall 10 !!!

Although we found it amusing that his intended bile lead to a 10 score, it does go to show guests are confused.

1
La Corbeliere

We have just had our first review under the new system and we are definitely not happy, so unhappy in fact that we are considering withdrawing our property from Booking.com.

In our latest review, we got 10 for all 6 categories, but the review score was only a 9.

For nearly 4 years we have consistently been rated at 9.9 and for the past year we have achieved a perfect 10.

It seems that all of our hard work and dedication to our guests will have be for nothing more than to receive a lower review score. It's a joke.

We know the old face system wasn't perfect and also know that some clients didn't fully understand it, but the new system seems like a backwards step that will do nothing more than annoy Booking.com partners.

4
Jennifer and A…

We are also thinking of removing our property if this is not corrected asap. We cannot risk losing our reputation because of their trials and errors.

1
N M Hunter

We have just had our first review on the new system - 3 x 10 and 3 x 7.5 on the categories. Overall score - 5!! On the old system this would be 8.75 average but now all we get is a 5. The booking.com information on the new review system says that they will work to resolve inconsistencies, but having just checked up on this, all they will do is send a message to the guest to suggest that they re-do the review and only if we ask in each individual case. This doesn't help us at all. The guest could refuse, or not get around to it and we are left sitting with a really low score on our reviews.

I should add that the negative comments on the review were 'we thought it was a hotel so we had to go out to eat again'. We are a clearly listed as a B&B, but apparently we are to blame because the guest didn't read the listing. The opening sentence on our listing declares the distance we are from the centre of the city, but it is a daily event that a guest arrives here saying 'we didn't realise you weren't in the city centre'!!

Please booking.com, recognise that if a guest can't be bothered to read what they are booking, the chances are fairly high that they won't read carefully what they are scoring either!! And our livelihoods and reputations depend on this!

Please, please, please revert back to the old system, or at the very least, leave in the option to score 1-10 on the categories but take the average as a score. This system is just not fair at all.

4
Jennifer and A…

We agree 100%. Please Booking.com, revert back to the old system. This new system is not coherent. By the way, it would be great if someone from Booking.com would actually respond to these comments and take this seriously. We've opened two additional conversations in the Partner feedback and Booking updates channels so that as hosts we can stick together and help get this changed back quickly. 

4
Jennifer and A…

Illaria, why is booking.com not responding to these comments and promising to look into this situation. Many hosts are upset with the new review system. Have you passed the message along? How does this forum work in terms of being heard by Booking.com?

B.
Ilaria - Commu…

Hi Jennifer and A…, thanks for the reminder.

We have passed your feedback and sentiment around the new review system to the relevant team.

As soon as they get back to us, will let you know. 

1
N M Hunter

Thank you Ilaria, that is good news.  To update you, Booking.com contacted the guest and suggested that they amend the overall score. They did - to a 7. This is good, but the overall score should have been 8.75 as that was the average of the categories, so we are still compromised by this new system. It is simply not fair on the hosts and leave the whole system open to confusion/abuse. Equally, not all hosts are getting the new system at the same time, so it is unfair that some hosts are subject to the new system and not others. This is not a level playing field! We would be really grateful if you could keep up the pressure on the relevant team to sort this out. 

8
pibomarco

So, who to please? 

Hosts that want final score 1 - 10 from the experience, or hosts that want an overall score calculated from the sections?

I guess to "satisfy" all of them would be, that a better score is picked. If host receive a better score from the sections comparing to overall experience, the score from the sections is registered and vice versa. 

I suggest that BDC make a review system from 9 - 10.. so that ALL properties on Booking.com will have a top score and treated equally, like in socialism ;)

8
fluff

I think you have the perfect answer pibomarco !!!

Joking aside...

This new system was in answer to many, myself included, objecting to the four smileys representing a score of 1 to 10. It didn't make any sense that scoring had to differ by a minimum of 2.5 per category. I for one was quite happy with averaging system over the multiple scores with the caveat of "location".

All that was asked for was the scale to be 1 to 10 instead of 4 = 10.

BDC then proceeded with this rather confusing, for the guest, system of mixing smileys with 1 to 10 & then separating an actual overall score and the smileys then not counting in the score at all. Convoluted and unnecessary.

In fact WEIRD and guaranteed to cause problems and anomalies. 

The logical change to a 1 - 10 system would have been retaining the category average but using 1 -10 to mean 1- 10 instead of 1 - 4 to mean 1 - 10.

I reiterate that BDC has my lowest score of the OTAs, this I put down to a system where if you are not all rainbows and unicorns then you drop a minimum of 2.5 point chunks instead of exactly what the guest chooses, whether that be less or more. 

At the best of times there is often no accounting for human nature in reviews but the previous and new systems make very little sense.

8
pibomarco

Our booking.com support / account manager told me that in near future sections will also be scored from 1 - 10 instead with 'smiles'. Can Ilaria - Community Manager  confirm this?

Will this really solve the problem though? I don't think so. I still beleive there will be a lot of complaining after that. :) 

4
Jennifer and A…

 

In the building we are 8 owners and we all agree with Fluff. Our main concern is that, as you point it Fluff, "separating an actual overall score and the smileys then not counting in the score at all" leaves more room for inaccuracy than the previous system, objectively, whether the error is advantageous or disadvantageous.

1
N M Hunter

Fluff, you are absolutely spot on. The system you describe is exactly what should have been provided. Logical, simple and would work perfectly. Minimal confusion for guests and fairer for us too. 

3
Andy

Hi, at the moment I am waiting to be convinced of the new rating process however one thing that is now lacking is that I at least am unable to comment on a review.  Out of politeness at least I liked the idea of being able to thank the guest for their review but now I am unable to.

1
La Corbeliere

We have just spoken to a guest who told us that he has never given a review in the past and the one he gave us was his first. He said he found it a bit complicated.

He scored us 10 for everything and commented that it was Brilliant. Our overall score was only 9 though. He couldn't understand why that was. Us neither!!!

3
Andy

Yes rather strange but suspect it has something to do with other businesses in the area dragging score down perhaps?  Seem to remember reading something along those lines but totally unfair if the case.  I reckon with the new system most people (guests) will go for an average thus the 7.5 smiley.

3
Andy

Well if guest said put 10 for everything and couldn't understand why came out as a 9 sounds unlikely put 9 as overall but then you never know with guests.

2
Bakuflats

That is exactly why the overall score should be automatically calculated by the system on the basis of individual scores. Not every guest has a good math skills to calculate a correct  overall score.

8
pibomarco

Well the guest obviously scored the sections 10 and overall score 9. He also mentioned that he never reviewed the property and that it was bit confusing for him, so it is a possiblity that he made a "mistake".

It is also a possibilty that the guest felt pressured if he was contacted or "confronted" by the host regarding the score and he acted a little "dumb" since he didn't have a good reason why he scored 9. 

"has something to do with other businesses in the area dragging score down perhaps?" I don't see this as a possibility at all. 

 

 

 

3
Andy

Quite agree, not sure why the host contacted guest as still a good review and as you say it puts pressure on the guest to possibly lie or play dumb.  As to peer group haven't a clue whether any truth in that but remembered reading posts about it and found them page 1 of this topic by: 

Callshannonfirst

Would be interesting getting a response from La Corbeliere who brought it up.

8
pibomarco

 

This will happen. Some people just don't rate their overall experience 10 by default even though they liked everything. It's just a matter of ones perspective. Considering also they are not that emotionally attached to reviews as we (hosts) are and from their point of view 9 is still considered amazing.. which it is.  

I highly doubt that the guest brought it up though based on his "defensive" reply / explanation. :) 

Callshannonfirst just doesn't know how the reviews are calculated and is not familiar with arithmetic mean. :) 

1
La Corbeliere

Just to clarify. Our guest didn't feel pressurised as we always contact guests that leave reviews via the messaging facility and not via the public review page. We do that because some of our guests are high profile and incognito, like actors, performers and politicians, etc.

The fact that we are the first place that he has stayed at and bothered to leave a review for and found it confusing says it all. The review scoring system is not perfect. End of!

2
Casa BuonVento
Booking review

Hi all, it's my first message here, just to tell that I'm feeling less alone reading you... As you can see, this guest rated 10 almost all the subcategories, but the overall score is 8.0 (our overall average score is 9.6). Booking.com, please solve the problem!  P.s.: I agree that a solution could be the following: guest rating 1-10 the sub-categories and overall score = automatic average of the sub-categories.  Regards, Elena

 

4
Jennifer and A…

Thank you Elena for the facts. Indeed, as it was before, automatic average of the sub-categories is better.  Here is a link where you can also vote to repair this asap. https://partner.booking.com/en-gb/community/booking-advice-and-updates/hosts-against-new-review-system

Here as well: https://partner.booking.com/en-gb/community/partner-feedback/new-rating-system We all expect Booking.com to appear in this chats and care. And remember to vote the first post, the one written by the community manager.

2
Bakuflats

This is exactly what my concern was when this system was introduced. The same problem exists in Airbnb, whose rating system booking.com decided to copy. Booking.com had it right with their original rating system, I have no clue why they decided to copy flawed system of their competitors. The overall score should be calculated automatically, it should not be left to the discretion of the guest. The ratings for subcategories could be from 1 to 10. I think booking.com should fix this problem asap.

8
pibomarco

The subcategories are also going to be scored from 1 - 10 (so they say), which I think it will bring more "lower" scores comparing to emojis.

We must take into consideration that those 10s were given with the emoji system. Meaning if subcategories would be set to 1 - 10, it's more likely that some of those categories could be rated 8 - 10.

I think this is just another indicator why emojis perform better with higer scores and the reason why we should expect more 8s and 9s instead of 10s with the new scoring system. 

The old review system was changed because hosts couldn't accept the score 7,5, and felt it was too low or unjust, but took 10s for granted. That's how I saw it. 

2
Pristine1917

The new, poorly crafted and absurd review system is compelling us to leave booking.com. Our letter to management explains why: Dear xxx: Not for the first time, booking.com has implemented sweeping changes based on some opaque, and probably faulty, understanding of the data it collects. Also not for the first time, we've concluded there is nothing we can do about it except document it. This time, however, our reputation is at stake, so unless there is a prompt correction, we are leaving booking.com for Expedia. Why are we upset? Take the review left by the guests for reservation number 2783724663 (our property ID is 1126597). It most ways it is typical for us. In five out six categories, we are given a perfect score. In one category, less than perfect (three stars out of 4, or 7.5). The guest took the time to write "everything was good breakfast good staff helpful", then added that he wished there was a coffee maker in the room. His intent is clear enough.

Under the old system, that would have yielded an overall score of 9.6, which accurately reflects the extraordinary work and investment we put into our property. Under the new system, we received a 2.0 -- an abysmal, failing grade. Obviously that's not the score the reviewer intended for us. With effort (and time we do not have), we could probably persuade him to amend it. But it is not his fault. It is because once again, someone imbued with far too much authority at booking.com has degraded functional policy at your company. This time, it is more serious than, say, removing the US state field from the guest address form, (costing us, on average, about 45 minutes a month). We simply cannot risk this kind of damage to our reputation. We have worked far too hard for too long to allow incompetence in some far away corporate office to destroy our efforts.

The sheer sloppiness of this latest effort is breathtaking: * The new review system replaces a score derived from six data points and replaces it with one derived from a single data point. As any competent analyst will point out, that is absurd. * Of course it is those "partners" with the highest aggregate scores who will be damaged --- the very partners that produce the most value for booking.com. Why? Because they typically achieve a mix of perfect (10) and "near perfect" (usually 7.5) review scores based on plural data points, yielding an aggregate score above 9.0. Under the new system, only a single data point matters. "Near perfect" is limited to a score of 9. As each reviewer is now asked to choose between perfection and near-perfection, and "near perfect" is no higher than nine, scores WILL migrate downward toward 9.0. (And that does not account for mistakes made by the reviewer in 2783724663, which will be far more frequent and harmful, now that a single data point determines the score). * The data presented review now presented to readers are a mix of scores derived from both the old system and the new. This, too, has no statistical validity. * The new system offers less information to the partners, not more. * And once again, the system was trotted out without testing the basic HTML code on the pages. The feedback button did not work for days. Users instructed to "hover over Peer Group, below" when the words "Peer Group" appeared nowhere on the page. And many other amateurish mistakes. Something appears deeply flawed in the corporate culture at booking.com, where senior managers appear compelled to compete to produce roll-outs like this one, yet are never accountable for the results. There's an obvious charlatanism at work here -- someone pretending to know more than they do. Booking.com should pull the plug on this fundamentally flawed review system -- and the executive who persuaded the company to waste resources on it -- and prove itself accountable for the risk and damage they offload onto the partners that generate wealth for the company. As it stands, booking.com is out only OTA, aside from Airbnb. But as we're now quite sure that our exposure on booking.com can only erode our reputation, we're finalizing arrangements with Expedia to move all our OTA business there.  Richard Latker (Former senior economic analyst at the Economist Group, London). Also ref Reservation IDs:***

 

2
Gustave Righetti

Thank you Pristine, thank you Richard Latker for illuminating the facts. We understand your decision to leave and are arranging to do the same, in the event Booking doesn't revert to the average calculation before December.

8
pibomarco

Of course it was his mistake/fault. We’re humans not robots. If you don’t want to notify the guest about his “mistake” in which he could be able to request a change, well that’s on you. It’s not the first and it wont be the last mistake made by guests.

2
Bakuflats

The system should not leave room for such mistakes. This could have been prevented, if the overall score had been calculated automatically.

8
pibomarco

I agree that the system could “warn” the guest when scores contradicts that much. 

2
Pristine1917

Of course the new system makes such mistakes all the easier -- and graver, given that there's only one data point instead of six. We do not intend to spend our time chasing guests to ask them to correct reviews. Our time is best spent running our property. Fortunately, we have been steadily reducing our reliance on booking.com over a long period anyway. 

8
pibomarco

There is no system where guest will make no mistakes.. specially if there are milions of people using the system. I head one guest last year that scored all subcategories 2,5 but he taught he gave the best score.

You pointed out 1 review with such an obvious mistake made by the guest. Lets see who and when will be next. This is a case per year.. at most. Considering the property acumulates cca 1000+ reviews.

2
Ivybridge Gues…

Booking.com is there for me because I am a B&B on the edge of the earth and many guests use their system to book! I am not in a position to pick and choose which third party to use; I use all that will bring me rooms from different regions! 

You should be on Expedia anyway, not burn bridges for scores! 

Have you ever bought things on line and looked through 5's, 3's and 2's just to see what people are writing. Well, what IF someone looks at a 2 score to see what is so bad about your place AND THE WRITE UP IS EXCELLENT? 

You win! : )

 

 

 

2
Pristine1917

Sorry, but that does not cut it for me. We've worked too hard to build this place to allow some faceless corporate bureaucracy to place our golden reputation at risk -- most likely just to protect the internal reputation of some overpaid executive who sold this dumb idea. The review launch is a disaster, but they will defend it no matter what evidence is presented to them, and what you and I say does not matter a whit. Fortunately, we ARE in a position to pick & choose. When booking.com summarily decided to curtail guest address data (removing the US state field, they claimed, would increase booking completions), and reneged on its longstanding promise to pre-vet credit cards, we took steps to reduce our reliance on it. When the company ignored us on these issues, we knew we would probably have to leave eventually. We've since boosted in-house bookings from about 12% to nearly 25%. Airbnb is also picking up steam, and we're confident we can replace booking.com with Expedia -- which I'm sure has its own problems, but for now there's a clean slate. We're headed into the down season, too, so its the perfect time to transit out. I just finished removing every reference to booking.com on our website. We've removed almost all our available rooms from the booking.com extranet calendar. If we don't receive an meaningful acknowledgement to our complaint about the new system and a plan to address it --- and we expect neither -- we will cancel our contract with the company. 

8
pibomarco

Pristine what is your property name.. How many reviews did you acumulate on BDC? :)

2
Pristine1917

You should be able to figure out who we are, if you are looking for some way to harass us. Over five years I'd say we've had about 500 reviews on BDC. Since we started reducing our reliance on BDC after the "address field" fiasco, we're getting far fewer reviews --- only 88 are currently active, average 9.6. Our average was 9.7 until the 2.0 review we got last week. Our business relies entirely on our reputation, as does our definition of success. From that standpoint, we find the latest error from BDC absolutely intolerable. 

8
pibomarco

No, just for a better understanding of your described situation. Surely the downside of the small properties is that negative reviews has a bigger impact on overall score and it can be frustrating wich is understandable. But all small properties are on the same boat fighting to keep a high overall score. Anyway as far as I see it you didn't loose any reputation. And generally looking overall score 9+ is definition of success.