Computer scientists develop technique to improve helpfulness of user-generated online reviews

“Our technique automatically analyses online user-generated reviews based on four key factors - reputation, social, sentiment and content - and determines how helpful the reviews are likely to be to customers before presenting the most helpful first - including both positive and negative reviews,” says UCD Professor Barry Smyth, Director of CLARITY, one of the researchers involved in the work.

User-generated reviews have become increasingly important when it comes to users making buying decisions. While online product or service reviews have become more common, they tend to vary greatly in their quality and helpfulness.

“Some online user-generated reviews can be biased or poorly written, while others can be balanced and insightful,” says Professor Smyth. “That is why we worked to develop an automated technique of accurately identifying helpful product or service reviews for the user.”

“Some online services are already trying to do this by allowing users to rate the helpfulness of a review, but this type of feedback tends to be sparse and varied. In fact, if the online reviews received enough helpfulness feedback from users there would be little need for our work in this area.”

According to the UCD computer scientists, after sampling 225,000 reviews by 45,000 distinct reviewers on 70,000 hotels on TripAdvisor, they discovered that about 65% of reviews did not receive sufficient feedback to allow for a straightforward determination of their helpfulness.”

“The infrequent helpfulness ratings that we identified would not give users a lot of confidence in the reviews,” explains Dr Michael O’Mahony, another of the UCD researchers involved in the work. “It’s about identifying which reviews are the most informative about the product or service. After all TripAdvisor currently hosts over 30 million user-generated reviews, so there is an amazing amount of latent value in existing online reviews.”

The system devised by the researchers applies machine learning techniques to discover the factors that make for a helpful review. It analyses reviews that have received feedback, and then applies the resulting models to evaluate the helpfulness of the majority of reviews that have little or no feedback.

The technique applies four key attributes to predict the helpfulness of a review: reputation, social, sentiment, and content. When added together, these elements offer a greatly improved helpfulness classification for online reviews.

Reputation captures a reviewer’s reputation as regards all previous reviews written by the same author. Social takes account of the fact that when a user writes their own reviews they also tend to be more likely to respond to existing reviews and thereby improving the quality of their own review. Sentiment considers that the score attributed to a product in the review is an indicator of review helpfulness, as users tend to respond more to positive reviews. Content accounts for how well the review is written, by way of form and length - poorly authored reviews are considered less favourable by users.

"One interesting result that we noticed is that users tend to be drawn more towards positive reviews and often ignore negative reviews," says Dr O'Mahony, "even though the negative reviews can be very revealing and informative. Thus one of the benefits of our system is that it can be used to promote helpful negative reviews that might otherwise be hidden within a mass of review content."

“Ultimately the user benefits by having access to more helpful reviews. And websites like TripAdvisor will benefit from a more satisfied user-base, and even the service providers (the hotel owners in this case) will benefit because the good services or products will have a better chance of attracting customers, while the poorer ones will have to be improved in order to attract better reviews.”

Provided by Univeristy College Dublin