The Engineer's Notebook Blog

The Engineer's Notebook

The Engineer's Notebook is a shared blog for entries that don't fit into a specific CR4 blog. Topics may range from grammar to physics and could be research or or an individual's thoughts - like you'd jot down in a well-used notebook.

Previous in Blog: Should Electronics Be Banned From Classrooms?   Next in Blog: Aitoliko, Greece: A Town Overrun by Spiderwebs
Close
Close
Close
8 comments

Algorithm Detects Fake Yelp, Trip Advisor Reviews

Posted September 22, 2018 12:00 AM by M-ReeD
Pathfinder Tags: AI Algorithm food

Some years ago, when Yelp was exciting and brand-new, a group of friends — not wanting to take a chance on wasting money at a tourist trap — consulted the novelty app for restaurant reviews of a little Italian restaurant in a huge metropolitan city overrun with little Italian restaurants.

One of the reviews called that particular restaurant a favorite of the notorious gangster John Gotti.

Whether this was a benchmark by which all Italian restaurants are measured, we did not know. Yet, because the Yelp post seemed full of other positives, we went inside.

Clearly, that Yelp review was packed with John Gotti-sized lies because the pricey meal was no better than what you would find at a fast food joint…or worse, at my house.

Fast forward some years later and it turns out that sites similar to Yelp, like TripAdvisor and Amazon, are packed with fake reviews meant to bolster a product or business.

“Misbehaving companies can either try to boost their sales by creating a positive brand image artificially or by generating fake negative reviews about a competitor. The motivation is, of course, money: online reviews are a big business for travel destinations, hotels, service providers and consumer products,” said Mika Juuti of Aalto University.

In fact, some of those reviews seem so legitimate, that people don’t even recognize that they are generated by an algorithm.

Now, researchers from Aalto University led by Juuti have developed an algorithm that can determine whether a review is a fake or a legitimate one.

To begin, researchers trained a machine learning model, a deep neural network, on a dataset of three million genuine restaurant ratings posted on Yelp. Once trained, the model produced fake restaurant reviews character by character using a text sequence of “review rating, restaurant name, city, state, and food tags.”

“In the user study we conducted, we showed participants real reviews written by humans and fake machine-generated reviews and asked them to identify the fakes. Up to 60% of the fake reviews were mistakenly thought to be real,” said Juuti.

From there, the team developed a “classifier” capable of spotting fake reviews. The classifier performed well, especially in scenarios where humans evaluating the legitimacy of a review were stumped.

Do you consult reviews before eating out or making a purchase?

Reply

Interested in this topic? By joining CR4 you can "subscribe" to
this discussion and receive notification when new comments are added.
Guru

Join Date: Apr 2010
Location: About 4000 miles from the center of the earth (+/-100 mi)
Posts: 8994
Good Answers: 1023
#1

Re: Algorithm Detects Fake Yelp, Trip Advisor Reviews

09/22/2018 11:00 AM

To begin, researchers trained a machine learning model, a deep neural network, on a dataset of three million genuine restaurant ratings posted on Yelp. Once trained, the model produced fake restaurant reviews character by character using a text sequence of “review rating, restaurant name, city, state, and food tags.”

“In the user study we conducted, we showed participants real reviews written by humans and fake machine-generated reviews and asked them to identify the fakes. Up to 60% of the fake reviews were mistakenly thought to be real,” said Juuti.

From there, the team developed a “classifier” capable of spotting fake reviews. The classifier performed well, especially in scenarios where humans evaluating the legitimacy of a review were stumped.

One problem I see is that the training set of "fake" reviews was generated by an algorithm. The "classifier" neural network apparently learned how this algorithm worked and was able to spot the fakes. The question is whether it would be as successful at spotting "genuine fakes" that are written by humans.

Reply
Associate

Join Date: May 2013
Posts: 25
#6
In reply to #1

Re: Algorithm Detects Fake Yelp, Trip Advisor Reviews

09/24/2018 8:45 AM

"In fact, some of those reviews seem so legitimate, that people don’t even recognize that they are generated by an algorithm."

It doesn't need to spot human-written fakes.

Reply
Guru

Join Date: Mar 2007
Location: at the beach in Florida
Posts: 30802
Good Answers: 1722
#2

Re: Algorithm Detects Fake Yelp, Trip Advisor Reviews

09/22/2018 11:32 AM

I always consult the reviews when buying, but you need to read them all, and look for consistency and spelling errors, and I might add, a return at no cost policy, and/or guarantee of satisfaction...false reviews are just a fact of life, just like Chinese fakes claiming to be genuine....It helps to do business through a moderator who guarantees the transaction....I do a lot of shopping on line, and have done so since the beginning, my advice is to take your time and research thoroughly...but have a guarantee to fall back on when you get fooled....and you will...

__________________
Break a sweat everyday doing something you enjoy
Reply
Guru

Join Date: Apr 2010
Location: About 4000 miles from the center of the earth (+/-100 mi)
Posts: 8994
Good Answers: 1023
#3

Re: Algorithm Detects Fake Yelp, Trip Advisor Reviews

09/22/2018 8:35 PM

I've never used Yelp much but usually look at Amazon ratings. I read the most negative ones to see what the rater's gripe is and whether it is important to me.

It doesn't cost anything to read ratings, and you get what you pay for. Yelp, I'm sure, does its best to filter out fakes because a loss of its reputation results in a loss in advertising revenue.

But any system that can be gamed will be gamed and crowdsourcing is no exception. It's human nature. Caveat Emptor.

Reply
Member

Join Date: Dec 2017
Posts: 9
Good Answers: 1
#4

Re: Algorithm Detects Fake Yelp, Trip Advisor Reviews

09/23/2018 8:33 AM

I do check Yelp before finding a place to dine in an unknown area. I've found that the locations with positive reviews numbering in the several hundreds or more are reliable. Still, I do read a number of positive and negative reviews to make sure that a location suits me and those I'm with.

Interestingly, I recently wrote a real review that rejected by Yelp. I tried again and it too was rejected and so I gave up. This is unfortunate for the business that deserves a good word. So I went to Google reviews and they took my review without any fuss. Does this mean that Google is less reliable or more reliable?

I don't question that there are fake reviews out there, but they will (should) only fool you once. No business can survive long on fake positive reviews as the negative reviews will win out with recurring details of exactly how a particular business failed to provide a pleasing experience.

Reply
Guru

Join Date: Apr 2010
Location: About 4000 miles from the center of the earth (+/-100 mi)
Posts: 8994
Good Answers: 1023
#5
In reply to #4

Re: Algorithm Detects Fake Yelp, Trip Advisor Reviews

09/23/2018 10:59 AM

Maybe when it comes to "faking", the negative reviews of competitors are more effective than positive reviews of one's own business The customer may learn first hand about a false positive review but, by avoiding a business with false negative reviews, never know the difference.

(Maybe the same principle works for political ads.)

Reply
Guru
Australia - Member - New Member

Join Date: Jun 2011
Location: Thailand
Posts: 631
Good Answers: 3
#7

Re: Algorithm Detects Fake Yelp, Trip Advisor Reviews

09/24/2018 12:18 PM

Some days ago - okay, late '70s - early 80's - Playboy (only for the stories, of course) had a story 'Have I found the best restaurant in the world?' about a restaurant in the SE of France called 'Troisgros' - Michelin 3 stars -

Found myself in Geneva in 84 with a bit of time & thought SE of France is not too far away - but there was just no way to get there. Don't know how far it was.

Anyway, learnt later that one of the criteria for getting Michelin stars (back then) was fairly predictable ;-)

__________________
Floss or die!
Reply
The Engineer
Engineering Fields - Engineering Physics - Physics... United States - Member - NY Popular Science - Genetics - Organic Chemistry... Popular Science - Cosmology - New Member Ingeniería en Español - Nuevo Miembro - New Member

Join Date: Feb 2005
Location: Albany, New York
Posts: 5170
Good Answers: 129
#8

Re: Algorithm Detects Fake Yelp, Trip Advisor Reviews

09/27/2018 8:37 AM

I'm always a little skeptical of an article like this because they don't mention the false positives. For instance, I could build an algorithm that identified fake restaurant reviews with 100% accuracy in 15 minutes. I would just write a program that marked all restaurant reviews as fake. My accuracy would be 100% but my false positives would be 100% as well which is useless (obviously). By describing the accuracy but not the false positives for a classifier, you're not really being told anything significant.

Reply
Reply to Blog Entry 8 comments
Interested in this topic? By joining CR4 you can "subscribe" to
this discussion and receive notification when new comments are added.
Copy to Clipboard

Users who posted comments:

Bayes (1); dougefresh (1); EinC (1); Rixter (3); SolarEagle (1); Stuart21 (1)

Previous in Blog: Should Electronics Be Banned From Classrooms?   Next in Blog: Aitoliko, Greece: A Town Overrun by Spiderwebs

Advertisement