Evaluating User Reviews

Over the past few weeks I have been surfing the web looking to make several on-line purchases. As I did this, I was paying attention to the heuristics involved in my decision-making. How did I get from Google to purchase? It turns out that I don’t have a set methodology for my searches and decision-making but there are some strategies I employ for the process. Visually, the process looks more like a Dallas highway interchange than a road through Kansas.

my process_opt

I thought it might look like a squiggly line but, I can’t use that metaphor because I rarely had one starting point and one ending point.

my squiggle_opt

 I have a few general observations but then I’ll tell you something I found really interesting. My reactions to user reviews. I bet there has to be a lot of science going on studying those!

Here are three of my general observations:

  • With Google (my default search engine) search results. I ignore the promoted links at the top and along the side. In thinking about it, I am ‘OK’ with the links along the side of the page. They typically don’t interest me but I don’t mind them being there. The promoted links at the top of the search results bother me. In my mind, the search is for me so it should return what I am looking for, not what someone wants me to see.

my search_opt

  • I never looked at pop-up ads (not the pop-up pages but the little ads placed on a web page. I believe they go into my eye but never reach my occipital lobe. I think this is because of how those ads were originally used. Mostly, they had little to do with what the page content was. My mental model hasn’t shifted in the last decade.
  • When I realized there was official lingo (e.g., the thing you hook your bicycle into to ride indoors during the cold winter months versus ‘bike trainer’) I was back at my Google search page doing a new search (another on-ramp).

So, on to the user reviews.  I spend a lot of time reading reviews. I appreciate the experiences other have had and the more specific the review the more I liked it. I was buying a coat for my daughter who goes to school in Chicago. If the review said ‘nice coat, keeps me warm’ and had 5 stars, that was ok. It was somewhat useful. If it said ‘Nice coat, keeps me warm on 10 degree days with just a t-shirt underneath. The hem hits me just below the knee.’ And the reviewer was from Chicago… I found that to be much more useful.

But, the interesting thing is that even a specific review was not evaluated the same way across all the products I was looking at. There were a few other factors that came into play.

 my reviews_opt

Risk: The risk involved in my purchase, either money or convenience (e.g., I need a book for class tomorrow) modified how I reacted to reviews. If there was a lot of risk, I tend to require more 5-star ratings and no 1- or 2-star ratings. And, I needed more details.

Familiarity: What I am talking about here is my experience with a product category (e.g., hammers, bike trainers, etc.), not necessarily a specific product (e.g., the Sears 16-inch, steel hammer). If I had experience with the category (hammers) I wanted the review to tell me about its overall quality. If it was a new category, like that bike trainer, I needed the review to tell me more. Did it work differently with different bikes? Did it perform well under load? Etc.

Judging the Review: Yes. I am judgmental and, truthfully, I only found this to be an issue on Amazon when I read book reviews. There were a couple of academically-oriented books I was looking at. I expected them to be written in an academic style. I’ll call that dense. If the reviewer gave the book 1-star and wrote, ‘dull, boring and hard to get through’ I judged. What were they expecting, Rick Pitino’s thoughts on leadership? I ignored the review and it didn’t have an impact on my overall decision-making.

Emotion within the Review: Without a doubt, this was the most fascinating observation for me. Once I started to think about it, it became very clear, very quickly. Reviews without any emotional content were useful, regardless of whether they were positive or negative. In general, they were informative. A positive review with some emotional content – smiling emoticons, upper case phrases, etc., gave the review a boost. A 5 star rating was, mentally, a 5+ star rating. They were extra informative.  Now the fascinating part…. Negative ratings with emotion were not extra informative. I thought they were just rants and tended to ignore them. Sometimes things break. No need for curse words, hand gestures, etc.

my matrix_opt

So, what do I do with this understanding of my heuristics.  Two things.  First, I’ll be paying more attention to how marketers are trying to capture my attention.  Second, the awareness of how I react to user reviews, bringing those mental models more to the forefront of my mind, will allow me to be more deliberate in evaluating those reviews

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Comment/Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s