All Ratings and Reviews Are Not Created Equal!

I was on a conference call last week with some industry executives, when the question of rating value came up. One of the executives wanted to know the monetary value of a single rating. He mentioned an recent conversation with a home improvement rating and review site executive. The site executive had created a hacked formula for assigning a value to a single rating. He took the recent acquisition price of InsiderPages.com, $13 million and the number of ratings they collected, 600K, mixed it with a little magical potion and came up with $2.00 per rating.

I am sure he put a great amount of thought into his calculation of this value. However, in my professional opinion he is not totally correct with his conclusion. We as humans have a need to instantly assign a value to everything. There is nothing wrong with doing so, however I believe value is based upon one interpretation of gathered information. For instance, the InsiderPages model could be considered voided, due to the popular opinion that the acquisition was nothing more than a LIQUIDATION PREFERENCE. If they were sold merely because the VCs decided they wanted out, then you can not consider this an acquisition, but more like a yard sale. Therefore, I can not accept this as a reliable resource for calculating the value of a rating.

The inability to assign a value to ratings does not ended there. The rating and review industry is broken into two areas: general and targeted. Sites like InsiderPages are considered general. They allow user to rate various consumer related experiences. Where sites like MyDealerReport.com targets auto consumer related experiences. And to further complicate matters, the surveys themselves are not uniformed, therefore one must determine the value of the density and quality of the questions.

So let us examine these specific factors. Are targeted website ratings more valuable than those of general websites? Probably! A good illustration of this argument would be the health industry. The family physician versus the specialist. One treats your overall general health, while the other focuses on a specific targeted area of your health. If you had a heart disease which physician would you prefer to be treated by? I hope you said the heart specialist. Now with that being said, which physician is typically the higher paid of the two? The specialist. Nevertheless, they both are physicians. Case and point, a single targeted rating has greater value than one that is general.

Where does density and quality factor. There are multiple forms of surveys being utilized by rating and review websites. Most of these sites ask one to five questions. MyDealerReport.com provides a more comprehensive 24 question survey. This leads us to the question, does survey density create greater value? The answer is yes and no. Low density rating produces a vague depiction of the user’s experience. While, a higher level of density may give a more comprehensive view of the user’s experience. However, too many question can dilute the experience, as well. It is my belief that low density surveys are designed for low impact life experiences, such as movies, dinner and books. Higher density is necessary for more complex purchases, such as insurance, home improvement and purchasing, auto purchasing and repair. Most general rating and review websites do not provide different density level surveys for different experiences. Instead they provide one size fits all. This creates a generic environment were a steak dinner and a car purchase are wieghed the same. Same type of concensus must be established within the overall market regarding the standards of rating and reviews or users will continue to be subjected to dubious rating results.

The market also lacks the standard of quality. All surveys are not competent. The types of questions presented to the user may affect the results. You must be careful to ask a question and not imply an answer. Example: Sallie needs a tissue, so she ask Frank the following question, ” You don’t have a tissue do you?”. Sallie has just implied that Frank does not have a tissue and if Frank is not motivated to help her, he will just give Sallie the expected response, NO. The quality of a survey’s questions are extremely vital. Users must consider the quality of the questions in a survey, before relying on the results. MyDealerReport.com allows our users to view the questions that were used in the calculation of our ratings.

So what is the monetary value of a single rating? Well, I think we will have to leave that up to the end users. Ratings are like art, it value is based on one’s personal preference. Therefore, users will let us know the monetary value in due time.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: