Can you really trust online reviews?

Key Concepts

• Subjectivity of Online Reviews: Reviews are based on personal opinions, leading to skewed distributions and potential inaccuracies.
• J-Curve Distribution: The typical shape of online review ratings, characterized by a high number of extreme (1 and 5-star) ratings and fewer moderate ratings.
• Reviewer Self-Selection Bias: People are more likely to leave reviews after exceptionally positive or negative experiences.
• Platform Norms & Reciprocity: Different platforms foster different review behaviors, with sharing economy platforms exhibiting more generous ratings due to potential for retaliation or reciprocal reviews.
• Informative Value vs. Objective Quality: A disconnect exists between aggregated opinions and objective measures of product or service quality.
• High-Volume Reviewers: Individuals who consistently leave reviews are more likely to provide a balanced and representative assessment.

The Reliability of Online Reviews: A Detailed Analysis

Introduction

The conversation centers around the question of whether online reviews can be trusted, acknowledging their pervasive influence on consumer decisions in areas like travel, accommodation, and product purchases. While recognizing that online reviews are “better than the alternative” – an absence of information – the discussion highlights significant flaws and biases inherent in the system.

The Problem of Subjectivity & Distribution

The core issue identified is the subjective nature of online reviews. The example of Goodreads and The Hunger Games illustrates how aggregated subjective opinions can lead to outcomes that don’t necessarily reflect universal agreement or objective quality. This subjectivity manifests in the distribution of ratings, which rarely follows a normal (bell curve) distribution. Instead, reviews tend to exhibit a “J-curve” – a disproportionate number of 1-star and 5-star ratings, with fewer moderate ratings. This skewed distribution arises because individuals are more motivated to review experiences that are either exceptionally good or exceptionally bad.

Research & Objective vs. Subjective Measures

A substantial body of research exists examining the reliability of online reviews, driven by the increasing importance of online commerce. Researchers have found a significant gap between customer reviews and objective measures of product quality, such as product testing results or resale values. This discrepancy underscores the limitations of relying solely on aggregated opinions.

Identifying Reliable Reviewers

The discussion proposes a strategy for improving the reliability of online reviews: focusing on the reviewer rather than the average rating. The key is to identify individuals who consistently leave a large number of reviews. This approach mitigates the self-selection bias inherent in the J-curve distribution. The reasoning is that forcing someone to review a wider range of experiences (e.g., the last book they read) would likely result in more moderate ratings (2, 3, or 4 stars) compared to someone freely choosing to review only experiences they strongly loved or hated (resulting in 5-star ratings).

Platform-Specific Norms & Reciprocity

Different online platforms cultivate different review norms. Research indicates that sharing economy platforms – such as Uber and Airbnb – tend to have more generous reviews compared to platforms like TripAdvisor. This is attributed to a sense of reciprocity and the potential for retaliation. In the context of Airbnb, the guest-host relationship creates a “slightly gray area” where guests may be hesitant to leave negative reviews to avoid jeopardizing the host’s future business. Airbnb attempts to mitigate this risk by concealing reviews from each other before they are published, but the potential for retaliation remains a subconscious influence.

The Role of Reciprocal Reviews

The conversation explicitly acknowledges the reciprocal nature of reviews on platforms like Airbnb, where both guests and hosts review each other. This creates a dynamic where both parties are incentivized to provide positive feedback.

Notable Quote

“It's better than the alternative. So, you know, that's where you have to start, right? An absence of information is definitely worse than um the situation we have now. They are more useful than nothing.” – Andrew, emphasizing the relative value of online reviews despite their flaws.

Technical Terms

• J-Curve Distribution: A statistical distribution where a large proportion of data points cluster at the extremes (low and high values) with fewer points in the middle.
• Self-Selection Bias: A systematic error in statistical sampling that occurs when the individuals selected for a study are not representative of the population being studied.
• Reciprocity: A social norm where individuals respond to positive actions with positive actions, and negative actions with negative actions.

Logical Connections

The conversation progresses logically from identifying the fundamental problem of subjectivity to exploring the underlying causes (J-curve distribution, self-selection bias) and proposing solutions (focusing on high-volume reviewers). It then expands to consider the influence of platform-specific norms and the impact of reciprocal reviews.

Conclusion

While online reviews are not a perfect source of information, they remain a valuable tool for consumers. However, it’s crucial to approach them with a critical mindset, recognizing their inherent biases and limitations. Focusing on reviewers who consistently provide feedback, and being aware of platform-specific norms, can help mitigate these biases and improve the reliability of the information obtained. The key takeaway is to supplement online reviews with other sources of information and exercise informed judgment.