Ratings play an enormous role in our lives. Ratings made by critics, judges, and evaluators determine a range of outcomes, from the seemingly trivial (which wine you pick for dinner or which products you buy from Amazon) to the more consequential (which athletes win Olympic gold or which students get into top universities).
Research: Why Ratings on Everything from Wine to Amazon Products Improve Over Time
Ratings play an enormous role in our lives. Ratings made by critics, judges, and evaluators determine a range of outcomes, from the seemingly trivial (which wine you pick for dinner) to the more consequential (which students get into top universities). But how reliable are these ratings? In eight studies, researchers captured more than 12,000 sequential evaluations to see whether ratings changed as the rater gained more experience. The evaluations covered much territory: judges’ scores on the TV show Dancing with the Stars, student grades from university professors, and ratings for short stories and photographs by college students. Researchers also analyzed thousands of Amazon product reviews by devoted reviewers. Across the board, we found the same result: Ratings rise with experience.