Mean reciprocal rank (MRR) is an evaluation metric for ranking models that measures how much crap the user must get through before they find what they actually want. Let
While an excellent MRR is usually not sufficient to satisfy users of a search engine or recommendation system, it is definitely necessary. The primary example that Cory Doctorow gives in his discussion on the “enshittification” of Amazon points to MRR, which approaches zero:
Search Amazon for “cat beds” and the entire first screen is ads. One of them is an ad for a dog carrier, which Amazon itself manufactures and sells, competing with the other sellers who bought that placement.
Scroll down one screen and you get some “organic” results – that is, results that represent Amazon’s best guess at the best products for your query. Scroll once more and yup, another entire screen of ads, these ones labeled “Highly rated.” One more scroll, and another screenful of ads, one for a dog product.
Keep scrolling, you’ll keep seeing ads, including ads you’ve already scrolled past. “On these first five screens, more than 50 percent of the space was dedicated to ads and Amazon touting its own products.” Amazon is a cesspit of ads: twice as many as Target, four times as many as Walmart.