User feedback changed how I train my recommendation model

I run a small e-commerce site that uses an algorithm to suggest products. Last month, a customer emailed me saying our recommendations kept pushing baby toys to her. She had never shopped for kids stuff. I checked the logs and realized the model was picking up on her browsing of gifts for a friend's baby shower. That one email made me add a session filter so short term browsing doesn't mess up long term preferences. It also got me thinking about how our training data lumps together different intents. Has anyone else had to adjust their model because of a single user complaint?

2 comments

2 Comments

the_susan9d ago

That part about one complaint being a "loud minority thing" is worth a closer look. The user wasn't complaining about something small like a wrong color suggestion. She was getting recommendations that assumed she had a baby, which is a pretty big mistake for an e-commerce site to make. If you ignore that kind of feedback, you end up training your model on bad data that doesn't match what people actually want. One complaint can tell you about a pattern that affects way more people who just never bother to write in. It is not about the single voice being loud, it is about what that voice reveals about a flaw in your whole approach.

nathan1009d ago

Nah, I used to think one complaint was just a loud minority thing. But that baby shower example is exactly the kind of edge case that ruins your whole model if you ignore it. Totally changed my view on listening to single user feedback, even if it seems small.