Why does nobody talk about how long it takes to clean biased training data

I spent three weeks trying to fix a hate speech filter for a chatbot project. I thought it would take a weekend but every time I removed one bad example, two more popped up from the same dataset. Finally after 22 days of going through 15,000 flagged comments, I found the problem was in how the data was labeled originally. Has anyone else run into training data that just keeps spiraling out of control?

2 comments

2 Comments

claire_davis3114d ago

oh man, I spent a whole month untangling a toxicity dataset that somehow kept getting worse the more I cleaned it

jamie_webb6714d ago

@claire_davis31 I feel your pain on that one, I really do. My own little cleaning project turned into a full blown spiral too, where I'd delete a batch of bad entries and somehow the remaining ones started looking even worse by comparison (like cleaning one room in a hoarder house only to notice the rest more). The weirdest part was when I realized I was accidentally introducing new bias by being too aggressive with my own filters. By the end I just sat there staring at the screen, wondering if I was the biased one all along.