12
Hot take: algorithm bias is actually a people problem first
I was at a teacher conference last month in Portland and sat in on a talk about AI grading tools. The speaker showed how one system marked essays from minority students as lower quality 15 percent more often. But here's the thing, she traced it back to the training data which was pulled from old teacher grades that already had bias baked in. So the algorithm was just copying what we humans did in the first place. Has anyone else run into a situation where the fix needed to start with people not the code?
3 comments
Log in to join the discussion
Log In3 Comments
the_max1mo ago
Hard disagree honestly. The bias is in the code and the data selection process itself. Those old teacher grades were a choice, someone decided what data to feed the model maybe they picked a flawed dataset because it was convenient or cheap. The algorithm doesn't think or copy, it just does math on the inputs it gets. If you give it bad data you get bad results, but the people who built the system should have known better and tested for that. Fixing the people who wrote the old grades doesn't help when the real problem is the engineers who didn't filter their training data properly.
8
robert6591mo ago
I mean, you're right that the data matters a lot, but it's not quite that simple. The bias isn't just in the data or the engineers picking bad grades. It's also in how the algorithm itself learns patterns from that data, even if the data is cleaned up. Like, if you train a model on test scores that mostly come from schools with more funding, it can still pick up on stuff like "higher income neighborhoods = smarter students" and then amplify that. And the engineers can filter data all they want, but if the original data collection was biased to begin with, you can't just scrub that out perfectly. It's more like the algorithm is a mirror that reflects back what society already did wrong, not just a math bot copying bad input.
4