28
Remember when AI bias was just a theory? Now my city's hiring algorithm got flagged for discrimination.
Two years ago it was all talk in papers, but after the Denver audit found it screening out applicants from certain zip codes, it's a real lawsuit. Is this a tech problem or a people problem, and who should fix it?
3 comments
Log in to join the discussion
Log In3 Comments
lily572mo ago
My apartment's rent approval software got me thinking about this last month. It asked for my old address and I swear it slowed down when I put in my college neighborhood. Gavin228 is right that it's a people problem baked into tech, because that bias came from somewhere. I see it now in small ways, like how my friend's mom keeps getting ads for lower paying jobs than my dad does. The fix has to start with the people who make the rules, not just the people who write the code. They need to ask why those zip codes mattered in the first place, before they even turn on the computer.
10
shanef342mo ago
Man, that's exactly why you need to audit the training data first. We had a similar scare with a client's resume screener. The fix isn't just new code, it's digging into the old hiring records they fed the system. Those zip codes didn't just pop up by magic, the algorithm learned from past biased decisions. So the city has to clean its own data while the tech team rebuilds the model, or this just happens again.
3
gavin2282mo ago
Yeah, that's a tough spot for your city. It's wild how these systems can pick up on our own bad patterns without anyone meaning for it to happen. Feels like a people problem that got baked into a tech problem, so both sides need to clean it up. The companies building the tools and the folks using them have to own this together.
2