I was training a hiring algorithm on my own biased resume data for 2 years before I realized it

I work at a mid-size tech firm in Austin and I built our internal screening tool back in 2021. I fed it all the resumes I thought were good from my own hiring history. Last month, I ran a fairness audit and discovered it was ranking candidates from certain universities 40% lower just because I had never hired from those schools. The algorithm was just copying my blind spots. Has anyone else found their own bias baked into a system they designed?

3 comments

3 Comments

irisowens1mo agoTop Commenter

Fixed mine with a simple test. Had it score my own resume against a generic one. Turned out it hated my own alma mater.

the_piper1mo ago

That part about it hating your alma mater really got me thinking, @irisowens. Did you try swapping out just the school name for something generic to see if that was the exact trigger?

owens.anthony1mo ago

So like... did you actually dig into which specific keywords or phrases it was docking you for? Just curious if it was literally the school name or something more subtle...

-1