Saw a hiring algorithm in Phoenix toss out every resume from a certain community college

I was helping a friend's small tech firm in Phoenix set up a new resume filter last month, and we found it was automatically rejecting every single applicant from the local community college, about 200 people. The system's training data was full of resumes from big name schools, so it just learned to ignore anything else. Has anyone else seen a filter that basically built its own unfair zip code rule?

4 comments

4 Comments

michaelgrant2mo ago

Did you try adding a bunch of resumes from smaller schools to the training data?

wyatt1352mo ago

That's not a zip code rule, it's a bias from bad training data.

robertcarr2mo ago

Man, that's so messed up. It's scary how these filters just copy our worst habits. I saw something similar where a system kept picking guys named "John" for sales jobs. @michaelgrant has the right idea about fixing the data, but how many good people get passed over before anyone notices? It's like the algorithm decided that one school's name meant "not good enough" without ever looking at a single skill.

jessej2316d ago

Flipped through a design job board recently and saw a posting that literally said "Rutgers grads and similar pedigree only" in the requirements. Like, not even hidden behind some algorithm, just right out there in plain text. Makes you wonder if someone's still running the whole hiring process based on what college football teams they root for. I know a guy from a no name state school who's probably the best UX designer I've ever worked with, but he'd get filtered out before anyone even saw his portfolio. The whole thing feels like it's designed to miss the actual talent out there.