Had to choose between a black box model and an interpretable one for a loan approval system - picked the interpretable one and it was a nightmare

We were building a loan approval tool for a credit union in Portland last spring. The black box model gave us 92% accuracy on test data, but the interpretable logistic regression only hit 84%. Found out later that the black box was picking up on zip codes as a proxy for race (which we thought we had removed). The interpretable model had a hard time with some edge cases though, like people with thin credit files. We ended up sticking with the simpler model because we could actually explain denials to customers. Has anyone else dealt with this accuracy vs fairness trade-off in a real deployment?

2 comments

2 Comments

lily_torres3113d ago

White box model only hit 84%" - yeah that's a pretty big drop. Most people just want their loan approved not a lecture on algorithmic fairness. Real question is how many of those denied actually complained vs how many just moved on.

ivanbell12d ago

My buddy got denied last month by one of those black box models and he didn't even bother to appeal, just figured it wasn't worth his time. That's probably the case for a lot of people, they just move on and maybe try another lender or assume they'll get denied elsewhere too. The actual complaint numbers don't tell you anything about the real frustration people feel, it's just a quiet kind of defeat.