AI-powered lending will make financial deals even more unfair for women – here’s how to avoid it

It is well known that women on average receive less favorable lending terms than men from salespeople when they borrow money. This study of lending practices at U.S. car dealerships recently confirmed this. This has also been observed for many years in the field of bank lending and mortgage lending around the world.

Academic studies suggest that salespeople may offer women less favorable terms, believing they know less about the market and are therefore less able to assess whether they are getting a fair deal. It could also be that women are penalized because they are not as assertive as men.

An increasingly pressing question is how artificial intelligence (AI) will impact this sector as it plays a larger role in lending. While banks and other lenders may be coy about the extent to which they are using machine learning and generative AI in lending, it is already happening behind the scenes and is expected to become much more prominent over the next couple of years.

You might think that AI could reduce discrimination against women in credit, perhaps by counteracting bias among sales reps. In fact, a new study from my research group suggests that the situation could get worse. So why is this, and can it be avoided?

Our study looked at more than 50,000 auto loans in Canada and found new evidence of discrimination against women in credit. In credit research, the standard method for comparing loans is known as “expected utility.”

This measure measures the utility of a loan to a borrower by taking into account factors such as the interest rate, the likelihood of approval, and the effort the seller puts into the borrower. We found that the expected utility of loans was 68% lower for women than for men.

To see how AI could change the automotive industry, which is still in the early stages of adoption, we explored how machine learning could optimize the commissions lenders pay salespeople for arranging loans for car buyers. Commissions play a critical role in auto lending, influencing sales reps’ decisions on loan pricing and accounting for a substantial portion of dealership revenue.

In an ideal world, integrating AI into this process might automate loan pricing, removing salespeople from the equation, and eliminating their commissions. In reality, competition among lenders is fierce enough, and dealers make so much money from commissions, that they would likely prefer to do business with other lenders. So it’s unlikely that the loan commission model will change, either in the auto industry or in consumer lending in general.

Instead, lenders could use machine learning to optimize fees so that sales reps choose loan rates that generate higher expected profits for the lender and are motivated to put enough effort into the customer to get them to accept the deal. By doing so, we found that lenders could increase their profits by 8%. This, of course, comes at the expense of customers. We found that the expected utility of loans to customers decreases by 20% in this scenario.

However, when we compared male and female borrowers, we found that the decline was 42% for women, compared to just 17% for men. We didn’t test what exactly was happening, but it’s reasonable to assume that since the historical data was “contaminated” by dodgy loan offers to women, the AI ​​made this worse by assuming that women are more tolerant of worse offers than men.

The workaround

This confirms long-held fears among some industry observers that AI could end up widening discrimination in lending, not just to women but also to other groups that benefit from less favorable lending terms, such as some ethnic minorities.

It could be argued that the prudent thing for lenders to do is to stay away from AI. But we wondered if there was a compromise. Could we encourage lenders to use AI more responsibly, to change the trade-off between profits and social justice?

We tested this theory in our study by programming the machine learning algorithm to maximize profits without degrading the expected utility of loans for women. In other words, utility decreased only for men. Under this restriction, we found that lenders could still increase their profits by 4%.

This shows that, if used thoughtfully, AI can both benefit lenders and protect disadvantaged groups. In response to those who would prefer to keep AI out of financial services, it might be better to accept its inevitability and use it instead as a tool to make lending fairer.