Should credit checks be automated?

With the success of peer-to-peer lenders like Lending Club and Prosper, many lenders have experimented with alternative mechanisms beyond credit scoring.1 One such alternative is the increased reliance on soft information, which is subjective data that is difficult to interpret without loan officers. Loan officers, however, are not just costly in terms of compensation. Their characteristics – such as being cautious or having low ability – could distort their loan decisions to the detriment of the lender.

I consider the value of these loan officers using loan and repayment data from 2010–2013 for approximately 32,000 borrowers from a Chinese lender. This large lender specialises in unsecured, cash loans to households and small businesses. Loan officers view the borrower’s entire file including financial statements, references, notes, credit scores, and even photographs before choosing an approved loan amount.2

My job market paper calculates the value of hiring these loan officers by comparing them to an alternative where the lender only uses hard information such as income and credit scores (Wang 2014). It is important to note that this is not measuring the value of credit scoring. Risk-based pricing has been extensively used since at least the early 1990s (Johnson 1992) and their effectiveness is not in doubt. Einav et al. (2013) and Edelberg (2006) both provide compelling evidence of the strength of credit scoring versus exclusively subjective underwriting. My paper compares loan officers operating in conjunction with credit scoring versus credit scoring alone.

Despite the fact that loan applications are randomly assigned,3 there are differences across loan officers. Figure 1 shows the average approved loan amount plotted against credit quality. Notice that loan officer A approves a higher loan amount than loan officer B at every level. There are also differences in the variance of loan sizes as well as loan performance. To explain this, I lay out an empirical framework in the paper that can explain their behaviour, accounting for differences in risk attitudes, ability, and even overconfidence.

Figure 1. Average loan amount by credit quality

Notes: The graph displays the average approved loan amounts for 282 loans made by loan officers in June 2012 for a 48% APR, 24 month loan. Borrowers apply for a loan amount with a pre-set APR and payment length, and loan officers decide on an approved loan amount. Over 90% of borrowers are given loans much smaller than the amount applied. Credit quality is the lender’s internal proprietary measure of borrower risk. Higher values indicate safer borrowers.

Using the insights from the empirical framework, I calibrate an algorithm that takes into account only the borrower’s hard information, and then I compare to the loan officers’ actual performance. Figure 2 shows the additional profit per loan in dollars that each loan officer contributed above and beyond this alternative. While some loan officers are not profitable, the main result of the paper is that the average loan officer contributes three times his pay in additional annual profits.4 This implies that these loan officers are much more profitable than the lender operating by itself.

Figure 2. Loan officers’ additional per loan profit over algorithm

Notes: The graph displays the difference between the loan officers’ loan profits compared to an algorithm that only takes into account hard information for approximately 32,000 borrowers. The algorithm is calibrated using over 700 data points about each borrower and the actual repayment data. The average loan officer is paid roughly $11 per loan, and the average additional profit is just over $35.

One may wonder how much of this result comes from the effectiveness of the algorithm. Could a more predictive algorithm outperform these loan officers? The answer is likely to be no, because the algorithm is calibrated using the borrower’s actual repayment data.[5] This idealised algorithm is analogous to picking an investment portfolio in 2013 after observing data from 2014, which means that alternative algorithms developed at the time of a loan’s origination are likely to be worse. In addition, the algorithm takes into account more than 700 data points about each borrower including many that may be excluded in some settings such as gender, age, and ethnicity.

Conclusion

I have argued here that these loan officers are valuable. This is despite their biases and an automated lending model with access to extensive amounts of hard information and repayment data. More broadly, my job market paper provides an empirical framework for evaluating the contributions of subjective expertise that can be applied in other contexts such as asset managers or even admissions counsellors. While experts have been beaten in many fields such as chess and mutual fund management (Gruber 1996), this is one area where man can still beat the machines.

This article is published in collaboration with Vox EU. Publication does not imply endorsement of views by the World Economic Forum. 

To keep up with Forum:Agenda subscribe to our weekly newsletter.

Author: James Wang is a PhD job market candidate in Economics, University of Michigan

Image: A generic picture of a woman in an office using a computer mouse. OFFPO REUTERS/Catherine Benson