Evaluation of Neural Network and Logit Models for Classification of Default in Banking Loans

Abstract:

The purpose of the study was to evaluate the performance of neural networks as modern techniques to classify the risk of default against the traditional Logit statistical method, taking a Honduran bank as a case study. The data was obtained from its credit portfolio made up of 38,156 personal loans and 9 available characteristics, choosing the most representative independent variables to design a Multilayer Perceptron type base model and its Logit equivalent to which characteristics were added to analyze their impact on the classification of the dependent variable Default, leaving in the end a network with an input layer of 8 nodes, 4 hidden dense layers of 20 and 24 nodes, a central dropout layer and a node in the output layer as well as an equivalent logistic regression to compare the performance of both. The results with unbalanced data showed a superior performance of the networks, but when applying SMOTE oversampling, although there was no greater impact on the network, there was in the regressions, concluding that these learn to classify loan default better when the data subsets are balanced in the class of the response variable since its new results almost reached those of the neural network, which was finally chosen as the preferred model for its implementation with an accuracy of 99.16%, precision of 99.47%, sensitivity of 99.59%, specificity of 95.48 %, F1 score of 99.53% and ROC and PR curves with AUC of 98.68% and 97.69% respectively.

• Periodical: Engineering Headway (Volume 12)
• Pages: 97-107
• DOI: https://doi.org/10.4028/p-DXRV7c
• Online since: September 2024
• Authors: Walter Jeremías López Flores*
• Keywords: Credit Risk, Loans Default, Logistic Regression, Multilayer Perceptron Neural Network, Supervised Machine Learning

References

[1] A.K. Tiwari, "Machine Learning Application in Loan Default Prediction". Journal NX - A Multidisciplinary Peer Reviewed Journal, vol. 4, no. 5, pp.1-5, 2018. [Online]. Available: https://repo.journalnx.com/index.php/nx/article/view/(2008)

[2] D. M. Obare, G. G. Njoroge, M. M. Muraya, "Analysis of Individual Loan Defaults Using Logit under Supervised Machine Learning Approach". Asian Journal of Probability and Statistics, vol. 3, no. 4, pp.1-12, 2019

DOI: 10.9734/AJPAS/2019/v3i430100

[3] A. Lahsasna, R. N. Ainon, T. Y. Wah, "Credit Scoring Models Using Soft Computing Methods: A Survey". The International Arab Journal of Information Technology, vol. 7, no. 2, pp.115-123, 2010. [Online]. Available: https://iajit.org/PDF/vol.7,no.2/712.pdf

[4] H. A. Bekhet, S. F. Kamel Eletter, "Credit risk assessment model for Jordanian commercial banks: Neural scoring approach". Review of Development Finance, vol. 4, pp.20-28, 2014

DOI: 10.1016/j.rdf.2014.03.002

[5] M. L. Saavedra García, M. J. Saavedra García, "Modelos para medir el riesgo de crédito de la banca". Cuadernos de Administración, vol. 23, no. 40, pp.295-319, (2010)

[6] D.M. Obare, M.M. Muraya, "Comparison of Accuracy of Support Vector Machine Model and Logistic Regression Model in Predicting Individual Loan Defaults". American Journal of Applied Mathematics and Statistics, vol.6, no.6, pp.266-271, 2018

[7] J. W. do Prado, F. D. Carvalho, G. C. Benedicto, A. L. Lima, "Analysis of credit risk faced by public companies in Brazil, an approach based on discriminant analysis, logistic regression and artificial neural networks". Estudios Gerenciales, vol. 35, no. 153, pp.347-360, 2019

DOI: 10.18046/j.estger.2019.153.3151

[8] C.F. Tsai, J.W. Wu, "Using neural network ensembles for bankruptcy prediction and credit scoring". Expert Systems with Applications, vol. 34, pp.2639-2649, 2008

DOI: 10.1016/j.eswa.2007.05.019

[9] D. West, "Neural network credit scoring models". Computers & Operations Research, vol. 27, pp.1131-1152, 2000.

DOI: 10.1016/s0305-0548(99)00149-5

[10] A. Khashman, "Neural Networks for credit risk evaluation: Investigation of different neural models and learning schemes". Expert Systems with Applications, vol. 37, pp.6233-6239, 2010

DOI: 10.1016/j.eswa.2010.02.101

[11] P. Golbayani, D. Wang, I. Florescu, "Application of Deep Neural Networks to assess corporate Credit Rating". arXiv, 2020. [Online]. Available: https://arxiv.org/abs/2003.02334

[12] P. Golbayani, I. Florescu, R. Chatterjee, "A comparative study of forecasting Corporate Credit Ratings using Neural Networks, Support Vector Machines, and Decision Trees". arXiv, 2020. [Online]. Available: https://arxiv.org/abs/2007.06617

DOI: 10.1016/j.najef.2020.101251

[13] S. Haykin, Neural Networks and Learning Machines, 3rd ed, Pearson, 2009.

[14] J. Brownlee, "How to Choose an Activation Function for Deep Learning". Machine Learning. https://machinelearningmastery.com/choose-an-activation-function-for-deep-learning/ (accessed Oct. 5, 2023).

[15] T. Babs, "The mathematics of Neural Networks". Medium. https://medium.com/coinmonks/ the-mathematics-of-neural-network-60a112dd3e05 (accessed Oct. 6, 2024).

[16] J. Brownlee, "Logistic Regression for Machine Learning". Machine Learning. 2023 https://machinelearningmastery.com/logistic-regression-for-machine-learning (accessed Oct. 5, 2023).

DOI: 10.2174/9789815124422123010005

[17] F.E. Salazar Villano, "Cuantificación del riesgo de incumplimiento en créditos de libre inversión: un ejercicio econométrico para una entidad bancaria del municipio de Popayán". Estudios Gerenciales, vol. 29, pp.416-427, 2013.

DOI: 10.1016/j.estger.2013.11.007

[18] P. Jiang, J. Zhang, J. Zou, "Credit Card Fraud Detection Using Autoencoder Neural Network". arXiv, 2019. [Online]. Available: https://arxiv.org/abs/1908.11553

[19] M. Senoae Santos, J. Pompeu Soares, P. Henriquez Abreu, H. Araújo, J. Santos, "Cross-Validation for Imbalanced Datasets: Avoiding Overoptimistic and Overfitting Approaches". IEEE Computational Inteligence Magazine, vol. 13, no. 4, pp.59-76, 2018

DOI: 10.1109/MCI.2018.2866730

[20] D.R. Wilson, T.R. Martínez, "Improved Heterogeneous Distance Functions". Journal of Artificial Intelligence Research, vol. 6, pp.1-34, 1997.

[21] F. Aguilar, "SMOTE-NC in ML Categorization Models for Imbalanced Datasets". Medium Analytics Vidhya. https://medium.com/analytics-vidhya/smote-nc-in-ml-categorization-models-for-imbalanced-datasets-8adbdcf08c25 (accessed Oct. 10, 2023).

[22] J. Davis, M. Goadrich, "The Relationship Between Precision-Recall and ROC Curves". IEEE Proceedings of the 23rd International Conference on Machine Learning - ICML '06, 2006

DOI: 10.1145/1143844.1143874

[23] S. Narkhede, "Understanding Confusion Matrix". Towards Data Science. https://towardsdatascience.com/understanding-confusion-matrix-a9ad42dcfd62 (accessed Oct. 10, 2023).

[24] Z. Lipton, C. Elkan, B. Narayanaswamy, "Thresholding Classifiers to Maximize F1 Score". arXiv, 2014. [Online]. Available: https://arxiv.org/abs/1402.1892

[25] V. Lendave, "Python Guide to Precision-Recall Tradeoff". Analytics India Magazine. https://analyticsindiamag.com/python-guide-to-precision-recall-tradeoff/ (accessed Oct. 11, 2023).

[26] CNBS, "Manual de Reporte de Datos de Crédito". Comisión Nacional de Bancos y Seguros. https://cnbs.gob.hn/wp-content/uploads/2020/02/Ver_2_0_40_MRDC-Capturador-Datos-Credito-SL_NIIF_V_20191209.pdf (accessed Sep. 11, 2023).