Feature Importance And Model Performance Analysis Of Logistic Regression And Multilayer Perceptron In Alternative Credit Classification With Telecommunications Data

Almas Mirzandi Ramadhan, Dr. Mardhani Riasetiawan, SE Ak, M.T.

2025 | Skripsi | ILMU KOMPUTER

In the evolving landscape of financial services, the need for fair and transparent credit scoring models is growing, especially in regions striving for greater financial inclusion. In the world of credit scoring, credit classification models are employed as one of the aspects of assessment. It will assess whether a loan receiver will default or not. Default, in this case is when a loan receiver fails to pay back the loan that was given by the related financial institution.

 This research examines how telecommunications data can enhance credit classification by comparing the performance and interpretability of traditional Logistic Regression (LR) and Multilayer Perceptron (MLP) models.

Focusing on Indonesia, the study addresses challenges in using alternative data. By exploring the differences in feature importance between these models, the research sheds light on their ability to provide actionable insights for creditworthiness assessment. Performance is evaluated through key metrics, ensuring a balanced view of each model's strengths and limitations. The results highlight the potential of alternative data to support financial inclusion efforts, offering pathways to more transparent and effective credit scoring solutions.

In summary, the research reveals that the MLP model consistently outperforms LR. Without SMOTE, MLP achieved an AUC of 0.6618 and a K-S score of 0.2688, compared to LR’s AUC of 0.628 and K-S of 0.2081; with SMOTE, these improved to 0.7443 and 0.3750 for MLP, and 0.7293 and 0.3649 for LR. Both models heavily rely on the average amount of transaction per day as their top predictor—showing SHAP ranges of about ?0.06 to +0.07 in MLP and ?0.04 to +0.05 in LR—while variables regarding the products people buy (comprising over 25% of features) and a 360-day behavioral window also significantly contribute to predicting default risk.

In the evolving landscape of financial services, the need for fair and transparent credit scoring models is growing, especially in regions striving for greater financial inclusion. In the world of credit scoring, credit classification models are employed as one of the aspects of assessment. It will assess whether a loan receiver will default or not. Default, in this case is when a loan receiver fails to pay back the loan that was given by the related financial institution.

 This research examines how telecommunications data can enhance credit classification by comparing the performance and interpretability of traditional Logistic Regression (LR) and Multilayer Perceptron (MLP) models.

Focusing on Indonesia, the study addresses challenges in using alternative data. By exploring the differences in feature importance between these models, the research sheds light on their ability to provide actionable insights for creditworthiness assessment. Performance is evaluated through key metrics, ensuring a balanced view of each model's strengths and limitations. The results highlight the potential of alternative data to support financial inclusion efforts, offering pathways to more transparent and effective credit scoring solutions.

In summary, the research reveals that the MLP model consistently outperforms LR. Without SMOTE, MLP achieved an AUC of 0.6618 and a K-S score of 0.2688, compared to LR’s AUC of 0.628 and K-S of 0.2081; with SMOTE, these improved to 0.7443 and 0.3750 for MLP, and 0.7293 and 0.3649 for LR. Both models heavily rely on the average amount of transaction per day as their top predictor—showing SHAP ranges of about ?0.06 to +0.07 in MLP and ?0.04 to +0.05 in LR—while variables regarding the products people buy (comprising over 25% of features) and a 360-day behavioral window also significantly contribute to predicting default risk.

Kata Kunci : Logistic Regression, Multilayer Perceptron, SHAP values, Credit Scoring, Alternative Credit Scoring, Feature Importance, Model Explainability, Financial Inclusion