Analisis Performa Bidders Recommender dengan Clustering Data Penyedia Dan KBKI pada Beberapa Metode Machine Learning. Studi Kasus: Pengadaan Publik di Indonesia

Ribut Aji Kasmiadi, Dr.Eng. Silmi Fauziati, S.T., M.T.; Dr. Indriana Hidayah, S.T., M.T.

2024 | Tesis | S2 Teknologi Informasi

Pengadaan publik di Indonesia, meskipun telah dilakukan secara elektronik melalui Sistem Pengadaan Secara Elektronik (SPSE), masih sering mengalami kendala, salah satunya yaitu kurangnya partisipasi penyedia barang/jasa. Berdasarkan laporan LKPP, pada tahun 2022 baru sekitar 45,52% penyedia yang terdaftar dalam SPSE yang aktif ikut dalam proses pengadaan. Bidders Recommender memanfaatkan algoritme Random Forest untuk memprediksi pemenang tender, kemudian memberikan daftar rekomendasi penyedia yang memiliki kompetensi sejenis atau lebih baik dibandingkan penyedia terprediksi untuk ikut serta dalam proses pengadaan. Bidders Recommender sangat bergantung kepada data pengadaan dan data penyedia pada suatu negara sehingga tidak dapat diterapkan secara langsung, butuh penyesuaian data.  

Pada penelitian ini dilakukan penyesuaian data, replikasi, serta pengujian beberapa algoritme klasifikasi untuk implementasi Bidders Recommender pada data pengadaan di Indonesia. Penyesuaian data dilakukan dengan menambahkan label Klasifikasi Baku Komoditi Indonesia (KBKI), clustering data penyedia, dan mengubah algoritme menjadi multi-label classificiation. Algoritme machine learning yang dibandingkan adalah Extreme Gradient Boosting (XGBoost), Neural Network, Light Gradient Boosting (Light GBM), Adaptive Boosting (AdaBoost), dan Stacking method yang menggabungkan Light GBM dan AdaBoost serta menggunakan K-NN sebagai final estimator. Setiap algoritme diuji sebanyak 10x dan diukur performa masing-masing dengan pengaturan n_cluster = 12, 25, 50, dan 100, menggunakan metrik precision, recall, dan f1-score, dengan metode penghitungan rata-rata yaitu micro average, macro average, dan weighted average. Hasil penghitungan performa dibandingkan menggunakan uji statistik ANOVA dan uji post-hoc menggunakan Tukey’s Test.

Berdasarkan hasil penelitian, saat replikasi, accuracy yang dihasilkan lebih rendah dibandingkan penelitian sebelumnya, yaitu 0,25%. Saat memanfaatkan dataset penawar pengadaan, accuracy meningkat menjadi 20%. Kemudian setelah dilakukan clustering dan diubah menjadi multi-label,  performa Bidders Recommender yaitu precision 59,96% hingga 67,33%, recall 13,24% hingga 41,97%, dan f1-score 21,69% hingga 51,71%. Dengan n_cluster = 12, memberikan performa paling baik secara signifikan pada seluruh algoritme yang dibandingkan. Algoritme Light GBM (57,42% - 64,51%), XGBoost (62,97% - 64,27%), dan Neural Network (61,35% - 63,66%) memberikan performa lebih baik. Namun pada metrik recall dan f1-score belum ada metode yang lebih baik secara signifikan dibandingkan Random Forest. 

Public procurement in Indonesia, even though it has been carried out electronically through the Electronic Procurement System (SPSE), it still faces obstacles, i.e. the lack of participation of goods/service providers. Based on the LKPP report, in 2022 only around 45.52% of providers registered in SPSE actively participated in the procurement process. Bidders Recommender utilizes the Random Forest algorithm to predict tender winners, then provides a list of recommendations for providers who have similar or better competencies than the predicted providers to participate in the procurement process. Bidders Recommender is very dependent on procurement data and provider data in a country so it cannot be applied directly, it requires data adjustments.

In this research, data adjustment, replication, and testing of several classification algorithms were carried out to implement Bidders Recommender on procurement data in Indonesia. Data adjustments were made by adding Indonesian Commodity Standard Classification (KBKI) labels, clustering providers data, and changing the algorithm to multi-label classification. The compared machine learning algorithms were Extreme Gradient Boosting (XGBoost), Neural Network, Light Gradient Boosting (Light GBM), Adaptive Boosting (AdaBoost), and Stacking method which combines Light GBM and AdaBoost and using K-NN as final estimator. Each algorithm was tested 10x and its performance was measured respectively with settings n_cluster = 12, 25, 50, and 100, using the precision, recall, and f1-score metrics, with the average calculation method: micro average, macro average, and weighted average. The performance calculation results were compared using the ANOVA statistical test and post-hoc test using Tukey's Test.

Based on the research results, during replication, the accuracy was lower than previous research, which was 0.25%. When utilizing the procurement bidder's dataset, accuracy increased to 20%. Then after doing clustering and changing it to multi-label, the precision of Bidders Recommender was 59.96% to 67.33%, recall 13 .24% to 41.97%, and f1-score 21.69% to 51.71%. With n_cluster = 12, it provided significantly better performance across all compared algorithms. The algorithms Light GBM (57.42% - 64.51%), XGBoost (62.97% - 64.27%), and Neural Network (61, 35% - 63.66%) provided better performance. However, for the recall and f1-score metrics, there was no method that was significantly better than Random Forest.

Kata Kunci : Pengadaan publik, e-procurement, machine learning, Random Forest