The Evaluation of the Effects of Contrast Enhancement and Class-Balanced Oversampling on Transfer-Learned Models for Facial Emotion Recognition in the Wild

Azzahra Adissa Sharya Darsono, Wahyono, S.Kom., Ph.D.

2026 | Skripsi | ILMU KOMPUTER

Pengenalan Emosi Wajah (FER) pada kondisi dunia nyata menghadapi tantangan pencahayaan dan ketidakseimbangan kelas, sehingga penelitian ini mengevaluasi pengaruh Contrast Limited Adaptive Histogram Equalization (CLAHE) dan Class-Balanced Oversampling (CBO) pada model VGG-16. Hasil eksperimen pada sepuluh konfigurasi menunjukkan bahwa CLAHE moderat (clip limit 0.5) memberikan kinerja terbaik dengan akurasi 0,6545. Temuan krusial dari uji tambahan (M8-M10) mengungkapkan adanya kekurangan dari oversampling, di mana model CLAHE saja secara konsisten mengungguli model kombinasi pada intensitas kontras yang sama karena CBO cenderung memperkenalkan noise saat sinyal visual dimodifikasi secara agresif. Meskipun demikian, CBO tetap esensial untuk menstabilkan perilaku kelas minoritas, mencapai skor F1 tertinggi pada kelas “Angry”. Sebaliknya, kontras berlebihan (clip limit > 1.0) menurunkan kinerja secara signifikan akibat amplifikasi noise digital. Penelitian ini menyimpulkan bahwa prapemrosesan visual adalah pendorong utama akurasi, sementara penyeimbangan data diperluka

Facial Emotion Recognition (FER) in-the-wild remains challenging due to inconsistent lighting and class imbalance, prompting this study to examine the independent and combined effects of Contrast Limited Adaptive Histogram Equalization (CLAHE) and Class-Balanced Oversampling (CBO) on a transfer-learned VGG-16 model. By conducting ten experimental configurations, the research identified that moderate contrast enhancement (CLAHE clip limit 0.5) yielded the highest overall performance, with an accuracy of 0.6545 and a macro F1-score of 0.6373. Crucially, it was revealed CLAHE-only models consistently outperformed their oversampled counterparts at identical clip limit values, suggesting a disadvantage of CBO where it may introduce noise or overfitting that hampers overall accuracy when the visual signal is heavily modified. Despite this, CBO remained essential for stabilizing minority class behavior, achieving the peak F1-score for the “Angry” class when paired with mild contrast. In contrast, aggressive enhancement (clip limit > 1.0) was found to degrade performance by over-amplifying digital noise and distorting subtle emotional cues. Ultimately, these findings demonstrate that while image clarity via minimal preprocessing is the primary driver of accuracy, balanced sampling is necessary to ensure fair and stable recognition across underrepresented emotion categories in uncontrolled environments. 

Kata Kunci : Facial Emotion Recognition, CLAHE, Class-balanced Oversampling, Deep Learning, Transfer Learning