Adaptive Indoor Digital Signage Using Audience Context Analysis and Emotional Feedback

Lana Fitria Dewani, Dr. Ir. Rudy Hartanto, M.T., IPM.; Dr. Ir. Ahmad Nasikun, S.T., M.Sc., IPM., ASEAN Eng.

2026 | Skripsi | TEKNOLOGI INFORMASI

Perkembangan Digital Out-of-Home (DOOH) dan digital signage mendorong kebutuhan terhadap sistem reklame digital yang lebih adaptif dan responsif terhadap kondisi audiens. Namun, sebagian besar reklame digital indoor masih beroperasi secara satu arah dan belum mampu menyesuaikan konten berdasarkan kondisi aktual audiens maupun mengevaluasi respons emosional terhadap iklan yang ditampilkan. Penelitian ini bertujuan mengembangkan prototipe papan reklame digital interaktif di dalam ruang berbasis analisis visual audiens yang mampu melakukan adaptasi konten iklan secara near real-time dengan tetap menjaga privasi melalui pendekatan non-identifikasi.

Metode penelitian menggunakan pendekatan pengembangan prototipe berbasis computer vision dengan arsitektur modular yang terdiri atas Expression Engine, Context Engine, Ad Decision Engine, Recommendation Stabilizer, dan Switch Decision. Sistem memanfaatkan analisis ekspresi wajah, deteksi audiens valid, estimasi demografi agregat, analisis tingkat keramaian, deteksi objek kontekstual, dan atribut visual sebagai dasar pengambilan keputusan iklan berbasis rule-based scoring. Pengujian dilakukan melalui eksperimen model ekspresi wajah, pengujian fungsional modul, pengujian integrasi, serta evaluasi performa real-time pada beberapa perangkat komputasi.

Hasil penelitian menunjukkan bahwa sistem berhasil melakukan adaptasi konten iklan berdasarkan kondisi audiens dan lingkungan secara near real-time. Model ResNet-18 pada dataset Balanced AffectNet tiga kelas memperoleh performa terbaik pada data citra dengan accuracy sebesar 86,49%  dan macro F1-score sebesar 85,32%. Namun, pada pengujian video real-time, model FER library/original memberikan performa terbaik dengan rata-rata accuracy sebesar 74,95%. Sistem juga berhasil menerapkan mekanisme stabilisasi rekomendasi dan pergantian iklan sehingga perubahan iklan berlangsung lebih stabil tanpa memotong video yang sedang diputar.

Penelitian ini menyimpulkan bahwa data visual non-identifikasi dan respons emosional audiens dapat dimanfaatkan untuk mendukung reklame digital adaptif yang kontekstual, responsif, dan menjaga privasi pengguna. Hasil penelitian diharapkan menjadi kontribusi awal bagi pengembangan sistem reklame digital berbasis computer vision yang seimbang antara adaptivitas, kemampuan real-time, efisiensi komputasi, dan perlindungan privasi visual. 

The rapid growth of Digital Out-of-Home (DOOH) and digital signage technologies has increased the demand for advertising systems that are adaptive and responsive to audience conditions. However, most indoor digital advertising systems still operate in a one-way manner and lack the ability to adjust content according to real audience conditions or evaluate audience emotional responses to displayed advertisements. This study aims to develop an indoor interactive digital billboard prototype based on visual audience analysis that can adapt advertisement content in near real-time while preserving privacy through a non-identification approach.

This research adopts a computer vision-based prototyping approach using a modular architecture consisting of the Expression Engine, Context Engine, Ad Decision Engine, Recommendation Stabilizer, and Switch Decision modules. The system utilizes facial expression analysis, valid audience detection, aggregate demographic estimation, crowd analysis, contextual object detection, and visual attributes as inputs for rule-based advertisement decision making. The system was evaluated through facial expression model experiments, functional testing, integration testing, and real-time performance evaluation across multiple computing devices.

The results demonstrate that the system successfully adapts advertisement content based on audience and environmental conditions in near real-time. The ResNet-18 model trained on the three-class Balanced AffectNet dataset achieved the best image-based performance with an accuracy of 86.49% and a macro F1-score of 85.32%. However, in video-based real-time testing, the original FER library achieved the highest average accuracy of 74.95%. The proposed system also successfully implemented recommendation stabilization and advertisement switching mechanisms, enabling stable content transitions without interrupting the currently playing advertisement.

This study concludes that non-identifiable visual data and audience emotional responses can support adaptive digital advertising systems that are contextual, responsive, and privacy-preserving. The proposed approach contributes to the development of computer vision-based digital advertising systems that balance adaptability, real-time capability, computational efficiency, and visual privacy protection.

Kata Kunci : Computer Vision, Digital Signage, Facial Expression Recognition, Adaptive Advertising, Audience Analysis