Sentiment Analysis on IMDb Movie Reviews Using TF-IDF+SVM and Word2Vec-BiLSTM Models

Nashifa Ammara Fawziya Muchtar, Moh. Edi Wibowo, S.Kom., M.Kom., Ph.D.

2025 | Skripsi | ILMU KOMPUTER

Dengan meningkatnya popularitas platform ulasan film daring seperti IMDb, ulasan yang dihasilkan oleh pengguna telah menjadi sumber yang berpengaruh terhadap opini publik. Mengklasifikasikan sentimen dari ulasan-ulasan tersebut secara otomatis dapat memberikan wawasan yang berharga bagi penonton maupun pembuat film.

Penelitian ini mengeksplorasi analisis sentimen pada ulasan film IMDb dengan membandingkan kinerja tiga model: Term Frequency-Inverse Document Frequency (TF-IDF) dengan Support Vector Machine (SVM), Word2Vec dengan Bidirectional Long Short-Term Memory (BiLSTM), dan Word2Vec-BiLSTM dengan attention layer. Dua jenis attention mechanism diuji, yaitu additive attention dan scaled dot-product attention. Penelitian ini bertujuan untuk mengevaluasi performa dalam hal akurasi, F1-score, dan training time serta untuk menentukan apakah penambahan attention mechanism dapat meningkatkan performa BiLSTM. Setiap model diuji coba menggunakan 3-fold cross-validation untuk memastikan ketahanan performa hasil.

Hasil penelitian menunjukkan bahwa model TF-IDF+SVM yang ringan secara komputasi mengungguli model Word2Vec-BiLSTM yang lebih kompleks di ketiga metrik, mencapai 89,48% ± 0,17 untuk akurasi dan F1-score, dengan training time selama 25 menit. Model dasar Word2Vec-BiLSTM memiliki performa terburuk dan menunjukkan variabilitas tinggi di seluruh lipatan. Meskipun model Word2Vec-BiLSTM dengan attention mechanism tidak melampaui performa model SVM, keduanya secara signifikan meningkatkan akurasi pada fold dengan performa buruk. Mekanisme additive attention meningkatkan akurasi dari 73,97% menjadi 88,29%, dan F1-score dari 69,94% menjadi 88,49%. Demikian pula, scaled dot-product attention meningkatkan akurasi dan F1-score menjadi 88,38?n 88,60%. Temuan ini menyoroti keefektifan pendekatan tradisional dan manfaat selektif dari attention mechanism dalam menangani pembagian data yang sulit untuk klasifikasi sentiment.

With the rise of online movie review platforms like IMDb, user-generated reviews have become an influential source of public opinion. Automatically classifying the sentiment of these reviews can provide valuable insights for both viewers and filmmakers.

This research explores sentiment analysis on IMDb movie reviews by comparing the performance of three models: Term Frequency-Inverse Document Frequency (TF-IDF) with Support Vector Machine (SVM), Word2Vec with Bidirectional Long Short-Term Memory (BiLSTM), and Word2Vec-BiLSTM with an attention layer. Two attention mechanisms were tested, additive attention and scaled dot-product attention. The study aims to evaluate performance in terms of accuracy, F1-score, and training time, as well as to determine whether the addition of attention improves BiLSTM performance. Each model was trained using 3-fold cross-validation to ensure robustness.

Results show that the lightweight TF-IDF+SVM model outperformed the more complex Word2Vec-BiLSTM models in all three metrics, achieving 89.48% ± 0.17 for both accuracy and F1-score, with a training time of 25 minutes. The baseline Word2Vec-BiLSTM model performed the worst, showing high variability across folds. While the attention-enhanced Word2Vec-BiLSTM models did not surpass the SVM model, they significantly improved results on underperforming folds. Additive attention increased accuracy from 73.97% to 88.29%, and F1-score from 69.94% to 88.49%. Similarly, scaled dot-product attention improved accuracy and F1-score to 88.38% and 88.60%, respectively. These findings highlight the effectiveness of traditional approaches and the selective benefits of attention mechanisms in handling difficult data splits for sentiment classification.

Kata Kunci : Sentiment Analysis,TF-IDF,SVM,Word2Vec Embedding,BiLSTM,Attention Mechanism,Movie Reviews