PESAT: PERSONALISED ENGLISH-SPEAKING AGENTIC TRAINER A Multi-Agent Orchestration Framework for Rubric-Aligned and Localized IELTS Speaking Training for Indonesian EFL Learners

Ismail Seto Adi, Dr. Mardhani Riasetiawan, SE Ak, M.T

2026 | Skripsi | ILMU KOMPUTER

INTISARI

Tes IELTS Speaking merupakan tantangan bagi banyak pembelajar Indonesia karena menilai kefasihan dan koherensi, kosakata, tata bahasa, serta pelafalan secara bersamaan, sementara umpan balik yang jelas dan terjangkau sulit diperoleh, terutama di luar kota-kota besar dengan jumlah tutor IELTS berkualifikasi yang terbatas.

Skripsi ini memperkenalkan PESAT (Personalised English-Speaking Agentic Trainer), sebuah sistem pembelajaran formatif berbasis kecerdasan buatan bagi pembelajar EFL Indonesia. PESAT tidak dirancang terutama sebagai sistem penilaian otomatis; estimasi band-nya merupakan salah satu fitur pendukung yang membantu menyusun umpan balik berbasis bukti dan mengidentifikasi area perbaikan utama setiap pembelajar. Setiap sesi menyediakan penjelasan yang disesuaikan bagi pembelajar Indonesia, rekomendasi latihan terarah, playback heatmap, serta catatan kemajuan dari waktu ke waktu. Estimasi tersebut hanya bersifat panduan dan bukan skor IELTS resmi.

PESAT menerapkan IELTS Speaking Band Descriptors resmi yang tersedia untuk publik melalui large language model secara zero-shot, tanpa contoh berskor, fine-tuning, maupun algoritme penilaian khusus. Multi-Agent Orchestrator berbasis LangGraph mengoordinasikan agen-agen terspesialisasi: Interviewer Agent menjalankan simulasi tes tiga bagian dengan pewaktuan, Input Processing Agent mempertahankan keraguan, pengulangan, kesalahan, kata yang tidak pasti, dan audio asli, empat Evaluation Agent menilai kriteria IELTS secara paralel, dan Analyst Agent mengambil penjelasan interferensi bahasa pertama (L1) Indonesia yang terverifikasi serta aktivitas latihan dari basis pengetahuan yang dibekukan. Audit deterministik lima pemeriksaan memverifikasi skema yang disyaratkan, sumber deskriptor resmi atau basis pengetahuan, bukti pembelajar, dan konfigurasi terekam pada setiap hasil; keluaran yang gagal tidak ditampilkan. Pemeriksaan ini menjamin ketertelusuran, bukan akurasi penilaian, sehingga estimasi juga dievaluasi secara empiris.

PESAT merupakan alat formatif, bukan pengganti asesmen resmi. Purwarupa (MVP) sistem ini lulus 87 pengujian otomatis yang mencakup provenance, pemrosesan pelafalan, isolasi antarpengguna, dan persistensi data; eksekusi berulang pada konfigurasi beku cfg-1.1 menghasilkan skor identik pada keempat kriteria. Sebuah tolok ukur awal juga membandingkan estimasi keseluruhan PESAT dengan skor penguji terpublikasi dari 17 sampel resmi IDP IELTS.

Kontribusi utama skripsi ini adalah kerangka pelatihan yang dapat diaudit, terpersonalisasi, dan terlokalisasi secara budaya, yang menjadikan latihan IELTS Speaking berbasis bukti lebih mudah diakses oleh pembelajar Indonesia.

ABSTRACT

The IELTS Speaking test is challenging for many Indonesian learners because it assesses several abilities at the same time: fluency and coherence, vocabulary, grammar, and pronunciation. Many learners also have difficulty accessing clear and affordable feedback, especially outside major cities where qualified IELTS tutors may be limited.

This thesis presents PESAT, the Personalised English-Speaking Agentic Trainer, an AI-based formative learning system for Indonesian EFL learners. As an innovative-solution thesis, PESAT is not mainly designed as an automated scoring system. Its band estimates are only one supporting feature. They help the system organise evidence-based feedback and identify the learner's main areas for improvement. Each session also provides explanations tailored to Indonesian learners, targeted practice recommendations, a playback heatmap, and a record of progress over time. The estimates are provided only for guidance and are not official IELTS scores.

PESAT applies the official public IELTS Speaking Band Descriptors through a zero-shot large language model. It does not use scored examples, fine-tuning, or a custom scoring algorithm. Because band estimates may vary across models, rubrics, accents, recording conditions, and speaking samples, this component was evaluated directly. The system records and checks the source of every band estimate, explanation, and recommendation. A deterministic five-check audit verifies the required schema, official descriptor or knowledge-base source, learner evidence, and recorded configuration. Results that do not pass are withheld or marked as insufficient. A real-time telemetry HUD shows the pipeline while it runs, and the “Verify now” and “Inject fault” controls allow an observer to repeat the checks against stored data and test whether the audit detects deliberate errors. These checks establish traceability, not scoring accuracy, so the estimates were also evaluated empirically.

PESAT uses a LangGraph-based Multi-Agent Orchestrator to coordinate specialised agents. The Interviewer Agent runs a timed three-part mock test, while the Input Processing Agent preserves hesitations, repetitions, errors, uncertain words, and original audio. Four Evaluation Agents assess IELTS criteria in parallel using official Band Descriptors and learner evidence. The Analyst Agent then identifies recurring errors and retrieves verified Indonesian L1 explanations and practice activities from frozen knowledge bases.

PESAT is a formative IELTS Speaking tool, not a replacement for official assessment. It provides criterion-level band estimates, timestamped feedback, localised explanations, targeted practice, playback heatmaps, and session history. The MVP passed 87 automated tests covering provenance, pronunciation processing, cross-user isolation, and data persistence. Under configuration cfg-1.1, repeated runs produced identical scores across all four criteria. A preliminary benchmark also compared PESAT’s overall estimates with published examiner scores from 17 official IDP IELTS samples.

The main contribution is an auditable, personalised, and culturally localised training framework that makes evidence-based IELTS Speaking practice more accessible to Indonesian learners.

Kata Kunci : IELTS Speaking, Indonesian EFL Learners, Formative Learning, Auditability, Personalised Feedback, Multi-Agent Orchestration, Zero-Shot Large Language Model, Retrieval-Augmented Generation, First-Language Interference, Targeted Practice Recommendations.