Kendali Quadcopter Menggunakan State Feedback dan Auto-Tuning Proportional-Derivative (PD) Controller Berbasis Reinforcement Learning untuk Proses Hovering

Fiorella Averina Gunawan, Ahmad Ataka Awwalur Rizqi, S.T., Ph.D; Dr.Eng. Ir. Adha Imam Cahyadi, S.T., M.Eng., IPM.

2024 | Skripsi | TEKNIK ELEKTRO

Quadcopter merupakan salah satu jenis robot bergerak berupa pesawat tanpa awak (unmanned aerial vehicle atau UAV). Quadcopter dapat membantu manusia dalam berbagai hal, seperti pemantauan area yang sulit dijangkau. Walaupun quadcopter memiliki banyak kegunaan, terdapat kesulitan dalam mengendalikannya karena dinamikanya yang tidak linear. Kendali dengan state feedback melibatkan linearisasi yang dapat menghilangkan sebagian dinamika internal sistem. Kendali dengan proportional-integral-derivative (PID) controller memiliki tantangan pada proses tuning parameter gain-nya, dan seberapa tahan sistem kendali tersebut terhadap gangguan. Kendali dengan adaptive filter serta kendali keseluruhan sistem quadcopter dengan reinforcement learning (RL) memiliki beban komputasi yang berat.

Oleh karena itu, pada penelitian ini, dikembangkan sebuah metode yang berupa gabungan antara state feedback serta auto-tuning proportional-derivative (PD) controller berbasis RL untuk mengendalikan quadcopter agar dapat melakukan hovering. State feedback bertanggung jawab dalam pengendalian ketinggian (altitude), dan auto-tuning PD controller mengendalikan orientasi (attitude) dari quadcopter. Dengan demikian, tuning PD controller dapat terbantu RL, dan data yang terlibat dalam proses training model RL menjadi lebih sedikit. Sistem kendali ini juga tidak melibatkan linearisasi. Algoritma RL yang digunakan adalah Proximal Policy Optimization (PPO). Metodologi penelitian meliputi instalasi NVIDIA driver dan CUDA Toolkit untuk menunjang proses training, persiapan Python dan dependencies pada sebuah virtual environment, pembuatan program environment Gym, training model RL, testing model RL hasil training, serta evaluasi sistem kendali hasil training dan analisis.

Hasil penelitian menunjukkan bahwa gabungan antara state feedback dan auto-tuning PD controller berbasis RL sudah mampu mengendalikan ketinggian dan orientasi quadcopter untuk proses hovering. Lalu, PD controller berbasis RL sudah dapat menemukan keseimbangan antara respon transien sistem (yang dievaluasi dengan penghitungan rise time serta settling time) serta energi kinetik yang dibutuhkan (yang dianalisis dengan penghitungan jumlah kuadrat torsi).

Quadcopter is a type of unmanned aerial vehicle (UAV), belonging to the mobile robotics family. Quadcopter is helpful for humans in many areas, such as performing surveillance on areas that are hard to reach. Despite its beneficial advantages, there are challenges in controlling of a quadcopter, due to its inherent non-linear dynamics. State feedback control involves linearisation that could eliminate some of the system’s internal dynamics. A proportional-integral-derivative (PID) controller has problems regarding its gain parameters’ tuning process, and its robustness against disturbances. Adaptive filter control and control based on reinforcement learning (RL) for the whole quadcopter system are both demanding in terms of computational load.

In this research, the combination of a state feedback and an RL-based auto-tuning proportional-derivative (PD) controller is developed to control a quadcopter’s hovering. While the state feedback is responsible for altitude (elevation) control, the auto-tuning PD controller handles the attitude (orientation) control. That way, RL could assist the tuning process of the PD controller, and reduce the data required for model training. Linearisation is also not involved in this control system. The RL algorithm used is Proximal Policy Optimization (PPO). The research methodology includes the installation of NVIDIA driver and CUDA Toolkit to enhance RL training performance, preparation of a Python virtual environment that contains dependencies, development of a Gym environment program, training and testing of the RL model, and evaluation as well as analysis of the control system’s performance.

Results show that combination of state feedback and an RL-based auto-tuning PD controller could effectively maintain both of the quadcopter’s altitude and attitude for hovering. Moreover, the RL-based auto-tuning PD controller could also find the balance between the system’s transient response (evaluated by calculating rise time and settling time) and the required kinetic energy (analysed by computing the sum of squared torques).

Kata Kunci : Autonomous aerial vehicle, State feedback, PD control, Reinforcement learning, Robot learning