Adaptive PID Control for Quadrotor UAVs Using Feedback Linearization and Deep Reinforcement Learning

This repository contains MATLAB and Simulink files for the implementation of an adaptive control strategy for a nonlinear quadrotor UAV. The core objective is to improve trajectory tracking and robustness using a Deep Reinforcement Learning (DRL) agent—specifically,Deep Deterministic Policy Gradient (DDPG) and Twin Delayed Deep Deterministic Policy Gradient (TD3)—to dynamically tune the gains of PID controllers in real time.

---

Project Overview

Quadrotor UAVs are nonlinear, underactuated systems with significant sensitivity to external disturbances. Classical controllers like PID and MPC are limited by their fixed gain structures and offline tuning methods. This project integrates DDPG and TD3 with:

• Adaptive PID Control: TD3 tunes $K_p, K_i, K_d$ gains online.
• Feedback Linearization (FBL): Used to linearize quadrotor dynamics before controller design.
• Simulink Integration: Realistic nonlinear model simulation of a quadrotor using FBL.

---

Directory Structure

├── Environment Setup/
│   ├── Environment_Setup.m         # Defines observation, action, and reward logic
│   ├── ResetFunction.m             # Custom reset logic for episodic training
│   └── TrainingOptions.m           # Training hyperparameters (batch size, steps, etc.)
│   └── TD3Agent_Setup.m            # Configuration of TD3 agent (agent options, networks)

├── Simulink Model/
│   └── FBL_ON_UAV_Final.slx        # Simulink model of feedback-linearized UAV

├── TD3_Networks/
│   ├── ActorNetwork_TD3.m          # Actor neural network architecture
│   ├── CriticNetwork_TD3.m         # Critic network design
│   └── TD3Agent_Setup.m            

├── Trained TD3 Agent/
│   ├── AgentDDPG_Final.mat         # Pre-trained DDPG agent
│   ├── AgentTD3_Final2.mat         # Pre-trained TD3 agent
│   ├── BlockDiagram.png            # System block representation
│   ├── Compiled_Code.mlx           # Consolidated code to reproduce results
│   ├── Tracking performance.png    # Trajectory tracking performance plots

├── ResearchProject_Report.pdf      # Complete project report with results and analysis
├── LICENSE                         # Open source license (customize as needed)
└── README.md                       # Project overview and instructions

---

Setup & Execution

1. Requirements:

   • MATLAB R2022b or later
   • Reinforcement Learning Toolbox
   • Simulink (for .slx model)

2. Steps to Reproduce:

   • Open Compiled_Code.mlx or run sections from:

     • Environment_Setup.m to define the training environment.
     • TD3Agent_Setup.m to configure the TD3 agent.
     • TrainingOptions.m to modify learning parameters.

   • Load the Simulink model: FBL_ON_UAV_Final.slx.

   • Use AgentTD3_Final2.mat to directly evaluate a trained agent.

   • Run simulations and visualize tracking performance.

   • A pre-trained DDPG agent is included for benchmarking and comparative analysis against the TD3-based adaptive PID controller.

---

Results

• TD3-PID shows strong disturbance rejection and smooth recovery in altitude/attitude.
• Comparative analysis in the report shows clear performance gain over fixed-gain counterparts.

---

📘 Documentation For complete theoretical background, design equations, training pipeline, and result analysis, refer to:

📄 ResearchProject_Report.pdf

---

Citations & References

Key techniques referenced in this project:

• Feedback Linearization for UAV control
• Deep Deterministic Policy Gradient (DDPG) and TD3
• Adaptive control using DRL agents

See project report for complete references.

---

Author

Shreehan S Kate

Contributing

Work is being done in drafting the results obtained after running the DDPG-PID controller. Pull requests are welcome! If you'd like to extend this framework (e.g., test on a different plant, add LQR baseline, etc.), please feel free to fork and improve.

---

License

This project is open-source under the MIT License. Please check the terms before reuse.