Agile Drone Path Planning Based on Reinforcement Learning Algorithms

One of the significant challenges that arise when working with autonomous drone systems is the dynamic nature of the environment. When the environment is not entirely static, and other objects such as the goal object are moving, it requires the implementation of different algorithms for various tasks such as object detection, state estimation, and trajectory planning. This is a complex task that requires advanced techniques and algorithms. Several solutions are available for state estimation of moving objects in dynamic environments, one of which is using Visual-Inertial Odometry (VIO) cameras. They work by using multiple cameras to capture images of markers placed on the object of interest. These markers are small, highly reflective, and typically placed in a known pattern on the object. The system compares the images captured by each camera and calculates the distance between the markers by using the principle of triangulation, thus providing a precise and accurate 3D position and orientation of the object. The use of VIO cameras in autonomous drone systems provides a robust and efficient solution for state estimation of moving objects in dynamic environments, as it is able to track the object's position and orientation in real-time, even under challenging conditions. The use of Visual Odometry sensors for state estimation in model predictive control for trajectory planning of autonomous drones in dynamic environments can be challenging, as two main problems need to be addressed. These problems are: - Handling the continuous action spaces: The action space for controlling the drone's trajectory is continuous and high-dimensional, making it challenging to find the optimal policy using traditional techniques. - Dealing with uncertainty and non-stationary environments: The environment in which the drone operates is dynamic and uncertain, making it challenging to predict the system's future state and plan a trajectory that avoids collisions by VIO systems. One solution to these problems is to use Kalman Filters for state estimation, by assuming that computer vision algorithms can extract the position of the gate centers without knowing anything about the dynamic model of the gates. Kalman Filters are a powerful tool for state estimation in dynamic systems. It is widely used in various fields such as control systems, navigation, and robotics. Once the position of the gate centers is estimated using Kalman Filters, the trajectory of the drone for passing through that center can be derived. This trajectory can then be fed to Advanced Actor-Critic Reinforcement Learning Algorithms such as DDPG, SAC, and PPO to derive the best policy. When Kalman filtering and RL algorithms are combined, the Kalman filter can provide accurate and precise state estimates, while the RL algorithm can learn the optimal policy for the agent to make decisions based on these estimates. Combining these two techniques can lead to improved performance in tasks such as trajectory planning, control, and decision-making. The built model of the KFRL algorithm is a continuous state-action environment, which means that the state of the system and the actions taken by the agent are continuous variables. In this model, the agent can take any action in a continuous range of values, rather than only a discrete set of actions. This allows for more flexibility and precision in controlling the drone's trajectory.