Physics-Informed Model-Based Reinforcement Learning for Soft Robot control

Soft Robots are gaining popularity in the scientific community, due to the numerous applications for which they outperform their rigid counterparts. The trade-off stands in the difficulty to model and control them. In the last decade, the community has developed various advanced techniques to improve our ability to model them, with the purpose of developing effective controllers. This work presents a strategy to derive a model of the soft system and a controller for it: the model is obtained by learning the forward dynamics of the system purely from kinematic observational data and the controller comes from Model-Based Deep Reinforcement Learning using the learned forward dynamics. The model regression makes use of Physics-based inductive biases to learn plausible behaviors, following the paradigm of Physics-Informed Neural Networks (PINNs), in particular that of Deep Lagrangian Networks (DeLaNs). The proposed solution is composed of three stages: i) collection of the data samples from the real (simulated) system using a random policy; ii) training of a set of simple MLPs to learn the coefficients of the Euler-Lagrange equations of the system; iii) training of the RL-based controller via the interaction with the learned model, to get a robust policy with minimal interaction with the real system. Ultimately, an iterative re-application of these three steps, by replacing the random policy with the current one, is investigated, with the goal of obtaining an effective policy with less interactions. Experimental validation has been carried out for a single-segment Piecewise Constant Strain soft rod, to make its tip reach an arbitrary point in space. This approach combines, for the first time on a soft robotics system, the ability of Deep Reinforcement Learning to generate robust controllers requiring minimal knowledge of the real system, with the power of DeLaNs to learn solutions that comply with fundamental physical laws.