This thesis explores the geometric properties of the reinforcement learning algorithm Proximal Policy Optimization (PPO) in the context of continuous action spaces and isotropic deep Gaussian policies, with standard deviation σ. Despite the popularity of PPO in this setting, few works examine theoretical and geometric implications of the algorithm structure. However, these can prove fundamental to improve the existing heuristics and to develop better strategies. Our focus is on the characterization of the role of σ, which is often assumed to only affect exploration. However, we show that it adds relevant geometric implications. Firstly, we develop novel bounds for the gradient norm and eigenvalues of the loss characterizing its optimization landscape.