Hybrid dynamical systems, such as quadruped robots, humanoids and aerial manipulators, pose unique control challenges due to their coexistence of continuous and discrete dynamics and in response to these challenges, this thesis presents a novel control algorithm for hybrid dynamical systems. Based on the iterative linear quadratic regulator algorithm adapted for hybrid systems through the use of the saltation matrix, we further extend the approach to effectively accommodate state and input constraints. Our methodology integrates both an Interior Point method and an Augmented Lagrangian method to ensure the generation of trajectories within the safe operational set, thereby enhancing system stability and safety.