This thesis focuses on training surrogate models based on Deep Learning (DL) to predict the solutions of parametric Partial Differential Equation (PDE) problems. In particular, the PDE problems we consider have most of their parameters used to characterize the boundary conditions, not only the physical properties embedded in the differential equations. The scenarios examined involve two types of problems, one purely diffusion-based and one involving both diffusion and convection. Several kinds of DL models are taken into account, including a novel spatial-based graph network called Graph-Informed Neural Network (GINN). Error statistics are computed to understand how the models' predictions are affected by the model architecture, the amount of training data, the hyperparameters of the network, and the physical parameters of the problem.