Gene expression is the process by which the information encoded in the genes is transcribed and translated into a functional product, allowing cells to react to external impulses and carry out their main functions. Hence, gene expression has primary importance in life, and it is not difficult to imagine that fully understanding such a phenomenon may help cancer diagnosis and drug discovery. State-of-the-art Deep Learning techniques such as Xpresso and Expecto focus on gene expression level prediction by analyzing the raw DNA sequences (tens of thousands of base-pairs long) of each gene extracted from the reference genome, exploiting Convolutional layers. Learning from long sequences is challenging due to the intrinsic nature of the DNA.