The rapid development of Large Language Models (LLMs) has revolutionized natural language processing, enabling sophisticated human-computer interactions. However, the widespread deployment of these models in cloud environments raises significant concerns about privacy, data security and computational efficiency. This thesis, conducted at Tether's Data Division, focuses on the development of privacy-preserving AI solutions through the creation of efficient Small Language Models (SLMs) with 1-3 billion parameters, optimized for local deployment on edge devices. This research provides two main contributions to the development of high performance SLMs. Firstly, it introduces novel pipelines for high-quality post-training synthetic data generation, which is essential for improving model performance in following instructions while reducing hallucinations.