The evolution of Large Language Models (LLMs) has driven significant progress in natural language understanding and generation. However, deploying and experimenting with these models in production-like environment, such as those supporting Amazon Alexa, remains operationally complex. Machine learning (ML) scientists need isolated, production-grade “sandbox” environments that allow them to safely experiment with LLMs and adjust inference parameters (e.g., temperature, top-k) without impacting end users. These parameters are critical controls that influence output quality, creativity, and contextual relevance. Traditional approaches often rely on manual provisioning of GPU instances and static configuration of generation parameters, leading to high engineering overhead, slow iteration cycles, limited scalability, inefficient resource utilization, and restricted flexibility in real-time parameter tuning.