Evaluating AI-Based Code Generation Models for the Travel Industry

In this work, we present a code generation model incorporating natural language understanding. Our model uses an auto-regressive language model with NTP as the objective to learn both the natural language description and corresponding code. We post-processed the XL- Cost dataset to enable the use of a functional correctness metric, and our finetuning methods were successful in improving model performance. Additionally, we proposed a novel application of Constrained Beam Search for code generation, which improved the performance of our model on the Multi-Turn Programming Benchmark (MTPB) dataset. We also conducted benchmark evaluations on the throughput and cloud cost of our model.