Large Language Models (LLMs) have shown remarkable capabilities in gener- ative tasks, but their application in data analysis and querying tasks is still limited by issues such as hallucinations, restricted context windows, and lack of transpar- ent reasoning. To address these challenges, this work investigates the integration of Retrieval-Augmented Generation (RAG) with data provenance concepts to en- hance the accuracy and explainability of querying using LLM in closed-domain settings. We design and evaluate a customizable RAG pipeline over structured datasets, including a university schema and a scaled version of the TPC-H bench- mark. Our approach introduces provenance explanations, using witness sets to justify query results, and explores iterative strategies to improve reasoning: (i) an iterative RAG pipeline that refines answers across multiple steps, and (ii) a plan generation method that decomposes complex questions into atomic substep inspired by the Plan of SQL framework.