Autonomous driving has seen significant advancements over the past two decades, thanks to progress in artificial intelligence (AI). However, the decision-making processes of autonomous vehicles (AVs) remain largely opaque, creating a barrier to societal trust and regulatory acceptance due to concerns over trustworthiness, safety and accountability. This thesis explores the application of a novel explainable AI (XAI) technique, Policy Graphs (PGs), in the domain of autonomous driving. Policy Graphs represent an agent's policy as a directed graph with natural language descriptors, providing a human-readable explanation of the agent’s behaviour. This framework is further enhanced by incorporating notions of Theory of Mind, to understand these systems as if they possessed beliefs, desires and intentions, and enabling the graph to capture not just what the agent does, but also what it desires and intends to do.