A Novel AI Based Algorithm for Automatic Reordering of Firewall Rules

Packet filtering firewalls are the backbone of every modern network infrastructure; however, increasing demands for network efficiency pose significant challenges. As security policies grow and evolve in both number and complexity, the order in which rules are organized within the firewall often emerges as a critical performance bottleneck. This thesis tackles the reordering problem with the goal of improving packet processing time while preserving the original security semantics. We frame rule ordering as a sequential decision task and adopt Reinforcement Learning to obtain an adaptive, data-driven reordering that follows live traffic. In particular, we implement a lightweight tabular Q-Learning agent that balances exploration and exploitation and continuously adjusts the relative priority of rules as the distribution of flows changes over time. Moreover we adopt two deployment variants: the first one is an integrated smart firewall , where the learning agent runs alongside the filtering engine and updates the rule order continuously as packets arrive; the second one is a digital twin composition, in which a learning agent observes the same packet stream in parallel, periodically proposes an optimised permutation of the existing rules, and a conventional firewall enforces decisions. Two deployment models are studied. The first is an integrated smart firewall, where the learning agent runs alongside the filtering engine and updates the rule order continuously as packets arrive. The second is a digital twin composition: a learning agent observes the same packet stream in parallel, periodically proposes an optimised permutation of the existing rules, and a conventional firewall enforces decisions. This separation keeps enforcement simple and auditable while allowing ongoing optimisation on a controlled cadence. We compare these models on two primary outcomes: packet processing latency, the time required to process a fixed batch of packets, and the incidence of misses, packets handled suboptimally due to order. The results reveal a clear trade-off: the digital twin achieves lower latency, while the smart firewall shows fewer misses. Overall, the study demonstrates that adaptive reordering via Reinforcement Learning is a practical and effective way to accelerate packet filtering without altering policy semantics.