Enhancing Network Interception with Mitmproxy: An Open Source Solution for Transparent Proxy Mode on macOS and Linux

Mitmproxy is an open-source tool designed for intercepting and manipulating HTTPS traffic. It allows users to intercept traffic from an entire machine or a single, specific process, offering flexibility in a range of operational modes: regular, reverse, upstream, SOCKS, DNS and transparent. In transparent mode, the tool operates at the operating system (OS) layer, making it OS-specific and increasing the complexity of implementation. On macOS, two possible approaches were examined: the first was to redirect any packet on the User Tunnel (UTUN) interface: from UTUN packets could be managed and sent to Mitmproxy. The code was written exclusively in Rust, leveraging its capabilities to operate at a low level; having only one language to maintain might be an advantage but on macOS, conditionally redirections of packets are complex, this forced the redirection of all traffic to Mitmproxy only filtering requests later. This behavior is suboptimal because Mitmproxy bears the burden of processing the entire traffic even when it is only concerned with a specific process. The second approach, currently taken, involves a combination of Rust and Swift. Swift side is the actual Redirector: a companion app that exploits Apple Network Extensions (NE). This allows to read the process identifier (PID) and the process name of the flow source app, deciding which packet to send to Mitmproxy and which packet to ignore. Swift is a good choice to have a perfect fit with the Apple systems. Rust side runs the Redirector sending all configuration details and forward packets received from the swift side to the core of Mitmproxy. The initial version of this approach used Unix pipes, a simplex inter-process communication (IPC) system, but three separate channels are required: one for configuration details, another for inbound packets from the Rust side to the Swift Side, and a third for outbound packets from the Swift side to the Rust side. To enhance communication efficiency, pipes have been replaced with Unix sockets, which support full-duplex communication, consolidating the three separate pipes into a single socket. The serialization and the deserialization of data, both for configuration and packets, are implemented with Protocol Buffers (Protobuf): a language-neutral, platform-neutral, extensible mechanism for serializing structured data. On Linux, the development stage is more immature than on macOS. Because of the absence of direct APIs, the strategy is to take advantage of the EBPF, allowing programs to run directly in kernel space and exploiting two particular types of eBPF programs called: TC and KProbe, making them work together.