mirror of
https://github.com/outbackdingo/firezone.git
synced 2026-01-27 18:18:55 +00:00
c3a45f53df
In `connlib`, traffic is sent through sockets via one of three ways: 1. Direct p2p traffic between clients and gateways: For these, we always explicitly set the source IP (and thus interface). 2. UDP traffic to the relays: For these, we let the OS pick an appropriate source interface. 3. WebSocket traffic over TCP to the portal: For this too, we let the OS pick the source interface. For (2) and (3), it is possible to run into routing loops, depending on the routes that we have configured on the TUN device. In Linux, we can prevent routing loops by marking a socket [0] and repeating the mark when we add routes [1]. Packets sent via a marked socket won't be routed by a rule that contains this mark. On Android, we can do something similar by "protecting" a socket via a syscall on the Java side [2]. On Windows, routing works slightly different. There, the source interface is determined based on a computed metric [3] [4]. To prevent routing loops on Windows, we thus need to find the "next best" interface after our TUN interface. We can achieve this with a combination of several syscalls: 1. List all interfaces on the machine 2. Ask Windows for the best route on each interface, except our TUN interface. 3. Sort by Windows' routing metric and pick the lowest one (lower is better). Thanks to the abstraction of `SocketFactory` that we already previously introduced, Integrating this into `connlib` isn't too difficult: 1. For TCP sockets, we simply resolve the best route after creating the socket and then bind it to that local interface. That way, all packets will always going via that interface, regardless of which routes are present on our TUN interface. 2. UDP is connection-less so we need to decide per-packet, which interface to use. "Pick the best interface for me" is modelled in `connlib` via the `DatagramOut::src` field being `None`. - To ensure those packets don't cause a routing loop, we introduce a "source IP resolver" for our `UdpSocket`. This function gets called every time we need to send a packet without a source IP. - For improved performance, we cache these results. The Windows client uses this source IP resolver to use the above devised strategy to find a suitable source IP. - In case the source IP resolution fails, we don't send the packet. This is important, otherwise, the kernel might choose our TUN interface again and trigger a routing loop. The last remark to make here is that this also works for connection roaming. The TCP socket gets thrown away when we reconnect to the portal. Thus, the new socket will pick the new best interface as it is re-created. The UDP sockets also get thrown away as part of roaming. That clears the above cache which is what we want: Upon roaming, the best interface for a given destination IP will likely have changed. [0]:59014a9622/rust/headless-client/src/linux.rs (L19-L29)[1]:59014a9622/rust/bin-shared/src/tun_device_manager/linux.rs (L204-L224)[2]:59014a9622/rust/connlib/clients/android/src/lib.rs (L535-L549)[3]: https://learn.microsoft.com/en-us/previous-versions/technet-magazine/cc137807(v=msdn.10)?redirectedfrom=MSDN [4]: https://learn.microsoft.com/en-us/windows-server/networking/technologies/network-subsystem/net-sub-interface-metric Fixes: #5955. --------- Signed-off-by: Thomas Eizinger <thomas@eizinger.io> Co-authored-by: Thomas Eizinger <thomas@eizinger.io>