3c7ac084c0
## Abstract This pull-request implements the first stage of off-loading routing of TURN data channel messages to the kernel via an eBPF XDP program. In particular, the eBPF kernel implemented here **only** handles the decapsulation of IPv4 data channel messages into their embedded UDP payload. Implementation of other data paths, such as the receiving of UDP traffic on an allocation and wrapping it in a TURN channel data message is deferred to a later point for reasons explained further down. As it stands, this PR implements the bare minimum for us to start experimenting and benefiting from eBPF. It is already massive as it is due to the infrastructure required for actually doing this. Let's dive into it! ## A refresher on TURN channel-data messages TURN specifies a channel-data message for relaying data between two peers. A channel data message has a fixed 4-byte header: - The first two bytes specify the channel number - The second two bytes specify the length of the encapsulated payload Like all TURN traffic, channel data messages run over UDP by default, meaning this header sits at the very front of the UDP payload. This will be important later. After making an allocation with a TURN server (i.e. reserving a port on the TURN server's interfaces), a TURN client can bind channels on that allocation. As such, channel numbers are scoped to a client's allocation. Channel numbers are allocated by the client within a given range (0x4000 - 0x4FFF). When binding a channel, the client specifies the remote's peer address that they'd like the data sent on the channel to be sent to. Given this setup, when a TURN server receives a channel data message, it first looks at the sender's IP + port to infer the allocation (a client can only ever have 1 allocation at a time). Within that allocation, the server then looks for the channel number and retrieves the target socket address from that. The allocation itself is a port on the relay's interface. With that, we can now "unpack" the payload of the channel data message and rewrite it to the new receiver: - The new source IP can be set from the old dst IP (when operating in user-space mode this is irrelevant because we are working with the socket API). - The new source port is the client's allocation. - The new destination IP is retrieved from the mapping retrieved via the channel number. - The new destination port is retrieved from the mapping retrieved via the channel number. Last but not least, all that is left is removing the channel data header from the UDP payload and we can send out the packet. In other words, we need to cut off the first 4 bytes of the UDP payload. ## User-space relaying At present, we implement the above flow in user-space. This is tricky to do because we need to bind _many_ sockets, one for each possible allocation port (of which there can be 16383). The actual work to be done on these packets is also extremely minimal. All we do is cut off (or add on) the data-channel header. Benchmarks show that we spend pretty much all of our time copying data between user-space and kernel-space. Cutting this out should give us a massive increase in performance. ## Implementing an eBPF XDP TURN router eBPF has been shown to be a very efficient way of speeding up a TURN server [0]. After many failed experiments (e.g. using TC instead of XDP) and countless rabbit-holes, we have also arrived at the design documented within the paper. Most notably: - The eBPF program is entirely optional. We try to load it on startup, but if that fails, we will simply use the user-space mode. - Retaining the user-space mode is also important because under certain circumstances, the eBPF kernel needs to pass on the packet, for example, when receiving IPv4 packets with options. Those make the header dynamically-sized which makes further processing difficult because the eBPF verifier disallows indexing into the packet with data derived from the packet itself. - In order to add/remove the channel-data header, we shift the packet headers backwards / forwards and leave the payload in place as the packet headers are constant in size and can thus easily and cheaply be copied out. In order to perform the relaying flow explained above, we introduce maps that are shared with user-space. These maps go from a tuple of (client-socket, channel-number) to a tuple of (allocation-port, peer-socket) and thus give us all the data necessary to rewrite the packet. ## Integration with our relay Last but not least, to actually integrate the eBPF kernel with our relay, we need to extend the `Server` with two more events so we can learn, when channel bindings are created and when they expire. Using these events, we can then update the eBPF maps accordingly and therefore influence the routing behaviour in the kernel. ## Scope What is implemented here is only one of several possible data paths. Implementing the others isn't conceptually difficult but it does increase the scope. Landing something that already works allows us to gain experience running it in staging (and possibly production). Additionally, I've hit some issues with the eBPF verifier when adding more codepaths to the kernel. I expect those to be possible to resolve given sufficient debugging but I'd like to do so after merging this. --- Depends-On: #8506 Depends-On: #8507 Depends-On: #8500 Resolves: #8501 [0]: https://dl.acm.org/doi/pdf/10.1145/3609021.3609296
A modern alternative to legacy VPNs.
Overview
Firezone is an open source platform to securely manage remote access for any-sized organization. Unlike most VPNs, Firezone takes a granular, least-privileged approach to access management with group-based policies that control access to individual applications, entire subnets, and everything in between.
Features
Firezone is:
- Fast: Built on WireGuard® to be 3-4 times faster than OpenVPN.
- Scalable: Deploy two or more gateways for automatic load balancing and failover.
- Private: Peer-to-peer, end-to-end encrypted tunnels prevent packets from routing through our infrastructure.
- Secure: Zero attack surface thanks to Firezone's holepunching tech which establishes tunnels on-the-fly at the time of access.
- Open: Our entire product is open-source, allowing anyone to audit the codebase.
- Flexible: Authenticate users via email, Google Workspace, Okta, Entra ID, or OIDC and sync users and groups automatically.
- Simple: Deploy gateways and configure access in minutes with a snappy admin UI.
Firezone is not:
- A tool for creating bi-directional mesh networks
- A full-featured router or firewall
- An IPSec or OpenVPN server
Contents of this repository
This is a monorepo containing the full Firezone product, marketing website, and product documentation, organized as follows:
- elixir: Control plane and internal Elixir libraries:
- elixir/apps/web: Admin UI
- elixir/apps/api: API for Clients, Relays and Gateways.
- rust/: Data plane and internal Rust libraries:
- rust/gateway: Gateway - Tunnel server based on WireGuard and deployed to your infrastructure.
- rust/relay: Relay - STUN/TURN server to facilitate holepunching.
- rust/headless-client: Cross-platform CLI client.
- rust/gui-client: Cross-platform GUI client.
- swift/: macOS / iOS clients.
- kotlin/: Android / ChromeOS clients.
- website/: Marketing website and product documentation.
- terraform/: Terraform files for various example deployments.
- terraform/examples/google-cloud/nat-gateway: Example Terraform configuration for deploying a cluster of Firezone Gateways behind a NAT gateway on GCP with a single egress IP.
- terraform/modules/google-cloud/apps/gateway-region-instance-group: Production-ready Terraform module for deploying regional Firezone Gateways to Google Cloud Compute using Regional Instance Groups.
Quickstart
The quickest way to get started with Firezone is to sign up for an account at https://app.firezone.dev/sign_up.
Once you've signed up, follow the instructions in the welcome email to get started.
Frequently asked questions (FAQ)
Can I self-host Firezone?
Our license won't stop you from self-hosting the entire Firezone product top to bottom, but our internal APIs are changing rapidly so we can't meaningfully support self-hosting Firezone in production at this time.
If you're feeling especially adventurous and want to self-host Firezone for educational or hobby purposes, follow the instructions to spin up a local development environment in CONTRIBUTING.md.
The latest published clients (on App Stores and on
releases) are only guaranteed
to work with the managed version of Firezone and may not work with a self-hosted
portal built from this repository. This is because Apple and Google can
sometimes delay updates to their app stores, and so the latest published version
may not be compatible with the tip of main from this repository.
Therefore, if you're experimenting with self-hosting Firezone, you will probably want to use clients you build and distribute yourself as well.
See the READMEs in the following directories for more information on building each client:
- macOS / iOS: swift/apple
- Android / ChromeOS: kotlin/android
- Windows / Linux: rust/gui-client
How long will 0.7 be supported until?
Firezone 0.7 is currently end-of-life and has stopped receiving updates as of
January 31st, 2024. It will continue to be available indefinitely from the
legacy branch of this repo under the Apache 2.0 license.
How much does it cost?
We offer flexible per-seat monthly and annual plans for the cloud-managed version of Firezone, with optional invoicing for larger organizations. See our pricing page for more details.
Those experimenting with self-hosting can use Firezone for free without feature or seat limitations, but we can't provide support for self-hosted installations at this time.
Documentation
Additional documentation on general usage, troubleshooting, and configuration can be found at https://www.firezone.dev/kb.
Get Help
If you're looking for help installing, configuring, or using Firezone, check our community support options:
- Discussion Forums: Ask questions, report bugs, and suggest features.
- Join our Discord Server: Join live discussions, meet other users, and chat with the Firezone team.
- Open a PR: Contribute a bugfix or make a contribution to Firezone.
If you need help deploying or maintaining Firezone for your business, consider contacting our sales team to speak with a Firezone expert.
See all support options on our main support page.
Star History
Developing and Contributing
See CONTRIBUTING.md.
Security
See SECURITY.md.
License
Portions of this software are licensed as follows:
- All content residing under the "elixir/" directory of this repository, if that directory exists, is licensed under the "Elastic License 2.0" license defined in "elixir/LICENSE".
- All third party components incorporated into the Firezone Software are licensed under the original license provided by the owner of the applicable component.
- Content outside of the above mentioned directories or restrictions above is available under the "Apache 2.0 License" license as defined in "LICENSE".
WireGuard® is a registered trademark of Jason A. Donenfeld.