90cf191a7c
## Context At present, we only have a single thread that reads and writes to the TUN device on all platforms. On Linux, it is possible to open the file descriptor of a TUN device multiple times by setting the `IFF_MULTI_QUEUE` option using `ioctl`. Using multi-queue, we can then spawn multiple threads that concurrently read and write to the TUN device. This is critical for achieving a better throughput. ## Solution `IFF_MULTI_QUEUE` is a Linux-only thing and therefore only applies to headless-client, GUI-client on Linux and the Gateway (it may also be possible on Android, I haven't tried). As such, we need to first change our internal abstractions a bit to move the creation of the TUN thread to the `Tun` abstraction itself. For this, we change the interface of `Tun` to the following: - `poll_recv_many`: An API, inspired by tokio's `mpsc::Receiver` where multiple items in a channel can be batch-received. - `poll_send_ready`: Mimics the API of `Sink` to check whether more items can be written. - `send`: Mimics the API of `Sink` to actually send an item. With these APIs in place, we can implement various (performance) improvements for the different platforms. - On Linux, this allows us to spawn multiple threads to read and write from the TUN device and send all packets into the same channel. The `Io` component of `connlib` then uses `poll_recv_many` to read batches of up to 100 packets at once. This ties in well with #7210 because we can then use GSO to send the encrypted packets in single syscalls to the OS. - On Windows, we already have a dedicated recv thread because `WinTun`'s most-convenient API uses blocking IO. As such, we can now also tie into that by batch-receiving from this channel. - In addition to using multiple threads, this API now also uses correct readiness checks on Linux, Darwin and Android to uphold backpressure in case we cannot write to the TUN device. ## Configuration Local testing has shown that 2 threads give the best performance for a local `iperf3` run. I suspect this is because there is only so much traffic that a single application (i.e. `iperf3`) can generate. With more than 2 threads, the throughput actually drops drastically because `connlib`'s main thread is too busy with lock-contention and triggering `Waker`s for the TUN threads (which mostly idle around if there are 4+ of them). I've made it configurable on the Gateway though so we can experiment with this during concurrent speedtests etc. In addition, switching `connlib` to a single-threaded tokio runtime further increased the throughput. I suspect due to less task / context switching. ## Results Local testing with `iperf3` shows some very promising results. We now achieve a throughput of 2+ Gbit/s. ``` Connecting to host 172.20.0.110, port 5201 Reverse mode, remote host 172.20.0.110 is sending [ 5] local 100.80.159.34 port 57040 connected to 172.20.0.110 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 274 MBytes 2.30 Gbits/sec [ 5] 1.00-2.00 sec 279 MBytes 2.34 Gbits/sec [ 5] 2.00-3.00 sec 216 MBytes 1.82 Gbits/sec [ 5] 3.00-4.00 sec 224 MBytes 1.88 Gbits/sec [ 5] 4.00-5.00 sec 234 MBytes 1.96 Gbits/sec [ 5] 5.00-6.00 sec 238 MBytes 2.00 Gbits/sec [ 5] 6.00-7.00 sec 229 MBytes 1.92 Gbits/sec [ 5] 7.00-8.00 sec 222 MBytes 1.86 Gbits/sec [ 5] 8.00-9.00 sec 223 MBytes 1.87 Gbits/sec [ 5] 9.00-10.00 sec 217 MBytes 1.82 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.30 GBytes 1.98 Gbits/sec 22247 sender [ 5] 0.00-10.00 sec 2.30 GBytes 1.98 Gbits/sec receiver iperf Done. ``` This is a pretty solid improvement over what is in `main`: ``` Connecting to host 172.20.0.110, port 5201 [ 5] local 100.65.159.3 port 56970 connected to 172.20.0.110 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 90.4 MBytes 758 Mbits/sec 1800 106 KBytes [ 5] 1.00-2.00 sec 93.4 MBytes 783 Mbits/sec 1550 51.6 KBytes [ 5] 2.00-3.00 sec 92.6 MBytes 777 Mbits/sec 1350 76.8 KBytes [ 5] 3.00-4.00 sec 92.9 MBytes 779 Mbits/sec 1800 56.4 KBytes [ 5] 4.00-5.00 sec 93.4 MBytes 783 Mbits/sec 1650 69.6 KBytes [ 5] 5.00-6.00 sec 90.6 MBytes 760 Mbits/sec 1500 73.2 KBytes [ 5] 6.00-7.00 sec 87.6 MBytes 735 Mbits/sec 1400 76.8 KBytes [ 5] 7.00-8.00 sec 92.6 MBytes 777 Mbits/sec 1600 82.7 KBytes [ 5] 8.00-9.00 sec 91.1 MBytes 764 Mbits/sec 1500 70.8 KBytes [ 5] 9.00-10.00 sec 92.0 MBytes 771 Mbits/sec 1550 85.1 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 917 MBytes 769 Mbits/sec 15700 sender [ 5] 0.00-10.00 sec 916 MBytes 768 Mbits/sec receiver iperf Done. ```
A modern alternative to legacy VPNs.
Overview
Firezone is an open source platform to securely manage remote access for any-sized organization. Unlike most VPNs, Firezone takes a granular, least-privileged approach to access management with group-based policies that control access to individual applications, entire subnets, and everything in between.
Features
Firezone is:
- Fast: Built on WireGuard® to be 3-4 times faster than OpenVPN.
- Scalable: Deploy two or more gateways for automatic load balancing and failover.
- Private: Peer-to-peer, end-to-end encrypted tunnels prevent packets from routing through our infrastructure.
- Secure: Zero attack surface thanks to Firezone's holepunching tech which establishes tunnels on-the-fly at the time of access.
- Open: Our entire product is open-source, allowing anyone to audit the codebase.
- Flexible: Authenticate users via email, Google Workspace, Okta, Entra ID, or OIDC and sync users and groups automatically.
- Simple: Deploy gateways and configure access in minutes with a snappy admin UI.
Firezone is not:
- A tool for creating bi-directional mesh networks
- A full-featured router or firewall
- An IPSec or OpenVPN server
Contents of this repository
This is a monorepo containing the full Firezone product, marketing website, and product documentation, organized as follows:
- elixir: Control plane and internal Elixir libraries:
- elixir/apps/web: Admin UI
- elixir/apps/api: API for Clients, Relays and Gateways.
- rust/: Data plane and internal Rust libraries:
- rust/gateway: Gateway - Tunnel server based on WireGuard and deployed to your infrastructure.
- rust/relay: Relay - STUN/TURN server to facilitate holepunching.
- rust/headless-client: Cross-platform CLI client.
- rust/gui-client: Cross-platform GUI client.
- swift/: macOS / iOS clients.
- kotlin/: Android / ChromeOS clients.
- website/: Marketing website and product documentation.
- terraform/: Terraform files for various example deployments.
- terraform/examples/google-cloud/nat-gateway: Example Terraform configuration for deploying a cluster of Firezone Gateways behind a NAT gateway on GCP with a single egress IP.
- terraform/modules/google-cloud/apps/gateway-region-instance-group: Production-ready Terraform module for deploying regional Firezone Gateways to Google Cloud Compute using Regional Instance Groups.
Quickstart
The quickest way to get started with Firezone is to sign up for an account at https://app.firezone.dev/sign_up.
Once you've signed up, follow the instructions in the welcome email to get started.
Frequently asked questions (FAQ)
Can I self-host Firezone?
Our license won't stop you from self-hosting the entire Firezone product top to bottom, but our internal APIs are changing rapidly so we can't meaningfully support self-hosting Firezone in production at this time.
If you're feeling especially adventurous and want to self-host Firezone for educational or hobby purposes, follow the instructions to spin up a local development environment in CONTRIBUTING.md.
The latest published clients (on App Stores and on
releases) are only guaranteed
to work with the managed version of Firezone and may not work with a self-hosted
portal built from this repository. This is because Apple and Google can
sometimes delay updates to their app stores, and so the latest published version
may not be compatible with the tip of main from this repository.
Therefore, if you're experimenting with self-hosting Firezone, you will probably want to use clients you build and distribute yourself as well.
See the READMEs in the following directories for more information on building each client:
- macOS / iOS: swift/apple
- Android / ChromeOS: kotlin/android
- Windows / Linux: rust/gui-client
How long will 0.7 be supported until?
Firezone 0.7 is currently end-of-life and has stopped receiving updates as of
January 31st, 2024. It will continue to be available indefinitely from the
legacy branch of this repo under the Apache 2.0 license.
How much does it cost?
We offer flexible per-seat monthly and annual plans for the cloud-managed version of Firezone, with optional invoicing for larger organizations. See our pricing page for more details.
Those experimenting with self-hosting can use Firezone for free without feature or seat limitations, but we can't provide support for self-hosted installations at this time.
Documentation
Additional documentation on general usage, troubleshooting, and configuration can be found at https://www.firezone.dev/kb.
Get Help
If you're looking for help installing, configuring, or using Firezone, check our community support options:
- Discussion Forums: Ask questions, report bugs, and suggest features.
- Join our Discord Server: Join live discussions, meet other users, and chat with the Firezone team.
- Open a PR: Contribute a bugfix or make a contribution to Firezone.
If you need help deploying or maintaining Firezone for your business, consider contacting our sales team to speak with a Firezone expert.
Star History
Developing and Contributing
See CONTRIBUTING.md.
Security
See SECURITY.md.
License
Portions of this software are licensed as follows:
- All content residing under the "elixir/" directory of this repository, if that directory exists, is licensed under the "Elastic License 2.0" license defined in "elixir/LICENSE".
- All third party components incorporated into the Firezone Software are licensed under the original license provided by the owner of the applicable component.
- Content outside of the above mentioned directories or restrictions above is available under the "Apache 2.0 License" license as defined in "LICENSE".
WireGuard® is a registered trademark of Jason A. Donenfeld.