a4851ee76f
This PR implements the "reverse path" of handling TURN traffic, i.e. UDP datagrams that arrive on an allocation port and need to be wrapped in a channel-data message to be sent to the TURN client. In order to achieve that, I had to rewrite most of the TURN code to not use the `etherparse` crate. I couldn't quite figure out the details but the eBPF verifier rejected my code in mysterious ways that I didn't understand. Commenting out random code-paths seemed to make it happy but all code-paths combined caused an error. Eventually, I decided that we simply have to use less abstractions to implement the same logic. All the "parsing" code is now using types inspired by `network-types`. The only modification here is that we use byte-arrays within our structs in order to directly receive them in big-endian ordering. `network-types` uses `u16`s and `u32`s which get interpreted as little-endian on x86. Instead of converting around between the endianness, constructing those values where we want them using the right endianness is deemed much simpler. I opened an issue with upstream which - if accepted - will allow us to remove our own structs and instead depend on upstream again. I also had to aggressively add `#[inline(always)]` to several functions, otherwise the compiler would not optimise away our function calls, causing the linker and / or eBPF verifier to fail. This PR also fixes numerous bugs that I've found in the already existing eBPF code. The number of bugs makes me question how this has been working so far at all! - We did not swap the Ethernet source and destination MAC address when re-routing the packet. The integration-test didn't catch this because it only operates on the loopback interface. Further testing on staging should allow us to confirm that this is indeed working now. - The UDP checksum update did not incorporate the new src and dst port. The integration-test didnt' catch that because it has UDP checksumming disabled. We need to have that disabled in the test because UDP checksumming is typically offloaded to the NIC and packets on the loopback interface never leave the device. Related: https://github.com/vadorovsky/network-types/issues/32. Related: #7518
relay
This crate houses a minimalistic STUN & TURN server.
Features
We aim to support the following feature set:
- STUN binding requests
- TURN allocate requests
- TURN refresh requests
- TURN channel bind requests
- TURN channel data requests
Relaying of data through other means such as DATA frames is not supported.
Building
You can build the relay using: cargo build --release --bin firezone-relay
You should then find a binary in target/release/firezone-relay.
Running
The Firezone Relay supports Linux only. To run the Relay binary on your Linux host:
- Generate a new Relay token from the "Relays" section of the admin portal and save it in your secrets manager.
- Ensure the
FIREZONE_TOKEN=<relay_token>environment variable is set securely in your Relay's shell environment. The Relay expects this variable at startup. - Now, you can start the Firezone Relay with:
firezone-relay
To view more advanced configuration options pass the --help flag:
firezone-relay --help
Ports
By default, the relay listens on port udp/3478. This is the standard port for
STUN/TURN. Additionally, the relay needs to have access to the port range
49152 - 65535 for the allocations.
Portal Connection
When given a token, the relay will connect to the Firezone portal and wait for
an init message before commencing relay operations.
Metrics
The relay parses the OTLP_GRPC_ENDPOINT env variable.
Traces and metrics will be sent to an OTLP collector listening on that endpoint.
It is recommended to set additional environment variables to scope your metrics:
OTEL_SERVICE_NAME: Translates to theservice.name.OTEL_RESOURCE_ATTRIBUTES: Additional, comma-separated key=value attributes.
By default, we set the following OTEL attributes:
service.name=relayservice.namespace=firezone
The docker-init-relay.sh script integrates with GCE.
When OTEL_METADATA_DISCOVERY_METHOD=gce_metadata, the service.instance.id
variables is set to the instance ID of the VM.
Design
The relay is designed in a sans-IO fashion, meaning the core components do not cause side effects but operate as pure, synchronous state machines. They take in data and emit commands: wake me at this point in time, send these bytes to this peer, etc.
This allows us to very easily unit-test all kinds of scenarios because all inputs are simple values.
The main server runs in a single task and spawns one additional task for each allocation. Incoming data that needs to be relayed is forwarded to the main task where it gets authenticated and relayed on success.