Firezone's UDP connections are designed to be idempotent. If a Client discards its "half" of the connection but the Gateway still keeps the state around, a subsequent connection setup by the Client will reuse connection state on the Gateway. To fully support this, `snownet` re-sends all its local candidates to the remote peer whenever a connection gets upserted. The current `seed_agent_with_local_candidates` function attempts to do this job but its design overlooked a crucial detail: Re-adding a candidate that the `IceAgent` already knows about is considered to be redundant. As such, the candidate is not re-signalled to the remote! The real-world consequences for this are subtle. `str0m`'s support for peer-reflexive candidate means that incoming STUN binding requests are still answered, even if they come from an address that the agent doesn't know anything about, i.e. it has never been told about that candidate. Thus, what happens right now is that when a Client re-creates a connection that is still present on the Gateway, it will start receiving STUN binding requests for candidates it doesn't know about and create peer-reflexive candidates for them. Where this does show up is in our test-suite which has fairly strict timing constraints. When we simulate the re-deploy of relays, we expect connections to be migrated to a new relay immediately. To support this, the current relay candidates are invalidated on both sides. This however only works if the current candidate is correctly recognised by the local ICE agent. Peer-reflexive candidates are created on-demand and typically only serve a placeholder-like role until we learn about the real candidate that is being used. Due to the above described behaviour of `seed_agent_with_local_candidates`, this however may not happen at all. As a result, attempting to invalidate a relay candidate fails because we don't recognise the relay candidate as we only have a peer-reflexive one. Putting all of this together, whilst not re-sending all candidates doesn't cause immediate issues for a connection, it may cause problems at a later point when we are trying to invalidate a currently active candidate to achieve a speedy failover to a new one.
Rust development guide
Firezone uses Rust for all data plane components. This directory contains the Linux and Windows clients, and low-level networking implementations related to STUN/TURN.
We target the last stable release of Rust using rust-toolchain.toml.
If you are using rustup, that is automatically handled for you.
Otherwise, ensure you have the latest stable version of Rust installed.
Reading Client logs
The Client logs are written as JSONL for machine-readability.
To make them more human-friendly, pipe them through jq like this:
cd path/to/logs # e.g. `$HOME/.cache/dev.firezone.client/data/logs` on Linux
cat *.log | jq -r '"\(.time) \(.severity) \(.message)"'
Resulting in, e.g.
2024-04-01T18:25:47.237661392Z INFO started log
2024-04-01T18:25:47.238193266Z INFO GIT_VERSION = 1.0.0-pre.11-35-gcc0d43531
2024-04-01T18:25:48.295243016Z INFO No token / actor_name on disk, starting in signed-out state
2024-04-01T18:25:48.295360641Z INFO null
Benchmarking on Linux
The recommended way for benchmarking any of the Rust components is Linux' perf utility.
For example, to attach to a running application, do:
- Ensure the binary you are profiling is compiled with the
releaseprofile. sudo perf record -g --freq 10000 --pid $(pgrep <your-binary>).- Run the speed test or whatever load-inducing task you want to measure.
sudo perf script > profile.perf- Open profiler.firefox.com and load
profile.perf
Instead of attaching to a process with --pid, you can also specify the path to executable directly.
That is useful if you want to capture perf data for a test or a micro-benchmark.