Ubuntu 22.04 is over 3 years old and therefore ships with quite an old
kernel. Our production VMs (for relays) all run Ubuntu 24.04 so it makes
sense to build and test them on the same kernel / OS release. For
consistency reasons, we therefore bump all runners to 24.04.
When we receive a DNS query for a DNS resource in Firezone, we take the
next available 4 IPs from the CG-NAT range and assign them to the domain
name. For example, if `example.com` is a DNS resource and it is the
first resource being queried in a Firezone session, we will assigned the
IPs `100.96.0.1` - `100.96.0.4` to it. If the user now restarts Firezone
or signs out and back in, this state is lost and we assign those same
IPs to the next DNS query coming in.
This creates a problem for applications that do not re-query DNS very
often or never. They expect these IPs to not change. Restarting software
or signing out and back in is a common approach to fixing software
problems, yet in this specific case, doing so may create even more
problems for the user.
To mitigate this, `ClientState` introduce a new event
`DnsRecordsChanged` that gets emitted to the event-loop every time we
assign new records. The event-loop then caches this in memory and reuses
it in case a new session is initiated. The records are only stored
in-memory and not on disk. Most likely, the tunnel process will be alive
for the entire OS session.
To verify this behaviour, we add a new `RestartClient` transition to our
proptests. In the proptests, we already keep a mapping of all DNS names
we ever resolved, including DNS resources. When generating IP traffic,
we sample from this list of IPs and then expect the packet to be routed.
By replacing the `ClientState` as part of this transition and re-seeding
it with the previously exported DNS records, we can verify that packets
to IPs resolved from a previous session still get successfully routed to
the resource.
Related: #5498
To deploy the relays on Azure, we need to make sure the binaries are
copied there, similar to GCP. This adds a job step to do just that,
placing them into a storage account + container using new infra
provisioned in Azure.
In CI, eBPF in driver mode actually functions just fine with no changes
to our existing tests, given we apply a few workarounds and bugfixes:
- The interface learning mechanism had two flaws: (1) it only learned
per-CPU, which meant the risk for a missing entry grew as the core count
of the relay host grew, and (2) it did not filter for unicast IPs, so it
picked up broadcast and link-local addresses, causing cross-relay paths
to fail occasionally
- The `relay-relay` candidate where the two relays are the same relay
causes packet drops / loops in the Docker bridge setup, and possibly in
GCP too. I'm not sure this is a valid path that solves a real
connectivity issue in the wild. I can understand relay-relay paths where
two relays are different hosts, and the client and gateway both talk
over their TURN channel to each other (i.e. WireGuard is blocked in each
of their networks), but I can't think of an advantage for a relay-relay
candidate where the traffic simply hairpins (or is dropped) off the
nearest switch. This has been now detected with a new `PacketLoop` error
that triggers whenever source_ip == dest_ip.
- The relays in CI need a common next-hop to talk to for the MAC address
swapping to work. A simple router service is added which functions as a
basic L3 router (no NAT) that allows the MAC swapping to work.
- The `veth` driver has some peculiar requirements to allow it to
function with XDP_TX. If you send a packet out of one interface of a
veth pair with XDP_TX, you need to either make sure both interfaces have
GRO enabled, or you need to attach a dummy XDP program that simply does
XDP_PASS to the other interface so that the sk_buff is allocated before
going up the stack to the Docker bridge. The GRO method was unreliable
and didn't work in our case, causing massive packet delays and
unpredictable bursts that prevented ICE from working, so we use the
XDP_PASS method instead. A simple docker image is built and lives at
https://github.com/firezone/xdp-pass to handle this.
Related: #10138
Related: #10260
- Removes the swift DerivedData cache. This was added to attempt to
speed up the Swift builds in CI but in reality, those are already fast
and the cache did not speed them up.
- Removes the runner.os/arch specifier from the Webview installer cache
key. The binary download is hardcoded for a specific windows version /
arch already so the cache key just adds unneeded complexity.
These caches are getting saved on PR runs which consumes excess GHA
cache storage.
To avoid burning Azure credits, we move the runners back down to the
free tier. Now that caching is properly set up, this should incur only a
minor increase in CI time.
The COS images we currently use to run our Relays ship with an older
Linux kernel that doesn't have some of the nice verifier improvements
for our eBPF relay.
To fix this, we need to use Ubuntu 24.04. To keep things simple there,
we would like to avoid installing Docker on that image and instead run
the Relay raw. To support that, we first need to push the built relay
binary to our staging cloud storage bucket.
Related: #10177
Related: https://github.com/firezone/infra/pull/116
---------
Signed-off-by: Jamil <jamilbk@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
We are _very much_ over our GHA cache limit of 10 GB so in an effort to
keep evictions to a minimum, we update the Rust SCCACHE to only write on
`main` and the Docker elixir and data plane image build steps to do the
same.
Fixes#10145
In the real world, it's entirely possible that the latency between
clients, gateways, and relays is much lower than the latency to the API
nodes. This added latency will test that we can handle such cases
reliably.
---------
Co-authored-by: Thomas Eizinger <thomas@eizinger.io>
Right now, `snownet` de-multiplexes WireGuard packets based on their
source tuple (IP + port) to the _first_ connection that would like to
handle this traffic. What appears to be happening based on observation
from customer logs is that we sometimes dispatch the traffic to the
wrong connection.
The WireGuard packet format uses session indices to declare, which
session a packet is for. The local session index is selected during the
handshake for a particular session.
By associating the different session indices (we can have up to 8 in
parallel per peer) with our Firezone-specific connection ID, we can
change our de-multiplexing scheme to uses these indices instead of the
source tuple. This is especially important for Gateways as those talk to
multiple different clients.
The session index is a 32-bit integer where the top 24 bits identify the
connection and the bottom 8 bits are used in a round-robin fashion to
identify individual sessions within the connection. Thus, to find the
correct connection, we right-shift the session index of an incoming
packet to arrive back at the 24-bit connection identifier.
In environments with a limited number of ports outside the NAT, a
connection from a new Client may come from a source tuple of a previous
Client. In such a case, we'd dispatch the packets to the wrong
connection, causing the Client to not be able to handshake a tunnel.
Bumps
[taiki-e/install-action](https://github.com/taiki-e/install-action) from
2.55.3 to 2.57.5.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/taiki-e/install-action/releases">taiki-e/install-action's
releases</a>.</em></p>
<blockquote>
<h2>2.57.5</h2>
<ul>
<li>Update <code>vacuum@latest</code> to 0.17.7.</li>
</ul>
<h2>2.57.4</h2>
<ul>
<li>Update <code>trivy@latest</code> to 0.65.0.</li>
</ul>
<h2>2.57.3</h2>
<ul>
<li>Update <code>syft@latest</code> to 1.29.1.</li>
</ul>
<h2>2.57.2</h2>
<ul>
<li>
<p>Update <code>grcov@latest</code> to 0.10.3.</p>
</li>
<li>
<p>Update <code>cargo-shear@latest</code> to 1.4.1.</p>
</li>
</ul>
<h2>2.57.1</h2>
<ul>
<li>
<p>Update <code>git-cliff@latest</code> to 2.10.0.</p>
</li>
<li>
<p>Update <code>cargo-binstall@latest</code> to 1.14.2.</p>
</li>
</ul>
<h2>2.57.0</h2>
<ul>
<li>Support <code>mdbook-alerts</code>. (<a
href="https://redirect.github.com/taiki-e/install-action/pull/1060">#1060</a>,
thanks <a
href="https://github.com/CommanderStorm"><code>@CommanderStorm</code></a>)</li>
</ul>
<h2>2.56.24</h2>
<ul>
<li>Update <code>just@latest</code> to 1.42.4.</li>
</ul>
<h2>2.56.23</h2>
<ul>
<li>Update <code>release-plz@latest</code> to 0.3.139.</li>
</ul>
<h2>2.56.22</h2>
<ul>
<li>Update <code>wasmtime@latest</code> to 35.0.0.</li>
</ul>
<h2>2.56.21</h2>
<ul>
<li>Improve error message for unsupported host architectures.</li>
</ul>
<h2>2.56.20</h2>
<ul>
<li>Update <code>syft@latest</code> to 1.29.0.</li>
</ul>
<h2>2.56.19</h2>
<ul>
<li>Update <code>cargo-llvm-cov@latest</code> to 0.6.18.</li>
</ul>
<h2>2.56.18</h2>
<ul>
<li>Update <code>just@latest</code> to 1.42.3.</li>
</ul>
<h2>2.56.17</h2>
<ul>
<li>Update <code>wasmtime@latest</code> to 34.0.2.</li>
</ul>
<h2>2.56.16</h2>
<ul>
<li>
<p>Update <code>cargo-zigbuild@latest</code> to 0.20.1.</p>
</li>
<li>
<p>Update <code>cargo-lambda@latest</code> to 1.8.6.</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md">taiki-e/install-action's
changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<p>All notable changes to this project will be documented in this
file.</p>
<p>This project adheres to <a href="https://semver.org">Semantic
Versioning</a>.</p>
<!-- raw HTML omitted -->
<h2>[Unreleased]</h2>
<h2>[2.57.5] - 2025-08-01</h2>
<ul>
<li>Update <code>vacuum@latest</code> to 0.17.7.</li>
</ul>
<h2>[2.57.4] - 2025-07-31</h2>
<ul>
<li>Update <code>trivy@latest</code> to 0.65.0.</li>
</ul>
<h2>[2.57.3] - 2025-07-31</h2>
<ul>
<li>Update <code>syft@latest</code> to 1.29.1.</li>
</ul>
<h2>[2.57.2] - 2025-07-29</h2>
<ul>
<li>
<p>Update <code>grcov@latest</code> to 0.10.3.</p>
</li>
<li>
<p>Update <code>cargo-shear@latest</code> to 1.4.1.</p>
</li>
</ul>
<h2>[2.57.1] - 2025-07-27</h2>
<ul>
<li>
<p>Update <code>git-cliff@latest</code> to 2.10.0.</p>
</li>
<li>
<p>Update <code>cargo-binstall@latest</code> to 1.14.2.</p>
</li>
</ul>
<h2>[2.57.0] - 2025-07-26</h2>
<ul>
<li>Support <code>mdbook-alerts</code>. (<a
href="https://redirect.github.com/taiki-e/install-action/pull/1060">#1060</a>,
thanks <a
href="https://github.com/CommanderStorm"><code>@CommanderStorm</code></a>)</li>
</ul>
<h2>[2.56.24] - 2025-07-25</h2>
<ul>
<li>Update <code>just@latest</code> to 1.42.4.</li>
</ul>
<h2>[2.56.23] - 2025-07-24</h2>
<ul>
<li>Update <code>release-plz@latest</code> to 0.3.139.</li>
</ul>
<h2>[2.56.22] - 2025-07-24</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="d31232495a"><code>d312324</code></a>
Release 2.57.5</li>
<li><a
href="00ad1b8748"><code>00ad1b8</code></a>
Update <code>vacuum@latest</code> to 0.17.7</li>
<li><a
href="e8c1cf74a6"><code>e8c1cf7</code></a>
Release 2.57.4</li>
<li><a
href="f5b10fbf06"><code>f5b10fb</code></a>
Update <code>trivy@latest</code> to 0.65.0</li>
<li><a
href="17ad3887d7"><code>17ad388</code></a>
Release 2.57.3</li>
<li><a
href="450b647d5c"><code>450b647</code></a>
Update <code>syft@latest</code> to 1.29.1</li>
<li><a
href="bbdef1c33c"><code>bbdef1c</code></a>
Release 2.57.2</li>
<li><a
href="c01bd8006a"><code>c01bd80</code></a>
Update <code>grcov@latest</code> to 0.10.3</li>
<li><a
href="658daa5fc2"><code>658daa5</code></a>
Update <code>cargo-shear@latest</code> to 1.4.1</li>
<li><a
href="a416ddeedb"><code>a416dde</code></a>
Release 2.57.1</li>
<li>Additional commits viewable in <a
href="https://github.com/taiki-e/install-action/compare/v2.55.3...d31232495ad76f47aad66e3501e47780b49f0f3e">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This fixes an issue introduced when we moved to GHCR hosting where the
`latest` tag was being applied to each `main` build of the `client` and
`gateway` instead of publish builds only.
With the removal of the NAT64/46 modules, we can now simplify the
internals of our `IpPacket` struct. The requirements for our `IpPacket`
struct are somewhat delicate.
On the one hand, we don't want to be overly restrictive in our parsing /
validation code because there is a lot of broken software out there that
doesn't necessarily follow RFCs. Hence, we want to be as lenient as
possible in what we accept.
On the other hand, we do need to verify certain aspects of the packet,
like the payload lengths. At the moment, we are somewhat too lenient
there which causes errors on the Gateway where we have to NAT or
otherwise manipulate the packets. See #9567 or #9552 for example.
To fix this, we make the parsing in the `IpPacket` constructor more
restrictive. If it is a UDP, TCP or ICMP packet, we attempt to fully
parse its headers and validate the payload lengths.
This parsing allows us to then rely on the integrity of the packet as
part of the implementation. This does create several code paths that can
in theory panic but in practice, should be impossible to hit. To ensure
that this does in fact not happen, we also tackle an issue that is long
overdue: Fuzzing.
Resolves: #6667Resolves: #9567Resolves: #9552
Packet loss is a reality on the modern internet. Ideally, Firezone
should be able to handle some level of packet loss and still function
reliably, especially considering all of the UDP-based protocols we rely
on.
To test this, we set an extreme packet loss of 20% and perform a 10 MB
download through Firezone. Doing so actually exposed a bug:
For DNS resources, we need to set up the DNS resource NAT on the Gateway
which happens through the p2p control protocol. This packet is resent at
most every 2s but only if there are any other DNS queries. If we don't
receive another DNS query but get traffic for the resource, we keep
buffering those packets without trying to re-send the `AssignedIp`s
packet.
In #6876, we added functionality that would only make use of new remote
candidates whilst we haven't nominated a socket yet with the remote. The
reason for that was because in the described edge-case where relays
reboot or get replaced whilst the client is partitioned from the portal
(or we experience a connection hiccup), only one of the two peers, i.e.
Client or Gateway would migrate to the new relay, leaving the other one
in an inconsistent state.
Looking at recent customer logs, I've been seeing a lot of these
messages:
> Unknown connection or socket has already been nominated
For this particular customer, these are then very quickly followed by
ICE timeouts, leaving the connection unusable.
Considering that, I no longer think that the above change was a good
idea and we should instead always make use of all candidates that we are
given. What we are seeing is that in deployment scenarios where the
latency link between Client and Gateway is very short (5-10ms) yet the
latency to the portal is longer (~30-50ms), we trigger a race condition
where we are temporarily nominating a _peer-reflexive_ candidate pair
instead of a regular one. This happens because with such a short latency
link, Client and Gateway are _faster_ in sending back and forth several
STUN bindings than the control plane is in delivering all the
candidates.
Due to the functionality added in #6876, this then results in us not
accepting the candidates. It further appears that a nominated
peer-reflexive candidate does not provide a stable connection which is
why we then run into an ICE timeout, requiring Firezone to establish a
new connection only to have the same thing happen again.
This is very disruptive for the user experience as the connection only
works for a few moments at a time.
With #9793, we have actually added a feature that is also at play here.
Now that we don't immediately act on an ICE timeout, it is actually
possible for both Client and Gateway to migrate a connection to a
different relay, should the one that they are using get disconnected. In
#9793, we added a timeout of 2s for this.
To make this fully work, we need to patch str0m to transition to
`Checking` early. Presently, str0m would directly transition from
`Disconnected` to `Connected` in this case which in some of the
high-latency scenarios that we are testing in CI is not enough to
recover the connection within 2s. By transitioning to `Checking` early,
we abort this timer.
Related: https://github.com/algesten/str0m/pull/676
Bumps
[taiki-e/install-action](https://github.com/taiki-e/install-action) from
2.55.3 to 2.56.19.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/taiki-e/install-action/releases">taiki-e/install-action's
releases</a>.</em></p>
<blockquote>
<h2>2.56.19</h2>
<ul>
<li>Update <code>cargo-llvm-cov@latest</code> to 0.6.18.</li>
</ul>
<h2>2.56.18</h2>
<ul>
<li>Update <code>just@latest</code> to 1.42.3.</li>
</ul>
<h2>2.56.17</h2>
<ul>
<li>Update <code>wasmtime@latest</code> to 34.0.2.</li>
</ul>
<h2>2.56.16</h2>
<ul>
<li>
<p>Update <code>cargo-zigbuild@latest</code> to 0.20.1.</p>
</li>
<li>
<p>Update <code>cargo-lambda@latest</code> to 1.8.6.</p>
</li>
<li>
<p>Update <code>vacuum@latest</code> to 0.17.6.</p>
</li>
<li>
<p>Update <code>earthly@latest</code> to 0.8.16.</p>
</li>
</ul>
<h2>2.56.15</h2>
<ul>
<li>
<p>Fix <code>cargo-valgrind</code> installation error due to their tag
rename.</p>
</li>
<li>
<p>Update <code>cargo-valgrind@latest</code> to 2.3.2.</p>
</li>
<li>
<p>Update <code>just@latest</code> to 1.42.2.</p>
</li>
</ul>
<h2>2.56.14</h2>
<ul>
<li>
<p>Update <code>zola@latest</code> to 0.21.0.</p>
</li>
<li>
<p>Update <code>wait-for-them@latest</code> to 0.5.1.</p>
</li>
<li>
<p>Update <code>mdbook@latest</code> to 0.4.52.</p>
</li>
<li>
<p>Update <code>just@latest</code> to 1.42.1.</p>
</li>
<li>
<p>Update <code>cargo-shear@latest</code> to 1.4.0.</p>
</li>
<li>
<p>Update <code>cyclonedx@latest</code> to 0.29.0.</p>
</li>
</ul>
<h2>2.56.13</h2>
<ul>
<li>Update <code>cargo-nextest@latest</code> to 0.9.101.</li>
</ul>
<h2>2.56.12</h2>
<ul>
<li>Update <code>cargo-hack@latest</code> to 0.6.37.</li>
</ul>
<h2>2.56.11</h2>
<ul>
<li>
<p>Update <code>osv-scanner@latest</code> to 2.1.0.</p>
</li>
<li>
<p>Update <code>cargo-no-dev-deps@latest</code> to 0.2.16.</p>
</li>
<li>
<p>Update <code>cargo-minimal-versions@latest</code> to 0.1.31.</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md">taiki-e/install-action's
changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<p>All notable changes to this project will be documented in this
file.</p>
<p>This project adheres to <a href="https://semver.org">Semantic
Versioning</a>.</p>
<!-- raw HTML omitted -->
<h2>[Unreleased]</h2>
<h2>[2.56.19] - 2025-07-19</h2>
<ul>
<li>Update <code>cargo-llvm-cov@latest</code> to 0.6.18.</li>
</ul>
<h2>[2.56.18] - 2025-07-19</h2>
<ul>
<li>Update <code>just@latest</code> to 1.42.3.</li>
</ul>
<h2>[2.56.17] - 2025-07-18</h2>
<ul>
<li>Update <code>wasmtime@latest</code> to 34.0.2.</li>
</ul>
<h2>[2.56.16] - 2025-07-18</h2>
<ul>
<li>
<p>Update <code>cargo-zigbuild@latest</code> to 0.20.1.</p>
</li>
<li>
<p>Update <code>cargo-lambda@latest</code> to 1.8.6.</p>
</li>
<li>
<p>Update <code>vacuum@latest</code> to 0.17.6.</p>
</li>
<li>
<p>Update <code>earthly@latest</code> to 0.8.16.</p>
</li>
</ul>
<h2>[2.56.15] - 2025-07-16</h2>
<ul>
<li>
<p>Fix <code>cargo-valgrind</code> installation error due to their tag
rename.</p>
</li>
<li>
<p>Update <code>cargo-valgrind@latest</code> to 2.3.2.</p>
</li>
<li>
<p>Update <code>just@latest</code> to 1.42.2.</p>
</li>
</ul>
<h2>[2.56.14] - 2025-07-15</h2>
<ul>
<li>
<p>Update <code>zola@latest</code> to 0.21.0.</p>
</li>
<li>
<p>Update <code>wait-for-them@latest</code> to 0.5.1.</p>
</li>
<li>
<p>Update <code>mdbook@latest</code> to 0.4.52.</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c99cc51b30"><code>c99cc51</code></a>
Release 2.56.19</li>
<li><a
href="35aa282e0f"><code>35aa282</code></a>
Update <code>cargo-llvm-cov@latest</code> to 0.6.18</li>
<li><a
href="8962e8bc90"><code>8962e8b</code></a>
Release 2.56.18</li>
<li><a
href="86ed27786e"><code>86ed277</code></a>
Update <code>just@latest</code> to 1.42.3</li>
<li><a
href="d8fcd11e5f"><code>d8fcd11</code></a>
Release 2.56.17</li>
<li><a
href="03efa19be6"><code>03efa19</code></a>
Update readme</li>
<li><a
href="14dc975de9"><code>14dc975</code></a>
ci: Fix debian 10 setup</li>
<li><a
href="00c7072f52"><code>00c7072</code></a>
Update wasmtime manifest</li>
<li><a
href="8520ed0913"><code>8520ed0</code></a>
Release 2.56.16</li>
<li><a
href="56de642f63"><code>56de642</code></a>
Update <code>cargo-zigbuild@latest</code> to 0.20.1</li>
<li>Additional commits viewable in <a
href="9ca1734d89...c99cc51b30">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Google Cloud Artifact registry and Cloud storage is a significant cost.
GitHub, on the other hand, is completely free due to our being a public
repository. Hence, it makes sense to ditch GCP for GHCR.
To do this, we move all "staging" artifacts to GHCR. These will then be
used in the infra repo to push to GCP for deploys - we probably still
want pulls for our infra to hit GCP and not GitHub.
One big element of this is that we potentially lose sccache, so I'll be
checking the compile time of this PR and looking for alternatives that
don't involve such a massive cloud bill.
When receiving an `init` message from the portal, we will now revoke all
authorizations not listed in the `authorizations` list of the `init`
message.
We (partly) test this by introducing a new transition in our proptests
that de-authorizes a certain resource whilst the Gateway is simulated to
be partitioned. It is difficult to test that we cannot make a connection
once that has happened because we would have to simulate a malicious
client that knows about resources / connections or ignores the "remove
resource" message.
Testing this is deferred to a dedicated task. We do test that we hit the
code path of revoking the resource authorization and because the other
resources keep working, we also test that we are at least not revoking
the wrong ones.
Resolves: #9892
In Docker environments, applying iptables rules to filter
container-container traffic on the Docker bridged network is not
reliable, leading to direct connections being established in our relayed
tests. To fix this, we insert the rules directly from the client
container itself.
---------
Co-authored-by: Jamil Bou Kheir <jamilbk@users.noreply.github.com>
The current Git tag for releases of the Apple client is out-of-line with
the naming of rest of the repository. Ideally, the tag would be renamed
to `apple-client-X.Y.Z` as it represents the version for both the macOS
and iOS client.
I am not familiar with the redirect system on our website to
confidentially do this without breaking anything, so the easiest fix
here is to employ the same hack we already do for Sentry where we
special-case the `macos-client` tag.
Resolves: #9871
In the DNS resource NAT table, we track parts of the layer 4 protocol of
the connection in order to map packets back to the correct proxy IP in
case multiple DNS names resolve to the same real IP. The involvement of
layer 4 means we need to perform some packet inspection in case we
receive ICMP errors from an upstream router.
Presently, the only ICMP error we handle here is destination
unreachable. Those are generated e.g. when we are trying to contact an
IPv6 address but we don't have an IPv6 egress interface. An additional
error that we want to handle here is "time exceeded":
Time exceeded is sent when the TTL of a packet reaches 0. Typically,
TTLs are set high enough such that the packet makes it to its
destination. When using tools such as `tracepath` however, the TTL is
specifically only incremented one-by-one in order to resolve the exact
hops a packet is taking to a destination. Without handling the time
exceeded ICMP error, using `tracepath` through Firezone is broken
because the packets get dropped at the DNS resource NAT.
With this PR, we generalise the functionality of detecting destination
unreachable ICMP errors to also handle time-exceeded errors, allowing
tools such as `tracepath` to somewhat work:
```
❯ sudo docker compose exec --env RUST_LOG=info -it client /bin/sh -c 'tracepath -b example.com'
1?: [LOCALHOST] pmtu 1280
1: 100.82.110.64 (100.82.110.64) 0.795ms
1: 100.82.110.64 (100.82.110.64) 0.593ms
2: example.com (100.96.0.1) 0.696ms asymm 45
3: example.com (100.96.0.1) 5.788ms asymm 45
4: example.com (100.96.0.1) 7.787ms asymm 45
5: example.com (100.96.0.1) 8.412ms asymm 45
6: example.com (100.96.0.1) 9.545ms asymm 45
7: example.com (100.96.0.1) 7.312ms asymm 45
8: example.com (100.96.0.1) 8.779ms asymm 45
9: example.com (100.96.0.1) 9.455ms asymm 45
10: example.com (100.96.0.1) 14.410ms asymm 45
11: example.com (100.96.0.1) 24.244ms asymm 45
12: example.com (100.96.0.1) 31.286ms asymm 45
13: no reply
14: example.com (100.96.0.1) 303.860ms asymm 45
15: no reply
16: example.com (100.96.0.1) 135.616ms (This broken router returned corrupted payload) asymm 45
17: no reply
18: example.com (100.96.0.1) 161.647ms asymm 45
19: no reply
20: no reply
21: no reply
22: example.com (100.96.0.1) 238.066ms reached
Resume: pmtu 1280 hops 22 back 45
```
We say "somewhat work" because due to the NAT that is in place for DNS
resources, the output does not disclose the intermediary hops beyond the
Gateway.
Co-authored-by: Antoine Labarussias <antoinelabarussias@gmail.com>
---------
Co-authored-by: Antoine Labarussias <antoinelabarussias@gmail.com>
With this patch, we sample a list of DNS resources on each test run and
create a "TCP service" for each of their addresses. Using this list of
resources, we then change the `SendTcpPayload` transition to
`ConnectTcp` and establish TCP connections using `smoltcp` to these
services.
For now, we don't send any data on these connections but we do set the
keep-alive interval to 5s, meaning `smoltcp` itself will keep these
connections alive. We also set the timeout to 30s and after each
transition in a test-run, we assert that all TCP sockets are still in
their expected state:
- `ESTABLISHED` for most of them.
- `CLOSED` for all sockets where we ended up sampling an IPv4 address
but the DNS resource only supports IPv6 addresses (or vice-versa). In
these cases, we use the ICMP error to sent by the Gateway to assert that
the socket is `CLOSED`. Unfortunately, `smoltcp` currently does not
handle ICMP messages for its sockets, so we have to call `abort`
ourselves.
Overall, this should assert that regardless of whether we roam networks,
switch relays or do other kind of stuff with the underlying connection,
the tunneled TCP connection stays alive.
In order to make this work, I had to tweak the timeouts when we are
on-demand refreshing allocations. This only happens in one particular
case: When we are being given new relays by the portal, we refresh all
_other_ relays to make sure they are still present. In other words, all
relays that we didn't remove and didn't just add but still had in-memory
are refreshed. This is important for cases where we are
network-partitioned from the portal whilst relays are deployed or reset
their state otherwise. Instead of the previous 8s max elapsed time of
the exponential backoff like we have it for other requests, we now only
use a single message with a 1s timeout there. With the increased ICE
timeout of 15s, a TCP connection with a 30s timeout would otherwise not
survive such an event. This is because it takes the above mentioned 8s
for us to remove a non-functioning relay, all whilst trying to establish
a new connection (which also incurs its own ICE timeout then).
With the reduced timeout on the on-demand refresh of 1s, we detect the
disappeared relay much quicker and can immediately establish a new
connection via one of the new ones. As always with reduced timeouts,
this can create false-positives if the relay doesn't reply within 1s for
some reason.
Resolves: #9531
The tunnel service creates the Firezone ID upon start-up. With recent
changes to the GUI client, we now require reading the ID file when
starting the GUI client.
This exposes a race condition in our smoke-tests where we start them
both at roughly the same time.
To fix this, we sleep for 500ms after starting the tunnel process.
