mirror of
https://github.com/outbackdingo/firezone.git
synced 2026-03-22 11:41:52 +00:00
d6a1966a42e7aaaffc051ce4dfd2ee945ce7ea01
38 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Thomas Eizinger
|
416e320319 |
revert: bump netlink-packet-route and rtnetlink (#7899)
Reverts: #6694 Related: https://github.com/rust-netlink/netlink-packet-route/issues/140 |
||
|
dependabot[bot]
|
0779757646 |
build(deps): netlink-packet-route and rtnetlink (#6694)
`rtnetlink` has some breaking changes in their latest version. To avoid waiting until they actually cut a release, we temporarily depend on their `main` branch. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Thomas Eizinger <thomas@eizinger.io> |
||
|
Thomas Eizinger
|
90cf191a7c |
feat(linux): multi-threaded TUN device operations (#7449)
## Context At present, we only have a single thread that reads and writes to the TUN device on all platforms. On Linux, it is possible to open the file descriptor of a TUN device multiple times by setting the `IFF_MULTI_QUEUE` option using `ioctl`. Using multi-queue, we can then spawn multiple threads that concurrently read and write to the TUN device. This is critical for achieving a better throughput. ## Solution `IFF_MULTI_QUEUE` is a Linux-only thing and therefore only applies to headless-client, GUI-client on Linux and the Gateway (it may also be possible on Android, I haven't tried). As such, we need to first change our internal abstractions a bit to move the creation of the TUN thread to the `Tun` abstraction itself. For this, we change the interface of `Tun` to the following: - `poll_recv_many`: An API, inspired by tokio's `mpsc::Receiver` where multiple items in a channel can be batch-received. - `poll_send_ready`: Mimics the API of `Sink` to check whether more items can be written. - `send`: Mimics the API of `Sink` to actually send an item. With these APIs in place, we can implement various (performance) improvements for the different platforms. - On Linux, this allows us to spawn multiple threads to read and write from the TUN device and send all packets into the same channel. The `Io` component of `connlib` then uses `poll_recv_many` to read batches of up to 100 packets at once. This ties in well with #7210 because we can then use GSO to send the encrypted packets in single syscalls to the OS. - On Windows, we already have a dedicated recv thread because `WinTun`'s most-convenient API uses blocking IO. As such, we can now also tie into that by batch-receiving from this channel. - In addition to using multiple threads, this API now also uses correct readiness checks on Linux, Darwin and Android to uphold backpressure in case we cannot write to the TUN device. ## Configuration Local testing has shown that 2 threads give the best performance for a local `iperf3` run. I suspect this is because there is only so much traffic that a single application (i.e. `iperf3`) can generate. With more than 2 threads, the throughput actually drops drastically because `connlib`'s main thread is too busy with lock-contention and triggering `Waker`s for the TUN threads (which mostly idle around if there are 4+ of them). I've made it configurable on the Gateway though so we can experiment with this during concurrent speedtests etc. In addition, switching `connlib` to a single-threaded tokio runtime further increased the throughput. I suspect due to less task / context switching. ## Results Local testing with `iperf3` shows some very promising results. We now achieve a throughput of 2+ Gbit/s. ``` Connecting to host 172.20.0.110, port 5201 Reverse mode, remote host 172.20.0.110 is sending [ 5] local 100.80.159.34 port 57040 connected to 172.20.0.110 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 274 MBytes 2.30 Gbits/sec [ 5] 1.00-2.00 sec 279 MBytes 2.34 Gbits/sec [ 5] 2.00-3.00 sec 216 MBytes 1.82 Gbits/sec [ 5] 3.00-4.00 sec 224 MBytes 1.88 Gbits/sec [ 5] 4.00-5.00 sec 234 MBytes 1.96 Gbits/sec [ 5] 5.00-6.00 sec 238 MBytes 2.00 Gbits/sec [ 5] 6.00-7.00 sec 229 MBytes 1.92 Gbits/sec [ 5] 7.00-8.00 sec 222 MBytes 1.86 Gbits/sec [ 5] 8.00-9.00 sec 223 MBytes 1.87 Gbits/sec [ 5] 9.00-10.00 sec 217 MBytes 1.82 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 2.30 GBytes 1.98 Gbits/sec 22247 sender [ 5] 0.00-10.00 sec 2.30 GBytes 1.98 Gbits/sec receiver iperf Done. ``` This is a pretty solid improvement over what is in `main`: ``` Connecting to host 172.20.0.110, port 5201 [ 5] local 100.65.159.3 port 56970 connected to 172.20.0.110 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 90.4 MBytes 758 Mbits/sec 1800 106 KBytes [ 5] 1.00-2.00 sec 93.4 MBytes 783 Mbits/sec 1550 51.6 KBytes [ 5] 2.00-3.00 sec 92.6 MBytes 777 Mbits/sec 1350 76.8 KBytes [ 5] 3.00-4.00 sec 92.9 MBytes 779 Mbits/sec 1800 56.4 KBytes [ 5] 4.00-5.00 sec 93.4 MBytes 783 Mbits/sec 1650 69.6 KBytes [ 5] 5.00-6.00 sec 90.6 MBytes 760 Mbits/sec 1500 73.2 KBytes [ 5] 6.00-7.00 sec 87.6 MBytes 735 Mbits/sec 1400 76.8 KBytes [ 5] 7.00-8.00 sec 92.6 MBytes 777 Mbits/sec 1600 82.7 KBytes [ 5] 8.00-9.00 sec 91.1 MBytes 764 Mbits/sec 1500 70.8 KBytes [ 5] 9.00-10.00 sec 92.0 MBytes 771 Mbits/sec 1550 85.1 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 917 MBytes 769 Mbits/sec 15700 sender [ 5] 0.00-10.00 sec 916 MBytes 768 Mbits/sec receiver iperf Done. ``` |
||
|
Thomas Eizinger
|
dd6b52b236 | chore(rust): share edition key via workspace table (#7451) | ||
|
Thomas Eizinger
|
2c26fc9c0e |
ci: lint Rust dependencies using cargo deny (#7390)
One of Rust's promises is "if it compiles, it works". However, there are certain situations in which this isn't true. In particular, when using dynamic typing patterns where trait objects are downcast to concrete types, having two versions of the same dependency can silently break things. This happened in #7379 where I forgot to patch a certain Sentry dependency. A similar problem exists with our `tracing-stackdriver` dependency (see #7241). Lastly, duplicate dependencies increase the compile-times of a project, so we should aim for having as few duplicate versions of a particular dependency as possible in our dependency graph. This PR introduces `cargo deny`, a linter for Rust dependencies. In addition to linting for duplicate dependencies, it also enforces that all dependencies are compatible with an allow-list of licenses and it warns when a dependency is referred to from multiple crates without introducing a workspace dependency. Thanks to existing tooling (https://github.com/mainmatter/cargo-autoinherit), transitioning all dependencies to workspace dependencies was quite easy. Resolves: #7241. |
||
|
dependabot[bot]
|
4014373dc2 |
build(deps): Bump clap from 4.5.20 to 4.5.21 in /rust (#7369)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.20 to 4.5.21. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/clap-rs/clap/releases">clap's releases</a>.</em></p> <blockquote> <h2>v4.5.21</h2> <h2>[4.5.21] - 2024-11-13</h2> <h3>Fixes</h3> <ul> <li><em>(parser)</em> Ensure defaults are filled in on error with <code>ignore_errors(true)</code></li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's changelog</a>.</em></p> <blockquote> <h2>[4.5.21] - 2024-11-13</h2> <h3>Fixes</h3> <ul> <li><em>(parser)</em> Ensure defaults are filled in on error with <code>ignore_errors(true)</code></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Thomas Eizinger
|
48ba2869a8 |
chore(rust): ban the use of .unwrap except in tests (#7319)
Using the clippy lint `unwrap_used`, we can automatically lint against all uses of `.unwrap()` on `Result` and `Option`. This turns up quite a few results actually. In most cases, they are invariants that can't actually be hit. For these, we change them to `Option`. In other cases, they can actually be hit. For example, if the user supplies an invalid log-filter. Activating this lint ensures the compiler will yell at us every time we use `.unwrap` to double-check whether we do indeed want to panic here. Resolves: #7292. |
||
|
Thomas Eizinger
|
e261cb3c27 |
chore: remove git_version! (#7270)
Reading the Git version requires the entire Git repository to be present, including all tags. The tags are only created _after_ the artifact is being built, when we publish the release. Therefore, these tags are never included in the actual released binary. For Sentry, we use the `CARGO_PKG_VERSION` variable instead. This doesn't tell us whether somebody built a client from source and then used it so there could be some confusion in Sentry events. It is quite unlikely that this happens though so for the majority of Sentry alerts, this will give us the correct version. For the Android client, we also depend on the `GITHUB_SHA` env variable at compile-time. We do the same thing for the GUI client here. Resolves: #6925. |
||
|
dependabot[bot]
|
a2828a217b |
build(deps): Bump thiserror from 1.0.64 to 1.0.68 in /rust (#7260)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.64 to 1.0.68. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/dtolnay/thiserror/releases">thiserror's releases</a>.</em></p> <blockquote> <h2>1.0.68</h2> <ul> <li>Handle incomplete expressions more robustly in format arguments, such as while code is being typed (<a href="https://redirect.github.com/dtolnay/thiserror/issues/341">#341</a>, <a href="https://redirect.github.com/dtolnay/thiserror/issues/344">#344</a>)</li> </ul> <h2>1.0.67</h2> <ul> <li>Improve expression syntax support inside format arguments (<a href="https://redirect.github.com/dtolnay/thiserror/issues/335">#335</a>, <a href="https://redirect.github.com/dtolnay/thiserror/issues/337">#337</a>, <a href="https://redirect.github.com/dtolnay/thiserror/issues/339">#339</a>, <a href="https://redirect.github.com/dtolnay/thiserror/issues/340">#340</a>)</li> </ul> <h2>1.0.66</h2> <ul> <li>Improve compile error on malformed format attribute (<a href="https://redirect.github.com/dtolnay/thiserror/issues/327">#327</a>)</li> </ul> <h2>1.0.65</h2> <ul> <li>Ensure OUT_DIR is left with deterministic contents after build script execution (<a href="https://redirect.github.com/dtolnay/thiserror/issues/325">#325</a>)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Thomas Eizinger
|
73eebd2c4d |
refactor(rust): consistently record errors as tracing::Value (#7104)
Our logging library, `tracing` supports structured logging. This is useful because it preserves the more than just the string representation of a value and thus allows the active logging backend(s) to capture more information for a particular value. In the case of errors, this is especially useful because it allows us to capture the sources of a particular error. Unfortunately, recording an error as a tracing value is a bit cumbersome because `tracing::Value` is only implemented for `&dyn std::error::Error`. Casting an error to this is quite verbose. To make it easier, we introduce two utility functions in `firezone-logging`: - `std_dyn_err` - `anyhow_dyn_err` Tracking errors as correct `tracing::Value`s will be especially helpful once we enable Sentry's `tracing` integration: https://docs.rs/sentry-tracing/latest/sentry_tracing/#tracking-errors |
||
|
dependabot[bot]
|
28c8b676fb |
build(deps): Bump axum from 0.7.6 to 0.7.7 in /rust (#6871)
Bumps [axum](https://github.com/tokio-rs/axum) from 0.7.6 to 0.7.7. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/tokio-rs/axum/releases">axum's releases</a>.</em></p> <blockquote> <h2>axum-extra - v0.7.7</h2> <ul> <li><strong>added:</strong> <code>Clone</code> implementation for <code>ErasedJson</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/2142">#2142</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/2142">#2142</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/2142">tokio-rs/axum#2142</a></p> <h2>axum v0.7.7</h2> <ul> <li><strong>change</strong>: Remove manual tables of content from the documentation, since rustdoc now generates tables of content in the sidebar (<a href="https://redirect.github.com/tokio-rs/axum/issues/2921">#2921</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/2921">#2921</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/2921">tokio-rs/axum#2921</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
1140656b83 |
build(deps): Bump clap from 4.5.18 to 4.5.19 in /rust (#6950)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.18 to 4.5.19. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/clap-rs/clap/releases">clap's releases</a>.</em></p> <blockquote> <h2>v4.5.19</h2> <h2>[4.5.19] - 2024-10-01</h2> <h3>Internal</h3> <ul> <li>Update dependencies</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's changelog</a>.</em></p> <blockquote> <h2>[4.5.19] - 2024-10-01</h2> <h3>Internal</h3> <ul> <li>Update dependencies</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
fec6cc9923 |
build(deps): Bump clap from 4.5.4 to 4.5.13 in /rust (#6800)
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.4 to 4.5.13. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/clap-rs/clap/releases">clap's releases</a>.</em></p> <blockquote> <h2>v4.5.13</h2> <h2>[4.5.13] - 2024-07-31</h2> <h3>Fixes</h3> <ul> <li><em>(derive)</em> Improve error message when <code>#[flatten]</code>ing an optional <code>#[group(skip)]</code></li> <li><em>(help)</em> Properly wrap long subcommand descriptions in help</li> </ul> <h2>v4.5.12</h2> <h2>[4.5.12] - 2024-07-31</h2> <h2>v4.5.10</h2> <h2>[4.5.10] - 2024-07-23</h2> <h2>v4.5.9</h2> <h2>[4.5.9] - 2024-07-09</h2> <h3>Fixes</h3> <ul> <li><em>(error)</em> When defining a custom help flag, be sure to suggest it like we do the built-in one</li> </ul> <h2>v4.5.8</h2> <h2>[4.5.8] - 2024-06-28</h2> <h3>Fixes</h3> <ul> <li>Reduce extra flushes</li> </ul> <h2>v4.5.7</h2> <h2>[4.5.7] - 2024-06-10</h2> <h3>Fixes</h3> <ul> <li>Clean up error message when too few arguments for <code>num_args</code></li> </ul> <h2>v4.5.6</h2> <h2>[4.5.6] - 2024-06-06</h2> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's changelog</a>.</em></p> <blockquote> <h2>[4.5.13] - 2024-07-31</h2> <h3>Fixes</h3> <ul> <li><em>(derive)</em> Improve error message when <code>#[flatten]</code>ing an optional <code>#[group(skip)]</code></li> <li><em>(help)</em> Properly wrap long subcommand descriptions in help</li> </ul> <h2>[4.5.12] - 2024-07-31</h2> <h2>[4.5.11] - 2024-07-25</h2> <h2>[4.5.10] - 2024-07-23</h2> <h2>[4.5.9] - 2024-07-09</h2> <h3>Fixes</h3> <ul> <li><em>(error)</em> When defining a custom help flag, be sure to suggest it like we do the built-in one</li> </ul> <h2>[4.5.8] - 2024-06-28</h2> <h3>Fixes</h3> <ul> <li>Reduce extra flushes</li> </ul> <h2>[4.5.7] - 2024-06-10</h2> <h3>Fixes</h3> <ul> <li>Clean up error message when too few arguments for <code>num_args</code></li> </ul> <h2>[4.5.6] - 2024-06-06</h2> <h2>[4.5.5] - 2024-06-06</h2> <h3>Fixes</h3> <ul> <li>Allow <code>exclusive</code> to override <code>required_unless_present</code>, <code>required_unless_present_any</code>, <code>required_unless_present_all</code></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
6ae769c996 |
build(deps): Bump wintun from 0.4.0 to 0.5.0 in /rust (#6696)
Bumps [wintun](https://github.com/nulldotblack/wintun) from 0.4.0 to 0.5.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/nulldotblack/wintun/releases">wintun's releases</a>.</em></p> <blockquote> <h2>v0.5.0</h2> <h2>What's Changed</h2> <ul> <li>docs: correct "between min and min" to "between min and max" by <a href="https://github.com/ReactorScram"><code>@ReactorScram</code></a> in <a href="https://redirect.github.com/nulldotblack/wintun/pull/22">nulldotblack/wintun#22</a></li> <li>Upgrade to windows-sys crate by <a href="https://github.com/ssrlive"><code>@ssrlive</code></a> in <a href="https://redirect.github.com/nulldotblack/wintun/pull/20">nulldotblack/wintun#20</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/nulldotblack/wintun/compare/v0.4.0...v0.5.0">https://github.com/nulldotblack/wintun/compare/v0.4.0...v0.5.0</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/nulldotblack/wintun/blob/main/CHANGELOG.md">wintun's changelog</a>.</em></p> <blockquote> <h1>ChangeLog</h1> <p>This format is based on <a href="https://keepachangelog.com/">Keep a Changelog</a> and this project adheres to <a href="https://semver.org">Semantic Versioning</a>.</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
94cef31d52 |
build(deps): Bump known-folders from 1.1.0 to 1.2.0 in /rust (#6641)
Bumps [known-folders](https://github.com/artichoke/known-folders-rs) from 1.1.0 to 1.2.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/artichoke/known-folders-rs/releases">known-folders's releases</a>.</em></p> <blockquote> <h2>v1.2.0</h2> <h2>What's Changed</h2> <ul> <li>Update Ruby and bundler version by <a href="https://github.com/lopopolo"><code>@lopopolo</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/32">artichoke/known-folders-rs#32</a></li> <li>Bump the gha-deps group with 1 update by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/34">artichoke/known-folders-rs#34</a></li> <li>Bump the bundler-deps group with 1 update by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/33">artichoke/known-folders-rs#33</a></li> <li>Bump the bundler-deps group with 1 update by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/35">artichoke/known-folders-rs#35</a></li> <li>Bump the gha-deps group with 3 updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/36">artichoke/known-folders-rs#36</a></li> <li>Bump the gha-deps group with 3 updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/38">artichoke/known-folders-rs#38</a></li> <li>Bump the bundler-deps group with 1 update by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/37">artichoke/known-folders-rs#37</a></li> <li>Bump the bundler-deps group with 1 update by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/40">artichoke/known-folders-rs#40</a></li> <li>Bump the gha-deps group with 4 updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/39">artichoke/known-folders-rs#39</a></li> <li>Bump the bundler-deps group with 2 updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/42">artichoke/known-folders-rs#42</a></li> <li>Bump the gha-deps group with 5 updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/41">artichoke/known-folders-rs#41</a></li> <li>Bump rexml from 3.2.6 to 3.2.8 by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/43">artichoke/known-folders-rs#43</a></li> <li>Bump rubocop from 1.63.4 to 1.64.1 in the bundler-deps group by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/45">artichoke/known-folders-rs#45</a></li> <li>Bump the gha-deps group across 1 directory with 2 updates by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/47">artichoke/known-folders-rs#47</a></li> <li>Upgrade Ruby and bundler by <a href="https://github.com/lopopolo"><code>@lopopolo</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/48">artichoke/known-folders-rs#48</a></li> <li>Bump rubocop from 1.64.1 to 1.65.1 in the bundler-deps group by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/49">artichoke/known-folders-rs#49</a></li> <li>Update windows-sys requirement from 0.52.0 to 0.59.0 in the cargo-deps group by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/artichoke/known-folders-rs/pull/50">artichoke/known-folders-rs#50</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/artichoke/known-folders-rs/compare/v1.1.0...v1.2.0">https://github.com/artichoke/known-folders-rs/compare/v1.1.0...v1.2.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Reactor Scram
|
f507a01f9f |
fix(windows): prevent routing loops for TCP connections (#6584)
In #6032, we attempted to fix routing loops for Windows and did so successfully for UDP packets. For TCP sockets, we believed that binding the socket to an interface is enough to prevent routing loops. This assumptions is wrong. > On Windows, a call to bind() affects card selection only incoming traffic, not outgoing traffic. > > Thus, on a client running in a multi-homed system (i.e., more than one interface card), it's the network stack that selects the card to use, and it makes its selection based solely on the destination IP, which in turn is based on the routing table. A call to bind() will not affect the choice of the card in any way. On most of our testing machines, this problem didn't surface but it turns out that on some machines, especially with WiFi cards there is a conflict between the routes added on the system. In particular, with the Internet resource active, we also add a catch-all route that we _want_ to have the most priority, i.e. Windows SHOULD send all traffic to our TUN device. Except for traffic that we generate, like TCP connections to the portal or UDP packets sent to gateways, relays or DNS servers. It appears that on some systems, mostly with Ethernet adapters, Windows picks the "correct" interface for our socket and sends traffic via that but on other systems, it doesn't. TCP sockets are only used for the WebSocket connection to the portal. Without that one, Firezone completely stops working because we can't send any control messages. To reliably fix this issue, we need to add a dedicated route for the target IP of each TCP socket that is more specific than the Internet resource route (`0.0.0.0/0`) but otherwise identical. We do this as part of creating a new TCP socket. This route is for the _default_ interface and thus, doesn't get automatically removed when Firezone exits. We implement a RAII guard that attempts to drop the route on a best-effort basis. Despite this RAII guard, this route can linger around in case Firezone is being forced to exit or exits in otherwise unclean ways. To avoid lingering routes, we always delete all routing table entries matching the IP of the portal just before we are about to add one. Fixes: #6591. [0]: https://forums.codeguru.com/showthread.php?487139-Socket-binding-with-routing-table&s=a31637836c1bf7f0bc71c1955e47bdf9&p=1891235#post1891235 --------- Signed-off-by: Thomas Eizinger <thomas@eizinger.io> Co-authored-by: Thomas Eizinger <thomas@eizinger.io> Co-authored-by: Foo Bar <foo@bar.com> Co-authored-by: conectado <gabrielalejandro7@gmail.com> |
||
|
Reactor Scram
|
d7810ef9c0 |
chore(rust/gui-client/windows): update windows to 0.58 (#6565)
Updates `windows` crates to 0.58 without the bug in #6551. Supersedes #6556. The bug was calling `try_send()?` on an MPSC channel of capacity 1, which would bail out of the worker thread if we got 2 DNS change notifications faster than the controller task / thread could process the first one. |
||
|
Reactor Scram
|
1505b699e5 |
fix(client/windows): Revert "chore(rust/gui-client/windows): update windows to 0.58 (#6506)" (#6555)
This reverts commit
|
||
|
Reactor Scram
|
d8f25f9bf8 |
chore(rust/gui-client/windows): update windows to 0.58 (#6506)
Supersedes #5913 This required a big refactor because `HANDLE` is no longer `Send` and was never supposed to be. So we add a worker thread for listening to DNS changes, since that requires us to hold a `HANDLE` across `await` points and I couldn't find any simpler way to do it. I could add integration tests for this in a future PR that prove the notifiers work by poking the registry or setting DNS servers and seeing if we pick up the changes on time. But setting DNS servers without the tunnel up may be tricky, so I left it out of scope for this PR. ```[tasklist] - [x] Fix force-kill bug ``` |
||
|
dependabot[bot]
|
217faeabf0 |
build(deps): Bump nix from 0.28.0 to 0.29.0 in /rust (#6343)
Bumps [nix](https://github.com/nix-rust/nix) from 0.28.0 to 0.29.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/nix-rust/nix/blob/master/CHANGELOG.md">nix's changelog</a>.</em></p> <blockquote> <h2>[0.29.0] - 2024-05-24</h2> <h3>Added</h3> <ul> <li>Add <code>getregset()/setregset()</code> for Linux/glibc/x86/x86_64/aarch64/riscv64 and <code>getregs()/setregs()</code> for Linux/glibc/aarch64/riscv64 (<a href="https://redirect.github.com/nix-rust/nix/pull/2044">#2044</a>)</li> <li>Add socket option Ipv6Ttl for apple targets. (<a href="https://redirect.github.com/nix-rust/nix/pull/2287">#2287</a>)</li> <li>Add socket option UtunIfname. (<a href="https://redirect.github.com/nix-rust/nix/pull/2325">#2325</a>)</li> <li>make SigAction repr(transparent) & can be converted to the libc raw type (<a href="https://redirect.github.com/nix-rust/nix/pull/2326">#2326</a>)</li> <li>Add <code>From</code> trait implementation for conversions between <code>sockaddr_in</code> and <code>SockaddrIn</code>, <code>sockaddr_in6</code> and <code>SockaddrIn6</code> (<a href="https://redirect.github.com/nix-rust/nix/pull/2328">#2328</a>)</li> <li>Add socket option ReusePortLb for FreeBSD. (<a href="https://redirect.github.com/nix-rust/nix/pull/2332">#2332</a>)</li> <li>Added support for openat2 on linux. (<a href="https://redirect.github.com/nix-rust/nix/pull/2339">#2339</a>)</li> <li>Add if_indextoname function. (<a href="https://redirect.github.com/nix-rust/nix/pull/2340">#2340</a>)</li> <li>Add <code>mount</code> and <code>unmount</code> API for apple targets. (<a href="https://redirect.github.com/nix-rust/nix/pull/2347">#2347</a>)</li> <li>Added <code>_PC_MIN_HOLE_SIZE</code> for <code>pathconf</code> and <code>fpathconf</code>. (<a href="https://redirect.github.com/nix-rust/nix/pull/2349">#2349</a>)</li> <li>Added <code>impl AsFd for pty::PtyMaster</code> (<a href="https://redirect.github.com/nix-rust/nix/pull/2355">#2355</a>)</li> <li>Add <code>open</code> flag <code>O_SEARCH</code> to AIX, Empscripten, FreeBSD, Fuchsia, solarish, WASI (<a href="https://redirect.github.com/nix-rust/nix/pull/2374">#2374</a>)</li> <li>Add prctl function <code>prctl_set_vma_anon_name</code> for Linux/Android. (<a href="https://redirect.github.com/nix-rust/nix/pull/2378">#2378</a>)</li> <li>Add <code>sync(2)</code> for <code>apple_targets/solarish/haiku/aix/hurd</code>, <code>syncfs(2)</code> for <code>hurd</code> and <code>fdatasync(2)</code> for <code>aix/hurd</code> (<a href="https://redirect.github.com/nix-rust/nix/pull/2379">#2379</a>)</li> <li>Add fdatasync support for Apple targets. (<a href="https://redirect.github.com/nix-rust/nix/pull/2380">#2380</a>)</li> <li>Add <code>fcntl::OFlag::O_PATH</code> for FreeBSD and Fuchsia (<a href="https://redirect.github.com/nix-rust/nix/pull/2382">#2382</a>)</li> <li>Added <code>PathconfVar::MIN_HOLE_SIZE</code> for apple_targets. (<a href="https://redirect.github.com/nix-rust/nix/pull/2388">#2388</a>)</li> <li>Add <code>open</code> flag <code>O_SEARCH</code> to apple_targets (<a href="https://redirect.github.com/nix-rust/nix/pull/2391">#2391</a>)</li> <li><code>O_DSYNC</code> may now be used with <code>aio_fsync</code> and <code>fcntl</code> on FreeBSD. (<a href="https://redirect.github.com/nix-rust/nix/pull/2404">#2404</a>)</li> <li>Added <code>Flock::relock</code> for upgrading and downgrading locks. (<a href="https://redirect.github.com/nix-rust/nix/pull/2407">#2407</a>)</li> </ul> <h3>Changed</h3> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Thomas Eizinger
|
0abbf6bba9 |
refactor(rust): inline http-health-check crate into bin-shared (#6258)
Now that we have the `bin-shared` crate, it is easy to move the health-check functionality into there. That allows us to get rid of a crate which makes navigating the workspace a bit easier. |
||
|
Thomas Eizinger
|
bed625a312 |
chore(rust): make logging more ergonomic (#6237)
Setting up a logger is something that pretty much every entrypoint needs to do, be it a test, a shared library embedded in another app or a standalone application. Thus, it makes sense to introduce a dedicated crate that allows us to bundle all the things together, how we want to do logging. This allows us to introduce convenience functions like `firezone_logging::test` which allow you to construct a logger for a test as a one-liner. Crucially though, introducing `firezone-logging` gives us a place to store a default log directive that silences very noisy crates. When looking into a problem, it is common to start by simply setting the log-filter to `debug`. Without further action, this floods the output with logs from crates like `netlink_proto` on Linux. It is very unlikely that those are the logs that you want to see. Without a preset filter, the only alternative here is to explicitly turn off the log filter for `netlink_proto` by typing something like `RUST_LOG=netlink_proto=off,debug`. Especially when debugging issues with customers, this is annoying. Log filters can be overridden, i.e. a 2nd filter that matches the exact same scope overrides a previous one. Thus, with this design it is still possible to activate certain logs at runtime, even if they have silenced by default. I'd expect `firezone-logging` to attract more functionality in the future. For example, we want to support re-loading of log-filters on other platforms. Additionally, where logs get stored could also be defined in this crate. --------- Signed-off-by: Thomas Eizinger <thomas@eizinger.io> Co-authored-by: Reactor Scram <ReactorScram@users.noreply.github.com> |
||
|
Thomas Eizinger
|
a87728b791 |
chore: remove connlib-shared dependency from bin-shared (#6229)
The `firezone-bin-shared` crate is meant to house non-tunnel related things. That allows it to compile in parallel to everything else. It currently only depends on `connlib-shared` to access the `DEFAULT_MTU` constant. We can remove that by requiring the MTU as a ctor parameter of `TunDeviceManager`. A longer write-up of the intended dependency structure is in #4470. |
||
|
Reactor Scram
|
68d934ee59 |
refactor(headless-client): remove unnecessary layering (#6211)
Refs #5754 The IPC service is still layered, but moving it around is more difficult than moving the headless Client. |
||
|
Thomas Eizinger
|
128d0eb407 |
feat(connlib): transparently forward non-resources DNS queries (#6181)
Currently, `connlib` depends on `hickory-resolver` to perform DNS queries for non-resources. This is unnecessary. Instead of buffering the original UDP DNS query, consulting hickory to resolve the name and mapping the response back, we can simply take the UDP payload and send it via our protected socket directly to the original upstream DNS server. This ensures `connlib` is as transparent as possible for DNS queries for non-resources. Additionally, it removes a lot of error handling and other cruft that we currently have to perform because we are using hickory. For example, hickory will automatically retry a DNS query after a certain timeout. However, the OS / client talking to `connlib` will also retry after a certain timeout because it is making DNS queries over an unreliable transport (UDP). It is thus unnecessary for us to do that internally. To correctly test this change, our test-suite needed some refactoring. Specifically, DNS servers are now modelled as dedicated `Host`s that can receive (UDP) traffic. Lastly, we can remove our dependency on `hickory-proto` and `hickory-resolver` everywhere and only use `domain` for parsing DNS messages. Resolves: #6141. Related: #6033. Related: #4800. (Impossible to happen with this design) |
||
|
Reactor Scram
|
5eb2bba47b |
feat(headless-client): use systemd-resolved DNS control by default (#6163)
Closes #5063, supersedes #5850 Other refactors and changes made as part of this: - Adds the ability to disable DNS control on Windows - Removes the spooky-action-at-a-distance `from_env` functions that used to be buried in `tunnel` - `FIREZONE_DNS_CONTROL` is now a regular `clap` argument again --------- Signed-off-by: Reactor Scram <ReactorScram@users.noreply.github.com> |
||
|
Gabi
|
5841f297a5 |
fix(gateway): prevent routing loops (#6096)
In some weird conditions there might be routing loops in the gateway too, so this fixes it and it doesn't do any harm. Could be the cause behind [these logs](https://github.com/firezone/firezone/issues/6067#issuecomment-2259081958) |
||
|
Reactor Scram
|
e6cbb5fa8a |
feat(gui-client/linux): network roaming (#5978)
Closes #5846 Will be moved down to the IPC service eventually. The goal for connection roaming is not for totally transparent "Change Wi-Fi networks without dropping SSH" handoffs, but just for Firezone to re-connect itself as quickly as possible so that everything above us can re-connect as quickly as it times out, and won't be hung up with a broken tunnel. |
||
|
Gabi
|
c3a45f53df |
fix(connlib): prevent routing loops on windows (#6032)
In `connlib`, traffic is sent through sockets via one of three ways: 1. Direct p2p traffic between clients and gateways: For these, we always explicitly set the source IP (and thus interface). 2. UDP traffic to the relays: For these, we let the OS pick an appropriate source interface. 3. WebSocket traffic over TCP to the portal: For this too, we let the OS pick the source interface. For (2) and (3), it is possible to run into routing loops, depending on the routes that we have configured on the TUN device. In Linux, we can prevent routing loops by marking a socket [0] and repeating the mark when we add routes [1]. Packets sent via a marked socket won't be routed by a rule that contains this mark. On Android, we can do something similar by "protecting" a socket via a syscall on the Java side [2]. On Windows, routing works slightly different. There, the source interface is determined based on a computed metric [3] [4]. To prevent routing loops on Windows, we thus need to find the "next best" interface after our TUN interface. We can achieve this with a combination of several syscalls: 1. List all interfaces on the machine 2. Ask Windows for the best route on each interface, except our TUN interface. 3. Sort by Windows' routing metric and pick the lowest one (lower is better). Thanks to the abstraction of `SocketFactory` that we already previously introduced, Integrating this into `connlib` isn't too difficult: 1. For TCP sockets, we simply resolve the best route after creating the socket and then bind it to that local interface. That way, all packets will always going via that interface, regardless of which routes are present on our TUN interface. 2. UDP is connection-less so we need to decide per-packet, which interface to use. "Pick the best interface for me" is modelled in `connlib` via the `DatagramOut::src` field being `None`. - To ensure those packets don't cause a routing loop, we introduce a "source IP resolver" for our `UdpSocket`. This function gets called every time we need to send a packet without a source IP. - For improved performance, we cache these results. The Windows client uses this source IP resolver to use the above devised strategy to find a suitable source IP. - In case the source IP resolution fails, we don't send the packet. This is important, otherwise, the kernel might choose our TUN interface again and trigger a routing loop. The last remark to make here is that this also works for connection roaming. The TCP socket gets thrown away when we reconnect to the portal. Thus, the new socket will pick the new best interface as it is re-created. The UDP sockets also get thrown away as part of roaming. That clears the above cache which is what we want: Upon roaming, the best interface for a given destination IP will likely have changed. [0]: |
||
|
Reactor Scram
|
05e3a38701 |
refactor(bin-shared): remove CommonArgs (#6068)
Closes #6025 It was only used in the Gateway, so we inline it there and remove `clap` as a dep for ~~that crate~~ `bin-shared` |
||
|
Reactor Scram
|
cc1478adc2 |
feat(headless-client/windows): add DNS change / network change listening to the Headless Client (#6022)
Note that for GUI Clients, listening is still done by the GUI process, not the IPC service. Yak shave towards #5846. This allows for faster dev cycles since I won't have to compile all the GUI stuff. Some changes in here were extracted from other draft PRs. Changes: - Remove `thiserror` that was never matched on - Don't return the DNS resolvers from the notifier directly, just send a notification and allow the caller to check the resolvers itself if needed - Rename `DnsListener` to `DnsNotifier` - Rename `Worker` to `NetworkNotifier` - remove `unwrap_or_default` when getting resolvers. I don't know why it's there, if there's a good reason then it should be handled inside the function, not in the caller ```[tasklist] ### Tasks - [x] Rename `*Listener` to `*Notifier` - [x] (not needed) ~~Support `/etc/resolv.conf` DNS control method too?~~ ``` |
||
|
Reactor Scram
|
82b8de4c9c |
refactor(client/windows): de-dupe wintun.dll (#6020)
Closes #5977 Refactored some other stuff to make this work Also removed a redundant impl of `ensure_dll` in a benchmark |
||
|
dependabot[bot]
|
7be47f2c6e |
build(deps): Bump url from 2.5.0 to 2.5.2 in /rust (#6002)
Bumps [url](https://github.com/servo/rust-url) from 2.5.0 to 2.5.2. <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
dependabot[bot]
|
6d09344521 |
build(deps): Bump uuid from 1.8.0 to 1.10.0 in /rust (#6005)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.8.0 to 1.10.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/uuid-rs/uuid/releases">uuid's releases</a>.</em></p> <blockquote> <h2>1.10.0</h2> <h2>Deprecations</h2> <p>This release deprecates and renames the following functions:</p> <ul> <li><code>Builder::from_rfc4122_timestamp</code> -> <code>Builder::from_gregorian_timestamp</code></li> <li><code>Builder::from_sorted_rfc4122_timestamp</code> -> <code>Builder::from_sorted_gregorian_timestamp</code></li> <li><code>Timestamp::from_rfc4122</code> -> <code>Timestamp::from_gregorian</code></li> <li><code>Timestamp::to_rfc4122</code> -> <code>Timestamp::to_gregorian</code></li> </ul> <h2>What's Changed</h2> <ul> <li>Use const identifier in uuid macro by <a href="https://github.com/Vrajs16"><code>@Vrajs16</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/764">uuid-rs/uuid#764</a></li> <li>Rename most methods referring to RFC4122 by <a href="https://github.com/Mikopet"><code>@Mikopet</code></a> / <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/765">uuid-rs/uuid#765</a></li> <li>prepare for 1.10.0 release by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/766">uuid-rs/uuid#766</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/Vrajs16"><code>@Vrajs16</code></a> made their first contribution in <a href="https://redirect.github.com/uuid-rs/uuid/pull/764">uuid-rs/uuid#764</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/uuid-rs/uuid/compare/1.9.1...1.10.0">https://github.com/uuid-rs/uuid/compare/1.9.1...1.10.0</a></p> <h2>1.9.1</h2> <h2>What's Changed</h2> <ul> <li>Add an example of generating bulk v7 UUIDs by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/761">uuid-rs/uuid#761</a></li> <li>Avoid taking the shared lock when getting usable bits in Uuid::now_v7 by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/762">uuid-rs/uuid#762</a></li> <li>Prepare for 1.9.1 release by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/763">uuid-rs/uuid#763</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/uuid-rs/uuid/compare/1.9.0...1.9.1">https://github.com/uuid-rs/uuid/compare/1.9.0...1.9.1</a></p> <h2>1.9.0</h2> <h2><code>Uuid::now_v7()</code> is guaranteed to be monotonic</h2> <p>Before this release, <code>Uuid::now_v7()</code> would only use the millisecond-precision timestamp for ordering. It now also uses a global 42-bit counter that's re-initialized each millisecond so that the following will always pass:</p> <pre lang="rust"><code>let a = Uuid::now_v7(); let b = Uuid::now_v7(); <p>assert!(a < b);<br /> </code></pre></p> <h2>What's Changed</h2> <ul> <li>Add a get_node_id method for v1 and v6 UUIDs by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/748">uuid-rs/uuid#748</a></li> <li>Update atomic and zerocopy to latest by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/750">uuid-rs/uuid#750</a></li> <li>Add repository field to uuid-macro-internal crate by <a href="https://github.com/paolobarbolini"><code>@paolobarbolini</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/752">uuid-rs/uuid#752</a></li> <li>update docs to updated RFC (from 4122 to 9562) by <a href="https://github.com/Mikopet"><code>@Mikopet</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/753">uuid-rs/uuid#753</a></li> <li>Support counters in v7 UUIDs by <a href="https://github.com/KodrAus"><code>@KodrAus</code></a> in <a href="https://redirect.github.com/uuid-rs/uuid/pull/755">uuid-rs/uuid#755</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/paolobarbolini"><code>@paolobarbolini</code></a> made their first contribution in <a href="https://redirect.github.com/uuid-rs/uuid/pull/752">uuid-rs/uuid#752</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
||
|
Thomas Eizinger
|
50d6b865a1 |
refactor(connlib): move Tun implementations out of firezone-tunnel (#5903)
The different implementations of `Tun` are the last platform-specific code within `firezone-tunnel`. By introducing a dedicated crate and a `Tun` trait, we can move this code into (platform-specific) leaf crates: - `connlib-client-android` - `connlib-client-apple` - `firezone-bin-shared` Related: #4473. --------- Co-authored-by: Not Applicable <ReactorScram@users.noreply.github.com> |
||
|
Thomas Eizinger
|
a4a8221b8b |
refactor(connlib): explicitly initialise Tun (#5839)
Connlib's routing logic and networking code is entirely platform agnostic. The only platform-specific bit is how we interact with the TUN device. From connlib's perspective though, all it needs is an interface for reading and writing. How the device gets initialised and updated is client-business. For the most part, this is the same on all platforms: We call callbacks and the client updates the state accordingly. The only annoying bit here is that Android recreates the TUN interface on every update and thus our old file descriptor is invalid. The current design works around this by returning the new file descriptor on Android. This is a problematic design for several reasons: - It forces the callback handler to finish synchronously, and halting connlib until this is complete. - The synchronous nature also means we cannot replace the callbacks with events as events don't have a return value. To fix this, we introduce a new `set_tun` method on `Tunnel`. This moves the business of how the `Tun` device is created up to the client. The clients are already platform-specific so this makes sense. In a future iteration, we can move all the various `Tun` implementations all the way up to the client-specific crates, thus co-locating the platform-specific code. Initialising `Tun` from the outside surfaces another issue: The routes are still set via the `Tun` handle on Windows. To fix this, we introduce a `make_tun` function on `TunDeviceManager` in order for it to remember the interface index on Windows and being able to move the setting of routes to `TunDeviceManager`. This simplifies several of connlib's APIs which are now infallible. Resolves: #4473. --------- Co-authored-by: Reactor Scram <ReactorScram@users.noreply.github.com> Co-authored-by: conectado <gabrielalejandro7@gmail.com> |
||
|
Thomas Eizinger
|
c92dd559f7 |
chore(rust): format Cargo.toml using cargo-sort (#5851)
|
||
|
Thomas Eizinger
|
960ce80680 |
refactor(connlib): move TunDeviceManager into firezone-bin-shared (#5843)
The `TunDeviceManager` is a component that the leaf-nodes of our dependency tree need: the binaries. Thus, it is misplaced in the `connlib-shared` crate which is at the very bottom of the dependency tree. This is necessary to allow the `TunDeviceManager` to actually construct a `Tun` (which currently lives in `firezone-tunnel`). Related: #5839. --------- Signed-off-by: Thomas Eizinger <thomas@eizinger.io> Co-authored-by: Reactor Scram <ReactorScram@users.noreply.github.com> |