github-personal/firezone
1
0
Fork 0
mirror of https://github.com/outbackdingo/firezone.git synced 2026-01-27 10:18:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(linux): try to set rmem_max and wmem_max on startup (#10349)

Browse Source 
The default send and receive buffer sizes on Linux are too small (only
~200 KB). Checking `nstat` after an iperf run revealed that the number
of dropped packets in the first interval directly correlates with the
number of receive buffer errors reported by `nstat`.

We already try to increase the send and receive buffer sizes for our UDP
socket but unfortunately, we cannot increase them beyond what the system
limits them to. To workaround this, we try to set `rmem_max` and
`wmem_max` during startup of the Linux headless client and Gateway. This
behaviour can be disabled by setting `FIREZONE_NO_INC_BUF=true`.

This doesn't work in Docker unfortunately, so we set the values manually
in the CI perf tests and verify after the test that we didn't encounter
any send and receive buffer errors.

It is yet to be determined how we should deal with this problem for all
the GUI clients. See #10350 as an issue tracking that.

Unfortunately, this doesn't fix all packet drops during the first iperf
interval. With this PR, we now see packet drops on the interface itself.
This commit is contained in:
Thomas Eizinger
2025-09-17 23:05:01 +00:00
committed by GitHub
parent 7222167b13
commit 3e6094af8d
9 changed files with 95 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
rust/gateway/src/main.rs
View File

@@ -101,6 +101,20 @@ async fn try_main(cli: Cli, telemetry: &mut Telemetry) -> Result<()> {
tracing::debug!(?cli);
if cfg!(target_os = "linux") && cli.is_inc_buf_allowed() {
let recv_buf_size = socket_factory::RECV_BUFFER_SIZE;
let send_buf_size = socket_factory::SEND_BUFFER_SIZE;
match tokio::fs::write("/proc/sys/net/core/rmem_max", recv_buf_size.to_string()).await {
Ok(()) => tracing::info!("Set `core.rmem_max` to {recv_buf_size}",),
Err(e) => tracing::info!("Failed to increase `core.rmem_max`: {e}"),
};
match tokio::fs::write("/proc/sys/net/core/wmem_max", send_buf_size.to_string()).await {
Ok(()) => tracing::info!("Set `core.wmem_max` to {send_buf_size}",),
Err(e) => tracing::info!("Failed to increase `core.wmem_max`: {e}"),
};
}
let firezone_id = get_firezone_id(cli.firezone_id.clone()).await
.context("Couldn't read FIREZONE_ID or write it to disk: Please provide it through the env variable or provide rw access to /var/lib/firezone/")?;
@@ -290,6 +304,10 @@ struct Cli {
default_value_t = false
)]
validate_checksums: bool,
/// Do not try to increase the `core.rmem_max` and `core.wmem_max` kernel parameters.
#[arg(long, env = "FIREZONE_NO_INC_BUF", default_value_t = false)]
no_inc_buf: bool,
}
#[derive(Debug, Clone, Copy, clap::ValueEnum)]
@@ -302,6 +320,10 @@ impl Cli {
fn is_telemetry_allowed(&self) -> bool {
!self.no_telemetry
}
fn is_inc_buf_allowed(&self) -> bool {
!self.no_inc_buf
}
}
/// An adapter struct around [`Tun`] that validates IPv4, UDP and TCP checksums.