In order to allow the portal to more easily classify, what kind of
component is connecting, we extend the `get_user_agent` header to
include a component type instead of the generic `connlib/`.
---------
Signed-off-by: Thomas Eizinger <thomas@eizinger.io>
Co-authored-by: Jamil <jamilbk@users.noreply.github.com>
With the introduction of the pre-resolved Sentry host, all Firezone
clients now require Internet on startup. That is a signficant usability
hit that we can easily fix by simply falling back to resolving the host
on-demand.
We always end up allow this lint when it pops up so we can also just
allow it for the whole repo in general. Most of the time, the reason for
too many arguments are borrow-checker limitations of Rust where mutable
references need to be tracked explicitly.
Right now, the Client event-loops have a channel with 1000 items for
sending new resource lists and updates to the TUN device to the host
app. This is kind of unnecessary as we always only care about the last
version of these. Intermediate updates that the host app doesn't process
are effectively irrelevant.
We've had an issue before where a bug in the portal caused us to receive
many updates to resources which ended up crashing Client apps because
this channel filled up.
To be more resilient on this front, we refactor the Client event loop to
use a `watch` channel for this. Watch channels only retain the last
value that got sent into them.
Our Sentry client needs to resolve DNS before being able to send logs or
errors to the backend. Currently, this DNS resolution happens on-demand
as we don't take any control of the underlying HTTP client.
In addition, this will use HTTP/1.1 by default which isn't as efficient
as it could be, especially with concurrent requests.
Finally, if we decide to ever proxy all Sentry for traffic through our
own domain, we have to take control of the underlying client anyway.
To resolve all of the above, we create a custom `TransportFactory` where
we reuse the existing `ReqwestHttpTransport` but provide an already
configured `reqwest::Client` that always uses HTTP/2 with a
pre-configured set of DNS records for the given ingest host.
By default, dropping a `tokio` runtime waits until all tasks have
finished. The tasks we spawn within `connlib` can have complex
dependencies with each other. To ensure that we can shut down in any
case and don't hang, we apply a timeout of 1s to the runtime.
These don't really tell us much. It appears that Windows is sometimes
failing to access the pipe but then succeeds on the next attempt, hence
why we have the retry loop in the first place. Logging a warning here
just spams Sentry unnecessarily.
Despite still being in development, the `tauri-specta` project already
proves to be quite useful. It allows us to generate TypeScript bindings
for our commands and events, creating a type-safe contract between the
frontend and the backend.
For example, this ensures that the TypeScript code calls a command
actually with the required parameters and thus avoids runtime failures.
Similarly, the frontend can listen on type-safe events without having to
use any magic strings.
When looking through customer logs, we see a lot of "Resolved best route
outside of tunnel" messages. Those get logged every time we need to
rerun our re-implementation of Windows' weighting algorithm as to which
source interface / IP a packet should be sent from.
Currently, this gets cached in every socket instance so for the
peer-to-peer socket, this is only computed once per destination IP.
However, for DNS queries, we make a new socket for every query. Using a
new source port DNS queries is recommended to avoid fingerprinting of
DNS queries. Using a new socket also means that we need to re-run this
algorithm every time we make a DNS query which is why we see this log so
often.
To fix this, we need to share this cache across all UDP sockets. Cache
invalidation is one of the hardest problems in computer science and this
instance is no different. This cache needs to be reset every time we
roam as that changes the weighting of which source interface to use.
To achieve this, we extend the `SocketFactory` trait with a `reset`
method. This method is called whenever we roam and can then reset a
shared cache inside the `UdpSocketFactory`. The "source IP resolver"
function that is passed to the UDP socket now simply accesses this
shared cache and inserts a new entry when it needs to resolve the IP.
As an added benefit, this may speed up DNS queries on Windows a bit
(although I haven't benchmarked it). It should certainly drastically
reduce the amount of syscalls we make on Windows.
Rust 1.88 has been released and brings with it a quite exciting feature:
let-chains! It allows us to mix-and-match `if` and `let` expressions,
therefore often reducing the "right-drift" of the relevant code, making
it easier to read.
Rust.188 also comes with a new clippy lint that warns when creating a
mutable reference from an immutable pointer. Attempting to fix this
revealed that this is exactly what we are doing in the eBPF kernel.
Unfortunately, it doesn't seem to be possible to design this in a way
that is both accepted by the borrow-checker AND by the eBPF verifier.
Hence, we simply make the function `unsafe` and document for the
programmer, what needs to be upheld.
At present, our primary indicator as to whether telemetry is active is
whether we have a Sentry session. For our analytics events however, we
currently require passing in the Firezone ID and API url again. This
makes it difficult to send analytics events from areas of the code that
don't have this information available.
To still allow for that, we integrate the `analytics` module more
tightly with the Sentry session. This allows us to drop two parameters
from the `$identify` event and also means we now respect the
`NO_TELEMETRY` setting for these events except for `new_session`. This
event is sent regardless because it allows us to track, how many on-prem
installations of Firezone are out there.
This log is misplaced within the current `try_connect` function because
that will also be called when we didn't have Internet while we tried to
first connect and then witnessed a network change (in which case the
token is cached in the `WaitingForNetwork` session state).
Thus, we move the log to the `Connect` msg handler where we shouldn't
have any existing session.
Instead of conditionally enabling the `logs` feature in the Sentry
client, we always enable it and control via the `tracing` integration,
which events should get forwarded to Sentry. The feature-flag check
accesses only shared-memory and is therefore really fast.
We already re-evaluate feature flags on a timer which means this boolean
will flip over automatically and logs will be streamed to Sentry.
Customer hit what seems to be a rare race condition where we try to
connect whilst we already have a session. I don't know which state it is
in so I am replacing it with a WARN log to learn more about this in
Sentry in case it gets hit again.
I believe some of the recent changes around how we load the
`firezone-id.json` from the GUI client surfaced that we in fact don't
always have access to it. Previously, this was silenced because we would
only optionally add it as context to the Sentry client.
Now, we need it to initialise telemetry so we know whether or not to
send logs to Sentry.
In order to be able to access the file, we need to change the config's
directory and the file to be owned by the `firezone-client` group.
The GUI client binary performs quite a few checks prior to setting up
logging. In order to log at least something, we have a bootstrap logger
config that logs to stdout based on the `RUST_LOG` env var.
However, in the context of an error, the logger guard was dropped to
early and therefore we couldn't actually see the error.
To fix this, we pass a mutable `Option` in to `try_main` instead. This
allows the function to drop the bootstrap logger once the real one is
set up but also keep logging using the bootstrap logger in case of an
error.
Originally, we introduced these to gather some data from logs / warnings
that we considered to be too spammy. We've since merged a
burst-protection that will at most submit the same event once every 5
minutes.
The data from the telemetry spans themselves have not been used at all.
I suspect that the new Windows runners are "too fast" and we hit a race
condition in the use of the keyring on Windows which causes failing CI
jobs. The attempt to fix this is to sleep for 1 seconds before every
assert in the test.
Sentry has a new "Logs" feature where we can stream logs directly to
Sentry. Doing this for all Clients and Gateways would be way too much
data to collect though.
In order to aid debugging from customer installations, we add a
PostHog-managed feature flag that - if set to `true` - enables the
streaming of logs to Sentry. This feature flag is evaluated every time
the telemetry context is initialised:
- For all FFI usages of connlib, this happens every time a new session
is created.
- For the Windows/Linux Tunnel service, this also happens every time we
create a new session.
- For the Headless Client and Gateway, it happens on startup and
afterwards, every minute. The feature-flag context itself is only
checked every 5 minutes though so it might take up to 5 minutes before
this takes effect.
The default value - like all feature flags - is `false`. Therefore, if
there is any issue with the PostHog service, we will fallback to the
previous behaviour where logs are simply stored locally.
Resolves: #9600
In order to more easily target customers with certain feature flags, we
include the `account_slug` in the `$identify` event to PostHog. This
will allow us to create Cohorts in PostHog and enable / disable feature
flags for all installations of Firezone for a particular customer.
At present, listening for DNS server change and network change events is
handled in the GUI client. Upon an event, a message is sent to the
tunnel service which then applies the new state to `connlib`.
We can avoid some of this boilerplate by moving these listeners to the
tunnel service as part of the handler. As a result, we get a few
improvements:
- We don't need to ignore these events if we don't have a session
because the lifetime of these listeners is tied to the IPC handler on
the service side.
- We need fewer IPC messages
- We can retry the connection directly from within the tunnel service in
case we have no Internet at the time of startup
- We can more easily model out the state machine of a connlib session in
the tunnel service
- On Linux, this means we no longer shell out to `resolvectl` from the
GUI process, unifying access to the "resolvers" from the tunnel service
- On Windows, we no longer need admin privileges on the GUI client for
optimized network-change detection. This now happens in the Tunnel
process which already runs as admin.
Resolves: #9465
With the introduction of the "connect on start" configuration option, we
introduced a bug where the GUI client said "Signed in as ..." even
though we did not have a `connlib` session. The tray-menu handles this
state correctly and clicking sign out and sign in restores Firezone to a
functional state.
This disparity happened because we assumed that having a token means we
must have a session.
To fix this, we introduce a new `SessionViewModel` that combines the
state of the auth session and the `connlib` state. Only if we have both
do we infer that we are "signed in". This also requires us to introduce
an intermediary state where we are "loading". This is represented as a
spinner in the UI.
Last but not least, this also removes the automated hiding of the client
window. In a prior design, the only job of this window was to show the
"Sign in" button so it wasn't useful beyond clicking that. Now that we
show more things in this window, automatically hiding it might confuse
the user.
Here is what this new design looks like:
[Login
flow](https://github.com/user-attachments/assets/276e390b-4837-48e2-aaf1-eea007472816)
As a result of other improvements around "zero-click sign-in", the user
often doesn't even have to switch to the browser window because sign-in
happens in the background. Unfortunately, the tab still remains open but
that is outside of our control (at least on Linux).
As part of investigating #9400, I refactored the startup code of the GUI
client to play around with the initialization order of the runtime and
other parts. After finding the root cause (fixed in #9469), I figured it
would still be nice to land these improvements.
This refactors the app startup:
- Only initialize what is absolutely necessary outside of `try_main`:
The runtime and the telemetry instance.
- Settings are loaded earlier
- Telemetry is initializer earlier
- Add a bootstrap logger that logs to stdout whilst we are booting
- Re-order some code inside `gui::run` to bundle the initialization of
Tauri into a single command chain
---------
Signed-off-by: Thomas Eizinger <thomas@eizinger.io>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
When introducing the MDM config, we took into account the log directives
from the config when applying it via the GUI but failed to apply the new
directives on startup.
The GUI client's update checker is a task that runs concurrently to the
main event-loop of the client. When a new update is found, it sends a
notification through a channel. When the update checker is disabled via
MDM, this channel is instantly closed, resulting in `None` being read as
part of the event-loop in the client.
Previously, we would then return
`Poll::Ready(EventloopTick::UpdateNotification(None))` and then simply
ignore this event. This however is wrong. Returning `Poll::Ready` means
that the future needs to be polled again, resulting in an effective
busy-loop where we constantly emit the above event and then ignore it.
Due to the priority order that was previously defined, this led to us
never checking another channel: The one where we receive deep-links from
2nd GUI instances.
As a result, signing into the client when the update checker was
disabled does not work and instead, the newly launched instance hangs
because it can never send over the deep-link.
Resolves: #9420
As part of the introduction of General settings, we split up "Advanced
settings" and also renamed one of the fields. Upon first start, the
settings are migrated to the new format. What we failed to notice is
that one the next subsequent start, the legacy settings struct will fail
to parse the now migrated configuration and fall back to the default.
This then appears as if the settings are not getting saved.
Resolves: #9417
---------
Co-authored-by: Jamil Bou Kheir <jamilbk@users.noreply.github.com>
This PR adds basic analytics to `connlib` by sending two events to
PostHog:
1. `new_session` which is sent every time we establish a new session
with a Firezone backend. This could be our production or staging
instance but also a session to an on-premise installation of Firezone.
We include the API URL in the event payload to further distinguish
these.
2. `$identify` to link the client + version as well as the operating
system to the user. The user is identified by the Firezone ID.
---------
Signed-off-by: Thomas Eizinger <thomas@eizinger.io>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Introducing a dedicated `Error` type allows us to directly serialise
`anyhow::Error`s to the GUI. Those will then be reported to Sentry on
the TS side.
As relict from very early designs of `connlib`, the `Callbacks` trait is
still present and defines how the host app receives events from a
running `Session`. Callbacks are not a great design pattern however
because they force the running code, i.e. `connlib`s event-loop to
execute unknown code. For example, if that code panics, all of `connlib`
is taken down. Additionally, not all consumers may want to receive
events via callbacks. The GUI and headless client for example already
have their own event-loop in which they process all kinds of things.
Having to deal with the `Callbacks` interface introduces an odd
indirection here.
To fix this, we instead return an `EventStream` when constructing a
`Session`. This essentially aligns the API of `Session` with that of a
channel. You receive two handles, one for sending in commands and one
for receiving events. A `Session` will automatically spawn itself onto
the given runtime so progress is made even if one does not poll on these
channel handles.
This greatly simplifies the code:
- We get to delete the `Callbacks` interface.
- We can delete the threaded callback adapter. This was only necessary
because we didn't want to block `connlib` with the handling of the
event. By using a channel for events, this is automatically guaranteed.
- The GUI and headless client can directly integrate the event handling
in their event-loop, without having to create an indirection with a
channel.
- It is now clear that only the Apple and Android FFI layers actually
use callbacks to communicate these events.
- We net-delete 100 LoC
With an increased amount of complexity in the frontend of the GUI
client, it pays off to initialise the Sentry React SDK to catch any
errors that might occur. In particular, any failing commands that we
issue to the backend will be caught that way as those rejected
`Promise`s will surface as uncaught exceptions.
Similar to the backend, Sentry in the frontend is only initialised for
known environments, i.e. our production and staging deployments of
Firezone. For on-premise installations, Sentry is disabled.
Related: #6138
The frontend of the GUI client is written in TypeScript and communicates
with the backend via event listeners. Currently, we only have
type-safety within either of those parts of the codebase but not across
it. The payloads of these events are JSON-encoded. Any change to this
interface therefore needs to be applied on either end.
To avoid this, we add `tslink` to the GUI client which generates
TypeScript interfaces from Rust structs. We still check those into Git
into order to make local builds easy (otherwise every dev would have to
set `TSLINK_BUILD=true` on their machine). Our Tauri CI build already
has a check to ensure the Git workspace isn't modified after building so
any changes to these generated files will fail CI.
This adds a bit more type-safety to the codebase and makes refactorings
on the GUI client easier.
This PR adds the equivalent MDM configuration that we already have for
MacOS & iOS for the GUI client on Windows. These options are retrieved
from the Windows registry when the Client is started. Specifically, the
key for these is: `HKEY_CURRENT_USER\Software\Policies\Firezone`.
At moment, these cannot be configured or seen by the user. They are also
not "watched" for whilst the Client is running. If an admin pushes a new
MDM configuration, the Client will have to be restarted in order for
that new config to take effect.
Windows Policy templates are structured into two files:
- An `.admx` file that defines the structure of the policy, like the
kinds of values it has, where it is stored, which versions it is
supported on and which category it belongs to.
- An `.adml` file that defines defines all strings and presentation
logic, like the actual text of the policies and how the values are
presented in the GUI in e.g. Intune.
Internally, we differentiate between `MdmSettings` and
`AdvancedSettings`. The `MdmSettings` are cross-platform, however on
Linux, we always fallback to the defaults and therefore, they are always
"unset". Eventually, it might make sense to wrap both of these into a
more general `Settings` struct that acts as as a proxy for the two.
Related: #4505
The GUI client currently has a bug that resets the favourites and the
status of the Internet Resource every time the advanced settings are
saved. This happens because those fields are annotated with
`#[serde(default)]` and are thus initialised to their default value when
the struct is deserialised from the frontend.
To mitigate this, we introduce a new `GeneralSettings` struct that holds
the status of the Internet Resource and the list of favourites. When a
client starts up, it will try to migrate the existing advanced settings
into the new split of general and advanced settings.
Ensuring that the "Settings" window always displays the latest state is
important. At the moment, we achieve this by fetching the settings from
Rust every time we know that they changed and when the window is opened.
Currently, the settings can only change as a result of the reset button.
Once we integrate MDM-controlled configuration into the GUI client, the
settings can change at any point when administrators push a new
configuration change. The window doesn't know that though.
To simplify the code and handle dynamic changes to the settings, we now
use an event to send the settings to the frontend.
For our telemetry sessions with Sentry, we need to know which
environment we are running in, i.e. staging, production or on-prem. The
GUI client's tunnel service doesn't have a concept of an environment
until a GUI connects and sends the `StartTelemetry` message. Therefore,
we should scope a telemetry session to a GUI being connected over IPC.
Any errors around setting up / tearing down the background service are a
catch-22. Until a GUI connects, we can't initialise the telemetry
connection but if we fail to set up the background service, no GUI can
ever connect. Hence, the current setup and tear down of the `Telemetry`
module around the `ipc_listen` calls can safely be removed as they are
effectively no-ops anyway.
