In order to properly handle SRV and TXT records on the clients, we need
to be able to pick a Gateway using the initial query itself. After that,
we need to know the Gateway Tunnel IPs we're connecting to so we can
have the query perform the lookup.
Fixes#8281
Why:
* After merging #8267 it was discovered that there was a race condition
that allowed a `resource_create` message to end up at the Gateway
Channel process. Previously, this message would not have ever arrived,
because we were replacing Resource IDs when a breaking change was made,
but since that is no longer the case, it is possible that a connection
could be established between the time the `delete_resource` and
`create_resource` messages are sent and the `create_resource` would end
up at the Gateway Channel process. This commit adds a no-op handler to
make sure the message gets processed without throwing an error.
Why:
* Rather than using a persistent_id field in Resources/Policies, it was
decided that we should allow "breaking changes" to these entities. This
means that Resources/Policies will now be able to update all fields on
the schema without changing the primary key ID of the entity.
* This change will greatly help the API and Terraform provider
development.
@jamilbk, would you like me to put a migration in this PR to actually
get rid of all of the existing soft deleted entities?
@thomaseizinger, I tagged you on this, because I wanted to make sure
that these changes weren't going to break any expectations in the client
and/or gateways.
---------
Signed-off-by: Brian Manifold <bmanifold@users.noreply.github.com>
Co-authored-by: Jamil <jamilbk@users.noreply.github.com>
Why:
* The fallback controller in the API was not catching `{:error,
:seat_limit_reached}` being returned and was then generating a 500
response when this happened. This commit adds the condition in the
fallback controller and adds a new template for a more specific error
message in the returned JSON.
Why:
* Currently, when using the API, a user has no way of easily identifying
what identities they are pulling back as the response only includes the
`provider_identifier` which for most of our AuthProviders is an ID for
the IdP and not an email address. Along with that, when adding users to
an OIDC provider within Firezone, there is no check for whether or not
an identity has already been added with a given email address. By
creating a separate email column on the `auth_identities` table, it will
be very straight forward to know whether an email address exists for a
given identity, return it in an API response and allow the admin of a
Firezone account to track users (Identities) by email rather than IdP
identifier.
Fixes#7392
Why:
* The API endpoint for updating Resources was using
`Resources.fetch_resource_by_id_or_persistent_id`, however that function
was fetching all Resources, which included deleted Resources. In order
to prevent an API user from attempting to update a Resource that is
deleted, a new function was added to fetch active Resources only.
Fixes: #7492
In order for the firezone terraform provider to work properly, the
Resources and Policies need to be able to be referenced by their
`persistent_id`, specifically in the portal API.
This PR implements the new idempotent control protocol for the gateway.
We retain backwards-compatibility with old clients to allow admins to
perform a disruption-free update to the latest version.
With this new control protocol, we are moving the responsibility of
exchanging the proxy IPs we assigned to DNS resources to a p2p protocol
between client and gateway. As a result, wildcard DNS resources only get
authorized on the first access. Accessing a new domain within the same
resource will thus no longer require a roundtrip to the portal.
Overall, users will see a greatly decreased connection setup latency. On
top of that, the new protocol will allow us to more easily implement
packet buffering which will be another UX boost for Firezone.
TODOs:
- [x] Switch to sending messages instead of replies
- [ ] Do not hide pre-filtered resources and render them with an error
instead (in case we will want to expose that on a client later)
- [x] Figure out how to generate PSK so that it stays across WS
connections
Now you can "edit" any fields on the policy, when one of fields that
govern the access is changed (resource, actor group or conditions) a new
policy will be created and an old one is deleted. This will be
broadcasted to the clients right away to minimize downtime. New policy
will have it's own flows to prevent confusion while auditing. To make
experience better for external systems we added `persistent_id` that
will be the same across all versions of a given policy.
Resources work in a similar fashion but when they are replaced we will
also replace all corresponding policies.
An additional nice effect of this approach is that we also got
configuration audit log for resources and policies.
Fixes#2504
Currently, the gateway requires a strict ordering of first receiving a
`request_connection` message, following by multiple `allow_access`
messages. Additionally, access can be granted as part of the initial
`request_connection` message too.
This isn't an ideal design. Setting up a new connection is infallible,
all we need to do is send our ICE credentials back to the client.
However, untangling that will require a bit more effort.
Starting with #6335, following this strict order on the client is a more
difficult. Whilst we can send them in order, it is harder to maintain
those ordering guarantees across all our systems.
To avoid this, we change the gateway to perform an upsert for its local
ACLs for a client. In case that an `allow_access` call would somehow get
to the gateway earlier, we can simply already create the `Peer` and only
set up the actual connection later.
---------
Signed-off-by: Jamil <jamilbk@users.noreply.github.com>
Co-authored-by: Jamil <jamilbk@users.noreply.github.com>
They will be sent in the API for connlib 1.3 and above.
I think in future we can make a whole menu section called "Internet
Security" which will be a specialized UI for the new resource type (and
now show it in Resources list) to improve the user experience around it.
Closes#5852
---------
Signed-off-by: Andrew Dryga <andrew@dryga.com>
Co-authored-by: Jamil <jamilbk@users.noreply.github.com>
If a new resource is created that will use format not supported by
previous client versions we temporarily show a warning:
<img width="683" alt="Screenshot 2024-08-07 at 2 28 57 PM"
src="https://github.com/user-attachments/assets/bbfdfc96-0c4b-4226-93c5-bc2b5fdb9d30">
It will also be excluded from `resources` list for older clients (below
1.2).
---------
Co-authored-by: Thomas Eizinger <thomas@eizinger.io>
(External contribution)
Hi, first thanks to @bmanifold for his awesome work! I've not yet tested
the API but here is a first PR fixing various small mistakes in the
generated openapi spec:
Schema names cannot contain spaces
Add missing path parameters in the spec
Remove duplicated endpoint for creating an identity (not sure about
that, I'll let you check)
If you want to validate the generated spec you can paste it here:
https://editor.swagger.io/ (or at the bottow of your swagger ui)
Please review commit by commit
Co-authored-by: Jamil <jamilbk@users.noreply.github.com>
Co-authored-by: Antoine Labarussias <antoinelabarussias@gmail.com>
Why:
* In order to manage a large number of Firezone Sites, Resources,
Policies, etc... a REST API is needed as clicking through the UI is too
time consuming, as well as prone to error. By providing a REST API
Firezone customers will be able to manage things within their Firezone
accounts with code.
`relays` will be removed from `prepare_connection` in a few weeks after
we release a version that reads them from `init` message. Keep in mind
technically `relays` list can be empty, it would be nice if clients
would log an error or show it in such cases.
On the domain side this PR extends `Domain.Repo` with filtering,
pagination, and ordering, along with some convention changes are
removing the code that is not needed since we have the filtering now.
This required to touch pretty much all contexts and code, but I went
through all public functions and added missing tests to make sure
nothing will be broken.
On the web side I've introduced a `<.live_table />` which is as close as
possible to being a drop-in replacement for the regular `<.table />`
(but requires to structure the LiveView module differently due to
assigns anyways). I've updated all the listing tables to use it.
Extracted out of #3391.
We don't actually need this for #3391 though because we've added a
compatibility layer during deserialization. But, it will be good to
remove that compat layer at some point which means we have to return the
addresses as plain socket addresses. Because that is a breaking change,
I decided to extract this into a different PR.
Co-authored-by: conectado <gabrielalejandro7@gmail.com>
---------
Co-authored-by: conectado <gabrielalejandro7@gmail.com>
# Gateways
- [x] When Gateway Group is deleted all gateways should be disconnected
- [x] When Gateway Group is updated (eg. routing) broadcast to all
affected gateway to disconnect all the clients
- [x] When Gateway is deleted it should be disconnected
- [x] When Gateway Token is revoked all gateways that use it should be
disconnected
# Relays
- [x] When Relay Group is deleted all relays should be disconnected
- [x] When Relay is deleted it should be disconnected
- [x] When Relay Token is revoked all gateways that use it should be
disconnected
# Clients
- [x] Remove Delete Client button, show clients using the token on the
Actors page (#2669)
- [x] When client is deleted disconnect it
- [ ] ~When Gateway is offline broadcast to the Clients connected to it
it's status~
- [x] Persist `last_used_token_id` in Clients and show it in tokens UI
# Resources
- [x] When Resource is deleted it should be removed from all gateways
and clients
- [x] When Resource connection is removed it should be deleted from
removed gateway groups
- [x] When Resource is updated (eg. traffic filters) all it's
authorizations should removed
# Authentication
- [x] When Token is deleted related sessions are terminated
- [x] When an Actor is deleted or disabled it should be disconnected
from browser and client
- [x] When Identity is deleted it's sessions should be disconnected from
browser and client
- [x] ^ Ensure the same happens for identities during IdP sync
- [x] When IdP is disabled act like all actors for it are disabled?
- [x] When IdP is deleted act like all actors for it are deleted?
# Authorization
- [x] When Policy is created clients that gain access to a resource
should get an update
- [x] When Policy is deleted we need to all authorizations it's made
- [x] When Policy is disabled we need to all authorizations it's made
- [x] When Actor Group adds or removes a user, related policies should
be re-evaluated
- [x] ^ Ensure the same happens for identities during IdP sync
# Settings
- [x] Re-send init message to Client when DNS settings change
# Code
- [x] Crear way to see all available topics and messages, do not use
binary topics any more
---------
Co-authored-by: conectado <gabrielalejandro7@gmail.com>
- [x] Introduce api_client actor type and code to create and
authenticate using it's token
- [x] Unify Tokens usage for Relays and Gateways
- [x] Unify Tokens usage for magic links
Closes#2367
Ref #2696
