From f379e85e9b060e3c4df11cf7520eed3feed366d1 Mon Sep 17 00:00:00 2001 From: Jamil Date: Fri, 18 Jul 2025 15:47:18 -0700 Subject: [PATCH] refactor(portal): cache access state in channel pids (#9773) When changes occur in the Firezone DB that trigger side effects, we need some mechanism to broadcast and handle these. Before, the system we used was: - Each process subscribes to a myriad of topics related to data it wants to receive. In some cases it would subscribe to new topics based on received events from existing topics (I.e. flows in the gateway channel), and sometimes in a loop. It would then need to be sure to _unsubscribe_ from these topics - Handle the side effect in the `after_commit` hook of the Ecto function call after it completes - Broadcast only a simply (thin) event message with a DB id - In the receiver, use the id(s) to re-evaluate, or lookup one or many records associated with the change - After the lookup completes, `push` the relevant message(s) to the LiveView, `client` pid, or `gateway` pid in their respective channel processes This system had a number of drawbacks ranging from scalability issues to undesirable access bugs: 1. The `after_commit` callback, on each App node, is not globally ordered. Since we broadcast a thin event schema and read from the DB to hydrate each event, this meant we had a `read after write` problem in our event architecture, leading to the potential for lost updates. Case in point: if a policy is updated from `resource_id-1` to `resource_id-2`, and then back to `resource_id-1`, it's possible that, given the right amount of delay, the gateway channel will receive two `reject_access` events for `resource_id-1`, as opposed to one for `resource_id-1` and one for `resource_id-2`, leading to the potential for unauthorized access. 1. It was very difficult to ensure that the correct topics were being subscribed to and unsubscribed from, and the correct number of times, leading to maintenance issues for other engineers. 1. We had a nasty N+1 query problem whenever memberships were added or removed that resolved in essentially all access related to that membership (so all Policies touching its actor group) to be re-evaluated, and broadcasted. This meant that any bulk addition or deletion of memberships would generate so many queries that they'd timeout or consume the entire connection pool. 1. We had no durability for side-effect processing. In some places, we were iterating over many returned records to send broadcasts. Broadcasting is not a zero-time operation, each call takes a small amount of CPU time to copy the message into the receiver's mailbox. If we deployed while this was happening, the state update would be lost forever. If this was a `reject_access` for a Gateway, the Gateway would never remove access for that particular flow. 1. On each flow authorization, we needed to hit `us-east1` not only to "authorize" the flow, but to log it as well. This incurs latency especially for users in other parts of the world, which happens on _each_ connection setup to a new resource. 1. Since we read and re-authorize access due to the thin events broadcasted from side effects, we risk hitting thundering herd problems (see the N+1 query problem above) where a single DB change could result in all receivers hitting the DB at once to "hydrate" their processing.ion 1. If an administrator modifies the DB directly, or, if we need to run a DB migration that involves side effects, they'll be lost, because the side effect triggers happened in `after_commit` hooks that are only available when querying the DB through Ecto. Manually deleting (or resurrecting) a policy, for example, would not have updated any connected clients or gateways with the new state. To fix all of the above, we move to the system introduced in this PR: - All changes are now serialized (for free) by Postgres and broadcasted as a single event stream - The number of topics has been reduced to just one, the `account_id` of an account. All receivers subscribe to this one topic for the lifetime of their pid and then only filter the events they want to act upon, ignoring all other messages - The events themselves have been turned into "fat" structs based on the schemas they present. By making them properly typed, we can apply things like the existing Policy authorizer functions to them as if we had just fetched them from the DB. - All flow creation now happens in memory and doesn't not need to incur a DB hit in `us-east1` to proceed. - Since clients and gateways now track state in a push-based manner from the DB, this means very few actual DB queries are needed to maintain state in the channel procs, and it also means we can be smarter about when to send `resource_deleted` and `resource_created_or_updated` appropriately, since we can always diff between what the client _had_ access to, and what they _now_ have access to. - All DB operations, whether they happen from the application code, a `psql` prompt, or even via Google SQL Studio in the GCP console, will trigger the _same_ side effects. - We now use a replication consumer based off Postgres logical decoding of the write-ahead log using a _durable slot_. This means that Postgres will retain _all events_ until they are acknowledged, giving us the ability to ensure at-least-once processing semantics for our system. Today, the ACK is simply, "did we broadcast this event successfully". But in the future, we can assert that replies are received before we acknowledge the event as processed back to Postgres. The tests in this PR have been updated to pass given the refactor. However, since we are tracking more state now in the channel procs, it would be a good idea to add more tests for those edge cases. That is saved as a later PR because (1) this one is already huge, and (2) we need to get this out to staging to smoke test everything anyhow. Fixes: #9908 Fixes: #9909 Fixes: #9910 Fixes: #9900 Related: #9501 --- elixir/apps/api/lib/api/client/channel.ex | 1572 +++++++++-------- elixir/apps/api/lib/api/gateway/channel.ex | 860 ++++----- elixir/apps/api/lib/api/gateway/views/flow.ex | 11 + .../apps/api/test/api/client/channel_test.exs | 1123 ++++++------ .../api/test/api/gateway/channel_test.exs | 767 ++++---- elixir/apps/domain/lib/domain/actors.ex | 7 + .../domain/lib/domain/actors/membership.ex | 5 +- elixir/apps/domain/lib/domain/clients.ex | 16 +- .../domain/lib/domain/clients/presence.ex | 3 +- .../domain/lib/domain/config/definitions.ex | 5 + elixir/apps/domain/lib/domain/events/hooks.ex | 6 +- .../lib/domain/events/hooks/accounts.ex | 48 +- .../events/hooks/actor_group_memberships.ex | 39 +- .../lib/domain/events/hooks/actor_groups.ex | 12 - .../domain/lib/domain/events/hooks/actors.ex | 12 - .../domain/events/hooks/auth_identities.ex | 12 - .../lib/domain/events/hooks/auth_providers.ex | 12 - .../domain/lib/domain/events/hooks/clients.ex | 30 +- .../domain/lib/domain/events/hooks/flows.ex | 24 + .../lib/domain/events/hooks/gateway_groups.ex | 21 +- .../lib/domain/events/hooks/gateways.ex | 14 +- .../lib/domain/events/hooks/policies.ex | 177 +- .../lib/domain/events/hooks/relay_groups.ex | 12 - .../domain/lib/domain/events/hooks/relays.ex | 12 - .../events/hooks/resource_connections.ex | 19 +- .../lib/domain/events/hooks/resources.ex | 80 +- .../domain/lib/domain/events/hooks/tokens.ex | 25 +- .../domain/events/replication_connection.ex | 13 +- .../apps/domain/lib/domain/events/topics.ex | 47 - elixir/apps/domain/lib/domain/flows.ex | 184 +- elixir/apps/domain/lib/domain/flows/flow.ex | 1 + .../domain/lib/domain/flows/flow/changeset.ex | 3 +- .../domain/lib/domain/flows/flow/query.ex | 20 + .../domain/lib/domain/gateways/presence.ex | 3 +- elixir/apps/domain/lib/domain/policies.ex | 30 +- .../lib/domain/policies/policy/query.ex | 5 + elixir/apps/domain/lib/domain/pubsub.ex | 311 ---- elixir/apps/domain/lib/domain/relays.ex | 2 + .../lib/domain/replication/connection.ex | 23 +- .../domain/lib/domain/replication/decoder.ex | 67 + elixir/apps/domain/lib/domain/resources.ex | 15 - elixir/apps/domain/lib/domain/tokens.ex | 30 - ...20250704201940_index_flows_on_token_id.exs | 17 + ...0323_add_id_to_actor_group_memberships.exs | 70 + ...l_flows_with_actor_group_membership_id.exs | 85 + elixir/apps/domain/priv/repo/seeds.exs | 7 +- .../apps/domain/test/domain/actors_test.exs | 90 - .../jobs/sync_directory_test.exs | 91 +- .../jumpcloud/jobs/sync_directory_test.exs | 92 +- .../jobs/sync_directory_test.exs | 91 +- .../okta/jobs/sync_directory_test.exs | 92 +- elixir/apps/domain/test/domain/auth_test.exs | 99 -- .../apps/domain/test/domain/clients_test.exs | 36 +- .../domain/events/hooks/accounts_test.exs | 104 +- .../hooks/actor_group_memberships_test.exs | 118 +- .../domain/events/hooks/actor_groups_test.exs | 26 - .../test/domain/events/hooks/actors_test.exs | 26 - .../events/hooks/auth_identities_test.exs | 26 - .../events/hooks/auth_providers_test.exs | 26 - .../test/domain/events/hooks/clients_test.exs | 107 +- .../test/domain/events/hooks/flows_test.exs | 49 + .../events/hooks/gateway_groups_test.exs | 42 +- .../domain/events/hooks/gateways_test.exs | 74 +- .../domain/events/hooks/policies_test.exs | 411 ++--- .../domain/events/hooks/relay_groups_test.exs | 26 - .../test/domain/events/hooks/relays_test.exs | 26 - .../hooks/resource_connections_test.exs | 54 +- .../domain/events/hooks/resources_test.exs | 337 ++-- .../test/domain/events/hooks/tokens_test.exs | 118 +- .../events/replication_connection_test.exs | 90 +- elixir/apps/domain/test/domain/flows_test.exs | 973 +++++----- .../jobs/outdated_gateways_test.exs | 5 +- .../domain/test/domain/resources_test.exs | 250 --- .../test/domain/schema_helpers_test.exs | 171 +- .../apps/domain/test/domain/tokens_test.exs | 25 +- .../domain/test/support/fixtures/flows.ex | 20 +- elixir/apps/web/lib/web.ex | 6 - elixir/apps/web/lib/web/live/actors/show.ex | 2 + .../apps/web/lib/web/live/policies/index.ex | 12 +- elixir/apps/web/lib/web/live/policies/show.ex | 13 +- .../apps/web/lib/web/live/resources/index.ex | 12 +- .../apps/web/lib/web/live/resources/show.ex | 13 +- .../web/test/web/acceptance/auth_test.exs | 12 +- .../web/test/web/live/actors/show_test.exs | 22 +- .../web/test/web/live/clients/show_test.exs | 20 +- .../web/test/web/live/resources/edit_test.exs | 12 +- elixir/config/config.exs | 7 +- 87 files changed, 4442 insertions(+), 5241 deletions(-) create mode 100644 elixir/apps/api/lib/api/gateway/views/flow.ex delete mode 100644 elixir/apps/domain/lib/domain/events/hooks/actor_groups.ex delete mode 100644 elixir/apps/domain/lib/domain/events/hooks/actors.ex delete mode 100644 elixir/apps/domain/lib/domain/events/hooks/auth_identities.ex delete mode 100644 elixir/apps/domain/lib/domain/events/hooks/auth_providers.ex create mode 100644 elixir/apps/domain/lib/domain/events/hooks/flows.ex delete mode 100644 elixir/apps/domain/lib/domain/events/hooks/relay_groups.ex delete mode 100644 elixir/apps/domain/lib/domain/events/hooks/relays.ex delete mode 100644 elixir/apps/domain/lib/domain/events/topics.ex create mode 100644 elixir/apps/domain/priv/repo/manual_migrations/20250704201940_index_flows_on_token_id.exs create mode 100644 elixir/apps/domain/priv/repo/migrations/20250712010323_add_id_to_actor_group_memberships.exs create mode 100644 elixir/apps/domain/priv/repo/migrations/20250712155618_backfill_flows_with_actor_group_membership_id.exs delete mode 100644 elixir/apps/domain/test/domain/events/hooks/actor_groups_test.exs delete mode 100644 elixir/apps/domain/test/domain/events/hooks/actors_test.exs delete mode 100644 elixir/apps/domain/test/domain/events/hooks/auth_identities_test.exs delete mode 100644 elixir/apps/domain/test/domain/events/hooks/auth_providers_test.exs create mode 100644 elixir/apps/domain/test/domain/events/hooks/flows_test.exs delete mode 100644 elixir/apps/domain/test/domain/events/hooks/relay_groups_test.exs delete mode 100644 elixir/apps/domain/test/domain/events/hooks/relays_test.exs diff --git a/elixir/apps/api/lib/api/client/channel.ex b/elixir/apps/api/lib/api/client/channel.ex index 52362defb..a2bc1da71 100644 --- a/elixir/apps/api/lib/api/client/channel.ex +++ b/elixir/apps/api/lib/api/client/channel.ex @@ -1,7 +1,21 @@ defmodule API.Client.Channel do use API, :channel alias API.Client.Views - alias Domain.{Accounts, Clients, Actors, PubSub, Resources, Gateways, Relays, Policies, Flows} + + alias Domain.{ + Accounts, + Clients, + Actors, + PubSub, + Resources, + Flows, + Gateways, + Relays, + Policies, + Flows, + Tokens + } + alias Domain.Relays.Presence.Debouncer require Logger require OpenTelemetry.Tracer @@ -21,39 +35,21 @@ defmodule API.Client.Channel do @impl true def join("client", _payload, socket) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) + with {:ok, socket} <- schedule_expiration(socket), + {:ok, gateway_version_requirement} <- + select_gateway_version_requirement(socket.assigns.client) do + socket = assign(socket, gateway_version_requirement: gateway_version_requirement) - OpenTelemetry.Tracer.with_span "client.join" do - opentelemetry_ctx = OpenTelemetry.Ctx.get_current() - opentelemetry_span_ctx = OpenTelemetry.Tracer.current_span_ctx() + send(self(), :after_join) - with {:ok, socket} <- schedule_expiration(socket), - {:ok, gateway_version_requirement} <- - select_gateway_version_requirement(socket.assigns.client) do - socket = - assign(socket, - opentelemetry_ctx: opentelemetry_ctx, - opentelemetry_span_ctx: opentelemetry_span_ctx, - gateway_version_requirement: gateway_version_requirement - ) - - send(self(), {:after_join, {opentelemetry_ctx, opentelemetry_span_ctx}}) - {:ok, socket} - end + {:ok, socket} end end - defp schedule_expiration(%{assigns: %{subject: %{expires_at: nil}}} = socket) do - {:ok, socket} - end - defp schedule_expiration(%{assigns: %{subject: %{expires_at: expires_at}}} = socket) do - expires_in = DateTime.diff(expires_at, DateTime.utc_now(), :millisecond) - # Protect from race conditions where the token might have expired during code execution - expires_in = max(0, expires_in) - # Expiration time is capped at 31 days even if IdP returns really long lived tokens - expires_in = min(expires_in, 2_678_400_000) + expires_in = + expires_at + |> DateTime.diff(DateTime.utc_now(), :millisecond) if expires_in > 0 do Process.send_after(self(), :token_expired, expires_in) @@ -63,93 +59,52 @@ defmodule API.Client.Channel do end end - defp init(socket) do - OpenTelemetry.Tracer.with_span "client.init" do - {:ok, resources} = - Resources.all_authorized_resources(socket.assigns.subject, - preload: [ - :gateway_groups - ] - ) - - # We pre-filter them before subscribing to events to avoid accidentally rendering them later - resources = Policies.pre_filter_non_conforming_resources(resources, socket.assigns.client) - - # We subscribe for all resource events but only care about update events, - # where resource might be renamed which should be propagated to the UI. - :ok = - Enum.each(resources, fn resource -> - # TODO: WAL - # Why are we unsubscribing and subscribing again? - :ok = PubSub.Resource.unsubscribe(resource.id) - :ok = PubSub.Resource.subscribe(resource.id) - end) - - # Subscribe for known gateway group names so that if they are updated - we can render change in the UI - :ok = - resources - |> Enum.flat_map(& &1.gateway_groups) - |> Enum.uniq() - |> Enum.each(fn gateway_group -> - # TODO: WAL - # Why are we unsubscribing and subscribing again? - :ok = PubSub.GatewayGroup.unsubscribe(gateway_group.id) - :ok = PubSub.GatewayGroup.subscribe(gateway_group.id) - end) - - # Return all connected relays for the account - {:ok, relays} = select_relays(socket) - :ok = Enum.each(relays, &Relays.subscribe_to_relay_presence/1) - :ok = maybe_subscribe_for_relays_presence(relays, socket) - - resources = - map_and_filter_compatible_resources(resources, socket.assigns.client.last_seen_version) - - push(socket, "init", %{ - resources: Views.Resource.render_many(resources), - relays: - Views.Relay.render_many( - relays, - socket.assigns.client.public_key, - socket.assigns.subject.expires_at - ), - interface: - Views.Interface.render(%{ - socket.assigns.client - | account: socket.assigns.subject.account - }) - }) - - # Cache new stamp secrets - socket = Debouncer.cache_stamp_secrets(socket, relays) - - {:ok, socket} - end - end - @impl true - def handle_info({:after_join, {opentelemetry_ctx, opentelemetry_span_ctx}}, socket) do - OpenTelemetry.Ctx.attach(opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(opentelemetry_span_ctx) - OpenTelemetry.Tracer.with_span "client.after_join" do - :ok = Clients.Presence.connect(socket.assigns.client) + # Called immediately after the client joins the channel + def handle_info(:after_join, socket) do + # Initialize the cache + socket = + socket + |> hydrate_policies_and_resources() + |> hydrate_membership_group_ids() - # Subscribe for account config updates - :ok = PubSub.Account.subscribe(socket.assigns.client.account_id) + # Initialize relays + {:ok, relays} = select_relays(socket) + :ok = Enum.each(relays, &Relays.subscribe_to_relay_presence/1) + :ok = maybe_subscribe_for_relays_presence(relays, socket) - # We subscribe for membership updates for all actor groups the client is a member of, - :ok = PubSub.Actor.Memberships.subscribe(socket.assigns.subject.actor.id) + # Initialize debouncer for flappy relays + socket = Debouncer.cache_stamp_secrets(socket, relays) - # We subscribe for policy access events for the actor and the groups the client is a member of, - actor_group_ids = Actors.all_actor_group_ids!(socket.assigns.subject.actor) - :ok = Enum.each(actor_group_ids, &PubSub.ActorGroup.Policies.subscribe/1) - :ok = PubSub.Actor.Policies.subscribe(socket.assigns.subject.actor.id) + # Track client's presence + :ok = Clients.Presence.connect(socket.assigns.client) - {:ok, socket} = init(socket) + # Subscribe to all account updates + :ok = PubSub.Account.subscribe(socket.assigns.client.account_id) - {:noreply, socket} - end + # Initialize resources + resources = authorized_resources(socket) + + # Delete any stale flows for resources we may not have access to anymore + Flows.delete_stale_flows_on_connect(socket.assigns.client, resources) + + push(socket, "init", %{ + resources: Views.Resource.render_many(resources), + relays: + Views.Relay.render_many( + relays, + socket.assigns.client.public_key, + socket.assigns.subject.expires_at + ), + interface: + Views.Interface.render(%{ + socket.assigns.client + | account: socket.assigns.subject.account + }) + }) + + {:noreply, socket} end # Called to actually push relays_presence with a disconnected relay to the client @@ -161,257 +116,394 @@ defmodule API.Client.Channel do ##### Reacting to domain events #### #################################### - # This message is sent using Clients.broadcast_to_client/1 eg. when the client is deleted - def handle_info("disconnect", socket) do - OpenTelemetry.Tracer.with_span "client.disconnect" do - push(socket, "disconnect", %{reason: :token_expired}) - send(socket.transport_pid, %Phoenix.Socket.Broadcast{event: "disconnect"}) - {:stop, :shutdown, socket} + # ACCOUNTS + + def handle_info( + {:updated, %Accounts.Account{} = old_account, %Accounts.Account{} = account}, + socket + ) do + socket = assign(socket, client: %{socket.assigns.client | account: account}) + + if old_account.config != account.config do + payload = %{interface: Views.Interface.render(socket.assigns.client)} + :ok = push(socket, "config_changed", payload) end - end - - # This event is broadcasted from the Accounts context whenever the account config is changed - def handle_info(:config_changed, socket) do - account = Accounts.fetch_account_by_id!(socket.assigns.client.account_id) - - :ok = - push(socket, "config_changed", %{ - interface: - Views.Interface.render(%{ - socket.assigns.client - | account: account - }) - }) {:noreply, socket} end + # ACTOR_GROUP_MEMBERSHIPS + + def handle_info( + {:created, %Actors.Membership{actor_id: actor_id, group_id: group_id}}, + %{assigns: %{client: %{actor_id: id}}} = socket + ) + when id == actor_id do + # 1. Get existing resources + existing_resources = authorized_resources(socket) + + # 2. Re-hydrate our policies and resources + # It's not ideal we're hitting the DB here, but in practice it shouldn't be an issue because + # periods of bursty membership creation typically only happen for new accounts or new directory + # syncs, which won't have any policies associated. + socket = hydrate_policies_and_resources(socket) + + # 3. Update our membership group IDs + ids = MapSet.put(socket.assigns.membership_group_ids, group_id) + socket = assign(socket, membership_group_ids: ids) + + # 3. Get new resources + new_resources = authorized_resources(socket) -- existing_resources + + # 4. Push new resources to the client + for resource <- new_resources do + push(socket, "resource_created_or_updated", Views.Resource.render(resource)) + end + + {:noreply, socket} + end + + def handle_info( + {:deleted, %Actors.Membership{actor_id: actor_id, group_id: group_id}}, + %{assigns: %{client: %{actor_id: id}}} = socket + ) + when id == actor_id do + # 1. Take a snapshot of all resource_ids we no longer have access to + deleted_resource_ids = + socket.assigns.policies + |> Enum.flat_map(fn {_id, policy} -> + if policy.actor_group_id == group_id, do: [policy.resource_id], else: [] + end) + |> Enum.uniq() + + # 2. Push deleted resources to the client + for resource_id <- deleted_resource_ids do + push(socket, "resource_deleted", resource_id) + end + + # 3. Update our state + policies = + socket.assigns.policies + |> Enum.filter(fn {_id, policy} -> policy.actor_group_id != group_id end) + |> Enum.into(%{}) + + r_ids = Enum.map(policies, fn {_id, policy} -> policy.resource_id end) |> Enum.uniq() + resources = Map.take(socket.assigns.resources, r_ids) + membership_group_ids = Map.delete(socket.assigns.membership_group_ids, group_id) + + socket = + socket + |> assign(policies: policies) + |> assign(resources: resources) + |> assign(membership_group_ids: membership_group_ids) + + {:noreply, socket} + end + + # CLIENTS + + # Changes in client verification can affect the list of allowed resources - send diff of resources + def handle_info( + {:updated, %Clients.Client{} = old_client, %Clients.Client{id: client_id} = client}, + %{assigns: %{client: %{id: id}}} = socket + ) + when id == client_id do + # 1. Snapshot existing authorized resources + existing_resources = authorized_resources(socket) + + # 2. Update our state + socket = assign(socket, client: client) + + # 3. If client's verification status changed, send diff of resources + if old_client.verified_at != client.verified_at do + resources = authorized_resources(socket) + + added_resources = resources -- existing_resources + removed_resources = existing_resources -- resources + + for resource <- added_resources do + push(socket, "resource_created_or_updated", Views.Resource.render(resource)) + end + + for resource <- removed_resources do + push(socket, "resource_deleted", resource.id) + end + end + + {:noreply, socket} + end + + def handle_info( + {:deleted, %Clients.Client{id: id}}, + %{assigns: %{client: %{id: client_id}}} = socket + ) + when id == client_id do + disconnect(socket) + end + + # FLOWS + + # If we're authorized for a resource for this flow, we need to push resource_deleted + # followed by resource_created_or_updated in order to reset the access state on the + # client. + def handle_info({:deleted, %Flows.Flow{} = flow}, socket) do + # 1. Check if this possibly affects us + if resource = socket.assigns.resources[flow.resource_id] do + # 1. If so, check if we're currently authorized for this resource + if resource.id in Enum.map(authorized_resources(socket), & &1.id) do + push(socket, "resource_deleted", resource.id) + push(socket, "resource_created_or_updated", Views.Resource.render(resource)) + end + end + + {:noreply, socket} + end + + # GATEWAY_GROUPS + + def handle_info( + {:updated, %Gateways.Group{} = old_group, %Gateways.Group{} = group}, + socket + ) do + resources = + socket.assigns.resources + |> Enum.map(fn {id, resource} -> + gateway_groups = + resource.gateway_groups + |> Enum.map(fn gg -> if gg.id == group.id, do: Map.merge(gg, group), else: gg end) + + # Send resource_created_or_updated for all resources that have this group if name has changed + if Enum.any?(gateway_groups, fn gg -> gg.name != old_group.name and gg.id == group.id end) do + push(socket, "resource_created_or_updated", Views.Resource.render(resource)) + end + + {id, %{resource | gateway_groups: gateway_groups}} + end) + |> Enum.into(%{}) + + socket = assign(socket, resources: resources) + + {:noreply, socket} + end + + # POLICIES + + def handle_info({:created, %Policies.Policy{} = policy}, socket) do + # 1. Check if this policy is for us + if MapSet.member?(socket.assigns.membership_group_ids, policy.actor_group_id) do + # 2. Snapshot existing resources + existing_resources = authorized_resources(socket) + + # 3. Hydrate a new resource if we aren't already tracking it + socket = + if Map.has_key?(socket.assigns.resources, policy.resource_id) do + # Resource already exists due to another policy + socket + else + {:ok, resource} = + Resources.fetch_resource_by_id(policy.resource_id, socket.assigns.subject, + preload: :gateway_groups + ) + + assign(socket, resources: Map.put(socket.assigns.resources, resource.id, resource)) + end + + # 4. Hydrate the new policy + socket = assign(socket, policies: Map.put(socket.assigns.policies, policy.id, policy)) + + # 5. Maybe send new resource + if resource = (authorized_resources(socket) -- existing_resources) |> List.first() do + push(socket, "resource_created_or_updated", Views.Resource.render(resource)) + end + + {:noreply, socket} + else + {:noreply, socket} + end + end + + def handle_info( + {:updated, + %Policies.Policy{ + resource_id: old_resource_id, + actor_group_id: old_actor_group_id, + conditions: old_conditions + } = old_policy, + %Policies.Policy{ + resource_id: resource_id, + actor_group_id: actor_group_id, + conditions: conditions, + disabled_at: disabled_at + } = policy}, + socket + ) + when old_resource_id != resource_id or old_actor_group_id != actor_group_id or + old_conditions != conditions do + # Breaking update - process this as a delete and then create + {:noreply, socket} = handle_info({:deleted, old_policy}, socket) + + if is_nil(disabled_at) do + handle_info({:created, policy}, socket) + else + {:noreply, socket} + end + end + + # Other update - just update our state if the policy is for us + def handle_info({:updated, %Policies.Policy{}, %Policies.Policy{} = policy}, socket) do + socket = + if Map.has_key?(socket.assigns.policies, policy.id) do + assign(socket, policies: Map.put(socket.assigns.policies, policy.id, policy)) + else + socket + end + + {:noreply, socket} + end + + def handle_info({:deleted, %Policies.Policy{} = policy}, socket) do + # 1. Check if this policy is for us + if Map.has_key?(socket.assigns.policies, policy.id) do + # 2. Snapshot existing resources + existing_resources = authorized_resources(socket) + + # 3. Update our state + socket = assign(socket, policies: Map.delete(socket.assigns.policies, policy.id)) + r_ids = Enum.map(socket.assigns.policies, fn {_id, p} -> p.resource_id end) |> Enum.uniq() + socket = assign(socket, resources: Map.take(socket.assigns.resources, r_ids)) + + # 4. Push deleted resource to the client if we lost access to it + if resource = (existing_resources -- authorized_resources(socket)) |> List.first() do + push(socket, "resource_deleted", resource.id) + end + + {:noreply, socket} + else + {:noreply, socket} + end + end + + # RESOURCE_CONNECTIONS + + def handle_info( + {:created, + %Resources.Connection{resource_id: resource_id, gateway_group_id: gateway_group_id}}, + socket + ) do + # 1. Check if this affects us + if resource = socket.assigns.resources[resource_id] do + # 2. Fetch the gateway_group to hydrate the site name + {:ok, gateway_group} = Gateways.fetch_group_by_id(gateway_group_id, socket.assigns.subject) + + # 3. Update our state + resource = %{resource | gateway_groups: resource.gateway_groups ++ [gateway_group]} + socket = assign(socket, resources: Map.put(socket.assigns.resources, resource_id, resource)) + + # 4. If resource is allowed, push + if resource in authorized_resources(socket) do + # Connlib doesn't handle resources changing sites, so we need to delete then create + # See https://github.com/firezone/firezone/issues/9881 + push(socket, "resource_deleted", resource.id) + push(socket, "resource_created_or_updated", Views.Resource.render(resource)) + end + + {:noreply, socket} + else + {:noreply, socket} + end + end + + # Resource connection is a required field on resources, so a delete will always be followed by a create. + # This means connlib *should* never see a resource with no sites (gateway_groups). + + def handle_info( + {:deleted, + %Resources.Connection{resource_id: resource_id, gateway_group_id: gateway_group_id}}, + socket + ) do + # 1. Check if this affects us + if resource = socket.assigns.resources[resource_id] do + # 2. Update our state + gateway_groups = Enum.reject(resource.gateway_groups, &(&1.id == gateway_group_id)) + resource = %{resource | gateway_groups: gateway_groups} + socket = assign(socket, resources: Map.put(socket.assigns.resources, resource_id, resource)) + + # 3. Tell connlib + push(socket, "resource_deleted", resource.id) + + # 4. If resource is allowed, and has at least one site connected, push + if resource.id in Enum.map(authorized_resources(socket), & &1.id) and + length(resource.gateway_groups) > 0 do + # Connlib doesn't handle resources changing sites, so we need to delete then create + # See https://github.com/firezone/firezone/issues/9881 + push(socket, "resource_created_or_updated", Views.Resource.render(resource)) + end + + {:noreply, socket} + else + {:noreply, socket} + end + end + + # RESOURCES + + def handle_info( + {:updated, %Resources.Resource{} = old_resource, %Resources.Resource{id: id} = resource}, + socket + ) do + # 1. Check if this affects us + if existing_resource = socket.assigns.resources[id] do + # 2. Update our state - take gateway_groups from existing resource + resource = %{resource | gateway_groups: existing_resource.gateway_groups} + + socket = + assign(socket, + resources: Map.put(socket.assigns.resources, id, resource) + ) + + # 3. If resource is allowed and had meaningful changes, push + # GatewayGroup changes are handled in the Resources.Connection handler + resource_changed? = + old_resource.ip_stack != resource.ip_stack or + old_resource.type != resource.type or + old_resource.filters != resource.filters or + old_resource.address != resource.address or + old_resource.address_description != resource.address_description or + old_resource.name != resource.name + + if resource.id in Enum.map(authorized_resources(socket), & &1.id) and resource_changed? do + push(socket, "resource_created_or_updated", Views.Resource.render(resource)) + end + + {:noreply, socket} + else + {:noreply, socket} + end + end + + # TOKENS + + def handle_info( + {:deleted, %Tokens.Token{type: :client, id: id}}, + %{assigns: %{subject: %{token_id: token_id}}} = socket + ) + when id == token_id do + disconnect(socket) + end + + #################################### + ##### Reacting to timed events ##### + #################################### + # Message is scheduled by schedule_expiration/1 on topic join to be sent # when the client token/subject expires def handle_info(:token_expired, socket) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) - - OpenTelemetry.Tracer.with_span "client.token_expired" do - push(socket, "disconnect", %{reason: :token_expired}) - {:stop, {:shutdown, :token_expired}, socket} - end + disconnect(socket) end - # This event is broadcasted when client was changed (eg. renamed, verified, etc.) - def handle_info({:updated, %Clients.Client{} = updated_client}, socket) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) - - OpenTelemetry.Tracer.with_span "client.updated" do - client = socket.assigns.client - socket = assign(socket, client: updated_client) - - socket = - if client.verified_at != updated_client.verified_at do - # Re-initialize since list of allowed_resources could have changed - {:ok, socket} = init(socket) - - socket - else - socket - end - - {:noreply, socket} - end - end - - # Resource is created - def handle_info({:create_resource, resource_id}, socket) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) - - OpenTelemetry.Tracer.with_span "client.resource_created", - attributes: %{resource_id: resource_id} do - with {:ok, resource} <- - Resources.fetch_and_authorize_resource_by_id(resource_id, socket.assigns.subject, - preload: [:gateway_groups] - ), - true <- - Policies.client_conforms_any_on_connect?( - socket.assigns.client, - resource.authorized_by_policies - ) do - case map_or_drop_compatible_resource( - resource, - socket.assigns.client.last_seen_version - ) do - {:cont, resource} -> - push( - socket, - "resource_created_or_updated", - Views.Resource.render(resource) - ) - - :drop -> - :ok - end - else - {:error, _reason} -> - :ok - - false -> - :ok - end - - {:noreply, socket} - end - end - - # Resource is updated, eg. renamed. We don't care about other changes - # as the access is dictated by the policy events - def handle_info({:update_resource, resource_id}, socket) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) - - OpenTelemetry.Tracer.with_span "client.resource_updated", - attributes: %{resource_id: resource_id} do - with {:ok, resource} <- - Resources.fetch_and_authorize_resource_by_id(resource_id, socket.assigns.subject, - preload: [:gateway_groups] - ), - true <- - Policies.client_conforms_any_on_connect?( - socket.assigns.client, - resource.authorized_by_policies - ) do - case map_or_drop_compatible_resource( - resource, - socket.assigns.client.last_seen_version - ) do - {:cont, resource} -> - push( - socket, - "resource_created_or_updated", - Views.Resource.render(resource) - ) - - :drop -> - :ok - end - else - {:error, _reason} -> - :ok - - false -> - :ok - end - - {:noreply, socket} - end - end - - # This event is ignored because we will receive a reject_access message from - # the Policies which will trigger a resource_deleted event - def handle_info({:delete_resource, _resource_id}, socket) do - {:noreply, socket} - end - - # Those events are broadcasted by Actors whenever a membership is created or deleted - def handle_info({:create_membership, _actor_id, group_id}, socket) do - :ok = PubSub.ActorGroup.Policies.subscribe(group_id) - {:noreply, socket} - end - - def handle_info({:delete_membership, _actor_id, group_id}, socket) do - :ok = PubSub.ActorGroup.Policies.unsubscribe(group_id) - {:noreply, socket} - end - - # This message is received when there is a policy created or enabled - # allowing access to a resource by a client actor group - def handle_info({:allow_access, policy_id, actor_group_id, resource_id}, socket) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) - - OpenTelemetry.Tracer.with_span "client.allow_access", - attributes: %{ - policy_id: policy_id, - actor_group_id: actor_group_id, - resource_id: resource_id - } do - # TODO: WAL - # Why are we unsubscribing and subscribing again? - :ok = PubSub.Resource.unsubscribe(resource_id) - :ok = PubSub.Resource.subscribe(resource_id) - - case Resources.fetch_and_authorize_resource_by_id(resource_id, socket.assigns.subject, - preload: [:gateway_groups] - ) do - {:ok, resource} -> - case map_or_drop_compatible_resource( - resource, - socket.assigns.client.last_seen_version - ) do - {:cont, resource} -> - push( - socket, - "resource_created_or_updated", - Views.Resource.render(resource) - ) - - :drop -> - :ok - end - - {:error, _reason} -> - :ok - end - - {:noreply, socket} - end - end - - # This message is received when the policy - # allowing access to a resource by a client actor group is deleted - def handle_info({:reject_access, policy_id, actor_group_id, resource_id}, socket) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) - - OpenTelemetry.Tracer.with_span "client.reject_access", - attributes: %{ - policy_id: policy_id, - actor_group_id: actor_group_id, - resource_id: resource_id - } do - :ok = PubSub.Resource.unsubscribe(resource_id) - - # We potentially can re-create the flow but this will require keep tracking of client connections to gateways, - # which is not worth it as this case should be pretty rare. Instead we just tell client to remove it - # and the recreate it right away if there is another allowing access to it. - push(socket, "resource_deleted", resource_id) - - case Resources.fetch_and_authorize_resource_by_id(resource_id, socket.assigns.subject, - preload: [:gateway_groups] - ) do - {:ok, resource} -> - case map_or_drop_compatible_resource( - resource, - socket.assigns.client.last_seen_version - ) do - {:cont, resource} -> - push( - socket, - "resource_created_or_updated", - Views.Resource.render(resource) - ) - - :drop -> - :ok - end - - {:error, _reason} -> - :ok - end - - {:noreply, socket} - end - end + #################################### + #### Reacting to relay presence #### + #################################### def handle_info( %Phoenix.Socket.Broadcast{ @@ -421,39 +513,31 @@ defmodule API.Client.Channel do }, socket ) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) - if Map.has_key?(leaves, relay_id) do - OpenTelemetry.Tracer.with_span "client.relays_presence", - attributes: %{ - relay_id: relay_id - } do - :ok = Relays.unsubscribe_from_relay_presence(relay_id) + :ok = Relays.unsubscribe_from_relay_presence(relay_id) - {:ok, relays} = select_relays(socket, [relay_id]) - :ok = maybe_subscribe_for_relays_presence(relays, socket) + {:ok, relays} = select_relays(socket, [relay_id]) + :ok = maybe_subscribe_for_relays_presence(relays, socket) - :ok = - Enum.each(relays, fn relay -> - # TODO: WAL - # Why are we unsubscribing and subscribing again? - :ok = Relays.unsubscribe_from_relay_presence(relay) - :ok = Relays.subscribe_to_relay_presence(relay) - end) + :ok = + Enum.each(relays, fn relay -> + # TODO: WAL + # Why are we unsubscribing and subscribing again? + :ok = Relays.unsubscribe_from_relay_presence(relay) + :ok = Relays.subscribe_to_relay_presence(relay) + end) - payload = %{ - disconnected_ids: [relay_id], - connected: - Views.Relay.render_many( - relays, - socket.assigns.client.public_key, - socket.assigns.subject.expires_at - ) - } + payload = %{ + disconnected_ids: [relay_id], + connected: + Views.Relay.render_many( + relays, + socket.assigns.client.public_key, + socket.assigns.subject.expires_at + ) + } - {:noreply, Debouncer.queue_leave(self(), socket, relay_id, payload)} - end + {:noreply, Debouncer.queue_leave(self(), socket, relay_id, payload)} else {:noreply, socket} end @@ -467,51 +551,48 @@ defmodule API.Client.Channel do }, socket ) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) - if Enum.count(joins) > 0 do - OpenTelemetry.Tracer.with_span "client.account_relays_presence" do + {:ok, relays} = select_relays(socket) + + if length(relays) > 0 do + :ok = Relays.unsubscribe_from_relays_presence_in_account(socket.assigns.subject.account) + + :ok = + Enum.each(relays, fn relay -> + # TODO: WAL + # Why are we unsubscribing and subscribing again? + :ok = Relays.unsubscribe_from_relay_presence(relay) + :ok = Relays.subscribe_to_relay_presence(relay) + end) + + # Cache new stamp secrets + socket = Debouncer.cache_stamp_secrets(socket, relays) + + # If a relay reconnects with a different stamp_secret, disconnect them immediately + joined_ids = Map.keys(joins) + + {socket, disconnected_ids} = + Debouncer.cancel_leaves_or_disconnect_immediately( + socket, + joined_ids, + socket.assigns.client.account_id + ) + {:ok, relays} = select_relays(socket) - if length(relays) > 0 do - :ok = Relays.unsubscribe_from_relays_presence_in_account(socket.assigns.subject.account) - - :ok = - Enum.each(relays, fn relay -> - # TODO: WAL - # Why are we unsubscribing and subscribing again? - :ok = Relays.unsubscribe_from_relay_presence(relay) - :ok = Relays.subscribe_to_relay_presence(relay) - end) - - # Cache new stamp secrets - socket = Debouncer.cache_stamp_secrets(socket, relays) - - # If a relay reconnects with a different stamp_secret, disconnect them immediately - joined_ids = Map.keys(joins) - - {socket, disconnected_ids} = - Debouncer.cancel_leaves_or_disconnect_immediately( - socket, - joined_ids, - socket.assigns.client.account_id + push(socket, "relays_presence", %{ + disconnected_ids: disconnected_ids, + connected: + Views.Relay.render_many( + relays, + socket.assigns.client.public_key, + socket.assigns.subject.expires_at ) + }) - {:ok, relays} = select_relays(socket) - - push(socket, "relays_presence", %{ - disconnected_ids: disconnected_ids, - connected: - Views.Relay.render_many( - relays, - socket.assigns.client.public_key, - socket.assigns.subject.expires_at - ) - }) - - {:noreply, socket} - end + {:noreply, socket} + else + {:noreply, socket} end else {:noreply, socket} @@ -524,104 +605,76 @@ defmodule API.Client.Channel do # This the list of ICE candidates gathered by the gateway and relayed to the client def handle_info( - {:ice_candidates, gateway_id, candidates, {opentelemetry_ctx, opentelemetry_span_ctx}}, - socket - ) do - OpenTelemetry.Ctx.attach(opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(opentelemetry_span_ctx) + {{:ice_candidates, client_id}, gateway_id, candidates}, + %{assigns: %{client: %{id: id}}} = socket + ) + when client_id == id do + push(socket, "ice_candidates", %{ + gateway_id: gateway_id, + candidates: candidates + }) - OpenTelemetry.Tracer.with_span "client.ice_candidates", - attributes: %{ - gateway_id: gateway_id, - candidates_length: length(candidates) - } do - push(socket, "ice_candidates", %{ - gateway_id: gateway_id, - candidates: candidates - }) - - {:noreply, socket} - end + {:noreply, socket} end def handle_info( - {:invalidate_ice_candidates, gateway_id, candidates, - {opentelemetry_ctx, opentelemetry_span_ctx}}, - socket - ) do - OpenTelemetry.Ctx.attach(opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(opentelemetry_span_ctx) + {{:invalidate_ice_candidates, client_id}, gateway_id, candidates}, + %{assigns: %{client: %{id: id}}} = socket + ) + when client_id == id do + push(socket, "invalidate_ice_candidates", %{ + gateway_id: gateway_id, + candidates: candidates + }) - OpenTelemetry.Tracer.with_span "client.invalidate_ice_candidates", - attributes: %{ - gateway_id: gateway_id, - candidates_length: length(candidates) - } do - push(socket, "invalidate_ice_candidates", %{ - gateway_id: gateway_id, - candidates: candidates - }) - - {:noreply, socket} - end + {:noreply, socket} end # DEPRECATED IN 1.4 # This message is sent by the gateway when it is ready to accept the connection from the client def handle_info( - {:connect, socket_ref, resource_id, gateway_public_key, payload, - {opentelemetry_ctx, opentelemetry_span_ctx}}, + {:connect, socket_ref, resource_id, gateway_public_key, payload}, socket ) do - OpenTelemetry.Ctx.attach(opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(opentelemetry_span_ctx) + reply( + socket_ref, + {:ok, + %{ + resource_id: resource_id, + persistent_keepalive: 25, + gateway_public_key: gateway_public_key, + gateway_payload: payload + }} + ) - OpenTelemetry.Tracer.with_span "client.connect", attributes: %{resource_id: resource_id} do - reply( - socket_ref, - {:ok, - %{ - resource_id: resource_id, - persistent_keepalive: 25, - gateway_public_key: gateway_public_key, - gateway_payload: payload - }} - ) - - {:noreply, socket} - end + {:noreply, socket} end def handle_info( {:connect, _socket_ref, resource_id, gateway_group_id, gateway_id, gateway_public_key, - gateway_ipv4, gateway_ipv6, preshared_key, ice_credentials, - {opentelemetry_ctx, opentelemetry_span_ctx}}, + gateway_ipv4, gateway_ipv6, preshared_key, ice_credentials}, socket ) do - OpenTelemetry.Ctx.attach(opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(opentelemetry_span_ctx) + reply_payload = %{ + resource_id: resource_id, + preshared_key: preshared_key, + client_ice_credentials: ice_credentials.client, + gateway_group_id: gateway_group_id, + gateway_id: gateway_id, + gateway_public_key: gateway_public_key, + gateway_ipv4: gateway_ipv4, + gateway_ipv6: gateway_ipv6, + gateway_ice_credentials: ice_credentials.gateway + } - OpenTelemetry.Tracer.with_span "client.connect", attributes: %{resource_id: resource_id} do - reply_payload = %{ - resource_id: resource_id, - preshared_key: preshared_key, - client_ice_credentials: ice_credentials.client, - gateway_group_id: gateway_group_id, - gateway_id: gateway_id, - gateway_public_key: gateway_public_key, - gateway_ipv4: gateway_ipv4, - gateway_ipv6: gateway_ipv6, - gateway_ice_credentials: ice_credentials.gateway - } + push(socket, "flow_created", reply_payload) - # We are pushing a message instead of replying for the sake of connlib message parsing convenience - push(socket, "flow_created", reply_payload) - # reply(socket_ref, {:ok, reply_payload}) - - {:noreply, socket} - end + {:noreply, socket} end + # Catch-all for messages we don't handle + def handle_info(_message, socket), do: {:noreply, socket} + #################################### ##### Client-initiated actions ##### #################################### @@ -635,99 +688,101 @@ defmodule API.Client.Channel do %{ "resource_id" => resource_id, "connected_gateway_ids" => connected_gateway_ids - } = attrs, + }, socket ) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) + location = { + socket.assigns.client.last_seen_remote_ip_location_lat, + socket.assigns.client.last_seen_remote_ip_location_lon + } - OpenTelemetry.Tracer.with_span "client.create_flow", attributes: attrs do - location = { - socket.assigns.client.last_seen_remote_ip_location_lat, - socket.assigns.client.last_seen_remote_ip_location_lon - } + with {:ok, resource} <- Map.fetch(socket.assigns.resources, resource_id), + {:ok, policy, expires_at} <- authorize_resource(socket, resource_id), + {:ok, gateways} when gateways != [] <- + Gateways.all_connected_gateways_for_resource(resource, socket.assigns.subject, + preload: :group + ), + {:ok, gateways} <- + filter_compatible_gateways(gateways, socket.assigns.gateway_version_requirement) do + gateway = Gateways.load_balance_gateways(location, gateways, connected_gateway_ids) - with {:ok, resource} <- - Resources.fetch_and_authorize_resource_by_id(resource_id, socket.assigns.subject), - {:ok, gateways} when gateways != [] <- - Gateways.all_connected_gateways_for_resource(resource, socket.assigns.subject, - preload: :group - ), - {:ok, gateways} <- - filter_compatible_gateways(gateways, socket.assigns.gateway_version_requirement), - OpenTelemetry.Tracer.set_attribute(:gateways_count, length(gateways)), - gateway = Gateways.load_balance_gateways(location, gateways, connected_gateway_ids), - OpenTelemetry.Tracer.set_attribute(:gateway_id, gateway.id), - {:ok, resource, flow, expires_at} <- - Flows.authorize_flow( - socket.assigns.client, - gateway, - resource_id, - socket.assigns.subject - ) do - OpenTelemetry.Tracer.set_attribute(:flow_id, flow.id) - opentelemetry_ctx = OpenTelemetry.Ctx.get_current() - opentelemetry_span_ctx = OpenTelemetry.Tracer.current_span_ctx() + # TODO: Optimization + # Move this to a Task.start that completes after broadcasting authorize_flow + {:ok, _flow} = + Flows.create_flow( + socket.assigns.client, + gateway, + resource_id, + policy, + socket.assigns.subject + ) - preshared_key = generate_preshared_key() - ice_credentials = generate_ice_credentials(socket.assigns.client, gateway) + preshared_key = generate_preshared_key() + ice_credentials = generate_ice_credentials(socket.assigns.client, gateway) - :ok = - PubSub.Gateway.broadcast( - gateway.id, - {:authorize_flow, {self(), socket_ref(socket)}, - %{ - client_id: socket.assigns.client.id, - resource_id: resource.id, - flow_id: flow.id, - authorization_expires_at: expires_at, - ice_credentials: ice_credentials, - preshared_key: preshared_key - }, {opentelemetry_ctx, opentelemetry_span_ctx}} - ) + :ok = + PubSub.Account.broadcast( + socket.assigns.client.account_id, + {{:authorize_flow, gateway.id}, {self(), socket_ref(socket)}, + %{ + client: socket.assigns.client, + resource: resource, + authorization_expires_at: expires_at, + ice_credentials: ice_credentials, + preshared_key: preshared_key + }} + ) + + {:noreply, socket} + else + {:error, :not_found} -> + push(socket, "flow_creation_failed", %{ + resource_id: resource_id, + reason: :not_found + }) {:noreply, socket} - else - {:error, :not_found} -> - OpenTelemetry.Tracer.set_status(:error, "not_found") - # We are pushing a message instead of replying for the sake of connlib message parsing convenience - # {:reply, {:error, %{reason: :not_found}}, socket} + {:error, :offline} -> + push(socket, "flow_creation_failed", %{ + resource_id: resource_id, + reason: :offline + }) - push(socket, "flow_creation_failed", %{ - resource_id: resource_id, - reason: :not_found - }) + {:noreply, socket} - {:noreply, socket} + {:error, :forbidden} -> + push(socket, "flow_creation_failed", %{ + resource_id: resource_id, + reason: :forbidden + }) - {:ok, []} -> - OpenTelemetry.Tracer.set_status(:error, "offline") + {:noreply, socket} - # We are pushing a message instead of replying for the sake of connlib message parsing convenience - # {:reply, {:error, %{reason: :offline}}, socket} + :error -> + push(socket, "flow_creation_failed", %{ + resource_id: resource_id, + reason: :not_found + }) - push(socket, "flow_creation_failed", %{ - resource_id: resource_id, - reason: :offline - }) + {:noreply, socket} - {:noreply, socket} + {:ok, []} -> + push(socket, "flow_creation_failed", %{ + resource_id: resource_id, + reason: :offline + }) - {:error, {:forbidden, violated_properties: violated_properties}} -> - OpenTelemetry.Tracer.set_status(:error, "forbidden") + {:noreply, socket} - # We are pushing a message instead of replying for the sake of connlib message parsing convenience - # {:reply, {:error, %{reason: :forbidden, violated_properties: violated_properties}}, socket} + {:error, {:forbidden, violated_properties: violated_properties}} -> + push(socket, "flow_creation_failed", %{ + resource_id: resource_id, + reason: :forbidden, + violated_properties: violated_properties + }) - push(socket, "flow_creation_failed", %{ - resource_id: resource_id, - reason: :forbidden, - violated_properties: violated_properties - }) - - {:noreply, socket} - end + {:noreply, socket} end end @@ -739,51 +794,52 @@ defmodule API.Client.Channel do # some of the gateways and can multiplex the connections. @impl true def handle_in("prepare_connection", %{"resource_id" => resource_id} = attrs, socket) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) + connected_gateway_ids = Map.get(attrs, "connected_gateway_ids", []) - OpenTelemetry.Tracer.with_span "client.prepare_connection", attributes: attrs do - connected_gateway_ids = Map.get(attrs, "connected_gateway_ids", []) + # TODO: Optimization + # Gateway selection and flow authorization shouldn't need to hit the DB + with {:ok, resource} <- Map.fetch(socket.assigns.resources, resource_id), + {:ok, _policy, _expires_at} <- authorize_resource(socket, resource_id), + {:ok, [_ | _] = gateways} <- + Gateways.all_connected_gateways_for_resource(resource, socket.assigns.subject, + preload: :group + ), + gateway_version_requirement = + maybe_update_gateway_version_requirement( + resource, + socket.assigns.gateway_version_requirement + ), + {:ok, gateways} <- filter_compatible_gateways(gateways, gateway_version_requirement) do + location = { + socket.assigns.client.last_seen_remote_ip_location_lat, + socket.assigns.client.last_seen_remote_ip_location_lon + } - with {:ok, resource} <- - Resources.fetch_and_authorize_resource_by_id(resource_id, socket.assigns.subject), - {:ok, [_ | _] = gateways} <- - Gateways.all_connected_gateways_for_resource(resource, socket.assigns.subject, - preload: :group - ), - gateway_version_requirement = - maybe_update_gateway_version_requirement( - resource, - socket.assigns.gateway_version_requirement - ), - {:ok, gateways} <- filter_compatible_gateways(gateways, gateway_version_requirement) do - location = { - socket.assigns.client.last_seen_remote_ip_location_lat, - socket.assigns.client.last_seen_remote_ip_location_lon - } + gateway = Gateways.load_balance_gateways(location, gateways, connected_gateway_ids) - OpenTelemetry.Tracer.set_attribute(:gateways_length, length(gateways)) - gateway = Gateways.load_balance_gateways(location, gateways, connected_gateway_ids) + reply = + {:ok, + %{ + resource_id: resource_id, + gateway_group_id: gateway.group_id, + gateway_id: gateway.id, + gateway_remote_ip: gateway.last_seen_remote_ip + }} - reply = - {:ok, - %{ - resource_id: resource_id, - gateway_group_id: gateway.group_id, - gateway_id: gateway.id, - gateway_remote_ip: gateway.last_seen_remote_ip - }} + {:reply, reply, socket} + else + {:ok, []} -> + {:reply, {:error, %{reason: :offline}}, socket} - {:reply, reply, socket} - else - {:ok, []} -> - OpenTelemetry.Tracer.set_status(:error, "offline") - {:reply, {:error, %{reason: :offline}}, socket} + {:error, :not_found} -> + {:reply, {:error, %{reason: :not_found}}, socket} - {:error, :not_found} -> - OpenTelemetry.Tracer.set_status(:error, "not_found") - {:reply, {:error, %{reason: :not_found}}, socket} - end + :error -> + {:reply, {:error, %{reason: :not_found}}, socket} + + {:error, {:forbidden, violated_properties: violated_properties}} -> + {:reply, {:error, %{reason: :forbidden, violated_properties: violated_properties}}, + socket} end end @@ -796,54 +852,49 @@ defmodule API.Client.Channel do "gateway_id" => gateway_id, "resource_id" => resource_id, "payload" => payload - } = attrs, + }, socket ) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) + with {:ok, resource} <- Map.fetch(socket.assigns.resources, resource_id), + {:ok, policy, expires_at} <- authorize_resource(socket, resource_id), + {:ok, gateway} <- Gateways.fetch_gateway_by_id(gateway_id, socket.assigns.subject), + true <- Gateways.gateway_can_connect_to_resource?(gateway, resource) do + # TODO: Optimization + {:ok, _flow} = + Flows.create_flow( + socket.assigns.client, + gateway, + resource_id, + policy, + socket.assigns.subject + ) - OpenTelemetry.Tracer.with_span "client.reuse_connection", attributes: attrs do - with {:ok, gateway} <- Gateways.fetch_gateway_by_id(gateway_id, socket.assigns.subject), - {:ok, resource, flow, _expires_at} <- - Flows.authorize_flow( - socket.assigns.client, - gateway, - resource_id, - socket.assigns.subject - ), - true <- Gateways.gateway_can_connect_to_resource?(gateway, resource) do - opentelemetry_ctx = OpenTelemetry.Ctx.get_current() - opentelemetry_span_ctx = OpenTelemetry.Tracer.current_span_ctx() + :ok = + PubSub.Account.broadcast( + socket.assigns.client.account_id, + {{:allow_access, gateway.id}, {self(), socket_ref(socket)}, + %{ + client: socket.assigns.client, + resource: resource, + authorization_expires_at: expires_at, + client_payload: payload + }} + ) - :ok = - PubSub.Gateway.broadcast( - gateway.id, - {:allow_access, {self(), socket_ref(socket)}, - %{ - client_id: socket.assigns.client.id, - resource_id: resource.id, - flow_id: flow.id, - authorization_expires_at: socket.assigns.subject.expires_at, - client_payload: payload - }, {opentelemetry_ctx, opentelemetry_span_ctx}} - ) + {:noreply, socket} + else + {:error, :not_found} -> + {:reply, {:error, %{reason: :not_found}}, socket} - {:noreply, socket} - else - {:error, :not_found} -> - OpenTelemetry.Tracer.set_status(:error, "not_found") - {:reply, {:error, %{reason: :not_found}}, socket} + :error -> + {:reply, {:error, %{reason: :not_found}}, socket} - {:error, {:forbidden, violated_properties: violated_properties}} -> - OpenTelemetry.Tracer.set_status(:error, "forbidden") + {:error, {:forbidden, violated_properties: violated_properties}} -> + {:reply, {:error, %{reason: :forbidden, violated_properties: violated_properties}}, + socket} - {:reply, {:error, %{reason: :forbidden, violated_properties: violated_properties}}, - socket} - - false -> - OpenTelemetry.Tracer.set_status(:error, "offline") - {:reply, {:error, %{reason: :offline}}, socket} - end + false -> + {:reply, {:error, %{reason: :offline}}, socket} end end @@ -860,53 +911,48 @@ defmodule API.Client.Channel do }, socket ) do - ctx_attrs = %{gateway_id: gateway_id, resource_id: resource_id} - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) + # Flow authorization can happen out-of-band since we just authorized the resource above + with {:ok, resource} <- Map.fetch(socket.assigns.resources, resource_id), + {:ok, policy, expires_at} <- authorize_resource(socket, resource_id), + {:ok, gateway} <- Gateways.fetch_gateway_by_id(gateway_id, socket.assigns.subject), + true <- Gateways.gateway_can_connect_to_resource?(gateway, resource) do + # TODO: Optimization + {:ok, _flow} = + Flows.create_flow( + socket.assigns.client, + gateway, + resource_id, + policy, + socket.assigns.subject + ) - OpenTelemetry.Tracer.with_span "client.request_connection", attributes: ctx_attrs do - with {:ok, gateway} <- Gateways.fetch_gateway_by_id(gateway_id, socket.assigns.subject), - {:ok, resource, flow, _expires_at} <- - Flows.authorize_flow( - socket.assigns.client, - gateway, - resource_id, - socket.assigns.subject - ), - true <- Gateways.gateway_can_connect_to_resource?(gateway, resource) do - opentelemetry_ctx = OpenTelemetry.Ctx.get_current() - opentelemetry_span_ctx = OpenTelemetry.Tracer.current_span_ctx() + :ok = + PubSub.Account.broadcast( + socket.assigns.client.account_id, + {{:request_connection, gateway.id}, {self(), socket_ref(socket)}, + %{ + client: socket.assigns.client, + resource: resource, + authorization_expires_at: expires_at, + client_payload: client_payload, + client_preshared_key: preshared_key + }} + ) - :ok = - PubSub.Gateway.broadcast( - gateway.id, - {:request_connection, {self(), socket_ref(socket)}, - %{ - client_id: socket.assigns.client.id, - resource_id: resource.id, - flow_id: flow.id, - authorization_expires_at: socket.assigns.subject.expires_at, - client_payload: client_payload, - client_preshared_key: preshared_key - }, {opentelemetry_ctx, opentelemetry_span_ctx}} - ) + {:noreply, socket} + else + {:error, :not_found} -> + {:reply, {:error, %{reason: :not_found}}, socket} - {:noreply, socket} - else - {:error, :not_found} -> - OpenTelemetry.Tracer.set_status(:error, "not_found") - {:reply, {:error, %{reason: :not_found}}, socket} + :error -> + {:reply, {:error, %{reason: :not_found}}, socket} - {:error, {:forbidden, violated_properties: violated_properties}} -> - OpenTelemetry.Tracer.set_status(:error, "forbidden") + {:error, {:forbidden, violated_properties: violated_properties}} -> + {:reply, {:error, %{reason: :forbidden, violated_properties: violated_properties}}, + socket} - {:reply, {:error, %{reason: :forbidden, violated_properties: violated_properties}}, - socket} - - false -> - OpenTelemetry.Tracer.set_status(:error, "offline") - {:reply, {:error, %{reason: :offline}}, socket} - end + false -> + {:reply, {:error, %{reason: :offline}}, socket} end end @@ -916,24 +962,15 @@ defmodule API.Client.Channel do %{"candidates" => candidates, "gateway_ids" => gateway_ids}, socket ) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) + :ok = + Enum.each(gateway_ids, fn gateway_id -> + PubSub.Account.broadcast( + socket.assigns.client.account_id, + {{:ice_candidates, gateway_id}, socket.assigns.client.id, candidates} + ) + end) - OpenTelemetry.Tracer.with_span "client.broadcast_ice_candidates" do - opentelemetry_ctx = OpenTelemetry.Ctx.get_current() - opentelemetry_span_ctx = OpenTelemetry.Tracer.current_span_ctx() - - :ok = - Enum.each(gateway_ids, fn gateway_id -> - PubSub.Gateway.broadcast( - gateway_id, - {:ice_candidates, socket.assigns.client.id, candidates, - {opentelemetry_ctx, opentelemetry_span_ctx}} - ) - end) - - {:noreply, socket} - end + {:noreply, socket} end def handle_in( @@ -941,24 +978,15 @@ defmodule API.Client.Channel do %{"candidates" => candidates, "gateway_ids" => gateway_ids}, socket ) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) + :ok = + Enum.each(gateway_ids, fn gateway_id -> + PubSub.Account.broadcast( + socket.assigns.client.account_id, + {{:invalidate_ice_candidates, gateway_id}, socket.assigns.client.id, candidates} + ) + end) - OpenTelemetry.Tracer.with_span "client.broadcast_invalidated_ice_candidates" do - opentelemetry_ctx = OpenTelemetry.Ctx.get_current() - opentelemetry_span_ctx = OpenTelemetry.Tracer.current_span_ctx() - - :ok = - Enum.each(gateway_ids, fn gateway_id -> - PubSub.Gateway.broadcast( - gateway_id, - {:invalidate_ice_candidates, socket.assigns.client.id, candidates, - {opentelemetry_ctx, opentelemetry_span_ctx}} - ) - end) - - {:noreply, socket} - end + {:noreply, socket} end # Catch-all for unknown messages @@ -977,8 +1005,6 @@ defmodule API.Client.Channel do socket.assigns.client.last_seen_remote_ip_location_lon } - OpenTelemetry.Tracer.set_attribute(:relays_length, length(relays)) - relays = Relays.load_balance_relays(location, relays) {:ok, relays} @@ -1120,4 +1146,100 @@ defmodule API.Client.Channel do gateway: %{username: gateway_username, password: gateway_password} } end + + defp disconnect(socket) do + push(socket, "disconnect", %{reason: :token_expired}) + send(socket.transport_pid, %Phoenix.Socket.Broadcast{event: "disconnect"}) + {:stop, :shutdown, socket} + end + + # TODO: Optimization + # We can reduce memory usage of this cache by an order of magnitude by storing + # optimized versions of the fields we need to evaluate policy conditions and + # render data to the client. + defp hydrate_policies_and_resources(socket) do + OpenTelemetry.Tracer.with_span "client.hydrate_policies_and_resources", + attributes: %{ + account_id: socket.assigns.client.account_id + } do + {_policies, acc} = + Policies.all_policies_for_actor!(socket.assigns.subject.actor) + |> Enum.map_reduce(%{policies: %{}, resources: %{}}, fn policy, acc -> + resources = Map.put(acc.resources, policy.resource_id, policy.resource) + + # Remove resource from policy to avoid storing twice + policies = Map.put(acc.policies, policy.id, Map.delete(policy, :resource)) + + {policy, Map.merge(acc, %{policies: policies, resources: resources})} + end) + + assign(socket, + policies: acc.policies, + resources: acc.resources + ) + end + end + + defp hydrate_membership_group_ids(socket) do + OpenTelemetry.Tracer.with_span "client.hydrate_membership_group_ids", + attributes: %{ + account_id: socket.assigns.client.account_id + } do + membership_group_ids = + Actors.all_actor_group_ids!(socket.assigns.subject.actor) + |> MapSet.new() + + assign(socket, membership_group_ids: membership_group_ids) + end + end + + defp authorized_resources(socket) do + OpenTelemetry.Tracer.with_span "client.authorized_resources", + attributes: %{ + account_id: socket.assigns.client.account_id + } do + client = socket.assigns.client + + resource_ids = + socket.assigns.policies + |> Map.values() + |> Policies.filter_by_conforming_policies_for_client(client) + |> Enum.map(& &1.resource_id) + |> Enum.uniq() + + socket.assigns.resources + |> Map.take(resource_ids) + |> Map.values() + |> map_and_filter_compatible_resources(client.last_seen_version) + end + end + + # Returns either the authorized resource or an error tuple of violated properties + defp authorize_resource(socket, resource_id) do + OpenTelemetry.Tracer.with_span "client.authorize_resource", + attributes: %{ + account_id: socket.assigns.client.account_id + } do + socket.assigns.policies + |> Enum.filter(fn {_id, policy} -> policy.resource_id == resource_id end) + |> Enum.map(fn {_id, policy} -> policy end) + |> Enum.reduce_while({:error, []}, fn policy, {:error, acc} -> + case Policies.ensure_client_conforms_policy_conditions(socket.assigns.client, policy) do + {:ok, expires_at} -> + {:halt, {:ok, policy, expires_at}} + + {:error, {:forbidden, violated_properties: violated_properties}} -> + {:cont, {:error, violated_properties ++ acc}} + end + end) + |> case do + {:error, violated_properties} -> + {:error, {:forbidden, violated_properties: violated_properties}} + + {:ok, policy, expires_at} -> + # Set a maximum expiration time for the authorization + {:ok, policy, expires_at || socket.assigns.subject.expires_at} + end + end + end end diff --git a/elixir/apps/api/lib/api/gateway/channel.ex b/elixir/apps/api/lib/api/gateway/channel.ex index ecebdc04f..2ec1bf619 100644 --- a/elixir/apps/api/lib/api/gateway/channel.ex +++ b/elixir/apps/api/lib/api/gateway/channel.ex @@ -1,29 +1,22 @@ defmodule API.Gateway.Channel do use API, :channel alias API.Gateway.Views - alias Domain.{Clients, Gateways, PubSub, Resources, Relays} + alias Domain.{Accounts, Flows, Gateways, PubSub, Relays, Resources, Tokens} alias Domain.Relays.Presence.Debouncer require Logger require OpenTelemetry.Tracer + # The interval at which the flow cache is pruned. + @prune_flow_cache_every :timer.minutes(1) + + # All relayed connections are dropped when this expires, so use + # a long expiration time to avoid frequent disconnections. + @relay_credentials_expire_in_hours 90 * 24 + @impl true def join("gateway", _payload, socket) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) - - OpenTelemetry.Tracer.with_span "gateway.join" do - opentelemetry_ctx = OpenTelemetry.Ctx.get_current() - opentelemetry_span_ctx = OpenTelemetry.Tracer.current_span_ctx() - send(self(), {:after_join, {opentelemetry_ctx, opentelemetry_span_ctx}}) - - socket = - assign(socket, - opentelemetry_ctx: opentelemetry_ctx, - opentelemetry_span_ctx: opentelemetry_span_ctx - ) - - {:ok, socket} - end + send(self(), :after_join) + {:ok, socket} end #################################### @@ -31,42 +24,44 @@ defmodule API.Gateway.Channel do #################################### @impl true - def handle_info({:after_join, {opentelemetry_ctx, opentelemetry_span_ctx}}, socket) do - OpenTelemetry.Ctx.attach(opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(opentelemetry_span_ctx) + def handle_info(:after_join, socket) do + # Initialize the cache + socket = hydrate_flows(socket) + Process.send_after(self(), :prune_flow_cache, @prune_flow_cache_every) - OpenTelemetry.Tracer.with_span "gateway.after_join" do - :ok = Gateways.Presence.connect(socket.assigns.gateway) + # Track gateway's presence + :ok = Gateways.Presence.connect(socket.assigns.gateway) - # Return all connected relays for the account - relay_credentials_expire_at = DateTime.utc_now() |> DateTime.add(90, :day) - {:ok, relays} = select_relays(socket) - :ok = Enum.each(relays, &Domain.Relays.subscribe_to_relay_presence/1) - :ok = maybe_subscribe_for_relays_presence(relays, socket) + # Subscribe to all account updates + :ok = PubSub.Account.subscribe(socket.assigns.gateway.account_id) - account = Domain.Accounts.fetch_account_by_id!(socket.assigns.gateway.account_id) + # Return all connected relays for the account + {:ok, relays} = select_relays(socket) + :ok = Enum.each(relays, &Domain.Relays.subscribe_to_relay_presence/1) + :ok = maybe_subscribe_for_relays_presence(relays, socket) - push(socket, "init", %{ - account_slug: account.slug, - interface: Views.Interface.render(socket.assigns.gateway), - relays: - Views.Relay.render_many( - relays, - socket.assigns.gateway.public_key, - relay_credentials_expire_at - ), - # These aren't used but needed for API compatibility - config: %{ - ipv4_masquerade_enabled: true, - ipv6_masquerade_enabled: true - } - }) + account = Domain.Accounts.fetch_account_by_id!(socket.assigns.gateway.account_id) - # Cache new stamp secrets - socket = Debouncer.cache_stamp_secrets(socket, relays) + init(socket, account, relays) - {:noreply, socket} - end + # Cache new stamp secrets + socket = Debouncer.cache_stamp_secrets(socket, relays) + + {:noreply, socket} + end + + def handle_info(:prune_flow_cache, socket) do + Process.send_after(self(), :prune_flow_cache, @prune_flow_cache_every) + + now = DateTime.utc_now() + + # Reject flows older than 14 days + flows = + socket.assigns.flows + |> Enum.reject(fn {_tuple, expires_at} -> DateTime.compare(expires_at, now) == :lt end) + |> Enum.into(%{}) + + {:noreply, assign(socket, flows: flows)} end # Called to actually push relays_presence with a disconnected relay to the gateway @@ -74,83 +69,77 @@ defmodule API.Gateway.Channel do {:noreply, Debouncer.handle_leave(socket, relay_id, stamp_secret, payload, &push/3)} end - def handle_info("disconnect", socket) do - OpenTelemetry.Tracer.with_span "gateway.disconnect" do - push(socket, "disconnect", %{"reason" => "token_expired"}) - send(socket.transport_pid, %Phoenix.Socket.Broadcast{event: "disconnect"}) - {:stop, :shutdown, socket} - end - end - #################################### ##### Reacting to domain events #### #################################### - # Resource create message is a no-op for the Gateway as the Resource - # details will be sent to the Gateway on an :authorize_flow message - def handle_info({:create_resource, _resource_id}, socket) do + # ACCOUNTS + + # Resend init when config changes so that new slug may be applied + def handle_info( + {:updated, %Accounts.Account{slug: old_slug}, %Accounts.Account{slug: slug} = account}, + socket + ) + when old_slug != slug do + {:ok, relays} = select_relays(socket) + init(socket, account, relays) + {:noreply, socket} end - # Resource is updated, eg. traffic filters are changed - def handle_info({:update_resource, resource_id}, socket) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) + # FLOWS - OpenTelemetry.Tracer.with_span "gateway.resource_updated", - attributes: %{resource_id: resource_id} do - resource = Resources.fetch_resource_by_id!(resource_id) + def handle_info({:deleted, %Flows.Flow{} = flow}, socket) do + tuple = {flow.client_id, flow.resource_id} - case API.Client.Channel.map_or_drop_compatible_resource( - resource, - socket.assigns.gateway.last_seen_version - ) do - {:cont, resource} -> - push(socket, "resource_updated", Views.Resource.render(resource)) + socket = + if Map.has_key?(socket.assigns.flows, tuple) do + push(socket, "reject_access", %{ + client_id: flow.client_id, + resource_id: flow.resource_id + }) - :drop -> - Logger.debug("Resource is not compatible with the gateway version", - gateway_id: socket.assigns.gateway.id, - resource_id: resource_id - ) + assign(socket, flows: Map.delete(socket.assigns.flows, tuple)) + else + socket end - {:noreply, socket} - end - end - - # This event is ignored because we will receive a reject_access message from - # the Flows which will trigger a reject_access event - def handle_info({:delete_resource, resource_id}, socket) do - :ok = PubSub.Resource.unsubscribe(resource_id) {:noreply, socket} end - # Flows context broadcasts this message when flow is expired, - # which happens when policy, resource, actor, group, identity or provider were - # disabled or deleted - def handle_info({:expire_flow, flow_id, client_id, resource_id}, socket) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) + # RESOURCES - OpenTelemetry.Tracer.with_span "gateway.reject_access", - attributes: %{ - flow_id: flow_id, - client_id: client_id, - resource_id: resource_id - } do - :ok = PubSub.Flow.unsubscribe(flow_id) + # The gateway only handles filter changes. Other breaking changes are handled by deleting + # relevant flows for the resource. + def handle_info( + {:updated, %Resources.Resource{filters: old_filters}, + %Resources.Resource{filters: filters, id: id} = resource}, + socket + ) + when old_filters != filters do + has_flows? = + socket.assigns.flows + |> Enum.any?(fn {{_client_id, resource_id}, _expires_at} -> resource_id == id end) - push(socket, "reject_access", %{ - flow_id: flow_id, - client_id: client_id, - resource_id: resource_id - }) - - {:noreply, socket} + if has_flows? do + push(socket, "resource_updated", Views.Resource.render(resource)) end + + {:noreply, socket} end + # TOKENS + + # Our gateway token was deleted - disconnect WebSocket + def handle_info({:deleted, %Tokens.Token{type: :gateway_group, id: id}}, socket) + when id == socket.assigns.token_id do + disconnect(socket) + end + + #################################### + #### Reacting to relay presence #### + #################################### + @impl true def handle_info( %Phoenix.Socket.Broadcast{ @@ -160,42 +149,34 @@ defmodule API.Gateway.Channel do }, socket ) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) - if Map.has_key?(leaves, relay_id) do - OpenTelemetry.Tracer.with_span "gateway.relays_presence", - attributes: %{ - relay_id: relay_id - } do - :ok = Domain.Relays.unsubscribe_from_relay_presence(relay_id) + :ok = Domain.Relays.unsubscribe_from_relay_presence(relay_id) - relay_credentials_expire_at = DateTime.utc_now() |> DateTime.add(90, :day) - {:ok, relays} = select_relays(socket, [relay_id]) - :ok = maybe_subscribe_for_relays_presence(relays, socket) + relay_credentials_expire_at = DateTime.utc_now() |> DateTime.add(90, :day) + {:ok, relays} = select_relays(socket, [relay_id]) + :ok = maybe_subscribe_for_relays_presence(relays, socket) - :ok = - Enum.each(relays, fn relay -> - # TODO: WAL - # Why are we unsubscribing and subscribing again? - :ok = Domain.Relays.unsubscribe_from_relay_presence(relay) - :ok = Domain.Relays.subscribe_to_relay_presence(relay) - end) + :ok = + Enum.each(relays, fn relay -> + # TODO: WAL + # Why are we unsubscribing and subscribing again? + :ok = Domain.Relays.unsubscribe_from_relay_presence(relay) + :ok = Domain.Relays.subscribe_to_relay_presence(relay) + end) - payload = %{ - disconnected_ids: [relay_id], - connected: - Views.Relay.render_many( - relays, - socket.assigns.gateway.public_key, - relay_credentials_expire_at - ) - } + payload = %{ + disconnected_ids: [relay_id], + connected: + Views.Relay.render_many( + relays, + socket.assigns.gateway.public_key, + relay_credentials_expire_at + ) + } - socket = Debouncer.queue_leave(self(), socket, relay_id, payload) + socket = Debouncer.queue_leave(self(), socket, relay_id, payload) - {:noreply, socket} - end + {:noreply, socket} else {:noreply, socket} end @@ -209,361 +190,252 @@ defmodule API.Gateway.Channel do }, socket ) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) - if Enum.count(joins) > 0 do - OpenTelemetry.Tracer.with_span "gateway.account_relays_presence" do - {:ok, relays} = select_relays(socket) + {:ok, relays} = select_relays(socket) - if length(relays) > 0 do - relay_credentials_expire_at = DateTime.utc_now() |> DateTime.add(90, :day) + if length(relays) > 0 do + relay_credentials_expire_at = DateTime.utc_now() |> DateTime.add(90, :day) - :ok = - Relays.unsubscribe_from_relays_presence_in_account(socket.assigns.gateway.account_id) + :ok = + Relays.unsubscribe_from_relays_presence_in_account(socket.assigns.gateway.account_id) - :ok = - Enum.each(relays, fn relay -> - # TODO: WAL - # Why are we unsubscribing and subscribing again? - :ok = Relays.unsubscribe_from_relay_presence(relay) - :ok = Relays.subscribe_to_relay_presence(relay) - end) + :ok = + Enum.each(relays, fn relay -> + # TODO: WAL + # Why are we unsubscribing and subscribing again? + :ok = Relays.unsubscribe_from_relay_presence(relay) + :ok = Relays.subscribe_to_relay_presence(relay) + end) - # Cache new stamp secrets - socket = Debouncer.cache_stamp_secrets(socket, relays) + # Cache new stamp secrets + socket = Debouncer.cache_stamp_secrets(socket, relays) - # If a relay reconnects with a different stamp_secret, disconnect them immediately - joined_ids = Map.keys(joins) + # If a relay reconnects with a different stamp_secret, disconnect them immediately + joined_ids = Map.keys(joins) - {socket, disconnected_ids} = - Debouncer.cancel_leaves_or_disconnect_immediately( - socket, - joined_ids, - socket.assigns.gateway.account_id + {socket, disconnected_ids} = + Debouncer.cancel_leaves_or_disconnect_immediately( + socket, + joined_ids, + socket.assigns.gateway.account_id + ) + + push(socket, "relays_presence", %{ + disconnected_ids: disconnected_ids, + connected: + Views.Relay.render_many( + relays, + socket.assigns.gateway.public_key, + relay_credentials_expire_at ) + }) - push(socket, "relays_presence", %{ - disconnected_ids: disconnected_ids, - connected: - Views.Relay.render_many( - relays, - socket.assigns.gateway.public_key, - relay_credentials_expire_at - ) - }) - - {:noreply, socket} - end + {:noreply, socket} end else {:noreply, socket} end end - ############################################################## - ##### Forwarding messages from the client to the gateway ##### - ############################################################## + ########################### + #### Connection setup ##### + ########################### def handle_info( - {:ice_candidates, client_id, candidates, {opentelemetry_ctx, opentelemetry_span_ctx}}, - socket - ) do - OpenTelemetry.Ctx.attach(opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(opentelemetry_span_ctx) - - OpenTelemetry.Tracer.with_span "gateway.ice_candidates", - attributes: %{ - client_id: client_id, - candidates_length: length(candidates) - } do - push(socket, "ice_candidates", %{ - client_id: client_id, - candidates: candidates - }) - - {:noreply, socket} - end - end - - def handle_info( - {:invalidate_ice_candidates, client_id, candidates, - {opentelemetry_ctx, opentelemetry_span_ctx}}, - socket - ) do - OpenTelemetry.Ctx.attach(opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(opentelemetry_span_ctx) - - OpenTelemetry.Tracer.with_span "gateway.invalidate_ice_candidates", - attributes: %{ - client_id: client_id, - candidates_length: length(candidates) - } do - push(socket, "invalidate_ice_candidates", %{ - client_id: client_id, - candidates: candidates - }) - - {:noreply, socket} - end - end - - def handle_info( - {:authorize_flow, {channel_pid, socket_ref}, payload, - {opentelemetry_ctx, opentelemetry_span_ctx}}, - socket - ) do - OpenTelemetry.Ctx.attach(opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(opentelemetry_span_ctx) - - %{ + {{:ice_candidates, gateway_id}, client_id, candidates}, + %{assigns: %{gateway: %{id: id}}} = socket + ) + when gateway_id == id do + push(socket, "ice_candidates", %{ client_id: client_id, - resource_id: resource_id, - flow_id: flow_id, + candidates: candidates + }) + + {:noreply, socket} + end + + def handle_info( + {{:invalidate_ice_candidates, gateway_id}, client_id, candidates}, + %{assigns: %{gateway: %{id: id}}} = socket + ) + when gateway_id == id do + push(socket, "invalidate_ice_candidates", %{ + client_id: client_id, + candidates: candidates + }) + + {:noreply, socket} + end + + def handle_info( + {{:authorize_flow, gateway_id}, {channel_pid, socket_ref}, payload}, + %{assigns: %{gateway: %{id: id}}} = socket + ) + when gateway_id == id do + %{ + client: client, + resource: resource, authorization_expires_at: authorization_expires_at, ice_credentials: ice_credentials, preshared_key: preshared_key } = payload - OpenTelemetry.Tracer.with_span "gateway.authorize_flow" do - :ok = PubSub.Flow.subscribe(flow_id) - - Logger.debug("Gateway authorizes a new network flow", - flow_id: flow_id, - client_id: client_id, - resource_id: resource_id - ) - - client = Clients.fetch_client_by_id!(client_id, preload: [:actor]) - resource = Resources.fetch_resource_by_id!(resource_id) - - # TODO: WAL - # Why are we unsubscribing and subscribing again? - :ok = PubSub.Resource.unsubscribe(resource_id) - :ok = PubSub.Resource.subscribe(resource_id) - - opentelemetry_headers = :otel_propagator_text_map.inject([]) - - ref = - encode_ref(socket, { - channel_pid, - socket_ref, - resource_id, - preshared_key, - ice_credentials, - opentelemetry_headers - }) - - push(socket, "authorize_flow", %{ - ref: ref, - flow_id: flow_id, - actor: Views.Actor.render(client.actor), - resource: Views.Resource.render(resource), - gateway_ice_credentials: ice_credentials.gateway, - client: Views.Client.render(client, preshared_key), - client_ice_credentials: ice_credentials.client, - expires_at: - if(authorization_expires_at, do: DateTime.to_unix(authorization_expires_at, :second)) + ref = + encode_ref(socket, { + channel_pid, + socket_ref, + resource.id, + preshared_key, + ice_credentials }) - Logger.debug("Awaiting gateway flow_authorized message", - client_id: client_id, - resource_id: resource_id, - flow_id: flow_id - ) + push(socket, "authorize_flow", %{ + ref: ref, + resource: Views.Resource.render(resource), + gateway_ice_credentials: ice_credentials.gateway, + client: Views.Client.render(client, preshared_key), + client_ice_credentials: ice_credentials.client, + # Gateway manages its own expiration + expires_at: + if(authorization_expires_at, + do: DateTime.to_unix(authorization_expires_at, :second) + ) + }) - {:noreply, socket} - end + # Start tracking flow + flows = Map.put(socket.assigns.flows, {client.id, resource.id}, authorization_expires_at) + socket = assign(socket, flows: flows) + + {:noreply, socket} end # DEPRECATED IN 1.4 def handle_info( - {:allow_access, {channel_pid, socket_ref}, attrs, - {opentelemetry_ctx, opentelemetry_span_ctx}}, - socket - ) do - OpenTelemetry.Ctx.attach(opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(opentelemetry_span_ctx) - + {{:allow_access, gateway_id}, {channel_pid, socket_ref}, attrs}, + %{assigns: %{gateway: %{id: id}}} = socket + ) + when gateway_id == id do %{ - client_id: client_id, - resource_id: resource_id, - flow_id: flow_id, + client: client, + resource: resource, authorization_expires_at: authorization_expires_at, client_payload: payload } = attrs - OpenTelemetry.Tracer.with_span "gateway.allow_access", - attributes: %{ - flow_id: flow_id, - client_id: client_id, - resource_id: resource_id - } do - :ok = PubSub.Flow.subscribe(flow_id) - - client = Clients.fetch_client_by_id!(client_id) - resource = Resources.fetch_resource_by_id!(resource_id) - - case API.Client.Channel.map_or_drop_compatible_resource( - resource, - socket.assigns.gateway.last_seen_version - ) do - {:cont, resource} -> - # TODO: WAL - # Why are we unsubscribing and subscribing again? - :ok = PubSub.Resource.unsubscribe(resource_id) - :ok = PubSub.Resource.subscribe(resource_id) - - opentelemetry_headers = :otel_propagator_text_map.inject([]) - ref = encode_ref(socket, {channel_pid, socket_ref, resource_id, opentelemetry_headers}) - - push(socket, "allow_access", %{ - ref: ref, - client_id: client_id, - flow_id: flow_id, - resource: Views.Resource.render(resource), - expires_at: DateTime.to_unix(authorization_expires_at, :second), - payload: payload, - client_ipv4: client.ipv4, - client_ipv6: client.ipv6 - }) - - Logger.debug("Awaiting gateway connection_ready message", - client_id: client_id, - resource_id: resource_id, - flow_id: flow_id + case API.Client.Channel.map_or_drop_compatible_resource( + resource, + socket.assigns.gateway.last_seen_version + ) do + {:cont, resource} -> + ref = + encode_ref( + socket, + {channel_pid, socket_ref, resource.id} ) - :drop -> - Logger.debug("Resource is not compatible with the gateway version", - gateway_id: socket.assigns.gateway.id, - client_id: client_id, - resource_id: resource_id, - flow_id: flow_id - ) - end + expires_at = DateTime.to_unix(authorization_expires_at, :second) - {:noreply, socket} + push(socket, "allow_access", %{ + ref: ref, + client_id: client.id, + resource: Views.Resource.render(resource), + expires_at: expires_at, + payload: payload, + client_ipv4: client.ipv4, + client_ipv6: client.ipv6 + }) + + # Start tracking the flow + flows = Map.put(socket.assigns.flows, {client.id, resource.id}, authorization_expires_at) + socket = assign(socket, flows: flows) + + {:noreply, socket} + + :drop -> + {:noreply, socket} end end # DEPRECATED IN 1.4 def handle_info( - {:request_connection, {channel_pid, socket_ref}, attrs, - {opentelemetry_ctx, opentelemetry_span_ctx}}, - socket - ) do - OpenTelemetry.Ctx.attach(opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(opentelemetry_span_ctx) - + {{:request_connection, gateway_id}, {channel_pid, socket_ref}, attrs}, + %{assigns: %{gateway: %{id: id}}} = socket + ) + when gateway_id == id do %{ - client_id: client_id, - resource_id: resource_id, - flow_id: flow_id, + client: client, + resource: resource, authorization_expires_at: authorization_expires_at, client_payload: payload, client_preshared_key: preshared_key } = attrs - OpenTelemetry.Tracer.with_span "gateway.request_connection" do - :ok = PubSub.Flow.subscribe(flow_id) - - Logger.debug("Gateway received connection request message", - client_id: client_id, - resource_id: resource_id - ) - - client = Clients.fetch_client_by_id!(client_id, preload: [:actor]) - resource = Resources.fetch_resource_by_id!(resource_id) - - case API.Client.Channel.map_or_drop_compatible_resource( - resource, - socket.assigns.gateway.last_seen_version - ) do - {:cont, resource} -> - # TODO: WAL - # Why are we unsubscribing and subscribing again? - :ok = PubSub.Resource.unsubscribe(resource_id) - :ok = PubSub.Resource.subscribe(resource_id) - - opentelemetry_headers = :otel_propagator_text_map.inject([]) - ref = encode_ref(socket, {channel_pid, socket_ref, resource_id, opentelemetry_headers}) - - push(socket, "request_connection", %{ - ref: ref, - flow_id: flow_id, - actor: Views.Actor.render(client.actor), - resource: Views.Resource.render(resource), - client: Views.Client.render(client, payload, preshared_key), - expires_at: DateTime.to_unix(authorization_expires_at, :second) - }) - - Logger.debug("Awaiting gateway connection_ready message", - client_id: client_id, - resource_id: resource_id, - flow_id: flow_id + case API.Client.Channel.map_or_drop_compatible_resource( + resource, + socket.assigns.gateway.last_seen_version + ) do + {:cont, resource} -> + ref = + encode_ref( + socket, + {channel_pid, socket_ref, resource.id} ) - :drop -> - Logger.debug("Resource is not compatible with the gateway version", - gateway_id: socket.assigns.gateway.id, - client_id: client_id, - resource_id: resource_id, - flow_id: flow_id - ) - end + expires_at = DateTime.to_unix(authorization_expires_at, :second) - {:noreply, socket} + push(socket, "request_connection", %{ + ref: ref, + resource: Views.Resource.render(resource), + client: Views.Client.render(client, payload, preshared_key), + expires_at: expires_at + }) + + # Start tracking the flow + flows = Map.put(socket.assigns.flows, {client.id, resource.id}, authorization_expires_at) + socket = assign(socket, flows: flows) + + {:noreply, socket} + + :drop -> + {:noreply, socket} end end + # Catch-all for messages we don't handle + def handle_info(_message, socket), do: {:noreply, socket} + @impl true def handle_in("flow_authorized", %{"ref" => signed_ref}, socket) do - OpenTelemetry.Tracer.with_span "gateway.flow_authorized" do - case decode_ref(socket, signed_ref) do - {:ok, - { - channel_pid, - socket_ref, - resource_id, - preshared_key, - ice_credentials, - opentelemetry_headers - }} -> - :otel_propagator_text_map.extract(opentelemetry_headers) + case decode_ref(socket, signed_ref) do + {:ok, + { + channel_pid, + socket_ref, + resource_id, + preshared_key, + ice_credentials + }} -> + send( + channel_pid, + { + :connect, + socket_ref, + resource_id, + socket.assigns.gateway.group_id, + socket.assigns.gateway.id, + socket.assigns.gateway.public_key, + socket.assigns.gateway.ipv4, + socket.assigns.gateway.ipv6, + preshared_key, + ice_credentials + } + ) - opentelemetry_ctx = OpenTelemetry.Ctx.get_current() - opentelemetry_span_ctx = OpenTelemetry.Tracer.current_span_ctx() + {:reply, :ok, socket} - send( - channel_pid, - { - :connect, - socket_ref, - resource_id, - socket.assigns.gateway.group_id, - socket.assigns.gateway.id, - socket.assigns.gateway.public_key, - socket.assigns.gateway.ipv4, - socket.assigns.gateway.ipv6, - preshared_key, - ice_credentials, - {opentelemetry_ctx, opentelemetry_span_ctx} - } - ) - - Logger.debug("Gateway replied to the Client with :authorize_flow message", - resource_id: resource_id, - channel_pid: inspect(channel_pid) - ) - - {:reply, :ok, socket} - - {:error, :invalid_ref} -> - OpenTelemetry.Tracer.set_status(:error, "invalid ref") - Logger.error("Gateway replied with an invalid ref") - {:reply, {:error, %{reason: :invalid_ref}}, socket} - end + {:error, :invalid_ref} -> + Logger.error("Gateway replied with an invalid ref") + {:reply, {:error, %{reason: :invalid_ref}}, socket} end end @@ -577,32 +449,18 @@ defmodule API.Gateway.Channel do }, socket ) do - OpenTelemetry.Tracer.with_span "gateway.connection_ready" do - case decode_ref(socket, signed_ref) do - {:ok, {channel_pid, socket_ref, resource_id, opentelemetry_headers}} -> - :otel_propagator_text_map.extract(opentelemetry_headers) + case decode_ref(socket, signed_ref) do + {:ok, {channel_pid, socket_ref, resource_id}} -> + send( + channel_pid, + {:connect, socket_ref, resource_id, socket.assigns.gateway.public_key, payload} + ) - opentelemetry_ctx = OpenTelemetry.Ctx.get_current() - opentelemetry_span_ctx = OpenTelemetry.Tracer.current_span_ctx() + {:reply, :ok, socket} - send( - channel_pid, - {:connect, socket_ref, resource_id, socket.assigns.gateway.public_key, payload, - {opentelemetry_ctx, opentelemetry_span_ctx}} - ) - - Logger.debug("Gateway replied to the Client with :connect message", - resource_id: resource_id, - channel_pid: inspect(channel_pid) - ) - - {:reply, :ok, socket} - - {:error, :invalid_ref} -> - OpenTelemetry.Tracer.set_status(:error, "invalid ref") - Logger.error("Gateway replied with an invalid ref") - {:reply, {:error, %{reason: :invalid_ref}}, socket} - end + {:error, :invalid_ref} -> + Logger.error("Gateway replied with an invalid ref") + {:reply, {:error, %{reason: :invalid_ref}}, socket} end end @@ -615,24 +473,15 @@ defmodule API.Gateway.Channel do %{"candidates" => candidates, "client_ids" => client_ids}, socket ) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) + :ok = + Enum.each(client_ids, fn client_id -> + PubSub.Account.broadcast( + socket.assigns.gateway.account_id, + {{:ice_candidates, client_id}, socket.assigns.gateway.id, candidates} + ) + end) - OpenTelemetry.Tracer.with_span "gateway.broadcast_ice_candidates" do - opentelemetry_ctx = OpenTelemetry.Ctx.get_current() - opentelemetry_span_ctx = OpenTelemetry.Tracer.current_span_ctx() - - :ok = - Enum.each(client_ids, fn client_id -> - PubSub.Client.broadcast( - client_id, - {:ice_candidates, socket.assigns.gateway.id, candidates, - {opentelemetry_ctx, opentelemetry_span_ctx}} - ) - end) - - {:noreply, socket} - end + {:noreply, socket} end def handle_in( @@ -640,24 +489,15 @@ defmodule API.Gateway.Channel do %{"candidates" => candidates, "client_ids" => client_ids}, socket ) do - OpenTelemetry.Ctx.attach(socket.assigns.opentelemetry_ctx) - OpenTelemetry.Tracer.set_current_span(socket.assigns.opentelemetry_span_ctx) + :ok = + Enum.each(client_ids, fn client_id -> + PubSub.Account.broadcast( + socket.assigns.gateway.account_id, + {{:invalidate_ice_candidates, client_id}, socket.assigns.gateway.id, candidates} + ) + end) - OpenTelemetry.Tracer.with_span "gateway.broadcast_invalidated_ice_candidates" do - opentelemetry_ctx = OpenTelemetry.Ctx.get_current() - opentelemetry_span_ctx = OpenTelemetry.Tracer.current_span_ctx() - - :ok = - Enum.each(client_ids, fn client_id -> - PubSub.Client.broadcast( - client_id, - {:invalidate_ice_candidates, socket.assigns.gateway.id, candidates, - {opentelemetry_ctx, opentelemetry_span_ctx}} - ) - end) - - {:noreply, socket} - end + {:noreply, socket} end # Catch-all for unknown messages @@ -702,13 +542,33 @@ defmodule API.Gateway.Channel do socket.assigns.gateway.last_seen_remote_ip_location_lon } - OpenTelemetry.Tracer.set_attribute(:relays_length, length(relays)) - relays = Relays.load_balance_relays(location, relays) {:ok, relays} end + defp init(socket, account, relays) do + relay_credentials_expire_at = + DateTime.utc_now() |> DateTime.add(@relay_credentials_expire_in_hours, :hour) + + push(socket, "init", %{ + authorizations: Views.Flow.render_many(socket.assigns.flows), + account_slug: account.slug, + interface: Views.Interface.render(socket.assigns.gateway), + relays: + Views.Relay.render_many( + relays, + socket.assigns.gateway.public_key, + relay_credentials_expire_at + ), + # These aren't used but needed for API compatibility + config: %{ + ipv4_masquerade_enabled: true, + ipv6_masquerade_enabled: true + } + }) + end + defp maybe_subscribe_for_relays_presence(relays, socket) do if length(relays) > 0 do :ok @@ -716,4 +576,24 @@ defmodule API.Gateway.Channel do Relays.subscribe_to_relays_presence_in_account(socket.assigns.gateway.account_id) end end + + defp hydrate_flows(socket) do + OpenTelemetry.Tracer.with_span "gateway.hydrate_flows", + attributes: %{ + gateway_id: socket.assigns.gateway.id, + account_id: socket.assigns.gateway.account_id + } do + flows = + Flows.all_gateway_flows_for_cache!(socket.assigns.gateway) + |> Map.new() + + assign(socket, flows: flows) + end + end + + defp disconnect(socket) do + push(socket, "disconnect", %{"reason" => "token_expired"}) + send(socket.transport_pid, %Phoenix.Socket.Broadcast{event: "disconnect"}) + {:stop, :shutdown, socket} + end end diff --git a/elixir/apps/api/lib/api/gateway/views/flow.ex b/elixir/apps/api/lib/api/gateway/views/flow.ex new file mode 100644 index 000000000..4e614fd0e --- /dev/null +++ b/elixir/apps/api/lib/api/gateway/views/flow.ex @@ -0,0 +1,11 @@ +defmodule API.Gateway.Views.Flow do + def render_many(flows) do + flows + |> Enum.map(fn {{client_id, resource_id}, _expires_at} -> + %{ + client_id: client_id, + resource_id: resource_id + } + end) + end +end diff --git a/elixir/apps/api/test/api/client/channel_test.exs b/elixir/apps/api/test/api/client/channel_test.exs index fb7c2c7dd..2f25d843a 100644 --- a/elixir/apps/api/test/api/client/channel_test.exs +++ b/elixir/apps/api/test/api/client/channel_test.exs @@ -1,6 +1,6 @@ defmodule API.Client.ChannelTest do use API.ChannelCase, async: true - alias Domain.{Clients, Events, PubSub} + alias Domain.Clients setup do account = @@ -19,7 +19,9 @@ defmodule API.Client.ChannelTest do actor_group = Fixtures.Actors.create_group(account: account) actor = Fixtures.Actors.create_actor(type: :account_admin_user, account: account) - Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) + + membership = + Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) identity = Fixtures.Auth.create_identity(actor: actor, account: account) subject = Fixtures.Auth.create_subject(identity: identity) @@ -115,11 +117,12 @@ defmodule API.Client.ChannelTest do ] ) - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: internet_resource - ) + internet_resource_policy = + Fixtures.Policies.create_policy( + account: account, + actor_group: actor_group, + resource: internet_resource + ) Fixtures.Policies.create_policy( account: account, @@ -134,8 +137,6 @@ defmodule API.Client.ChannelTest do {:ok, _reply, socket} = API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -150,6 +151,7 @@ defmodule API.Client.ChannelTest do client: client, gateway_group_token: gateway_group_token, gateway_group: gateway_group, + membership: membership, gateway: gateway, internet_gateway_group: internet_gateway_group, internet_gateway_group_token: internet_gateway_group_token, @@ -162,6 +164,7 @@ defmodule API.Client.ChannelTest do nonconforming_resource: nonconforming_resource, offline_resource: offline_resource, dns_resource_policy: dns_resource_policy, + internet_resource_policy: internet_resource_policy, socket: socket } end @@ -188,8 +191,6 @@ defmodule API.Client.ChannelTest do {:ok, _reply, _socket} = API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -210,16 +211,14 @@ defmodule API.Client.ChannelTest do {:ok, _reply, _socket} = API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) |> subscribe_and_join(API.Client.Channel, "client") assert_push "disconnect", %{reason: :token_expired}, 250 - assert_receive {:EXIT, _pid, {:shutdown, :token_expired}} - assert_receive {:socket_close, _pid, {:shutdown, :token_expired}} + assert_receive {:EXIT, _pid, :shutdown} + assert_receive {:socket_close, _pid, :shutdown} end test "selects compatible gateway versions", %{client: client, subject: subject} do @@ -228,8 +227,6 @@ defmodule API.Client.ChannelTest do {:ok, _reply, socket} = API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -242,8 +239,6 @@ defmodule API.Client.ChannelTest do {:ok, _reply, socket} = API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -255,8 +250,6 @@ defmodule API.Client.ChannelTest do assert API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -394,8 +387,6 @@ defmodule API.Client.ChannelTest do API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -467,8 +458,6 @@ defmodule API.Client.ChannelTest do API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -496,39 +485,6 @@ defmodule API.Client.ChannelTest do assert "us-east*-d.glob-example.com" not in resource_addresses end - test "subscribes for client events", %{ - client: client - } do - assert_push "init", %{} - Process.flag(:trap_exit, true) - PubSub.Client.broadcast(client.id, :token_expired) - assert_push "disconnect", %{reason: :token_expired}, 250 - end - - test "subscribes for resource events", %{ - dns_resource: resource, - subject: subject - } do - assert_push "init", %{} - - {:ok, _resource} = Domain.Resources.update_resource(resource, %{name: "foobar"}, subject) - - old_data = %{ - "id" => resource.id, - "account_id" => resource.account_id, - "address" => resource.address, - "name" => resource.name, - "type" => "dns", - "filters" => [], - "ip_stack" => "dual" - } - - data = Map.put(old_data, "name", "new name") - Events.Hooks.Resources.on_update(old_data, data) - - assert_push "resource_created_or_updated", %{} - end - test "subscribes for relays presence", %{client: client, subject: subject} do relay_group = Fixtures.Relays.create_global_group() @@ -554,8 +510,6 @@ defmodule API.Client.ChannelTest do API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -606,8 +560,6 @@ defmodule API.Client.ChannelTest do API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -670,8 +622,6 @@ defmodule API.Client.ChannelTest do API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -701,117 +651,149 @@ defmodule API.Client.ChannelTest do assert relay1_id == relay1.id end - - test "subscribes for policy events", %{ - dns_resource_policy: dns_resource_policy, - subject: subject - } do - assert_push "init", %{} - {:ok, policy} = Domain.Policies.disable_policy(dns_resource_policy, subject) - - # Simulate disable - old_data = %{ - "id" => policy.id, - "account_id" => policy.account_id, - "resource_id" => policy.resource_id, - "actor_group_id" => policy.actor_group_id, - "conditions" => [], - "disabled_at" => nil - } - - data = Map.put(old_data, "disabled_at", "2024-01-01T00:00:00Z") - Events.Hooks.Policies.on_update(old_data, data) - - assert_push "resource_deleted", _payload - refute_push "resource_created_or_updated", _payload - end end - describe "handle_info/2 :config_changed" do - test "sends updated configuration", %{ - account: account, - client: client, - socket: socket - } do - channel_pid = socket.channel_pid + describe "handle_info/2" do + # test "subscribes for client events", %{ + # client: client + # } do + # assert_push "init", %{} + # Process.flag(:trap_exit, true) + # PubSub.Client.broadcast(client.id, :token_expired) + # assert_push "disconnect", %{reason: :token_expired}, 250 + # end - Fixtures.Accounts.update_account( - account, - config: %{ - clients_upstream_dns: [ - %{protocol: "ip_port", address: "1.2.3.1"}, - %{protocol: "ip_port", address: "1.8.8.1:53"} - ], - search_domain: "example.com" - } - ) + # test "subscribes for resource events", %{ + # dns_resource: resource, + # subject: subject + # } do + # assert_push "init", %{} + # + # {:ok, _resource} = Domain.Resources.update_resource(resource, %{name: "foobar"}, subject) + # + # old_data = %{ + # "id" => resource.id, + # "account_id" => resource.account_id, + # "address" => resource.address, + # "name" => resource.name, + # "type" => "dns", + # "filters" => [], + # "ip_stack" => "dual" + # } + # + # data = Map.put(old_data, "name", "new name") + # Events.Hooks.Resources.on_update(old_data, data) + # + # assert_push "resource_created_or_updated", %{} + # end - send(channel_pid, :config_changed) + # test "subscribes for policy events", %{ + # dns_resource_policy: dns_resource_policy, + # subject: subject + # } do + # assert_push "init", %{} + # {:ok, policy} = Domain.Policies.disable_policy(dns_resource_policy, subject) + # + # # Simulate disable + # old_data = %{ + # "id" => policy.id, + # "account_id" => policy.account_id, + # "resource_id" => policy.resource_id, + # "actor_group_id" => policy.actor_group_id, + # "conditions" => [], + # "disabled_at" => nil + # } + # + # data = Map.put(old_data, "disabled_at", "2024-01-01T00:00:00Z") + # Events.Hooks.Policies.on_update(old_data, data) + # + # assert_push "resource_deleted", _payload + # refute_push "resource_created_or_updated", _payload + # end - assert_push "config_changed", %{interface: interface} + # describe "handle_info/2 :config_changed" do + # test "sends updated configuration", %{ + # account: account, + # client: client, + # socket: socket + # } do + # channel_pid = socket.channel_pid + # + # Fixtures.Accounts.update_account( + # account, + # config: %{ + # clients_upstream_dns: [ + # %{protocol: "ip_port", address: "1.2.3.1"}, + # %{protocol: "ip_port", address: "1.8.8.1:53"} + # ], + # search_domain: "example.com" + # } + # ) + # + # send(channel_pid, :config_changed) + # + # assert_push "config_changed", %{interface: interface} + # + # assert interface == %{ + # ipv4: client.ipv4, + # ipv6: client.ipv6, + # upstream_dns: [ + # %{protocol: :ip_port, address: "1.2.3.1:53"}, + # %{protocol: :ip_port, address: "1.8.8.1:53"} + # ], + # search_domain: "example.com" + # } + # end + # end - assert interface == %{ - ipv4: client.ipv4, - ipv6: client.ipv6, - upstream_dns: [ - %{protocol: :ip_port, address: "1.2.3.1:53"}, - %{protocol: :ip_port, address: "1.8.8.1:53"} - ], - search_domain: "example.com" - } - end - end + # describe "handle_info/2 {:updated, client}" do + # test "sends init message when breaking fields change", %{ + # socket: socket, + # client: client + # } do + # assert_push "init", %{} + # + # updated_client = %{client | verified_at: DateTime.utc_now()} + # send(socket.channel_pid, {:updated, updated_client}) + # assert_push "init", %{} + # end + # + # test "does not send init message when name changes", %{ + # socket: socket, + # client: client + # } do + # assert_push "init", %{} + # + # send(socket.channel_pid, {:updated, %{client | name: "New Name"}}) + # + # refute_push "init", %{} + # end + # end - describe "handle_info/2 {:updated, client}" do - test "sends init message when breaking fields change", %{ - socket: socket, - client: client - } do - assert_push "init", %{} + # describe "handle_info/2 :token_expired" do + # test "sends a token_expired messages and closes the socket", %{ + # socket: socket + # } do + # Process.flag(:trap_exit, true) + # channel_pid = socket.channel_pid + # + # send(channel_pid, :token_expired) + # assert_push "disconnect", %{reason: :token_expired} + # + # assert_receive {:EXIT, ^channel_pid, {:shutdown, :token_expired}} + # end + # end - updated_client = %{client | verified_at: DateTime.utc_now()} - send(socket.channel_pid, {:updated, updated_client}) - assert_push "init", %{} - end - - test "does not send init message when name changes", %{ - socket: socket, - client: client - } do - assert_push "init", %{} - - send(socket.channel_pid, {:updated, %{client | name: "New Name"}}) - - refute_push "init", %{} - end - end - - describe "handle_info/2 :token_expired" do - test "sends a token_expired messages and closes the socket", %{ - socket: socket - } do - Process.flag(:trap_exit, true) - channel_pid = socket.channel_pid - - send(channel_pid, :token_expired) - assert_push "disconnect", %{reason: :token_expired} - - assert_receive {:EXIT, ^channel_pid, {:shutdown, :token_expired}} - end - end - - describe "handle_info/2 :ice_candidates" do test "pushes ice_candidates message", %{ + client: client, gateway: gateway, socket: socket } do - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} - candidates = ["foo", "bar"] send( socket.channel_pid, - {:ice_candidates, gateway.id, candidates, otel_ctx} + {{:ice_candidates, client.id}, gateway.id, candidates} ) assert_push "ice_candidates", payload @@ -821,20 +803,17 @@ defmodule API.Client.ChannelTest do gateway_id: gateway.id } end - end - describe "handle_info/2 :invalidate_ice_candidates" do test "pushes invalidate_ice_candidates message", %{ + client: client, gateway: gateway, socket: socket } do - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} - candidates = ["foo", "bar"] send( socket.channel_pid, - {:invalidate_ice_candidates, gateway.id, candidates, otel_ctx} + {{:invalidate_ice_candidates, client.id}, gateway.id, candidates} ) assert_push "invalidate_ice_candidates", payload @@ -844,253 +823,245 @@ defmodule API.Client.ChannelTest do gateway_id: gateway.id } end - end - describe "handle_info/2 :create_resource" do - test "pushes message to the socket for authorized clients", %{ - gateway_group: gateway_group, - dns_resource: resource, - socket: socket - } do - send(socket.channel_pid, {:create_resource, resource.id}) + # test "pushes message to the socket for authorized clients", %{ + # gateway_group: gateway_group, + # dns_resource: resource, + # socket: socket + # } do + # send(socket.channel_pid, {:create_resource, resource.id}) + # + # assert_push "resource_created_or_updated", payload + # + # assert payload == %{ + # id: resource.id, + # type: :dns, + # ip_stack: :ipv4_only, + # name: resource.name, + # address: resource.address, + # address_description: resource.address_description, + # gateway_groups: [ + # %{id: gateway_group.id, name: gateway_group.name} + # ], + # filters: [ + # %{protocol: :tcp, port_range_end: 80, port_range_start: 80}, + # %{protocol: :tcp, port_range_end: 433, port_range_start: 433}, + # %{protocol: :udp, port_range_end: 200, port_range_start: 100}, + # %{protocol: :icmp} + # ] + # } + # end + # + # test "does not push resources that can't be access by the client", %{ + # nonconforming_resource: resource, + # socket: socket + # } do + # send(socket.channel_pid, {:create_resource, resource.id}) + # refute_push "resource_created_or_updated", %{} + # end + # end - assert_push "resource_created_or_updated", payload + # test "pushes message to the socket for authorized clients", %{ + # gateway_group: gateway_group, + # dns_resource: resource, + # socket: socket + # } do + # send(socket.channel_pid, {:update_resource, resource.id}) + # + # assert_push "resource_created_or_updated", payload + # + # assert payload == %{ + # id: resource.id, + # type: :dns, + # ip_stack: :ipv4_only, + # name: resource.name, + # address: resource.address, + # address_description: resource.address_description, + # gateway_groups: [ + # %{id: gateway_group.id, name: gateway_group.name} + # ], + # filters: [ + # %{protocol: :tcp, port_range_end: 80, port_range_start: 80}, + # %{protocol: :tcp, port_range_end: 433, port_range_start: 433}, + # %{protocol: :udp, port_range_end: 200, port_range_start: 100}, + # %{protocol: :icmp} + # ] + # } + # end + # + # test "does not push resources that can't be access by the client", %{ + # nonconforming_resource: resource, + # socket: socket + # } do + # send(socket.channel_pid, {:update_resource, resource.id}) + # refute_push "resource_created_or_updated", %{} + # end - assert payload == %{ - id: resource.id, - type: :dns, - ip_stack: :ipv4_only, - name: resource.name, - address: resource.address, - address_description: resource.address_description, - gateway_groups: [ - %{id: gateway_group.id, name: gateway_group.name} - ], - filters: [ - %{protocol: :tcp, port_range_end: 80, port_range_start: 80}, - %{protocol: :tcp, port_range_end: 433, port_range_start: 433}, - %{protocol: :udp, port_range_end: 200, port_range_start: 100}, - %{protocol: :icmp} - ] - } - end + # test "does nothing", %{ + # dns_resource: resource, + # socket: socket + # } do + # send(socket.channel_pid, {:delete_resource, resource.id}) + # refute_push "resource_deleted", %{} + # end - test "does not push resources that can't be access by the client", %{ - nonconforming_resource: resource, - socket: socket - } do - send(socket.channel_pid, {:create_resource, resource.id}) - refute_push "resource_created_or_updated", %{} - end - end + # test "subscribes for policy events for actor group", %{ + # account: account, + # gateway_group: gateway_group, + # actor: actor, + # socket: socket + # } do + # resource = + # Fixtures.Resources.create_resource( + # type: :ip, + # address: "192.168.100.2", + # account: account, + # connections: [%{gateway_group_id: gateway_group.id}] + # ) + # + # group = Fixtures.Actors.create_group(account: account) + # + # policy = + # Fixtures.Policies.create_policy( + # account: account, + # actor_group: group, + # resource: resource + # ) + # + # send(socket.channel_pid, {:create_membership, actor.id, group.id}) + # + # Fixtures.Policies.disable_policy(policy) + # + # # Simulate disable + # old_data = %{ + # "id" => policy.id, + # "account_id" => policy.account_id, + # "resource_id" => policy.resource_id, + # "actor_group_id" => policy.actor_group_id, + # "conditions" => [], + # "disabled_at" => nil + # } + # + # data = Map.put(old_data, "disabled_at", "2024-01-01T00:00:00Z") + # Events.Hooks.Policies.on_update(old_data, data) + # + # assert_push "resource_deleted", resource_id + # assert resource_id == resource.id + # + # refute_push "resource_created_or_updated", %{} + # end + # end - describe "handle_info/2 :update_resource" do - test "pushes message to the socket for authorized clients", %{ - gateway_group: gateway_group, - dns_resource: resource, - socket: socket - } do - send(socket.channel_pid, {:update_resource, resource.id}) + # test "allow_access pushes message to the socket", %{ + # account: account, + # gateway: gateway, + # gateway_group: gateway_group, + # dns_resource: resource, + # socket: socket + # } do + # group = Fixtures.Actors.create_group(account: account) + # + # policy = + # Fixtures.Policies.create_policy( + # account: account, + # actor_group: group, + # resource: resource + # ) + # + # send(socket.channel_pid, {:allow_access, policy.id, group.id, resource.id}) + # + # assert_push "resource_created_or_updated", payload + # + # assert payload == %{ + # id: resource.id, + # type: :dns, + # ip_stack: :ipv4_only, + # name: resource.name, + # address: resource.address, + # address_description: resource.address_description, + # gateway_groups: [ + # %{id: gateway_group.id, name: gateway_group.name} + # ], + # filters: [ + # %{protocol: :tcp, port_range_end: 80, port_range_start: 80}, + # %{protocol: :tcp, port_range_end: 433, port_range_start: 433}, + # %{protocol: :udp, port_range_end: 200, port_range_start: 100}, + # %{protocol: :icmp} + # ] + # } + # end - assert_push "resource_created_or_updated", payload - - assert payload == %{ - id: resource.id, - type: :dns, - ip_stack: :ipv4_only, - name: resource.name, - address: resource.address, - address_description: resource.address_description, - gateway_groups: [ - %{id: gateway_group.id, name: gateway_group.name} - ], - filters: [ - %{protocol: :tcp, port_range_end: 80, port_range_start: 80}, - %{protocol: :tcp, port_range_end: 433, port_range_start: 433}, - %{protocol: :udp, port_range_end: 200, port_range_start: 100}, - %{protocol: :icmp} - ] - } - end - - test "does not push resources that can't be access by the client", %{ - nonconforming_resource: resource, - socket: socket - } do - send(socket.channel_pid, {:update_resource, resource.id}) - refute_push "resource_created_or_updated", %{} - end - end - - describe "handle_info/2 :delete_resource" do - test "does nothing", %{ - dns_resource: resource, - socket: socket - } do - send(socket.channel_pid, {:delete_resource, resource.id}) - refute_push "resource_deleted", %{} - end - end - - describe "handle_info/2 :create_membership" do - test "subscribes for policy events for actor group", %{ - account: account, - gateway_group: gateway_group, - actor: actor, - socket: socket - } do - resource = - Fixtures.Resources.create_resource( - type: :ip, - address: "192.168.100.2", - account: account, - connections: [%{gateway_group_id: gateway_group.id}] - ) - - group = Fixtures.Actors.create_group(account: account) - - policy = - Fixtures.Policies.create_policy( - account: account, - actor_group: group, - resource: resource - ) - - send(socket.channel_pid, {:create_membership, actor.id, group.id}) - - Fixtures.Policies.disable_policy(policy) - - # Simulate disable - old_data = %{ - "id" => policy.id, - "account_id" => policy.account_id, - "resource_id" => policy.resource_id, - "actor_group_id" => policy.actor_group_id, - "conditions" => [], - "disabled_at" => nil - } - - data = Map.put(old_data, "disabled_at", "2024-01-01T00:00:00Z") - Events.Hooks.Policies.on_update(old_data, data) - - assert_push "resource_deleted", resource_id - assert resource_id == resource.id - - refute_push "resource_created_or_updated", %{} - end - end - - describe "handle_info/2 :allow_access" do - test "pushes message to the socket", %{ - account: account, - gateway_group: gateway_group, - dns_resource: resource, - socket: socket - } do - group = Fixtures.Actors.create_group(account: account) - - policy = - Fixtures.Policies.create_policy( - account: account, - actor_group: group, - resource: resource - ) - - send(socket.channel_pid, {:allow_access, policy.id, group.id, resource.id}) - - assert_push "resource_created_or_updated", payload - - assert payload == %{ - id: resource.id, - type: :dns, - ip_stack: :ipv4_only, - name: resource.name, - address: resource.address, - address_description: resource.address_description, - gateway_groups: [ - %{id: gateway_group.id, name: gateway_group.name} - ], - filters: [ - %{protocol: :tcp, port_range_end: 80, port_range_start: 80}, - %{protocol: :tcp, port_range_end: 433, port_range_start: 433}, - %{protocol: :udp, port_range_end: 200, port_range_start: 100}, - %{protocol: :icmp} - ] - } - end - end - - describe "handle_info/2 :reject_access" do - test "pushes message to the socket", %{ - account: account, - gateway_group: gateway_group, - socket: socket - } do - resource = - Fixtures.Resources.create_resource( - type: :ip, - address: "192.168.100.3", - account: account, - connections: [%{gateway_group_id: gateway_group.id}] - ) - - group = Fixtures.Actors.create_group(account: account) - - policy = - Fixtures.Policies.create_policy( - account: account, - actor_group: group, - resource: resource - ) - - send(socket.channel_pid, {:reject_access, policy.id, group.id, resource.id}) - - assert_push "resource_deleted", resource_id - assert resource_id == resource.id - - refute_push "resource_created_or_updated", %{} - end - - test "broadcasts a message to re-add the resource if other policy is found", %{ - account: account, - gateway_group: gateway_group, - dns_resource: resource, - socket: socket - } do - group = Fixtures.Actors.create_group(account: account) - - policy = - Fixtures.Policies.create_policy( - account: account, - actor_group: group, - resource: resource - ) - - send(socket.channel_pid, {:reject_access, policy.id, group.id, resource.id}) - - assert_push "resource_deleted", resource_id - assert resource_id == resource.id - - assert_push "resource_created_or_updated", payload - - assert payload == %{ - id: resource.id, - type: :dns, - ip_stack: :ipv4_only, - name: resource.name, - address: resource.address, - address_description: resource.address_description, - gateway_groups: [ - %{id: gateway_group.id, name: gateway_group.name} - ], - filters: [ - %{protocol: :tcp, port_range_end: 80, port_range_start: 80}, - %{protocol: :tcp, port_range_end: 433, port_range_start: 433}, - %{protocol: :udp, port_range_end: 200, port_range_start: 100}, - %{protocol: :icmp} - ] - } - end + # test "pushes message to the socket", %{ + # account: account, + # gateway_group: gateway_group, + # socket: socket + # } do + # resource = + # Fixtures.Resources.create_resource( + # type: :ip, + # address: "192.168.100.3", + # account: account, + # connections: [%{gateway_group_id: gateway_group.id}] + # ) + # + # group = Fixtures.Actors.create_group(account: account) + # + # policy = + # Fixtures.Policies.create_policy( + # account: account, + # actor_group: group, + # resource: resource + # ) + # + # send(socket.channel_pid, {:reject_access, policy.id, group.id, resource.id}) + # + # assert_push "resource_deleted", resource_id + # assert resource_id == resource.id + # + # refute_push "resource_created_or_updated", %{} + # end + # + # test "broadcasts a message to re-add the resource if other policy is found", %{ + # account: account, + # gateway_group: gateway_group, + # dns_resource: resource, + # socket: socket + # } do + # group = Fixtures.Actors.create_group(account: account) + # + # policy = + # Fixtures.Policies.create_policy( + # account: account, + # actor_group: group, + # resource: resource + # ) + # + # send(socket.channel_pid, {:reject_access, policy.id, group.id, resource.id}) + # + # assert_push "resource_deleted", resource_id + # assert resource_id == resource.id + # + # assert_push "resource_created_or_updated", payload + # + # assert payload == %{ + # id: resource.id, + # type: :dns, + # ip_stack: :ipv4_only, + # name: resource.name, + # address: resource.address, + # address_description: resource.address_description, + # gateway_groups: [ + # %{id: gateway_group.id, name: gateway_group.name} + # ], + # filters: [ + # %{protocol: :tcp, port_range_end: 80, port_range_start: 80}, + # %{protocol: :tcp, port_range_end: 433, port_range_start: 433}, + # %{protocol: :udp, port_range_end: 200, port_range_start: 100}, + # %{protocol: :icmp} + # ] + # } + # end + # end end describe "handle_in/3 create_flow" do @@ -1160,26 +1131,34 @@ defmodule API.Client.ChannelTest do actor_group: actor_group, gateway_group: gateway_group, gateway: gateway, + membership: membership, socket: socket } do + send(socket.channel_pid, {:created, membership}) + resource = Fixtures.Resources.create_resource( account: account, connections: [%{gateway_group_id: gateway_group.id}] ) - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource, - conditions: [ - %{ - property: :remote_ip_location_region, - operator: :is_not_in, - values: [client.last_seen_remote_ip_location_region] - } - ] - ) + send(socket.channel_pid, {:created, resource}) + + policy = + Fixtures.Policies.create_policy( + account: account, + actor_group: actor_group, + resource: resource, + conditions: [ + %{ + property: :remote_ip_location_region, + operator: :is_not_in, + values: [client.last_seen_remote_ip_location_region] + } + ] + ) + + send(socket.channel_pid, {:created, policy}) attrs = %{ "resource_id" => resource.id, @@ -1189,7 +1168,6 @@ defmodule API.Client.ChannelTest do :ok = Domain.Gateways.Presence.connect(gateway) push(socket, "create_flow", attrs) - # assert_reply ref, :error, %{reason: :not_found} assert_push "flow_creation_failed", %{ reason: :forbidden, @@ -1213,8 +1191,6 @@ defmodule API.Client.ChannelTest do "connected_gateway_ids" => [] }) - # assert_reply ref, :error, %{reason: :offline} - assert_push "flow_creation_failed", %{ reason: :offline, resource_id: resource_id @@ -1225,6 +1201,8 @@ defmodule API.Client.ChannelTest do test "returns online gateway connected to a resource", %{ dns_resource: resource, + dns_resource_policy: policy, + membership: membership, client: client, gateway_group_token: gateway_group_token, gateway: gateway, @@ -1246,33 +1224,42 @@ defmodule API.Client.ChannelTest do last_seen_at: DateTime.utc_now() |> DateTime.add(-10, :second) ) + :ok = Domain.PubSub.Account.subscribe(gateway.account_id) :ok = Domain.Gateways.Presence.connect(gateway) Domain.PubSub.subscribe(Domain.Tokens.socket_id(gateway_group_token)) + # Prime cache + send(socket.channel_pid, {:created, resource}) + send(socket.channel_pid, {:created, policy}) + send(socket.channel_pid, {:created, membership}) + push(socket, "create_flow", %{ "resource_id" => resource.id, "connected_gateway_ids" => [] }) - assert_receive {:authorize_flow, {_channel_pid, _socket_ref}, payload, _opentelemetry_ctx} + gateway_id = gateway.id + + assert_receive {{:authorize_flow, ^gateway_id}, {_channel_pid, _socket_ref}, payload} assert %{ - client_id: client_id, - resource_id: resource_id, - flow_id: _flow_id, + client: received_client, + resource: received_resource, authorization_expires_at: authorization_expires_at, ice_credentials: _ice_credentials, preshared_key: preshared_key } = payload - assert client_id == client.id - assert resource_id == resource.id + assert received_client.id == client.id + assert received_resource.id == resource.id assert authorization_expires_at == socket.assigns.subject.expires_at assert String.length(preshared_key) == 44 end test "returns online gateway connected to an internet resource", %{ account: account, + membership: membership, + internet_resource_policy: policy, internet_gateway_group_token: gateway_group_token, internet_gateway: gateway, internet_resource: resource, @@ -1297,6 +1284,8 @@ defmodule API.Client.ChannelTest do stamp_secret = Ecto.UUID.generate() :ok = Domain.Relays.connect_relay(global_relay, stamp_secret) + :ok = Domain.PubSub.Account.subscribe(account.id) + Fixtures.Relays.update_relay(global_relay, last_seen_at: DateTime.utc_now() |> DateTime.add(-10, :second) ) @@ -1304,24 +1293,29 @@ defmodule API.Client.ChannelTest do :ok = Domain.Gateways.Presence.connect(gateway) Domain.PubSub.subscribe(Domain.Tokens.socket_id(gateway_group_token)) + send(socket.channel_pid, {:created, resource}) + send(socket.channel_pid, {:created, policy}) + send(socket.channel_pid, {:created, membership}) + push(socket, "create_flow", %{ "resource_id" => resource.id, "connected_gateway_ids" => [] }) - assert_receive {:authorize_flow, {_channel_pid, _socket_ref}, payload, _opentelemetry_ctx} + gateway_id = gateway.id + + assert_receive {{:authorize_flow, ^gateway_id}, {_channel_pid, _socket_ref}, payload} assert %{ - client_id: client_id, - resource_id: resource_id, - flow_id: _flow_id, + client: recv_client, + resource: recv_resource, authorization_expires_at: authorization_expires_at, ice_credentials: _ice_credentials, preshared_key: preshared_key } = payload - assert client_id == client.id - assert resource_id == resource.id + assert recv_client.id == client.id + assert recv_resource.id == resource.id assert authorization_expires_at == socket.assigns.subject.expires_at assert String.length(preshared_key) == 44 end @@ -1329,6 +1323,7 @@ defmodule API.Client.ChannelTest do test "broadcasts authorize_flow to the gateway and flow_created to the client", %{ dns_resource: resource, dns_resource_policy: policy, + membership: membership, client: client, gateway_group_token: gateway_group_token, gateway: gateway, @@ -1346,6 +1341,11 @@ defmodule API.Client.ChannelTest do stamp_secret = Ecto.UUID.generate() :ok = Domain.Relays.connect_relay(global_relay, stamp_secret) + :ok = Domain.PubSub.Account.subscribe(gateway.account_id) + + send(socket.channel_pid, {:created, resource}) + send(socket.channel_pid, {:created, policy}) + send(socket.channel_pid, {:created, membership}) Fixtures.Relays.update_relay(global_relay, last_seen_at: DateTime.utc_now() |> DateTime.add(-10, :second) @@ -1359,19 +1359,23 @@ defmodule API.Client.ChannelTest do "connected_gateway_ids" => [] }) - assert_receive {:authorize_flow, {channel_pid, socket_ref}, payload, _opentelemetry_ctx} + gateway_id = gateway.id + + assert_receive {{:authorize_flow, ^gateway_id}, {channel_pid, socket_ref}, payload} assert %{ - client_id: client_id, - resource_id: resource_id, - flow_id: flow_id, + client: recv_client, + resource: recv_resource, authorization_expires_at: authorization_expires_at, ice_credentials: ice_credentials, preshared_key: preshared_key } = payload - assert flow = Repo.get(Domain.Flows.Flow, flow_id) - assert flow.client_id == client.id + client_id = recv_client.id + resource_id = recv_resource.id + + assert flow = Repo.get_by(Domain.Flows.Flow, client_id: client.id, resource_id: resource.id) + assert flow.client_id == client_id assert flow.resource_id == resource_id assert flow.gateway_id == gateway.id assert flow.policy_id == policy.id @@ -1381,12 +1385,10 @@ defmodule API.Client.ChannelTest do assert resource_id == resource.id assert authorization_expires_at == socket.assigns.subject.expires_at - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} - send( channel_pid, {:connect, socket_ref, resource_id, gateway.group_id, gateway.id, gateway.public_key, - gateway.ipv4, gateway.ipv6, preshared_key, ice_credentials, otel_ctx} + gateway.ipv4, gateway.ipv6, preshared_key, ice_credentials} ) gateway_group_id = gateway.group_id @@ -1403,7 +1405,10 @@ defmodule API.Client.ChannelTest do client_ice_credentials: %{username: client_ice_username, password: client_ice_password}, gateway_group_id: ^gateway_group_id, gateway_id: ^gateway_id, - gateway_ice_credentials: %{username: gateway_ice_username, password: gateway_ice_password}, + gateway_ice_credentials: %{ + username: gateway_ice_username, + password: gateway_ice_password + }, preshared_key: ^preshared_key } @@ -1418,6 +1423,8 @@ defmodule API.Client.ChannelTest do test "works with service accounts", %{ account: account, dns_resource: resource, + dns_resource_policy: policy, + membership: membership, gateway: gateway, gateway_group_token: gateway_group_token, actor_group: actor_group @@ -1432,8 +1439,6 @@ defmodule API.Client.ChannelTest do {:ok, _reply, socket} = API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -1458,20 +1463,31 @@ defmodule API.Client.ChannelTest do :ok = Domain.Gateways.Presence.connect(gateway) Domain.PubSub.subscribe(Domain.Tokens.socket_id(gateway_group_token)) + :ok = Domain.PubSub.Account.subscribe(account.id) + + send(socket.channel_pid, {:created, resource}) + send(socket.channel_pid, {:created, policy}) + send(socket.channel_pid, {:created, membership}) + push(socket, "create_flow", %{ "resource_id" => resource.id, "connected_gateway_ids" => [] }) - assert_receive {:authorize_flow, {_channel_pid, _socket_ref}, _payload, _opentelemetry_ctx} + gateway_id = gateway.id + + assert_receive {{:authorize_flow, ^gateway_id}, {_channel_pid, _socket_ref}, _payload} end test "selects compatible gateway versions", %{ account: account, gateway_group: gateway_group, dns_resource: resource, + dns_resource_policy: policy, + membership: membership, subject: subject, - client: client + client: client, + socket: socket } do global_relay_group = Fixtures.Relays.create_global_group() @@ -1503,11 +1519,15 @@ defmodule API.Client.ChannelTest do :ok = Domain.Gateways.Presence.connect(gateway) + :ok = Domain.PubSub.Account.subscribe(account.id) + + send(socket.channel_pid, {:created, resource}) + send(socket.channel_pid, {:created, policy}) + send(socket.channel_pid, {:created, membership}) + {:ok, _reply, socket} = API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -1543,13 +1563,17 @@ defmodule API.Client.ChannelTest do "connected_gateway_ids" => [] }) - assert_receive {:authorize_flow, {_channel_pid, _socket_ref}, _payload, _opentelemetry_ctx} + gateway_id = gateway.id + + assert_receive {{:authorize_flow, ^gateway_id}, {_channel_pid, _socket_ref}, _payload} end test "selects already connected gateway", %{ account: account, gateway_group: gateway_group, dns_resource: resource, + dns_resource_policy: policy, + membership: membership, socket: socket } do global_relay_group = Fixtures.Relays.create_global_group() @@ -1583,23 +1607,41 @@ defmodule API.Client.ChannelTest do :ok = Domain.Gateways.Presence.connect(gateway2) + :ok = Domain.PubSub.Account.subscribe(account.id) + + send(socket.channel_pid, {:created, resource}) + send(socket.channel_pid, {:created, policy}) + send(socket.channel_pid, {:created, membership}) + push(socket, "create_flow", %{ "resource_id" => resource.id, "connected_gateway_ids" => [gateway2.id] }) - assert_receive {:authorize_flow, {_channel_pid, _socket_ref}, %{flow_id: flow_id}, _} - assert flow = Repo.get(Domain.Flows.Flow, flow_id) - assert flow.gateway_id == gateway2.id + gateway_id = gateway2.id + + assert_receive {{:authorize_flow, ^gateway_id}, {_channel_pid, _socket_ref}, %{}} + + assert Repo.get_by(Domain.Flows.Flow, + resource_id: resource.id, + gateway_id: gateway2.id, + account_id: account.id + ) push(socket, "create_flow", %{ "resource_id" => resource.id, "connected_gateway_ids" => [gateway1.id] }) - assert_receive {:authorize_flow, {_channel_pid, _socket_ref}, %{flow_id: flow_id}, _} - assert flow = Repo.get(Domain.Flows.Flow, flow_id) - assert flow.gateway_id == gateway1.id + gateway_id = gateway1.id + + assert_receive {{:authorize_flow, ^gateway_id}, {_channel_pid, _socket_ref}, %{}} + + assert Repo.get_by(Domain.Flows.Flow, + resource_id: resource.id, + gateway_id: gateway1.id, + account_id: account.id + ) end end @@ -1709,6 +1751,7 @@ defmodule API.Client.ChannelTest do test "returns gateway that support the DNS resource address syntax", %{ account: account, actor_group: actor_group, + membership: membership, socket: socket } do global_relay_group = Fixtures.Relays.create_global_group() @@ -1738,11 +1781,12 @@ defmodule API.Client.ChannelTest do connections: [%{gateway_group_id: gateway_group.id}] ) - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource - ) + policy = + Fixtures.Policies.create_policy( + account: account, + actor_group: actor_group, + resource: resource + ) :ok = Domain.Gateways.Presence.connect(gateway) @@ -1765,6 +1809,11 @@ defmodule API.Client.ChannelTest do ) :ok = Domain.Gateways.Presence.connect(gateway) + :ok = Domain.PubSub.Account.subscribe(account.id) + + send(socket.channel_pid, {:created, resource}) + send(socket.channel_pid, {:created, policy}) + send(socket.channel_pid, {:created, membership}) ref = push(socket, "prepare_connection", %{"resource_id" => resource.id}) @@ -1859,8 +1908,6 @@ defmodule API.Client.ChannelTest do {:ok, _reply, socket} = API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -1928,8 +1975,6 @@ defmodule API.Client.ChannelTest do {:ok, _reply, socket} = API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -2003,6 +2048,7 @@ defmodule API.Client.ChannelTest do account: account, client: client, actor_group: actor_group, + membership: membership, gateway_group: gateway_group, gateway: gateway, socket: socket @@ -2013,18 +2059,19 @@ defmodule API.Client.ChannelTest do connections: [%{gateway_group_id: gateway_group.id}] ) - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource, - conditions: [ - %{ - property: :remote_ip_location_region, - operator: :is_not_in, - values: [client.last_seen_remote_ip_location_region] - } - ] - ) + policy = + Fixtures.Policies.create_policy( + account: account, + actor_group: actor_group, + resource: resource, + conditions: [ + %{ + property: :remote_ip_location_region, + operator: :is_not_in, + values: [client.last_seen_remote_ip_location_region] + } + ] + ) attrs = %{ "resource_id" => resource.id, @@ -2033,6 +2080,11 @@ defmodule API.Client.ChannelTest do } :ok = Domain.Gateways.Presence.connect(gateway) + :ok = Domain.PubSub.Account.subscribe(account.id) + + send(socket.channel_pid, {:created, resource}) + send(socket.channel_pid, {:created, policy}) + send(socket.channel_pid, {:created, membership}) ref = push(socket, "reuse_connection", attrs) @@ -2078,6 +2130,8 @@ defmodule API.Client.ChannelTest do test "broadcasts allow_access to the gateways and then returns connect message", %{ dns_resource: resource, + dns_resource_policy: policy, + membership: membership, gateway: gateway, client: client, socket: socket @@ -2087,6 +2141,11 @@ defmodule API.Client.ChannelTest do client_id = client.id :ok = Domain.Gateways.Presence.connect(gateway) + :ok = Domain.PubSub.Account.subscribe(resource.account_id) + + send(socket.channel_pid, {:created, resource}) + send(socket.channel_pid, {:created, policy}) + send(socket.channel_pid, {:created, membership}) attrs = %{ "resource_id" => resource.id, @@ -2096,22 +2155,24 @@ defmodule API.Client.ChannelTest do ref = push(socket, "reuse_connection", attrs) - assert_receive {:allow_access, {channel_pid, socket_ref}, payload, _opentelemetry_ctx} + gateway_id = gateway.id + + assert_receive {{:allow_access, ^gateway_id}, {channel_pid, socket_ref}, payload} assert %{ - resource_id: ^resource_id, - client_id: ^client_id, + resource: recv_resource, + client: recv_client, authorization_expires_at: authorization_expires_at, client_payload: "DNS_Q" } = payload + assert recv_resource.id == resource_id + assert recv_client.id == client_id assert authorization_expires_at == socket.assigns.subject.expires_at - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} - send( channel_pid, - {:connect, socket_ref, resource.id, gateway.public_key, "DNS_RPL", otel_ctx} + {:connect, socket_ref, resource.id, gateway.public_key, "DNS_RPL"} ) assert_reply ref, :ok, %{ @@ -2125,6 +2186,8 @@ defmodule API.Client.ChannelTest do test "works with service accounts", %{ account: account, dns_resource: resource, + dns_resource_policy: policy, + membership: membership, gateway: gateway, gateway_group_token: gateway_group_token, actor_group: actor_group @@ -2139,8 +2202,6 @@ defmodule API.Client.ChannelTest do {:ok, _reply, socket} = API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -2149,13 +2210,21 @@ defmodule API.Client.ChannelTest do :ok = Domain.Gateways.Presence.connect(gateway) Phoenix.PubSub.subscribe(Domain.PubSub, Domain.Tokens.socket_id(gateway_group_token)) + :ok = Domain.PubSub.Account.subscribe(account.id) + + send(socket.channel_pid, {:created, resource}) + send(socket.channel_pid, {:created, policy}) + send(socket.channel_pid, {:created, membership}) + push(socket, "reuse_connection", %{ "resource_id" => resource.id, "gateway_id" => gateway.id, "payload" => "DNS_Q" }) - assert_receive {:allow_access, _refs, _payload, _opentelemetry_ctx} + gateway_id = gateway.id + + assert_receive {{:allow_access, ^gateway_id}, _refs, _payload} end end @@ -2227,6 +2296,7 @@ defmodule API.Client.ChannelTest do account: account, client: client, actor_group: actor_group, + membership: membership, gateway_group: gateway_group, gateway: gateway, socket: socket @@ -2237,18 +2307,19 @@ defmodule API.Client.ChannelTest do connections: [%{gateway_group_id: gateway_group.id}] ) - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource, - conditions: [ - %{ - property: :remote_ip_location_region, - operator: :is_not_in, - values: [client.last_seen_remote_ip_location_region] - } - ] - ) + policy = + Fixtures.Policies.create_policy( + account: account, + actor_group: actor_group, + resource: resource, + conditions: [ + %{ + property: :remote_ip_location_region, + operator: :is_not_in, + values: [client.last_seen_remote_ip_location_region] + } + ] + ) attrs = %{ "resource_id" => resource.id, @@ -2259,6 +2330,12 @@ defmodule API.Client.ChannelTest do :ok = Domain.Gateways.Presence.connect(gateway) + :ok = Domain.PubSub.Account.subscribe(account.id) + + send(socket.channel_pid, {:created, resource}) + send(socket.channel_pid, {:created, policy}) + send(socket.channel_pid, {:created, membership}) + ref = push(socket, "request_connection", attrs) assert_reply ref, :error, %{ @@ -2297,6 +2374,8 @@ defmodule API.Client.ChannelTest do :ok = Domain.Gateways.Presence.connect(gateway) Domain.PubSub.subscribe(Domain.Tokens.socket_id(gateway_group_token)) + :ok = Domain.PubSub.Account.subscribe(resource.account_id) + attrs = %{ "resource_id" => resource.id, "gateway_id" => gateway.id, @@ -2306,23 +2385,26 @@ defmodule API.Client.ChannelTest do ref = push(socket, "request_connection", attrs) - assert_receive {:request_connection, {channel_pid, socket_ref}, payload, _opentelemetry_ctx} + gateway_id = gateway.id + + assert_receive {{:request_connection, ^gateway_id}, {channel_pid, socket_ref}, payload} assert %{ - resource_id: ^resource_id, - client_id: ^client_id, + resource: recv_resource, + client: recv_client, client_preshared_key: "PSK", client_payload: "RTC_SD", authorization_expires_at: authorization_expires_at } = payload - assert authorization_expires_at == socket.assigns.subject.expires_at + assert recv_resource.id == resource_id + assert recv_client.id == client_id - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} + assert authorization_expires_at == socket.assigns.subject.expires_at send( channel_pid, - {:connect, socket_ref, resource.id, gateway.public_key, "FULL_RTC_SD", otel_ctx} + {:connect, socket_ref, resource.id, gateway.public_key, "FULL_RTC_SD"} ) assert_reply ref, :ok, %{ @@ -2350,8 +2432,6 @@ defmodule API.Client.ChannelTest do {:ok, _reply, socket} = API.Client.Socket |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), client: client, subject: subject }) @@ -2360,6 +2440,8 @@ defmodule API.Client.ChannelTest do :ok = Domain.Gateways.Presence.connect(gateway) Phoenix.PubSub.subscribe(Domain.PubSub, Domain.Tokens.socket_id(gateway_group_token)) + :ok = Domain.PubSub.Account.subscribe(account.id) + push(socket, "request_connection", %{ "resource_id" => resource.id, "gateway_id" => gateway.id, @@ -2367,7 +2449,9 @@ defmodule API.Client.ChannelTest do "client_preshared_key" => "PSK" }) - assert_receive {:request_connection, _refs, _payload, _opentelemetry_ctx} + gateway_id = gateway.id + + assert_receive {{:request_connection, ^gateway_id}, _refs, _payload} end end @@ -2383,7 +2467,7 @@ defmodule API.Client.ChannelTest do } push(socket, "broadcast_ice_candidates", attrs) - refute_receive {:ice_candidates, _client_id, _candidates, _opentelemetry_ctx} + refute_receive {:ice_candidates, _client_id, _candidates} end test "broadcasts :ice_candidates message to all gateways", %{ @@ -2402,9 +2486,13 @@ defmodule API.Client.ChannelTest do :ok = Domain.Gateways.Presence.connect(gateway) Domain.PubSub.subscribe(Domain.Tokens.socket_id(gateway_group_token)) + :ok = Domain.PubSub.Account.subscribe(client.account_id) + push(socket, "broadcast_ice_candidates", attrs) - assert_receive {:ice_candidates, client_id, ^candidates, _opentelemetry_ctx}, 200 + gateway_id = gateway.id + + assert_receive {{:ice_candidates, ^gateway_id}, client_id, ^candidates}, 200 assert client.id == client_id end end @@ -2421,7 +2509,7 @@ defmodule API.Client.ChannelTest do } push(socket, "broadcast_invalidated_ice_candidates", attrs) - refute_receive {:invalidate_ice_candidates, _client_id, _candidates, _opentelemetry_ctx} + refute_receive {:invalidate_ice_candidates, _client_id, _candidates} end test "broadcasts :invalidate_ice_candidates message to all gateways", %{ @@ -2439,17 +2527,20 @@ defmodule API.Client.ChannelTest do :ok = Domain.Gateways.Presence.connect(gateway) Domain.PubSub.subscribe(Domain.Tokens.socket_id(gateway_group_token)) + :ok = Domain.PubSub.Account.subscribe(client.account_id) push(socket, "broadcast_invalidated_ice_candidates", attrs) - assert_receive {:invalidate_ice_candidates, client_id, ^candidates, _opentelemetry_ctx}, 200 + gateway_id = gateway.id + + assert_receive {{:invalidate_ice_candidates, ^gateway_id}, client_id, ^candidates}, 200 assert client.id == client_id end end # Debouncer tests - describe "handle_info/3 :push_leave" do - test "cancels leave if reconnecting with the same stamp secret" do + describe "handle_info/3" do + test "push_leave cancels leave if reconnecting with the same stamp secret" do relay_group = Fixtures.Relays.create_global_group() relay1 = Fixtures.Relays.create_relay(group: relay_group) @@ -2485,7 +2576,7 @@ defmodule API.Client.ChannelTest do relays_presence_timeout() + 10 end - test "disconnects immediately if reconnecting with a different stamp secret" do + test "push_leave disconnects immediately if reconnecting with a different stamp secret" do relay_group = Fixtures.Relays.create_global_group() relay1 = Fixtures.Relays.create_relay(group: relay_group) @@ -2524,7 +2615,7 @@ defmodule API.Client.ChannelTest do assert relay_id == relay1.id end - test "disconnects after the debounce timeout expires" do + test "push_leave disconnects after the debounce timeout expires" do relay_group = Fixtures.Relays.create_global_group() relay1 = Fixtures.Relays.create_relay(group: relay_group) @@ -2553,10 +2644,8 @@ defmodule API.Client.ChannelTest do assert relay_id == relay1.id end - end - describe "handle_in/3 for unknown messages" do - test "it doesn't crash", %{socket: socket} do + test "for unknown messages it doesn't crash", %{socket: socket} do ref = push(socket, "unknown_message", %{}) assert_reply ref, :error, %{reason: :unknown_message} end diff --git a/elixir/apps/api/test/api/gateway/channel_test.exs b/elixir/apps/api/test/api/gateway/channel_test.exs index 8460535fa..43752368b 100644 --- a/elixir/apps/api/test/api/gateway/channel_test.exs +++ b/elixir/apps/api/test/api/gateway/channel_test.exs @@ -1,6 +1,6 @@ defmodule API.Gateway.ChannelTest do use API.ChannelCase, async: true - alias Domain.{Events, Gateways, PubSub} + alias Domain.{Accounts, Events, Gateways, PubSub} setup do account = Fixtures.Accounts.create_account() @@ -17,9 +17,16 @@ defmodule API.Gateway.ChannelTest do connections: [%{gateway_group_id: gateway.group_id}] ) + token = + Fixtures.Gateways.create_token( + group: gateway_group, + account: account + ) + {:ok, _, socket} = API.Gateway.Socket |> socket("gateway:#{gateway.id}", %{ + token_id: token.id, gateway: gateway, gateway_group: gateway_group, opentelemetry_ctx: OpenTelemetry.Ctx.new(), @@ -46,7 +53,8 @@ defmodule API.Gateway.ChannelTest do resource: resource, relay: relay, global_relay: global_relay, - socket: socket + socket: socket, + token: token } end @@ -82,9 +90,62 @@ defmodule API.Gateway.ChannelTest do end end - describe "handle_info/2 :allow_access" do + describe "handle_info/2" do + test "resends init when account slug changes", %{ + account: account + } do + :ok = Domain.PubSub.Account.subscribe(account.id) + + old_data = %{ + "id" => account.id, + "slug" => account.slug + } + + data = %{ + "id" => account.id, + "slug" => "new-slug" + } + + Events.Hooks.Accounts.on_update(old_data, data) + + assert_receive {:updated, %Accounts.Account{}, %Accounts.Account{}} + + # Consume first init from join + assert_push "init", _payload + + assert_push "init", payload + + assert payload.account_slug == "new-slug" + end + + test "disconnects socket when token is deleted", %{ + account: account, + token: token + } do + # Prevents test from failing due to expected socket disconnect + Process.flag(:trap_exit, true) + + :ok = Domain.PubSub.Account.subscribe(account.id) + + data = %{ + "id" => token.id, + "account_id" => account.id, + "type" => "gateway_group" + } + + Events.Hooks.Tokens.on_delete(data) + + assert_receive {:deleted, deleted_token} + assert_push "disconnect", payload + assert_receive {:EXIT, _pid, _} + assert_receive {:socket_close, _pid, _} + assert deleted_token.id == token.id + assert payload == %{"reason" => "token_expired"} + end + test "pushes allow_access message", %{ client: client, + gateway: gateway, resource: resource, relay: relay, socket: socket @@ -92,8 +153,6 @@ defmodule API.Gateway.ChannelTest do channel_pid = self() socket_ref = make_ref() expires_at = DateTime.utc_now() |> DateTime.add(30, :second) - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} - flow_id = Ecto.UUID.generate() client_payload = "RTC_SD_or_DNS_Q" stamp_secret = Ecto.UUID.generate() @@ -101,14 +160,13 @@ defmodule API.Gateway.ChannelTest do send( socket.channel_pid, - {:allow_access, {channel_pid, socket_ref}, + {{:allow_access, gateway.id}, {channel_pid, socket_ref}, %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow_id, + client: client, + resource: resource, authorization_expires_at: expires_at, client_payload: client_payload - }, otel_ctx} + }} ) assert_push "allow_access", payload @@ -127,7 +185,6 @@ defmodule API.Gateway.ChannelTest do } assert payload.ref - assert payload.flow_id == flow_id assert payload.client_id == client.id assert payload.client_ipv4 == client.ipv4 assert payload.client_ipv6 == client.ipv6 @@ -137,6 +194,7 @@ defmodule API.Gateway.ChannelTest do test "pushes allow_access message for internet resource", %{ account: account, client: client, + gateway: gateway, relay: relay, socket: socket } do @@ -151,8 +209,6 @@ defmodule API.Gateway.ChannelTest do channel_pid = self() socket_ref = make_ref() expires_at = DateTime.utc_now() |> DateTime.add(30, :second) - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} - flow_id = Ecto.UUID.generate() client_payload = "RTC_SD_or_DNS_Q" stamp_secret = Ecto.UUID.generate() @@ -160,14 +216,13 @@ defmodule API.Gateway.ChannelTest do send( socket.channel_pid, - {:allow_access, {channel_pid, socket_ref}, + {{:allow_access, gateway.id}, {channel_pid, socket_ref}, %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow_id, + client: client, + resource: resource, authorization_expires_at: expires_at, client_payload: client_payload - }, otel_ctx} + }} ) assert_push "allow_access", payload @@ -178,17 +233,17 @@ defmodule API.Gateway.ChannelTest do } assert payload.ref - assert payload.flow_id == flow_id assert payload.client_id == client.id assert payload.client_ipv4 == client.ipv4 assert payload.client_ipv6 == client.ipv6 assert DateTime.from_unix!(payload.expires_at) == DateTime.truncate(expires_at, :second) end - test "subscribes for flow expiration event", %{ + test "handles flow deletion event", %{ account: account, client: client, resource: resource, + gateway: gateway, relay: relay, socket: socket, subject: subject @@ -196,7 +251,6 @@ defmodule API.Gateway.ChannelTest do channel_pid = self() socket_ref = make_ref() expires_at = DateTime.utc_now() |> DateTime.add(30, :second) - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} client_payload = "RTC_SD_or_DNS_Q" stamp_secret = Ecto.UUID.generate() @@ -210,80 +264,241 @@ defmodule API.Gateway.ChannelTest do resource: resource ) + data = %{ + "id" => flow.id, + "client_id" => client.id, + "resource_id" => resource.id, + "account_id" => account.id + } + send( socket.channel_pid, - {:allow_access, {channel_pid, socket_ref}, + {{:allow_access, gateway.id}, {channel_pid, socket_ref}, %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow.id, + client: client, + resource: resource, authorization_expires_at: expires_at, client_payload: client_payload - }, otel_ctx} + }} ) assert_push "allow_access", %{} - assert :ok = Domain.Flows.expire_flows_for(resource, subject) - - send(socket.channel_pid, {:expire_flow, flow.id, client.id, resource.id}) + Events.Hooks.Flows.on_delete(data) assert_push "reject_access", %{ - flow_id: flow_id, client_id: client_id, resource_id: resource_id } - assert flow_id == flow.id assert client_id == client.id assert resource_id == resource.id end - test "subscribes for resource events", %{ - account: account, - client: client, - resource: resource, - relay: relay, - socket: socket, - subject: subject - } do + test "ignores flow deletion for other flows", + %{ + account: account, + client: client, + resource: resource, + gateway: gateway, + relay: relay, + socket: socket, + subject: subject + } do channel_pid = self() socket_ref = make_ref() expires_at = DateTime.utc_now() |> DateTime.add(30, :second) - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} client_payload = "RTC_SD_or_DNS_Q" stamp_secret = Ecto.UUID.generate() :ok = Domain.Relays.connect_relay(relay, stamp_secret) - flow = + other_client = Fixtures.Clients.create_client(account: account, subject: subject) + + other_resource = + Fixtures.Resources.create_resource( + account: account, + connections: [%{gateway_group_id: gateway.group_id}] + ) + + other_flow1 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: other_client, + resource: resource + ) + + other_flow2 = Fixtures.Flows.create_flow( account: account, subject: subject, client: client, - resource: resource + resource: other_resource ) + # Build up flow cache send( socket.channel_pid, - {:allow_access, {channel_pid, socket_ref}, + {{:allow_access, gateway.id}, {channel_pid, socket_ref}, %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow.id, + client: client, + resource: resource, authorization_expires_at: expires_at, client_payload: client_payload - }, otel_ctx} + }} ) assert_push "allow_access", %{} - {:ok, resource} = - Domain.Resources.update_resource( - resource, - %{"name" => Ecto.UUID.generate()}, - subject - ) + send( + socket.channel_pid, + {{:allow_access, gateway.id}, {channel_pid, socket_ref}, + %{ + client: other_client, + resource: resource, + authorization_expires_at: expires_at, + client_payload: client_payload + }} + ) + + assert_push "allow_access", %{} + + send( + socket.channel_pid, + {{:allow_access, gateway.id}, {channel_pid, socket_ref}, + %{ + client: client, + resource: other_resource, + authorization_expires_at: expires_at, + client_payload: client_payload + }} + ) + + assert_push "allow_access", %{} + + assert %{assigns: %{flows: flows}} = + :sys.get_state(socket.channel_pid) + + assert flows == %{ + {client.id, resource.id} => expires_at, + {other_client.id, resource.id} => expires_at, + {client.id, other_resource.id} => expires_at + } + + data = %{ + "id" => other_flow1.id, + "client_id" => other_flow1.client_id, + "resource_id" => other_flow1.resource_id, + "account_id" => other_flow1.account_id + } + + Events.Hooks.Flows.on_delete(data) + + assert_push "reject_access", %{ + client_id: client_id, + resource_id: resource_id + } + + assert client_id == other_client.id + assert resource_id == resource.id + + data = %{ + "id" => other_flow2.id, + "client_id" => other_flow2.client_id, + "resource_id" => other_flow2.resource_id, + "account_id" => other_flow2.account_id + } + + Events.Hooks.Flows.on_delete(data) + + assert_push "reject_access", %{ + client_id: client_id, + resource_id: resource_id + } + + assert client_id == client.id + assert resource_id == other_resource.id + + refute_push "reject_access", _payload + end + + test "ignores other resource updates", %{ + client: client, + gateway: gateway, + resource: resource, + relay: relay, + socket: socket + } do + channel_pid = self() + socket_ref = make_ref() + expires_at = DateTime.utc_now() |> DateTime.add(30, :second) + client_payload = "RTC_SD_or_DNS_Q" + stamp_secret = Ecto.UUID.generate() + :ok = Domain.Relays.connect_relay(relay, stamp_secret) + + send( + socket.channel_pid, + {{:allow_access, gateway.id}, {channel_pid, socket_ref}, + %{ + client: client, + resource: resource, + authorization_expires_at: expires_at, + client_payload: client_payload + }} + ) + + assert_push "allow_access", %{} + + old_data = %{ + "id" => resource.id, + "account_id" => resource.account_id, + "name" => resource.name + } + + data = Map.put(old_data, "name", "New Resource Name") + + Events.Hooks.Resources.on_update(old_data, data) + + client_id = client.id + resource_id = resource.id + + assert %{ + assigns: %{ + flows: %{{^client_id, ^resource_id} => ^expires_at} + } + } = :sys.get_state(socket.channel_pid) + + refute_push "resource_updated", _payload + end + + test "sends resource_updated when filters change", %{ + client: client, + gateway: gateway, + resource: resource, + relay: relay, + socket: socket + } do + channel_pid = self() + socket_ref = make_ref() + expires_at = DateTime.utc_now() |> DateTime.add(30, :second) + client_payload = "RTC_SD_or_DNS_Q" + + stamp_secret = Ecto.UUID.generate() + :ok = Domain.Relays.connect_relay(relay, stamp_secret) + + send( + socket.channel_pid, + {{:allow_access, gateway.id}, {channel_pid, socket_ref}, + %{ + client: client, + resource: resource, + authorization_expires_at: expires_at, + client_payload: client_payload + }} + ) + + assert_push "allow_access", %{} old_data = %{ "id" => resource.id, @@ -295,7 +510,14 @@ defmodule API.Gateway.ChannelTest do "ip_stack" => "dual" } - data = Map.put(old_data, "name", "new name") + filters = [ + %{"protocol" => "tcp", "ports" => ["80", "433"]}, + %{"protocol" => "udp", "ports" => ["100-200"]}, + %{"protocol" => "icmp"} + ] + + data = Map.put(old_data, "filters", filters) + Events.Hooks.Resources.on_update(old_data, data) assert_push "resource_updated", payload @@ -435,76 +657,17 @@ defmodule API.Gateway.ChannelTest do }, relays_presence_timeout() + 10 end - end - describe "handle_info/2 :expire_flow" do - test "pushes message to the socket", %{ - client: client, - resource: resource, - socket: socket - } do - flow_id = Ecto.UUID.generate() - send(socket.channel_pid, {:expire_flow, flow_id, client.id, resource.id}) - - assert_push "reject_access", payload - assert payload == %{flow_id: flow_id, client_id: client.id, resource_id: resource.id} - end - end - - describe "handle_info/2 :update_resource" do - test "pushes message to the socket", %{ - resource: resource, - socket: socket - } do - send(socket.channel_pid, {:update_resource, resource.id}) - - assert_push "resource_updated", payload - - assert payload == %{ - address: resource.address, - id: resource.id, - name: resource.name, - type: :dns, - filters: [ - %{protocol: :tcp, port_range_end: 80, port_range_start: 80}, - %{protocol: :tcp, port_range_end: 433, port_range_start: 433}, - %{protocol: :udp, port_range_start: 100, port_range_end: 200}, - %{protocol: :icmp} - ] - } - end - end - - describe "handle_info/2 :create_resource" do - test "does nothing", %{ - resource: resource, - socket: socket - } do - send(socket.channel_pid, {:create_resource, resource.id}) - end - end - - describe "handle_info/2 :delete_resource" do - test "does nothing", %{ - resource: resource, - socket: socket - } do - send(socket.channel_pid, {:delete_resource, resource.id}) - end - end - - describe "handle_info/2 :ice_candidates" do test "pushes ice_candidates message", %{ client: client, + gateway: gateway, socket: socket } do - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} - candidates = ["foo", "bar"] send( socket.channel_pid, - {:ice_candidates, client.id, candidates, otel_ctx} + {{:ice_candidates, gateway.id}, client.id, candidates} ) assert_push "ice_candidates", payload @@ -514,20 +677,17 @@ defmodule API.Gateway.ChannelTest do client_id: client.id } end - end - describe "handle_info/2 :invalidate_ice_candidates" do test "pushes invalidate_ice_candidates message", %{ client: client, + gateway: gateway, socket: socket } do - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} - candidates = ["foo", "bar"] send( socket.channel_pid, - {:invalidate_ice_candidates, client.id, candidates, otel_ctx} + {{:invalidate_ice_candidates, gateway.id}, client.id, candidates} ) assert_push "invalidate_ice_candidates", payload @@ -537,12 +697,11 @@ defmodule API.Gateway.ChannelTest do client_id: client.id } end - end - describe "handle_info/2 :request_connection" do test "pushes request_connection message", %{ client: client, resource: resource, + gateway: gateway, global_relay: relay, socket: socket } do @@ -551,31 +710,25 @@ defmodule API.Gateway.ChannelTest do expires_at = DateTime.utc_now() |> DateTime.add(30, :second) preshared_key = "PSK" client_payload = "RTC_SD" - flow_id = Ecto.UUID.generate() - - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} stamp_secret = Ecto.UUID.generate() :ok = Domain.Relays.connect_relay(relay, stamp_secret) send( socket.channel_pid, - {:request_connection, {channel_pid, socket_ref}, + {{:request_connection, gateway.id}, {channel_pid, socket_ref}, %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow_id, + client: client, + resource: resource, authorization_expires_at: expires_at, client_payload: client_payload, client_preshared_key: preshared_key - }, otel_ctx} + }} ) assert_push "request_connection", payload assert is_binary(payload.ref) - assert payload.flow_id == flow_id - assert payload.actor == %{id: client.actor_id} assert payload.resource == %{ address: resource.address, @@ -606,9 +759,10 @@ defmodule API.Gateway.ChannelTest do DateTime.truncate(expires_at, :second) end - test "subscribes for flow expiration event", %{ + test "request_connection tracks flow and sends reject_access when flow is deleted", %{ account: account, client: client, + gateway: gateway, resource: resource, relay: relay, socket: socket, @@ -617,7 +771,6 @@ defmodule API.Gateway.ChannelTest do channel_pid = self() socket_ref = make_ref() expires_at = DateTime.utc_now() |> DateTime.add(30, :second) - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} client_payload = "RTC_SD_or_DNS_Q" preshared_key = "PSK" @@ -634,115 +787,39 @@ defmodule API.Gateway.ChannelTest do send( socket.channel_pid, - {:request_connection, {channel_pid, socket_ref}, + {{:request_connection, gateway.id}, {channel_pid, socket_ref}, %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow.id, + client: client, + resource: resource, authorization_expires_at: expires_at, client_payload: client_payload, client_preshared_key: preshared_key - }, otel_ctx} + }} ) assert_push "request_connection", %{} - assert :ok = Domain.Flows.expire_flows_for(resource, subject) + data = %{ + "id" => flow.id, + "client_id" => client.id, + "resource_id" => resource.id, + "account_id" => account.id + } - send(socket.channel_pid, {:expire_flow, flow.id, client.id, resource.id}) + Events.Hooks.Flows.on_delete(data) assert_push "reject_access", %{ - flow_id: flow_id, client_id: client_id, resource_id: resource_id } - assert flow_id == flow.id assert client_id == client.id assert resource_id == resource.id end - test "subscribes for resource events", %{ - account: account, - client: client, - resource: resource, - relay: relay, - socket: socket, - subject: subject - } do - channel_pid = self() - socket_ref = make_ref() - expires_at = DateTime.utc_now() |> DateTime.add(30, :second) - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} - client_payload = "RTC_SD_or_DNS_Q" - preshared_key = "PSK" - - stamp_secret = Ecto.UUID.generate() - :ok = Domain.Relays.connect_relay(relay, stamp_secret) - - flow = - Fixtures.Flows.create_flow( - account: account, - subject: subject, - client: client, - resource: resource - ) - - send( - socket.channel_pid, - {:request_connection, {channel_pid, socket_ref}, - %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow.id, - authorization_expires_at: expires_at, - client_payload: client_payload, - client_preshared_key: preshared_key - }, otel_ctx} - ) - - assert_push "request_connection", %{}, 200 - - {:ok, resource} = - Domain.Resources.update_resource( - resource, - %{"name" => Ecto.UUID.generate()}, - subject - ) - - old_data = %{ - "id" => resource.id, - "account_id" => resource.account_id, - "address" => resource.address, - "name" => resource.name, - "type" => "dns", - "filters" => [], - "ip_stack" => "dual" - } - - data = Map.put(old_data, "name", "new name") - Events.Hooks.Resources.on_update(old_data, data) - - assert_push "resource_updated", payload, 200 - - assert payload == %{ - address: resource.address, - id: resource.id, - name: resource.name, - type: :dns, - filters: [ - %{protocol: :tcp, port_range_end: 80, port_range_start: 80}, - %{protocol: :tcp, port_range_end: 433, port_range_start: 433}, - %{protocol: :udp, port_range_start: 100, port_range_end: 200}, - %{protocol: :icmp} - ] - } - end - end - - describe "handle_info/2 :authorize_flow" do test "pushes authorize_flow message", %{ client: client, + gateway: gateway, resource: resource, socket: socket } do @@ -750,33 +827,27 @@ defmodule API.Gateway.ChannelTest do socket_ref = make_ref() expires_at = DateTime.utc_now() |> DateTime.add(30, :second) preshared_key = "PSK" - flow_id = Ecto.UUID.generate() ice_credentials = %{ client: %{username: "A", password: "B"}, gateway: %{username: "C", password: "D"} } - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} - send( socket.channel_pid, - {:authorize_flow, {channel_pid, socket_ref}, + {{:authorize_flow, gateway.id}, {channel_pid, socket_ref}, %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow_id, + client: client, + resource: resource, authorization_expires_at: expires_at, ice_credentials: ice_credentials, preshared_key: preshared_key - }, otel_ctx} + }} ) assert_push "authorize_flow", payload assert is_binary(payload.ref) - assert payload.flow_id == flow_id - assert payload.actor == %{id: client.actor_id} assert payload.resource == %{ address: resource.address, @@ -808,13 +879,13 @@ defmodule API.Gateway.ChannelTest do test "pushes authorize_flow message for authorizations that do not expire", %{ client: client, + gateway: gateway, resource: resource, socket: socket } do channel_pid = self() socket_ref = make_ref() preshared_key = "PSK" - flow_id = Ecto.UUID.generate() ice_credentials = %{ client: %{username: "A", password: "B"}, @@ -823,23 +894,23 @@ defmodule API.Gateway.ChannelTest do send( socket.channel_pid, - {:authorize_flow, {channel_pid, socket_ref}, + {{:authorize_flow, gateway.id}, {channel_pid, socket_ref}, %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow_id, + client: client, + resource: resource, authorization_expires_at: nil, ice_credentials: ice_credentials, preshared_key: preshared_key - }, {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")}} + }} ) assert_push "authorize_flow", %{expires_at: nil} end - test "subscribes for flow expiration event", %{ + test "authorize_flow tracks flow and sends reject_access when flow is deleted", %{ account: account, client: client, + gateway: gateway, resource: resource, socket: socket, subject: subject @@ -847,7 +918,6 @@ defmodule API.Gateway.ChannelTest do channel_pid = self() socket_ref = make_ref() expires_at = DateTime.utc_now() |> DateTime.add(30, :second) - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} preshared_key = "PSK" ice_credentials = %{ @@ -865,118 +935,44 @@ defmodule API.Gateway.ChannelTest do send( socket.channel_pid, - {:authorize_flow, {channel_pid, socket_ref}, + {{:authorize_flow, gateway.id}, {channel_pid, socket_ref}, %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow.id, + client: client, + resource: resource, authorization_expires_at: expires_at, ice_credentials: ice_credentials, preshared_key: preshared_key - }, otel_ctx} + }} ) assert_push "authorize_flow", %{} - assert :ok = Domain.Flows.expire_flows_for(resource, subject) + data = %{ + "id" => flow.id, + "client_id" => client.id, + "resource_id" => resource.id, + "account_id" => account.id + } - send(socket.channel_pid, {:expire_flow, flow.id, client.id, resource.id}) + Events.Hooks.Flows.on_delete(data) assert_push "reject_access", %{ - flow_id: flow_id, client_id: client_id, resource_id: resource_id } - assert flow_id == flow.id assert client_id == client.id assert resource_id == resource.id end - - test "subscribes for resource events", %{ - account: account, - client: client, - resource: resource, - relay: relay, - socket: socket, - subject: subject - } do - channel_pid = self() - socket_ref = make_ref() - expires_at = DateTime.utc_now() |> DateTime.add(30, :second) - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} - preshared_key = "PSK" - - ice_credentials = %{ - client: %{username: "A", password: "B"}, - gateway: %{username: "C", password: "D"} - } - - stamp_secret = Ecto.UUID.generate() - :ok = Domain.Relays.connect_relay(relay, stamp_secret) - - flow = - Fixtures.Flows.create_flow( - account: account, - subject: subject, - client: client, - resource: resource - ) - - send( - socket.channel_pid, - {:authorize_flow, {channel_pid, socket_ref}, - %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow.id, - authorization_expires_at: expires_at, - ice_credentials: ice_credentials, - preshared_key: preshared_key - }, otel_ctx} - ) - - assert_push "authorize_flow", %{} - - {:ok, resource} = - Domain.Resources.update_resource( - resource, - %{"name" => Ecto.UUID.generate()}, - subject - ) - - old_data = %{ - "id" => resource.id, - "account_id" => resource.account_id, - "address" => resource.address, - "name" => resource.name, - "type" => "dns", - "filters" => [], - "ip_stack" => "dual" - } - - data = Map.put(old_data, "name", "new name") - Events.Hooks.Resources.on_update(old_data, data) - - assert_push "resource_updated", payload - - assert payload == %{ - address: resource.address, - id: resource.id, - name: resource.name, - type: :dns, - filters: [ - %{protocol: :tcp, port_range_end: 80, port_range_start: 80}, - %{protocol: :tcp, port_range_end: 433, port_range_start: 433}, - %{protocol: :udp, port_range_start: 100, port_range_end: 200}, - %{protocol: :icmp} - ] - } - end end - describe "handle_in/3 flow_authorized" do - test "forwards reply to the client channel", %{ + describe "handle_in/3" do + test "for unknown messages it doesn't crash", %{socket: socket} do + ref = push(socket, "unknown_message", %{}) + assert_reply ref, :error, %{reason: :unknown_message} + end + + test "flow_authorized forwards reply to the client channel", %{ client: client, resource: resource, gateway: gateway, @@ -984,7 +980,6 @@ defmodule API.Gateway.ChannelTest do } do channel_pid = self() socket_ref = make_ref() - flow_id = Ecto.UUID.generate() expires_at = DateTime.utc_now() |> DateTime.add(30, :second) preshared_key = "PSK" gateway_group_id = gateway.group_id @@ -999,19 +994,16 @@ defmodule API.Gateway.ChannelTest do gateway: %{username: "C", password: "D"} } - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("authorize_flow")} - send( socket.channel_pid, - {:authorize_flow, {channel_pid, socket_ref}, + {{:authorize_flow, gateway.id}, {channel_pid, socket_ref}, %{ - client_id: client.id, - resource_id: resource.id, - flow_id: flow_id, + client: client, + resource: resource, authorization_expires_at: expires_at, ice_credentials: ice_credentials, preshared_key: preshared_key - }, otel_ctx} + }} ) assert_push "authorize_flow", %{ref: ref} @@ -1029,14 +1021,11 @@ defmodule API.Gateway.ChannelTest do ^gateway_ipv4, ^gateway_ipv6, ^preshared_key, - ^ice_credentials, - {_opentelemetry_ctx, opentelemetry_span_ctx} + ^ice_credentials } - - assert elem(opentelemetry_span_ctx, 1) == otel_ctx |> elem(1) |> elem(1) end - test "pushes an error when ref is invalid", %{ + test "flow_authorized pushes an error when ref is invalid", %{ socket: socket } do push_ref = @@ -1046,10 +1035,8 @@ defmodule API.Gateway.ChannelTest do assert_reply push_ref, :error, %{reason: :invalid_ref} end - end - describe "handle_in/3 connection_ready" do - test "forwards RFC session description to the client channel", %{ + test "connection ready forwards RFC session description to the client channel", %{ client: client, resource: resource, relay: relay, @@ -1062,27 +1049,41 @@ defmodule API.Gateway.ChannelTest do preshared_key = "PSK" gateway_public_key = gateway.public_key payload = "RTC_SD" - flow_id = Ecto.UUID.generate() - - otel_ctx = {OpenTelemetry.Ctx.new(), OpenTelemetry.Tracer.start_span("connect")} stamp_secret = Ecto.UUID.generate() :ok = Domain.Relays.connect_relay(relay, stamp_secret) send( socket.channel_pid, - {:request_connection, {channel_pid, socket_ref}, + {{:request_connection, gateway.id}, {channel_pid, socket_ref}, %{ - client_id: client.id, - resource_id: resource.id, + client: client, + resource: resource, authorization_expires_at: expires_at, - flow_id: flow_id, client_payload: payload, client_preshared_key: preshared_key - }, otel_ctx} + }} ) - assert_push "request_connection", %{ref: ref, flow_id: ^flow_id} + assert_push "request_connection", %{ + ref: ref, + client: %{ + peer: peer, + id: client_id + }, + resource: re, + expires_at: ex + } + + assert is_binary(ref) + assert client_id == client.id + assert peer.ipv4 == client.ipv4 + assert peer.ipv6 == client.ipv6 + assert peer.public_key == client.public_key + assert peer.persistent_keepalive == 25 + assert peer.preshared_key == preshared_key + assert re.id == resource.id + assert DateTime.from_unix!(ex) == DateTime.truncate(expires_at, :second) push_ref = push(socket, "connection_ready", %{ @@ -1091,14 +1092,11 @@ defmodule API.Gateway.ChannelTest do }) assert_reply push_ref, :ok - - assert_receive {:connect, ^socket_ref, resource_id, ^gateway_public_key, ^payload, - _opentelemetry_ctx} - + assert_receive {:connect, ^socket_ref, resource_id, ^gateway_public_key, ^payload} assert resource_id == resource.id end - test "pushes an error when ref is invalid", %{ + test "connection_ready pushes an error when ref is invalid", %{ socket: socket } do push_ref = @@ -1109,24 +1107,25 @@ defmodule API.Gateway.ChannelTest do assert_reply push_ref, :error, %{reason: :invalid_ref} end - end - describe "handle_in/3 broadcast_ice_candidates" do - test "does nothing when gateways list is empty", %{ - socket: socket + test "broadcast ice candidates does nothing when gateways list is empty", %{ + socket: socket, + account: account } do candidates = ["foo", "bar"] + :ok = Domain.PubSub.Account.subscribe(account.id) + attrs = %{ "candidates" => candidates, "client_ids" => [] } push(socket, "broadcast_ice_candidates", attrs) - refute_receive {:ice_candidates, _client_id, _candidates, _opentelemetry_ctx} + refute_receive {:ice_candidates, _client_id, _candidates} end - test "broadcasts :ice_candidates message to all gateways", %{ + test "broadcasts :ice_candidates message to the target gateway", %{ client: client, gateway: gateway, subject: subject, @@ -1141,27 +1140,32 @@ defmodule API.Gateway.ChannelTest do :ok = Domain.Clients.Presence.connect(client) PubSub.subscribe(Domain.Tokens.socket_id(subject.token_id)) + :ok = Domain.PubSub.Account.subscribe(gateway.account_id) push(socket, "broadcast_ice_candidates", attrs) - assert_receive {:ice_candidates, gateway_id, ^candidates, _opentelemetry_ctx}, 200 + assert_receive {{:ice_candidates, client_id}, gateway_id, ^candidates}, + 200 + + assert client_id == client.id assert gateway.id == gateway_id end - end - describe "handle_in/3 broadcast_invalidated_ice_candidates" do - test "does nothing when gateways list is empty", %{ - socket: socket + test "broadcast_invalidated_ice_candidates does nothing when gateways list is empty", %{ + socket: socket, + account: account } do candidates = ["foo", "bar"] + :ok = Domain.PubSub.Account.subscribe(account.id) + attrs = %{ "candidates" => candidates, "client_ids" => [] } push(socket, "broadcast_invalidated_ice_candidates", attrs) - refute_receive {:invalidate_ice_candidates, _client_id, _candidates, _opentelemetry_ctx} + refute_receive {{:invalidate_ice_candidates, _client_id}, _gateway_id, _candidates} end test "broadcasts :invalidate_ice_candidates message to all gateways", %{ @@ -1177,21 +1181,23 @@ defmodule API.Gateway.ChannelTest do "client_ids" => [client.id] } + :ok = Domain.PubSub.Account.subscribe(gateway.account_id) :ok = Domain.Clients.Presence.connect(client) PubSub.subscribe(Domain.Tokens.socket_id(subject.token_id)) push(socket, "broadcast_invalidated_ice_candidates", attrs) - assert_receive {:invalidate_ice_candidates, gateway_id, ^candidates, _opentelemetry_ctx}, + assert_receive {{:invalidate_ice_candidates, client_id}, gateway_id, ^candidates}, 200 + assert client_id == client.id assert gateway.id == gateway_id end end # Debouncer tests - describe "handle_info/3 :push_leave" do - test "cancels leave if reconnecting with the same stamp secret" do + describe "handle_info/3" do + test "push_leave cancels leave if reconnecting with the same stamp secret" do relay_group = Fixtures.Relays.create_global_group() relay1 = Fixtures.Relays.create_relay(group: relay_group) @@ -1300,11 +1306,4 @@ defmodule API.Gateway.ChannelTest do defp relays_presence_timeout do Application.fetch_env!(:api, :relays_presence_debounce_timeout_ms) end - - describe "handle_in/3 for unknown messages" do - test "it doesn't crash", %{socket: socket} do - ref = push(socket, "unknown_message", %{}) - assert_reply ref, :error, %{reason: :unknown_message} - end - end end diff --git a/elixir/apps/domain/lib/domain/actors.ex b/elixir/apps/domain/lib/domain/actors.ex index c70cde538..0acfb21ee 100644 --- a/elixir/apps/domain/lib/domain/actors.ex +++ b/elixir/apps/domain/lib/domain/actors.ex @@ -38,6 +38,13 @@ defmodule Domain.Actors do end end + def fetch_membership_by_actor_id_and_group_id(actor_id, group_id) do + Membership.Query.all() + |> Membership.Query.by_actor_id(actor_id) + |> Membership.Query.by_group_id(group_id) + |> Repo.fetch(Membership.Query, []) + end + def list_groups(%Auth.Subject{} = subject, opts \\ []) do with :ok <- Auth.ensure_has_permissions(subject, Authorizer.manage_actors_permission()) do Group.Query.not_deleted() diff --git a/elixir/apps/domain/lib/domain/actors/membership.ex b/elixir/apps/domain/lib/domain/actors/membership.ex index 6832df8a7..a9297a3cb 100644 --- a/elixir/apps/domain/lib/domain/actors/membership.ex +++ b/elixir/apps/domain/lib/domain/actors/membership.ex @@ -1,10 +1,9 @@ defmodule Domain.Actors.Membership do use Domain, :schema - @primary_key false schema "actor_group_memberships" do - belongs_to :group, Domain.Actors.Group, primary_key: true - belongs_to :actor, Domain.Actors.Actor, primary_key: true + belongs_to :group, Domain.Actors.Group + belongs_to :actor, Domain.Actors.Actor belongs_to :account, Domain.Accounts.Account end diff --git a/elixir/apps/domain/lib/domain/clients.ex b/elixir/apps/domain/lib/domain/clients.ex index f166d4620..e9af27897 100644 --- a/elixir/apps/domain/lib/domain/clients.ex +++ b/elixir/apps/domain/lib/domain/clients.ex @@ -1,7 +1,7 @@ defmodule Domain.Clients do use Supervisor alias Domain.{Repo, Auth} - alias Domain.{Accounts, Actors, Flows} + alias Domain.{Accounts, Actors} alias Domain.Clients.{Client, Authorizer, Presence} require Ecto.Query @@ -211,16 +211,6 @@ defmodule Domain.Clients do with: &Client.Changeset.remove_verification(&1), preload: [:online?] ) - |> case do - # TODO: WAL - # Broadcast flow side effects directly - {:ok, client} -> - :ok = Flows.expire_flows_for(client) - {:ok, client} - - {:error, reason} -> - {:error, reason} - end end end @@ -252,6 +242,10 @@ defmodule Domain.Clients do end end + # TODO: Hard delete + # We don't necessarily want to delete associated tokens when deleting a client because + # that token could be a multi-owner token in the case of a headless client. + # Instead we need to introduce the concept of ephemeral clients/gateways and permanent ones. defp delete_clients(queryable, subject) do {_count, clients} = queryable diff --git a/elixir/apps/domain/lib/domain/clients/presence.ex b/elixir/apps/domain/lib/domain/clients/presence.ex index 4ff587178..63554516a 100644 --- a/elixir/apps/domain/lib/domain/clients/presence.ex +++ b/elixir/apps/domain/lib/domain/clients/presence.ex @@ -9,8 +9,7 @@ defmodule Domain.Clients.Presence do def connect(%Client{} = client) do with {:ok, _} <- __MODULE__.Account.track(client.account_id, client.id), {:ok, _} <- __MODULE__.Actor.track(client.actor_id, client.id) do - :ok = PubSub.Client.subscribe(client.id) - :ok = PubSub.Account.Clients.subscribe(client.account_id) + :ok end end diff --git a/elixir/apps/domain/lib/domain/config/definitions.ex b/elixir/apps/domain/lib/domain/config/definitions.ex index 218346d03..4de46c371 100644 --- a/elixir/apps/domain/lib/domain/config/definitions.ex +++ b/elixir/apps/domain/lib/domain/config/definitions.ex @@ -687,6 +687,11 @@ defmodule Domain.Config.Definitions do """ defconfig(:feature_idp_sync_enabled, :boolean, default: true) + @doc """ + Boolean flag to turn UI flow activities on/off for all accounts. + """ + defconfig(:feature_flow_activities_enabled, :boolean, default: false) + @doc """ Boolean flag to turn Account relays admin functionality on/off for all accounts. """ diff --git a/elixir/apps/domain/lib/domain/events/hooks.ex b/elixir/apps/domain/lib/domain/events/hooks.ex index 7ee419653..f6b1eaae5 100644 --- a/elixir/apps/domain/lib/domain/events/hooks.ex +++ b/elixir/apps/domain/lib/domain/events/hooks.ex @@ -3,7 +3,7 @@ defmodule Domain.Events.Hooks do A simple behavior to define hooks needed for processing WAL events. """ - @callback on_insert(data :: map()) :: :ok - @callback on_update(old_data :: map(), data :: map()) :: :ok - @callback on_delete(old_data :: map()) :: :ok + @callback on_insert(data :: map()) :: :ok | {:error, term()} + @callback on_update(old_data :: map(), data :: map()) :: :ok | {:error, term()} + @callback on_delete(old_data :: map()) :: :ok | {:error, term()} end diff --git a/elixir/apps/domain/lib/domain/events/hooks/accounts.ex b/elixir/apps/domain/lib/domain/events/hooks/accounts.ex index 029b1f461..feba1c03f 100644 --- a/elixir/apps/domain/lib/domain/events/hooks/accounts.ex +++ b/elixir/apps/domain/lib/domain/events/hooks/accounts.ex @@ -1,6 +1,6 @@ defmodule Domain.Events.Hooks.Accounts do @behaviour Domain.Events.Hooks - alias Domain.PubSub + alias Domain.{Accounts, PubSub, SchemaHelpers} require Logger @impl true @@ -9,34 +9,40 @@ defmodule Domain.Events.Hooks.Accounts do # Account slug changed - disconnect gateways for updated init @impl true - def on_update(%{"slug" => old_slug}, %{"slug" => slug, "id" => account_id} = _data) - when old_slug != slug do - # Technically we could push a :slug_changed message to the Gateways here, - # but at the time of writing, disconnecting and reconnecting is safer to ensure - # all relevant state on the Gateway is updated correctly. - PubSub.Account.Gateways.disconnect(account_id) - end - # Account disabled - disconnect clients - @impl true + # Account disabled - process as a delete def on_update( - %{"disabled_at" => nil} = _old_data, - %{"disabled_at" => disabled_at, "id" => account_id} = _data + %{"disabled_at" => nil} = old_data, + %{"disabled_at" => disabled_at} ) when not is_nil(disabled_at) do - PubSub.Account.Clients.disconnect(account_id) + on_delete(old_data) end - def on_update(%{"config" => old_config}, %{"config" => config, "id" => account_id}) do - if old_config != config do - PubSub.Account.broadcast(account_id, :config_changed) - else - :ok - end + # Account soft-deleted - process as a delete + def on_update( + %{"deleted_at" => nil} = old_data, + %{"deleted_at" => deleted_at} + ) + when not is_nil(deleted_at) do + on_delete(old_data) + end + + def on_update(old_data, data) do + old_account = SchemaHelpers.struct_from_params(Accounts.Account, old_data) + account = SchemaHelpers.struct_from_params(Accounts.Account, data) + PubSub.Account.broadcast(account.id, {:updated, old_account, account}) end @impl true - def on_delete(_old_data) do - :ok + + def on_delete(old_data) do + account = SchemaHelpers.struct_from_params(Accounts.Account, old_data) + + # TODO: Hard delete + # This can be removed upon implementation of hard delete + Domain.Flows.delete_flows_for(account) + + PubSub.Account.broadcast(account.id, {:deleted, account}) end end diff --git a/elixir/apps/domain/lib/domain/events/hooks/actor_group_memberships.ex b/elixir/apps/domain/lib/domain/events/hooks/actor_group_memberships.ex index 80ff29d56..3143900ac 100644 --- a/elixir/apps/domain/lib/domain/events/hooks/actor_group_memberships.ex +++ b/elixir/apps/domain/lib/domain/events/hooks/actor_group_memberships.ex @@ -1,43 +1,24 @@ defmodule Domain.Events.Hooks.ActorGroupMemberships do @behaviour Domain.Events.Hooks - alias Domain.{Flows, Policies, PubSub, Repo} + alias Domain.{Actors, SchemaHelpers, PubSub} @impl true - def on_insert(%{"actor_id" => actor_id, "group_id" => group_id} = _data) do - Task.start(fn -> - :ok = PubSub.Actor.Memberships.broadcast(actor_id, {:create_membership, actor_id, group_id}) - broadcast_access(:allow, actor_id, group_id) - end) - - :ok + def on_insert(data) do + membership = SchemaHelpers.struct_from_params(Actors.Membership, data) + PubSub.Account.broadcast(membership.account_id, {:created, membership}) end @impl true def on_update(_old_data, _data), do: :ok @impl true - def on_delete( - %{"account_id" => account_id, "actor_id" => actor_id, "group_id" => group_id} = _old_data - ) do - Task.start(fn -> - :ok = PubSub.Actor.Memberships.broadcast(actor_id, {:delete_membership, actor_id, group_id}) - broadcast_access(:reject, actor_id, group_id) + def on_delete(old_data) do + membership = SchemaHelpers.struct_from_params(Actors.Membership, old_data) - # TODO: WAL - # Broadcast flow side effects directly - :ok = Flows.expire_flows_for(account_id, actor_id, group_id) - end) + # TODO: Hard delete + # This can be removed upon implementation of hard delete + Domain.Flows.delete_flows_for(membership) - :ok - end - - defp broadcast_access(action, actor_id, group_id) do - Policies.Policy.Query.not_deleted() - |> Policies.Policy.Query.by_actor_group_id(group_id) - |> Repo.all(checkout_timeout: 30_000) - |> Enum.each(fn policy -> - payload = {:"#{action}_access", policy.id, policy.actor_group_id, policy.resource_id} - :ok = PubSub.Actor.Policies.broadcast(actor_id, payload) - end) + PubSub.Account.broadcast(membership.account_id, {:deleted, membership}) end end diff --git a/elixir/apps/domain/lib/domain/events/hooks/actor_groups.ex b/elixir/apps/domain/lib/domain/events/hooks/actor_groups.ex deleted file mode 100644 index ebf195081..000000000 --- a/elixir/apps/domain/lib/domain/events/hooks/actor_groups.ex +++ /dev/null @@ -1,12 +0,0 @@ -defmodule Domain.Events.Hooks.ActorGroups do - @behaviour Domain.Events.Hooks - - @impl true - def on_insert(_data), do: :ok - - @impl true - def on_update(_old_data, _data), do: :ok - - @impl true - def on_delete(_old_data), do: :ok -end diff --git a/elixir/apps/domain/lib/domain/events/hooks/actors.ex b/elixir/apps/domain/lib/domain/events/hooks/actors.ex deleted file mode 100644 index 325a09da2..000000000 --- a/elixir/apps/domain/lib/domain/events/hooks/actors.ex +++ /dev/null @@ -1,12 +0,0 @@ -defmodule Domain.Events.Hooks.Actors do - @behaviour Domain.Events.Hooks - - @impl true - def on_insert(_data), do: :ok - - @impl true - def on_update(_old_data, _data), do: :ok - - @impl true - def on_delete(_old_data), do: :ok -end diff --git a/elixir/apps/domain/lib/domain/events/hooks/auth_identities.ex b/elixir/apps/domain/lib/domain/events/hooks/auth_identities.ex deleted file mode 100644 index 8dd1cd6b8..000000000 --- a/elixir/apps/domain/lib/domain/events/hooks/auth_identities.ex +++ /dev/null @@ -1,12 +0,0 @@ -defmodule Domain.Events.Hooks.AuthIdentities do - @behaviour Domain.Events.Hooks - - @impl true - def on_insert(_data), do: :ok - - @impl true - def on_update(_old_data, _data), do: :ok - - @impl true - def on_delete(_old_data), do: :ok -end diff --git a/elixir/apps/domain/lib/domain/events/hooks/auth_providers.ex b/elixir/apps/domain/lib/domain/events/hooks/auth_providers.ex deleted file mode 100644 index 756b46aca..000000000 --- a/elixir/apps/domain/lib/domain/events/hooks/auth_providers.ex +++ /dev/null @@ -1,12 +0,0 @@ -defmodule Domain.Events.Hooks.AuthProviders do - @behaviour Domain.Events.Hooks - - @impl true - def on_insert(_data), do: :ok - - @impl true - def on_update(_old_data, _data), do: :ok - - @impl true - def on_delete(_old_data), do: :ok -end diff --git a/elixir/apps/domain/lib/domain/events/hooks/clients.ex b/elixir/apps/domain/lib/domain/events/hooks/clients.ex index b16c3a3e6..23e5c8fc4 100644 --- a/elixir/apps/domain/lib/domain/events/hooks/clients.ex +++ b/elixir/apps/domain/lib/domain/events/hooks/clients.ex @@ -1,25 +1,41 @@ defmodule Domain.Events.Hooks.Clients do @behaviour Domain.Events.Hooks - alias Domain.{Clients, PubSub, SchemaHelpers} + alias Domain.{Clients, SchemaHelpers, PubSub} @impl true def on_insert(_data), do: :ok - # Soft-delete @impl true - def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at} = _data) + + # Soft-delete + def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at}) when not is_nil(deleted_at) do on_delete(old_data) end # Regular update - def on_update(_old_data, data) do + def on_update(old_data, data) do + old_client = SchemaHelpers.struct_from_params(Clients.Client, old_data) client = SchemaHelpers.struct_from_params(Clients.Client, data) - PubSub.Client.broadcast(client.id, {:updated, client}) + + # Unverifying a client + # This is a special case - we need to delete associated flows when unverifying a client since + # it could affect connectivity if any policies are based on the verified status. + if not is_nil(old_client.verified_at) and is_nil(client.verified_at) do + Domain.Flows.delete_flows_for(client) + end + + PubSub.Account.broadcast(client.account_id, {:updated, old_client, client}) end @impl true - def on_delete(%{"id" => client_id} = _old_data) do - PubSub.Client.disconnect(client_id) + def on_delete(old_data) do + client = SchemaHelpers.struct_from_params(Clients.Client, old_data) + + # TODO: Hard delete + # This can be removed upon implementation of hard delete + Domain.Flows.delete_flows_for(client) + + PubSub.Account.broadcast(client.account_id, {:deleted, client}) end end diff --git a/elixir/apps/domain/lib/domain/events/hooks/flows.ex b/elixir/apps/domain/lib/domain/events/hooks/flows.ex new file mode 100644 index 000000000..98a295904 --- /dev/null +++ b/elixir/apps/domain/lib/domain/events/hooks/flows.ex @@ -0,0 +1,24 @@ +defmodule Domain.Events.Hooks.Flows do + @behaviour Domain.Events.Hooks + alias Domain.{Flows, PubSub, SchemaHelpers} + + @impl true + + # We don't react directly to flow creation events because connection setup + # is latency sensitive and we've already broadcasted the relevant message from + # client pid to gateway pid directly. + def on_insert(_data), do: :ok + + @impl true + + # Flows are never updated + def on_update(_old_data, _data), do: :ok + + @impl true + + # This will trigger reject_access for any subscribed gateways + def on_delete(old_data) do + flow = SchemaHelpers.struct_from_params(Flows.Flow, old_data) + PubSub.Account.broadcast(flow.account_id, {:deleted, flow}) + end +end diff --git a/elixir/apps/domain/lib/domain/events/hooks/gateway_groups.ex b/elixir/apps/domain/lib/domain/events/hooks/gateway_groups.ex index f717f7494..1b3c4f857 100644 --- a/elixir/apps/domain/lib/domain/events/hooks/gateway_groups.ex +++ b/elixir/apps/domain/lib/domain/events/hooks/gateway_groups.ex @@ -1,12 +1,31 @@ defmodule Domain.Events.Hooks.GatewayGroups do @behaviour Domain.Events.Hooks + alias Domain.{Gateways, PubSub, SchemaHelpers} @impl true def on_insert(_data), do: :ok + # Soft-delete @impl true - def on_update(_old_data, _data), do: :ok + def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at}) + when not is_nil(deleted_at) do + on_delete(old_data) + end + + # Regular update + def on_update(old_data, data) do + old_gateway_group = SchemaHelpers.struct_from_params(Gateways.Group, old_data) + gateway_group = SchemaHelpers.struct_from_params(Gateways.Group, data) + + PubSub.Account.broadcast( + gateway_group.account_id, + {:updated, old_gateway_group, gateway_group} + ) + end @impl true + + # Deleting a gateway group will delete the associated resource connection, where + # we handle removing it from the client's resource list. def on_delete(_old_data), do: :ok end diff --git a/elixir/apps/domain/lib/domain/events/hooks/gateways.ex b/elixir/apps/domain/lib/domain/events/hooks/gateways.ex index 6daad5e4c..a73925b0e 100644 --- a/elixir/apps/domain/lib/domain/events/hooks/gateways.ex +++ b/elixir/apps/domain/lib/domain/events/hooks/gateways.ex @@ -1,13 +1,13 @@ defmodule Domain.Events.Hooks.Gateways do @behaviour Domain.Events.Hooks - alias Domain.PubSub + alias Domain.{Gateways, PubSub, SchemaHelpers} @impl true def on_insert(_data), do: :ok # Soft-delete @impl true - def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at} = _data) + def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at}) when not is_nil(deleted_at) do on_delete(old_data) end @@ -16,7 +16,13 @@ defmodule Domain.Events.Hooks.Gateways do def on_update(_old_data, _data), do: :ok @impl true - def on_delete(%{"id" => gateway_id} = _old_data) do - PubSub.Gateway.disconnect(gateway_id) + def on_delete(old_data) do + gateway = SchemaHelpers.struct_from_params(Gateways.Gateway, old_data) + + # TODO: Hard delete + # This can be removed upon implementation of hard delete + Domain.Flows.delete_flows_for(gateway) + + PubSub.Account.broadcast(gateway.account_id, {:deleted, gateway}) end end diff --git a/elixir/apps/domain/lib/domain/events/hooks/policies.ex b/elixir/apps/domain/lib/domain/events/hooks/policies.ex index 0a2cd23ed..1a77125fa 100644 --- a/elixir/apps/domain/lib/domain/events/hooks/policies.ex +++ b/elixir/apps/domain/lib/domain/events/hooks/policies.ex @@ -1,175 +1,60 @@ defmodule Domain.Events.Hooks.Policies do @behaviour Domain.Events.Hooks - alias Domain.{Flows, PubSub} + alias Domain.{Policies, PubSub, SchemaHelpers} require Logger @impl true - def on_insert( - %{ - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id - } = - _data - ) do - # TODO: WAL - # Creating a policy should broadcast directly to subscribed clients/gateways - payload = {:create_policy, policy_id} - :ok = PubSub.Policy.broadcast(policy_id, payload) - :ok = PubSub.Account.Policies.broadcast(account_id, payload) - - payload = {:allow_access, policy_id, actor_group_id, resource_id} - :ok = PubSub.ActorGroup.Policies.broadcast(actor_group_id, payload) + def on_insert(data) do + policy = SchemaHelpers.struct_from_params(Policies.Policy, data) + PubSub.Account.broadcast(policy.account_id, {:created, policy}) end @impl true - # Enable - def on_update( - %{"disabled_at" => disabled_at} = _old_data, - %{ - "disabled_at" => nil, - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id - } = _data - ) - when not is_nil(disabled_at) do - # TODO: WAL - # Enabling a policy should broadcast directly to subscribed clients/gateways - payload = {:enable_policy, policy_id} - :ok = PubSub.Policy.broadcast(policy_id, payload) - :ok = PubSub.Account.Policies.broadcast(account_id, payload) + # Disable - process as delete - payload = {:allow_access, policy_id, actor_group_id, resource_id} - :ok = PubSub.ActorGroup.Policies.broadcast(actor_group_id, payload) + def on_update(%{"disabled_at" => nil}, %{"disabled_at" => disabled_at} = data) + when not is_nil(disabled_at) do + on_delete(data) end - # Disable - def on_update( - %{"disabled_at" => nil} = _old_data, - %{ - "disabled_at" => disabled_at, - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id - } = _data - ) + # Enable - process as insert + + def on_update(%{"disabled_at" => disabled_at}, %{"disabled_at" => nil} = data) when not is_nil(disabled_at) do - Task.start(fn -> - # TODO: WAL - # Disabling a policy should broadcast directly to the subscribed clients/gateways - payload = {:disable_policy, policy_id} - :ok = PubSub.Policy.broadcast(policy_id, payload) - :ok = PubSub.Account.Policies.broadcast(account_id, payload) - - payload = {:reject_access, policy_id, actor_group_id, resource_id} - :ok = PubSub.ActorGroup.Policies.broadcast(actor_group_id, payload) - - # TODO: WAL - # Broadcast flow side effects directly - :ok = Flows.expire_flows_for_policy_id(account_id, policy_id) - end) - - :ok + on_insert(data) end - # Soft-delete - def on_update( - %{ - "deleted_at" => nil - } = old_data, - %{"deleted_at" => deleted_at} = _data - ) + # Soft-delete - process as delete + def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at}) when not is_nil(deleted_at) do on_delete(old_data) end - # Breaking update - delete then create - def on_update( - %{ - "id" => old_policy_id, - "account_id" => old_account_id, - "actor_group_id" => old_actor_group_id, - "resource_id" => old_resource_id, - "conditions" => old_conditions - } = _old_data, - %{ - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id, - "conditions" => conditions - } = data - ) - when old_actor_group_id != actor_group_id or old_resource_id != resource_id or - old_conditions != conditions do - # Only act upon this if the policy is not deleted or disabled - if is_nil(data["deleted_at"]) and is_nil(data["disabled_at"]) do - Task.start(fn -> - # TODO: WAL - # Deleting a policy should broadcast directly to the subscribed clients/gateways - payload = {:delete_policy, old_policy_id} - :ok = PubSub.Policy.broadcast(old_policy_id, payload) - :ok = PubSub.Account.Policies.broadcast(old_account_id, payload) + # Regular update + def on_update(old_data, data) do + old_policy = SchemaHelpers.struct_from_params(Policies.Policy, old_data) + policy = SchemaHelpers.struct_from_params(Policies.Policy, data) - payload = {:reject_access, old_policy_id, old_actor_group_id, old_resource_id} - :ok = PubSub.ActorGroup.Policies.broadcast(old_actor_group_id, payload) - - payload = {:create_policy, policy_id} - :ok = PubSub.Policy.broadcast(policy_id, payload) - :ok = PubSub.Account.Policies.broadcast(account_id, payload) - - payload = {:allow_access, policy_id, actor_group_id, resource_id} - :ok = PubSub.ActorGroup.Policies.broadcast(actor_group_id, payload) - - # TODO: WAL - # Broadcast flow side effects directly - :ok = Flows.expire_flows_for_policy_id(account_id, policy_id) - end) - else - Logger.warning("Breaking update ignored for policy as it is deleted or disabled", - policy_id: policy_id - ) + # Breaking updates + # This is a special case - we need to delete related flows because connectivity has changed + if old_policy.conditions != policy.conditions or + old_policy.actor_group_id != policy.actor_group_id or + old_policy.resource_id != policy.resource_id do + Domain.Flows.delete_flows_for(policy) end - :ok - end - - # Regular update - name, description, etc - def on_update(_old_data, %{"id" => policy_id, "account_id" => account_id} = _data) do - payload = {:update_policy, policy_id} - :ok = PubSub.Policy.broadcast(policy_id, payload) - :ok = PubSub.Account.Policies.broadcast(account_id, payload) + PubSub.Account.broadcast(policy.account_id, {:updated, old_policy, policy}) end @impl true - def on_delete( - %{ - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id - } = _old_data - ) do - Task.start(fn -> - # TODO: WAL - # Deleting a policy should broadcast directly to the subscribed clients/gateways - payload = {:delete_policy, policy_id} - :ok = PubSub.Policy.broadcast(policy_id, payload) - :ok = PubSub.Account.Policies.broadcast(account_id, payload) + def on_delete(old_data) do + policy = SchemaHelpers.struct_from_params(Policies.Policy, old_data) - payload = {:reject_access, policy_id, actor_group_id, resource_id} - :ok = PubSub.ActorGroup.Policies.broadcast(actor_group_id, payload) + # TODO: Hard delete + # This can be removed upon implementation of hard delete + Domain.Flows.delete_flows_for(policy) - # TODO: WAL - # Broadcast flow side effects directly - :ok = Flows.expire_flows_for_policy_id(account_id, policy_id) - end) - - :ok + PubSub.Account.broadcast(policy.account_id, {:deleted, policy}) end end diff --git a/elixir/apps/domain/lib/domain/events/hooks/relay_groups.ex b/elixir/apps/domain/lib/domain/events/hooks/relay_groups.ex deleted file mode 100644 index 2dcda9a8d..000000000 --- a/elixir/apps/domain/lib/domain/events/hooks/relay_groups.ex +++ /dev/null @@ -1,12 +0,0 @@ -defmodule Domain.Events.Hooks.RelayGroups do - @behaviour Domain.Events.Hooks - - @impl true - def on_insert(_data), do: :ok - - @impl true - def on_update(_old_data, _data), do: :ok - - @impl true - def on_delete(_old_data), do: :ok -end diff --git a/elixir/apps/domain/lib/domain/events/hooks/relays.ex b/elixir/apps/domain/lib/domain/events/hooks/relays.ex deleted file mode 100644 index 48209a4fd..000000000 --- a/elixir/apps/domain/lib/domain/events/hooks/relays.ex +++ /dev/null @@ -1,12 +0,0 @@ -defmodule Domain.Events.Hooks.Relays do - @behaviour Domain.Events.Hooks - - @impl true - def on_insert(_data), do: :ok - - @impl true - def on_update(_old_data, _data), do: :ok - - @impl true - def on_delete(_old_data), do: :ok -end diff --git a/elixir/apps/domain/lib/domain/events/hooks/resource_connections.ex b/elixir/apps/domain/lib/domain/events/hooks/resource_connections.ex index 5c762fa09..2e38c3bc6 100644 --- a/elixir/apps/domain/lib/domain/events/hooks/resource_connections.ex +++ b/elixir/apps/domain/lib/domain/events/hooks/resource_connections.ex @@ -1,22 +1,19 @@ defmodule Domain.Events.Hooks.ResourceConnections do @behaviour Domain.Events.Hooks - alias Domain.Flows + alias Domain.{SchemaHelpers, Resources, PubSub} @impl true - def on_insert(_data), do: :ok + def on_insert(data) do + connection = SchemaHelpers.struct_from_params(Resources.Connection, data) + PubSub.Account.broadcast(connection.account_id, {:created, connection}) + end @impl true def on_update(_old_data, _data), do: :ok @impl true - def on_delete(%{"account_id" => account_id, "resource_id" => resource_id} = _old_data) do - # TODO: WAL - # Broadcast flow side effects directly - # This hook is called when resources change sites. - Task.start(fn -> - :ok = Flows.expire_flows_for_resource_id(account_id, resource_id) - end) - - :ok + def on_delete(old_data) do + connection = SchemaHelpers.struct_from_params(Resources.Connection, old_data) + PubSub.Account.broadcast(connection.account_id, {:deleted, connection}) end end diff --git a/elixir/apps/domain/lib/domain/events/hooks/resources.ex b/elixir/apps/domain/lib/domain/events/hooks/resources.ex index 7ca4f678e..7cf2615e8 100644 --- a/elixir/apps/domain/lib/domain/events/hooks/resources.ex +++ b/elixir/apps/domain/lib/domain/events/hooks/resources.ex @@ -1,71 +1,47 @@ defmodule Domain.Events.Hooks.Resources do @behaviour Domain.Events.Hooks - alias Domain.{Flows, PubSub} + alias Domain.{SchemaHelpers, PubSub, Resources} @impl true - def on_insert(%{"id" => resource_id, "account_id" => account_id} = _data) do - payload = {:create_resource, resource_id} - PubSub.Resource.broadcast(resource_id, payload) - PubSub.Account.Resources.broadcast(account_id, payload) + def on_insert(data) do + resource = SchemaHelpers.struct_from_params(Resources.Resource, data) + PubSub.Account.broadcast(resource.account_id, {:created, resource}) end @impl true - # Soft-delete - def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at} = _data) + # Soft-delete - process as delete + def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at}) when not is_nil(deleted_at) do on_delete(old_data) end - # Breaking update - expire flows so that new flows are created - def on_update( - %{ - "type" => old_type, - "address" => old_address, - "filters" => old_filters, - "ip_stack" => old_ip_stack - } = _old_data, - %{ - "type" => type, - "address" => address, - "filters" => filters, - "ip_stack" => ip_stack, - "id" => resource_id, - "account_id" => account_id - } = _data - ) - when old_type != type or - old_address != address or - old_filters != filters or - old_ip_stack != ip_stack do - # TODO: WAL - # Broadcast flow side effects directly - Task.start(fn -> - payload = {:delete_resource, resource_id} - PubSub.Resource.broadcast(resource_id, payload) - PubSub.Account.Resources.broadcast(account_id, payload) + # Regular update + def on_update(old_data, data) do + old_resource = SchemaHelpers.struct_from_params(Resources.Resource, old_data) + resource = SchemaHelpers.struct_from_params(Resources.Resource, data) - payload = {:create_resource, resource_id} - PubSub.Resource.broadcast(resource_id, payload) - PubSub.Account.Resources.broadcast(account_id, payload) + # Breaking updates + # This is a special case - we need to delete related flows because connectivity has changed + # Gateway _does_ handle resource filter changes so we don't need to delete flows + # for those changes + if old_resource.ip_stack != resource.ip_stack or + old_resource.type != resource.type or + old_resource.address != resource.address do + Domain.Flows.delete_flows_for(resource) + end - :ok = Flows.expire_flows_for_resource_id(account_id, resource_id) - end) - - :ok - end - - # Non-breaking update - for non-addressability changes - e.g. name, description, etc. - def on_update(_old_data, %{"id" => resource_id, "account_id" => account_id} = _data) do - payload = {:update_resource, resource_id} - PubSub.Resource.broadcast(resource_id, payload) - PubSub.Account.Resources.broadcast(account_id, payload) + PubSub.Account.broadcast(resource.account_id, {:updated, old_resource, resource}) end @impl true - def on_delete(%{"id" => resource_id, "account_id" => account_id} = _old_data) do - payload = {:delete_resource, resource_id} - PubSub.Resource.broadcast(resource_id, payload) - PubSub.Account.Resources.broadcast(account_id, payload) + def on_delete(old_data) do + resource = SchemaHelpers.struct_from_params(Resources.Resource, old_data) + + # TODO: Hard delete + # This can be removed upon implementation of hard delete + Domain.Flows.delete_flows_for(resource) + + PubSub.Account.broadcast(resource.account_id, {:deleted, resource}) end end diff --git a/elixir/apps/domain/lib/domain/events/hooks/tokens.ex b/elixir/apps/domain/lib/domain/events/hooks/tokens.ex index 1b03761ab..e5603a29d 100644 --- a/elixir/apps/domain/lib/domain/events/hooks/tokens.ex +++ b/elixir/apps/domain/lib/domain/events/hooks/tokens.ex @@ -1,6 +1,6 @@ defmodule Domain.Events.Hooks.Tokens do @behaviour Domain.Events.Hooks - alias Domain.PubSub + alias Domain.{PubSub, Tokens, SchemaHelpers} @impl true def on_insert(_data), do: :ok @@ -12,7 +12,7 @@ defmodule Domain.Events.Hooks.Tokens do def on_update(_old_data, %{"type" => "email"}), do: :ok - # Soft-delete + # Soft-delete - process as delete def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at}) when not is_nil(deleted_at) do on_delete(old_data) @@ -22,7 +22,24 @@ defmodule Domain.Events.Hooks.Tokens do def on_update(_old_data, _new_data), do: :ok @impl true - def on_delete(%{"id" => token_id}) do - PubSub.Token.disconnect(token_id) + def on_delete(old_data) do + token = SchemaHelpers.struct_from_params(Tokens.Token, old_data) + + # Disconnect all sockets using this token + disconnect_socket(token) + + # TODO: Hard delete + # This can be removed upon implementation of hard delete + Domain.Flows.delete_flows_for(token) + + PubSub.Account.broadcast(token.account_id, {:deleted, token}) + end + + # This is a special message that disconnects all sockets using this token, + # such as for LiveViews. + defp disconnect_socket(token) do + topic = Domain.Tokens.socket_id(token.id) + payload = %Phoenix.Socket.Broadcast{topic: topic, event: "disconnect"} + Domain.PubSub.broadcast(topic, payload) end end diff --git a/elixir/apps/domain/lib/domain/events/replication_connection.ex b/elixir/apps/domain/lib/domain/events/replication_connection.ex index 736dd8513..73d2376b7 100644 --- a/elixir/apps/domain/lib/domain/events/replication_connection.ex +++ b/elixir/apps/domain/lib/domain/events/replication_connection.ex @@ -5,13 +5,10 @@ defmodule Domain.Events.ReplicationConnection do @tables_to_hooks %{ "accounts" => Hooks.Accounts, "actor_group_memberships" => Hooks.ActorGroupMemberships, - "actor_groups" => Hooks.ActorGroups, - "actors" => Hooks.Actors, - "auth_identities" => Hooks.AuthIdentities, - "auth_providers" => Hooks.AuthProviders, "clients" => Hooks.Clients, - "gateway_groups" => Hooks.GatewayGroups, + "flows" => Hooks.Flows, "gateways" => Hooks.Gateways, + "gateway_groups" => Hooks.GatewayGroups, "policies" => Hooks.Policies, "resource_connections" => Hooks.ResourceConnections, "resources" => Hooks.Resources, @@ -21,9 +18,9 @@ defmodule Domain.Events.ReplicationConnection do def on_write(state, _lsn, op, table, old_data, data) do if hook = Map.get(@tables_to_hooks, table) do case op do - :insert -> hook.on_insert(data) - :update -> hook.on_update(old_data, data) - :delete -> hook.on_delete(old_data) + :insert -> :ok = hook.on_insert(data) + :update -> :ok = hook.on_update(old_data, data) + :delete -> :ok = hook.on_delete(old_data) end else log_warning(op, table) diff --git a/elixir/apps/domain/lib/domain/events/topics.ex b/elixir/apps/domain/lib/domain/events/topics.ex deleted file mode 100644 index 9a23e2412..000000000 --- a/elixir/apps/domain/lib/domain/events/topics.ex +++ /dev/null @@ -1,47 +0,0 @@ -defmodule Domain.Events.Topics do - @moduledoc """ - A simple module to house all of the topics and broadcasts so we can see - them and verify them in one place. - """ - alias Domain.PubSub - - defmodule Account do - def subscribe(account_id) do - account_id - |> topic() - |> PubSub.subscribe() - end - - defp topic(account_id) do - "accounts:" <> account_id - end - end - - defmodule Presence do - defmodule Account do - defmodule Clients do - def subscribe(account_id) do - account_id - |> topic() - |> PubSub.subscribe() - end - - defp topic(account_id) do - "presences:account_clients:" <> account_id - end - end - - defmodule Gateways do - def subscribe(account_id) do - account_id - |> topic() - |> PubSub.subscribe() - end - - defp topic(account_id) do - "presences:account_gateways:" <> account_id - end - end - end - end -end diff --git a/elixir/apps/domain/lib/domain/flows.ex b/elixir/apps/domain/lib/domain/flows.ex index 92bc0ec80..61c38c20b 100644 --- a/elixir/apps/domain/lib/domain/flows.ex +++ b/elixir/apps/domain/lib/domain/flows.ex @@ -1,45 +1,47 @@ defmodule Domain.Flows do alias Domain.Repo - alias Domain.{Auth, Actors, Clients, Gateways, Resources, Policies, Tokens} + alias Domain.{Auth, Actors, Clients, Gateways, Resources, Policies} alias Domain.Flows.{Authorizer, Flow} require Ecto.Query require Logger - def authorize_flow( + # TODO: Optimization + # Connection setup latency - doesn't need to block setting up flow. Authorizing the flow + # is now handled in memory and this only logs it, so these can be done in parallel. + def create_flow( %Clients.Client{ id: client_id, account_id: account_id, actor_id: actor_id - } = client, + }, %Gateways.Gateway{ id: gateway_id, last_seen_remote_ip: gateway_remote_ip, account_id: account_id }, resource_id, + %Policies.Policy{} = policy, %Auth.Subject{ account: %{id: account_id}, actor: %{id: actor_id}, - expires_at: expires_at, token_id: token_id, context: %Auth.Context{ remote_ip: client_remote_ip, user_agent: client_user_agent } - } = subject, - opts \\ [] + } = subject ) do with :ok <- Auth.ensure_has_permissions(subject, Authorizer.create_flows_permission()), - {:ok, resource} <- - Resources.fetch_and_authorize_resource_by_id(resource_id, subject, opts), - {:ok, policy, conformation_expires_at} <- fetch_conforming_policy(resource, client) do + {:ok, membership} <- + Actors.fetch_membership_by_actor_id_and_group_id(actor_id, policy.actor_group_id) do flow = Flow.Changeset.create(%{ token_id: token_id, policy_id: policy.id, client_id: client_id, gateway_id: gateway_id, - resource_id: resource.id, + resource_id: resource_id, + actor_group_membership_id: membership.id, account_id: account_id, client_remote_ip: client_remote_ip, client_user_agent: client_user_agent, @@ -47,28 +49,7 @@ defmodule Domain.Flows do }) |> Repo.insert!() - expires_at = conformation_expires_at || expires_at - - {:ok, resource, flow, expires_at} - end - end - - defp fetch_conforming_policy(%Resources.Resource{} = resource, client) do - Enum.reduce_while(resource.authorized_by_policies, {:error, []}, fn policy, {:error, acc} -> - case Policies.ensure_client_conforms_policy_conditions(client, policy) do - {:ok, expires_at} -> - {:halt, {:ok, policy, expires_at}} - - {:error, {:forbidden, violated_properties: violated_properties}} -> - {:cont, {:error, violated_properties ++ acc}} - end - end) - |> case do - {:error, violated_properties} -> - {:error, {:forbidden, violated_properties: violated_properties}} - - {:ok, policy, expires_at} -> - {:ok, policy, expires_at} + {:ok, flow} end end @@ -85,6 +66,14 @@ defmodule Domain.Flows do end end + def all_gateway_flows_for_cache!(%Gateways.Gateway{} = gateway) do + Flow.Query.all() + |> Flow.Query.by_account_id(gateway.account_id) + |> Flow.Query.by_gateway_id(gateway.id) + |> Flow.Query.for_cache() + |> Repo.all() + end + def list_flows_for(assoc, subject, opts \\ []) def list_flows_for(%Policies.Policy{} = policy, %Auth.Subject{} = subject, opts) do @@ -125,121 +114,62 @@ defmodule Domain.Flows do end end - # TODO: WAL - # Remove all of the indexes used for these after flow expiration is moved to state - # broadcasts - - def expire_flows_for(%Auth.Identity{} = identity) do + def delete_flows_for(%Domain.Accounts.Account{} = account) do Flow.Query.all() - |> Flow.Query.by_identity_id(identity.id) - |> expire_flows() + |> Flow.Query.by_account_id(account.id) + |> Repo.delete_all() end - def expire_flows_for(%Clients.Client{} = client) do + def delete_flows_for(%Domain.Actors.Membership{} = membership) do Flow.Query.all() + |> Flow.Query.by_account_id(membership.account_id) + |> Flow.Query.by_actor_group_membership_id(membership.id) + |> Repo.delete_all() + end + + def delete_flows_for(%Domain.Clients.Client{} = client) do + Flow.Query.all() + |> Flow.Query.by_account_id(client.account_id) |> Flow.Query.by_client_id(client.id) - |> expire_flows() + |> Repo.delete_all() end - def expire_flows_for(%Actors.Group{} = actor_group) do + def delete_flows_for(%Domain.Gateways.Gateway{} = gateway) do Flow.Query.all() - |> Flow.Query.by_policy_actor_group_id(actor_group.id) - |> expire_flows() + |> Flow.Query.by_account_id(gateway.account_id) + |> Flow.Query.by_gateway_id(gateway.id) + |> Repo.delete_all() end - def expire_flows_for(%Tokens.Token{} = token, %Auth.Subject{} = subject) do + def delete_flows_for(%Domain.Policies.Policy{} = policy) do Flow.Query.all() - |> Flow.Query.by_token_id(token.id) - |> expire_flows(subject) + |> Flow.Query.by_account_id(policy.account_id) + |> Flow.Query.by_policy_id(policy.id) + |> Repo.delete_all() end - def expire_flows_for(%Actors.Actor{} = actor, %Auth.Subject{} = subject) do - Flow.Query.all() - |> Flow.Query.by_actor_id(actor.id) - |> expire_flows(subject) - end - - def expire_flows_for(%Auth.Identity{} = identity, %Auth.Subject{} = subject) do - Flow.Query.all() - |> Flow.Query.by_identity_id(identity.id) - |> expire_flows(subject) - end - - def expire_flows_for(%Resources.Resource{} = resource, %Auth.Subject{} = subject) do + def delete_flows_for(%Domain.Resources.Resource{} = resource) do Flow.Query.all() + |> Flow.Query.by_account_id(resource.account_id) |> Flow.Query.by_resource_id(resource.id) - |> expire_flows(subject) + |> Repo.delete_all() end - def expire_flows_for(%Actors.Group{} = actor_group, %Auth.Subject{} = subject) do + def delete_flows_for(%Domain.Tokens.Token{} = token) do Flow.Query.all() - |> Flow.Query.by_policy_actor_group_id(actor_group.id) - |> expire_flows(subject) + |> Flow.Query.by_account_id(token.account_id) + |> Flow.Query.by_token_id(token.id) + |> Repo.delete_all() end - def expire_flows_for(%Auth.Provider{} = provider, %Auth.Subject{} = subject) do + def delete_stale_flows_on_connect(%Clients.Client{} = client, resources) + when is_list(resources) do + authorized_resource_ids = Enum.map(resources, & &1.id) + Flow.Query.all() - |> Flow.Query.by_identity_provider_id(provider.id) - |> expire_flows(subject) - end - - def expire_flows_for(account_id, actor_id, group_id) do - Flow.Query.all() - |> Flow.Query.by_account_id(account_id) - |> Flow.Query.by_actor_id(actor_id) - |> Flow.Query.by_policy_actor_group_id(group_id) - |> expire_flows() - end - - def expire_flows_for_resource_id(account_id, resource_id) do - Flow.Query.all() - |> Flow.Query.by_account_id(account_id) - |> Flow.Query.by_resource_id(resource_id) - |> expire_flows() - end - - def expire_flows_for_policy_id(account_id, policy_id) do - Flow.Query.all() - |> Flow.Query.by_account_id(account_id) - |> Flow.Query.by_policy_id(policy_id) - |> expire_flows() - end - - defp expire_flows(queryable, subject) do - with :ok <- Auth.ensure_has_permissions(subject, Authorizer.create_flows_permission()) do - queryable - |> Authorizer.for_subject(Flow, subject) - |> expire_flows() - end - end - - defp expire_flows(queryable) do - {:ok, :ok} = - Repo.transaction(fn -> - queryable - |> Repo.stream() - |> Stream.chunk_every(100) - |> Enum.each(fn chunk -> - Enum.each(chunk, &broadcast_flow_expiration/1) - end) - end) - - :ok - end - - defp broadcast_flow_expiration(flow) do - case Domain.PubSub.Flow.broadcast( - flow.id, - {:expire_flow, flow.id, flow.client_id, flow.resource_id} - ) do - :ok -> - :ok - - {:error, reason} -> - Logger.error("Failed to broadcast flow expiration", - reason: inspect(reason), - flow_id: flow.id - ) - end + |> Flow.Query.by_account_id(client.account_id) + |> Flow.Query.by_client_id(client.id) + |> Flow.Query.by_not_in_resource_ids(authorized_resource_ids) + |> Repo.delete_all() end end diff --git a/elixir/apps/domain/lib/domain/flows/flow.ex b/elixir/apps/domain/lib/domain/flows/flow.ex index 47d72108e..b5e421cc1 100644 --- a/elixir/apps/domain/lib/domain/flows/flow.ex +++ b/elixir/apps/domain/lib/domain/flows/flow.ex @@ -7,6 +7,7 @@ defmodule Domain.Flows.Flow do belongs_to :gateway, Domain.Gateways.Gateway belongs_to :resource, Domain.Resources.Resource belongs_to :token, Domain.Tokens.Token + belongs_to :actor_group_membership, Domain.Actors.Membership belongs_to :account, Domain.Accounts.Account diff --git a/elixir/apps/domain/lib/domain/flows/flow/changeset.ex b/elixir/apps/domain/lib/domain/flows/flow/changeset.ex index e2e1df0c6..012a3ddd0 100644 --- a/elixir/apps/domain/lib/domain/flows/flow/changeset.ex +++ b/elixir/apps/domain/lib/domain/flows/flow/changeset.ex @@ -2,7 +2,7 @@ defmodule Domain.Flows.Flow.Changeset do use Domain, :changeset alias Domain.Flows.Flow - @fields ~w[token_id policy_id client_id gateway_id resource_id + @fields ~w[token_id policy_id client_id gateway_id resource_id actor_group_membership_id account_id client_remote_ip client_user_agent gateway_remote_ip]a @@ -16,6 +16,7 @@ defmodule Domain.Flows.Flow.Changeset do |> assoc_constraint(:client) |> assoc_constraint(:gateway) |> assoc_constraint(:resource) + |> assoc_constraint(:actor_group_membership) |> assoc_constraint(:account) end end diff --git a/elixir/apps/domain/lib/domain/flows/flow/query.ex b/elixir/apps/domain/lib/domain/flows/flow/query.ex index be0f9dd8e..3a60303bf 100644 --- a/elixir/apps/domain/lib/domain/flows/flow/query.ex +++ b/elixir/apps/domain/lib/domain/flows/flow/query.ex @@ -21,12 +21,28 @@ defmodule Domain.Flows.Flow.Query do where(queryable, [flows: flows], flows.policy_id == ^policy_id) end + # Return the latest {client_id, resource_id} pairs over the last 14 days + def for_cache(queryable) do + cutoff = DateTime.utc_now() |> DateTime.add(-14, :day) + + where(queryable, [flows: flows], flows.inserted_at > ^cutoff) + |> group_by([flows: flows], [flows.client_id, flows.resource_id]) + |> select( + [flows: flows], + {{flows.client_id, flows.resource_id}, max(flows.inserted_at)} + ) + end + def by_policy_actor_group_id(queryable, actor_group_id) do queryable |> with_joined_policy() |> where([policy: policy], policy.actor_group_id == ^actor_group_id) end + def by_actor_group_membership_id(queryable, membership_id) do + where(queryable, [flows: flows], flows.actor_group_membership_id == ^membership_id) + end + def by_identity_id(queryable, identity_id) do queryable |> with_joined_client() @@ -43,6 +59,10 @@ defmodule Domain.Flows.Flow.Query do where(queryable, [flows: flows], flows.resource_id == ^resource_id) end + def by_not_in_resource_ids(queryable, resource_ids) do + where(queryable, [flows: flows], flows.resource_id not in ^resource_ids) + end + def by_client_id(queryable, client_id) do where(queryable, [flows: flows], flows.client_id == ^client_id) end diff --git a/elixir/apps/domain/lib/domain/gateways/presence.ex b/elixir/apps/domain/lib/domain/gateways/presence.ex index 6725b5ee5..1961be3ce 100644 --- a/elixir/apps/domain/lib/domain/gateways/presence.ex +++ b/elixir/apps/domain/lib/domain/gateways/presence.ex @@ -9,8 +9,7 @@ defmodule Domain.Gateways.Presence do def connect(%Gateway{} = gateway) do with {:ok, _} <- __MODULE__.Group.track(gateway.group_id, gateway.id), {:ok, _} <- __MODULE__.Account.track(gateway.account_id, gateway.id) do - :ok = PubSub.Gateway.subscribe(gateway.id) - :ok = PubSub.Account.Gateways.subscribe(gateway.account_id) + :ok end end diff --git a/elixir/apps/domain/lib/domain/policies.ex b/elixir/apps/domain/lib/domain/policies.ex index ab62b9d73..0be4bcf68 100644 --- a/elixir/apps/domain/lib/domain/policies.ex +++ b/elixir/apps/domain/lib/domain/policies.ex @@ -58,6 +58,22 @@ defmodule Domain.Policies do end end + def all_policies_for_actor!(%Actors.Actor{} = actor) do + Policy.Query.not_disabled() + |> Policy.Query.by_account_id(actor.account_id) + |> Policy.Query.by_actor_id(actor.id) + |> Policy.Query.with_preloaded_resource_gateway_groups() + |> Repo.all() + end + + def all_policies_for_actor_group_id!(account_id, actor_group_id) do + Policy.Query.not_disabled() + |> Policy.Query.by_account_id(account_id) + |> Policy.Query.by_actor_group_id(actor_group_id) + |> Policy.Query.with_preloaded_resource_gateway_groups() + |> Repo.all() + end + def new_policy(attrs, %Auth.Subject{} = subject) do Policy.Changeset.create(attrs, subject) end @@ -161,18 +177,8 @@ defmodule Domain.Policies do {:ok, policies} end - def pre_filter_non_conforming_resources(resources, %Clients.Client{} = client) do - resources - |> Enum.flat_map(fn resource -> - case client_conforms_any_on_connect?(client, resource.authorized_by_policies) do - true -> [resource] - false -> [] - end - end) - end - - def client_conforms_any_on_connect?(%Clients.Client{} = client, policies) do - Enum.any?(policies, fn policy -> + def filter_by_conforming_policies_for_client(policies, %Clients.Client{} = client) do + Enum.filter(policies, fn policy -> policy.conditions |> Enum.filter(&Condition.Evaluator.evaluable_on_connect?/1) |> Condition.Evaluator.ensure_conforms(client) diff --git a/elixir/apps/domain/lib/domain/policies/policy/query.ex b/elixir/apps/domain/lib/domain/policies/policy/query.ex index 69d8e6d76..dff77a258 100644 --- a/elixir/apps/domain/lib/domain/policies/policy/query.ex +++ b/elixir/apps/domain/lib/domain/policies/policy/query.ex @@ -108,6 +108,11 @@ defmodule Domain.Policies.Policy.Query do end) end + def with_preloaded_resource_gateway_groups(queryable) do + queryable + |> preload(resource: :gateway_groups) + end + # Pagination @impl Domain.Repo.Query diff --git a/elixir/apps/domain/lib/domain/pubsub.ex b/elixir/apps/domain/lib/domain/pubsub.ex index 23a9dd709..0e50af306 100644 --- a/elixir/apps/domain/lib/domain/pubsub.ex +++ b/elixir/apps/domain/lib/domain/pubsub.ex @@ -42,7 +42,6 @@ defmodule Domain.PubSub do Phoenix.PubSub.unsubscribe(__MODULE__, topic) end - # TODO: These are quite repetitive. We could simplify this with a `__using__` macro. defmodule Account do def subscribe(account_id) do account_id @@ -59,315 +58,5 @@ defmodule Domain.PubSub do defp topic(account_id) do Atom.to_string(__MODULE__) <> ":" <> account_id end - - defmodule Clients do - def subscribe(account_id) do - account_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def broadcast(account_id, payload) do - account_id - |> topic() - |> Domain.PubSub.broadcast(payload) - end - - def disconnect(account_id) do - account_id - |> topic() - |> Domain.PubSub.broadcast("disconnect") - end - - defp topic(account_id) do - Atom.to_string(__MODULE__) <> ":" <> account_id - end - end - - defmodule Gateways do - def subscribe(account_id) do - account_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def disconnect(account_id) do - account_id - |> topic() - |> Domain.PubSub.broadcast("disconnect") - end - - defp topic(account_id) do - Atom.to_string(__MODULE__) <> ":" <> account_id - end - end - - defmodule Policies do - def subscribe(account_id) do - account_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def broadcast(account_id, payload) do - account_id - |> topic() - |> Domain.PubSub.broadcast(payload) - end - - defp topic(account_id) do - Atom.to_string(__MODULE__) <> ":" <> account_id - end - end - - defmodule Resources do - def subscribe(account_id) do - account_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def broadcast(account_id, payload) do - account_id - |> topic() - |> Domain.PubSub.broadcast(payload) - end - - defp topic(account_id) do - Atom.to_string(__MODULE__) <> ":" <> account_id - end - end - end - - defmodule Actor do - def subscribe(actor_id) do - actor_id - |> topic() - |> Domain.PubSub.subscribe() - end - - defp topic(actor_id) do - Atom.to_string(__MODULE__) <> ":" <> actor_id - end - - defmodule Memberships do - def subscribe(actor_id) do - actor_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def broadcast(actor_id, payload) do - actor_id - |> topic() - |> Domain.PubSub.broadcast(payload) - end - - def broadcast_access(action, actor_id, group_id) do - Domain.Policies.Policy.Query.not_deleted() - |> Domain.Policies.Policy.Query.by_actor_group_id(group_id) - |> Domain.Repo.all() - |> Enum.each(fn policy -> - payload = {:"#{action}_access", policy.id, policy.actor_group_id, policy.resource_id} - :ok = Actor.Policies.broadcast(actor_id, payload) - end) - end - - defp topic(actor_id) do - Atom.to_string(__MODULE__) <> ":" <> actor_id - end - end - - defmodule Policies do - def subscribe(actor_id) do - actor_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def broadcast(actor_id, payload) do - actor_id - |> topic() - |> Domain.PubSub.broadcast(payload) - end - - defp topic(actor_id) do - Atom.to_string(__MODULE__) <> ":" <> actor_id - end - end - end - - defmodule ActorGroup do - defmodule Policies do - def subscribe(actor_group_id) do - actor_group_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def unsubscribe(actor_group_id) do - actor_group_id - |> topic() - |> Domain.PubSub.unsubscribe() - end - - def broadcast(actor_group_id, payload) do - actor_group_id - |> topic() - |> Domain.PubSub.broadcast(payload) - end - - defp topic(actor_group_id) do - Atom.to_string(__MODULE__) <> ":" <> actor_group_id - end - end - end - - defmodule Client do - def subscribe(client_id) do - client_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def broadcast(client_id, payload) do - client_id - |> topic() - |> Domain.PubSub.broadcast(payload) - end - - def disconnect(client_id) do - client_id - |> topic() - |> Domain.PubSub.broadcast("disconnect") - end - - defp topic(client_id) do - Atom.to_string(__MODULE__) <> ":" <> client_id - end - end - - defmodule Flow do - def subscribe(flow_id) do - flow_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def unsubscribe(flow_id) do - flow_id - |> topic() - |> Domain.PubSub.unsubscribe() - end - - def broadcast(flow_id, payload) do - flow_id - |> topic() - |> Domain.PubSub.broadcast(payload) - end - - defp topic(flow_id) do - Atom.to_string(__MODULE__) <> ":" <> flow_id - end - end - - defmodule GatewayGroup do - def subscribe(gateway_group_id) do - gateway_group_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def unsubscribe(gateway_group_id) do - gateway_group_id - |> topic() - |> Domain.PubSub.unsubscribe() - end - - defp topic(gateway_group_id) do - Atom.to_string(__MODULE__) <> ":" <> gateway_group_id - end - end - - defmodule Gateway do - def subscribe(gateway_id) do - gateway_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def broadcast(gateway_id, payload) do - gateway_id - |> topic() - |> Domain.PubSub.broadcast(payload) - end - - def disconnect(gateway_id) do - gateway_id - |> topic() - |> Domain.PubSub.broadcast("disconnect") - end - - defp topic(gateway_id) do - Atom.to_string(__MODULE__) <> ":" <> gateway_id - end - end - - defmodule Policy do - def subscribe(policy_id) do - policy_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def broadcast(policy_id, payload) do - policy_id - |> topic() - |> Domain.PubSub.broadcast(payload) - end - - defp topic(policy_id) do - Atom.to_string(__MODULE__) <> ":" <> policy_id - end - end - - defmodule Resource do - def subscribe(resource_id) do - resource_id - |> topic() - |> Domain.PubSub.subscribe() - end - - def unsubscribe(resource_id) do - resource_id - |> topic() - |> Domain.PubSub.unsubscribe() - end - - def broadcast(resource_id, payload) do - resource_id - |> topic() - |> Domain.PubSub.broadcast(payload) - end - - defp topic(resource_id) do - Atom.to_string(__MODULE__) <> ":" <> resource_id - end - end - - defmodule Token do - def disconnect(token_id) do - token_id - |> topic() - |> Domain.PubSub.broadcast(%Phoenix.Socket.Broadcast{ - topic: topic(token_id), - event: "disconnect" - }) - end - - defp topic(token_id) do - # This topic is managed by Phoenix - Domain.Tokens.socket_id(token_id) - end end end diff --git a/elixir/apps/domain/lib/domain/relays.ex b/elixir/apps/domain/lib/domain/relays.ex index 86f312208..2dbc7c56c 100644 --- a/elixir/apps/domain/lib/domain/relays.ex +++ b/elixir/apps/domain/lib/domain/relays.ex @@ -344,6 +344,8 @@ defmodule Domain.Relays do |> Enum.map(&Enum.random(elem(&1, 1))) end + # TODO: WAL + # Refactor to use new conventions def connect_relay(%Relay{} = relay, secret) do with {:ok, _} <- Presence.track(self(), group_presence_topic(relay.group_id), relay.id, %{}), diff --git a/elixir/apps/domain/lib/domain/replication/connection.ex b/elixir/apps/domain/lib/domain/replication/connection.ex index 56eff8fc0..a1a12f826 100644 --- a/elixir/apps/domain/lib/domain/replication/connection.ex +++ b/elixir/apps/domain/lib/domain/replication/connection.ex @@ -579,34 +579,13 @@ defmodule Domain.Replication.Connection do {:delete, old, nil} end - defp decode_value({value, %{type: type} = column}) - when type in ["json", "jsonb"] and is_binary(value) do - case JSON.decode(value) do - {:ok, decoded} -> - {column.name, decoded} - - {:error, reason} -> - Logger.warning("Could not decode JSONB value, using as-is", - reason: reason, - value: value, - column: column.name - ) - - {column.name, value} - end - end - - defp decode_value({value, column}) do - {column.name, value} - end - defp zip(nil, _), do: nil defp zip(tuple_data, columns) do tuple_data |> Tuple.to_list() |> Enum.zip(columns) - |> Map.new(&decode_value/1) + |> Map.new(&Decoder.decode_json/1) end end end diff --git a/elixir/apps/domain/lib/domain/replication/decoder.ex b/elixir/apps/domain/lib/domain/replication/decoder.ex index 62a4d7456..abc64922e 100644 --- a/elixir/apps/domain/lib/domain/replication/decoder.ex +++ b/elixir/apps/domain/lib/domain/replication/decoder.ex @@ -150,6 +150,73 @@ defmodule Domain.Replication.Decoder do alias Domain.Replication.OidDatabase + @doc """ + Helper for decoding JSON data inside messages. + """ + + # Postgrex uses `_jsonb` to mean `jsonb[]`. These array types are returned as string literals from + # Postgrex and need to be split, and then double-decoded. + def decode_json({value, %{type: type} = column}) + when type in ["_json", "_jsonb"] and is_binary(value) do + decoded_list = parse_postgres_jsonb_array(value) + {column.name, decoded_list} + end + + def decode_json({value, %{type: type} = column}) + when type in ["json", "jsonb"] and is_binary(value) do + case JSON.decode(value) do + {:ok, decoded} -> + {column.name, decoded} + + {:error, reason} -> + Logger.warning("Failed to decode JSON, using as-is", + json: value, + reason: reason + ) + + {column.name, value} + end + end + + def decode_json({value, column}) do + {column.name, value} + end + + defp parse_postgres_jsonb_array("{}"), do: [] + + defp parse_postgres_jsonb_array("{" <> content) do + content + |> String.trim_trailing("}") + |> split_json_array_elements() + |> Enum.map(&double_decode_json/1) + end + + defp parse_postgres_jsonb_array(_), do: [] + + # Split JSON elements in PostgreSQL array using regex + defp split_json_array_elements(content) do + ~r/,(?=(?:[^"]*"[^"]*")*[^"]*$)(?![^{]*})/ + |> Regex.split(content) + |> Enum.map(&String.trim/1) + |> Enum.reject(&(&1 == "")) + end + + # PostgreSQL double-encodes JSON in arrays, so we need to decode twice + defp double_decode_json(json_str) do + with {:ok, first} <- Jason.decode(json_str), + {:ok, second} <- Jason.decode(first) do + second + else + {:error, reason} -> + Logger.warning("Failed to decode JSON, using as-is", + json: json_str, + reason: reason + ) + + json_str + end + end + @doc """ Parses logical replication messages from Postgres diff --git a/elixir/apps/domain/lib/domain/resources.ex b/elixir/apps/domain/lib/domain/resources.ex index 3eedcfd70..0209eb955 100644 --- a/elixir/apps/domain/lib/domain/resources.ex +++ b/elixir/apps/domain/lib/domain/resources.ex @@ -80,21 +80,6 @@ defmodule Domain.Resources do end end - def fetch_and_authorize_resource_by_id(id, %Auth.Subject{} = subject, opts \\ []) do - with :ok <- - Auth.ensure_has_permissions(subject, Authorizer.view_available_resources_permission()), - true <- Repo.valid_uuid?(id) do - Resource.Query.not_deleted() - |> Resource.Query.by_id(id) - |> Resource.Query.by_account_id(subject.account.id) - |> Resource.Query.by_authorized_actor_id(subject.actor.id) - |> Repo.fetch(Resource.Query, opts) - else - false -> {:error, :not_found} - other -> other - end - end - def fetch_resource_by_id!(id) do if Repo.valid_uuid?(id) do Resource.Query.not_deleted() diff --git a/elixir/apps/domain/lib/domain/tokens.ex b/elixir/apps/domain/lib/domain/tokens.ex index b5b3892ec..a3961339a 100644 --- a/elixir/apps/domain/lib/domain/tokens.ex +++ b/elixir/apps/domain/lib/domain/tokens.ex @@ -183,10 +183,6 @@ defmodule Domain.Tokens do ]} with :ok <- Auth.ensure_has_permissions(subject, required_permissions) do - # TODO: WAL - # Broadcast flow side effects directly - :ok = Domain.Flows.expire_flows_for(token, subject) - Token.Query.not_deleted() |> Token.Query.by_id(token.id) |> Authorizer.for_subject(subject) @@ -203,23 +199,9 @@ defmodule Domain.Tokens do |> Token.Query.by_id(subject.token_id) |> Authorizer.for_subject(subject) |> delete_tokens() - |> case do - {:ok, [token]} -> - # TODO: WAL - # Broadcast flow side effects directly - :ok = Domain.Flows.expire_flows_for(token, subject) - {:ok, token} - - {:ok, []} -> - {:ok, []} - end end def delete_tokens_for(%Auth.Identity{} = identity) do - # TODO: WAL - # Broadcast flow side effects directly - :ok = Domain.Flows.expire_flows_for(identity) - Token.Query.not_deleted() |> Token.Query.by_identity_id(identity.id) |> delete_tokens() @@ -227,10 +209,6 @@ defmodule Domain.Tokens do def delete_tokens_for(%Actors.Actor{} = actor, %Auth.Subject{} = subject) do with :ok <- Auth.ensure_has_permissions(subject, Authorizer.manage_tokens_permission()) do - # TODO: WAL - # Broadcast flow side effects directly - :ok = Domain.Flows.expire_flows_for(actor, subject) - Token.Query.not_deleted() |> Token.Query.by_actor_id(actor.id) |> Authorizer.for_subject(subject) @@ -240,10 +218,6 @@ defmodule Domain.Tokens do def delete_tokens_for(%Auth.Identity{} = identity, %Auth.Subject{} = subject) do with :ok <- Auth.ensure_has_permissions(subject, Authorizer.manage_tokens_permission()) do - # TODO: WAL - # Broadcast flow side effects directly - :ok = Domain.Flows.expire_flows_for(identity, subject) - Token.Query.not_deleted() |> Token.Query.by_identity_id(identity.id) |> Authorizer.for_subject(subject) @@ -253,10 +227,6 @@ defmodule Domain.Tokens do def delete_tokens_for(%Auth.Provider{} = provider, %Auth.Subject{} = subject) do with :ok <- Auth.ensure_has_permissions(subject, Authorizer.manage_tokens_permission()) do - # TODO: WAL - # Broadcast flow side effects directly - :ok = Domain.Flows.expire_flows_for(provider, subject) - Token.Query.not_deleted() |> Token.Query.by_provider_id(provider.id) |> Authorizer.for_subject(subject) diff --git a/elixir/apps/domain/priv/repo/manual_migrations/20250704201940_index_flows_on_token_id.exs b/elixir/apps/domain/priv/repo/manual_migrations/20250704201940_index_flows_on_token_id.exs new file mode 100644 index 000000000..35df55c53 --- /dev/null +++ b/elixir/apps/domain/priv/repo/manual_migrations/20250704201940_index_flows_on_token_id.exs @@ -0,0 +1,17 @@ +defmodule Domain.Repo.Migrations.IndexFlowsOnTokenId do + use Ecto.Migration + + @disable_ddl_transaction true + + def up do + execute(""" + CREATE INDEX CONCURRENTLY IF NOT EXISTS flows_account_id_token_id_index ON flows USING BTREE (account_id, token_id, inserted_at DESC, id DESC); + """) + end + + def down do + execute(""" + DROP INDEX CONCURRENTLY IF EXISTS flows_account_id_token_id_index; + """) + end +end diff --git a/elixir/apps/domain/priv/repo/migrations/20250712010323_add_id_to_actor_group_memberships.exs b/elixir/apps/domain/priv/repo/migrations/20250712010323_add_id_to_actor_group_memberships.exs new file mode 100644 index 000000000..d4272d3f9 --- /dev/null +++ b/elixir/apps/domain/priv/repo/migrations/20250712010323_add_id_to_actor_group_memberships.exs @@ -0,0 +1,70 @@ +defmodule Domain.Repo.Migrations.AddIdToActorGroupMemberships do + use Ecto.Migration + + @moduledoc """ + This migration will lock the `actor_group_memberships` table, so it's + best to run this when a brief period of downtime is acceptable. + """ + + def up do + # Step 1: Add the new column with a default value + execute(""" + ALTER TABLE actor_group_memberships + ADD COLUMN IF NOT EXISTS id UUID DEFAULT uuid_generate_v4() + """) + + # Step 2: Backfill the new column for existing rows + execute(""" + UPDATE actor_group_memberships SET id = uuid_generate_v4() WHERE id IS NULL + """) + + # Step 3: Enforce the NOT NULL constraint + execute(""" + ALTER TABLE actor_group_memberships + ALTER COLUMN id SET NOT NULL + """) + + # Step 4: Drop the old composite primary key + execute(""" + ALTER TABLE actor_group_memberships + DROP CONSTRAINT IF EXISTS actor_group_memberships_pkey + """) + + # Step 5: Add the new primary key on the id column + execute(""" + ALTER TABLE actor_group_memberships + ADD PRIMARY KEY (id) + """) + + # Step 6: Recreate the actor_id, group_id index with unique constraint + execute(""" + CREATE UNIQUE INDEX IF NOT EXISTS actor_group_memberships_actor_id_group_id_index + ON actor_group_memberships (actor_id, group_id) + """) + end + + def down do + # Step 1: Drop the unique index on actor_id and group_id + execute(""" + DROP INDEX IF EXISTS actor_group_memberships_actor_id_group_id_index + """) + + # Step 2: Drop the new single-column primary key + execute(""" + ALTER TABLE actor_group_memberships + DROP CONSTRAINT IF EXISTS actor_group_memberships_pkey + """) + + # Step 3: Restore the original composite primary key + execute(""" + ALTER TABLE actor_group_memberships + ADD CONSTRAINT IF NOT EXISTS actor_group_memberships_pkey PRIMARY KEY (actor_id, group_id) + """) + + # Step 4: Drop the id column + execute(""" + ALTER TABLE actor_group_memberships + DROP COLUMN IF EXISTS id + """) + end +end diff --git a/elixir/apps/domain/priv/repo/migrations/20250712155618_backfill_flows_with_actor_group_membership_id.exs b/elixir/apps/domain/priv/repo/migrations/20250712155618_backfill_flows_with_actor_group_membership_id.exs new file mode 100644 index 000000000..2ab6c029d --- /dev/null +++ b/elixir/apps/domain/priv/repo/migrations/20250712155618_backfill_flows_with_actor_group_membership_id.exs @@ -0,0 +1,85 @@ +defmodule Domain.Repo.Migrations.BackfillFlowsWithActorGroupMembershipId do + use Ecto.Migration + + @moduledoc """ + This migration will lock the `flows` table, so it's best to run this when a brief period of + downtime is acceptable. + """ + + def up do + # Step 1: Truncate flows table to remove entries older than 14 days + execute(""" + DELETE FROM flows + WHERE inserted_at < NOW() - INTERVAL '14 days' + """) + + # Step 2: Add the new foreign key column if it doesn't already exist + execute(""" + ALTER TABLE flows + ADD COLUMN IF NOT EXISTS actor_group_membership_id UUID + """) + + # Step 3: Backfill the new column by finding the correct membership ID + execute(""" + UPDATE flows AS f + SET actor_group_membership_id = agm.id + FROM + clients AS c, + policies AS p, + actor_group_memberships AS agm + WHERE + f.client_id = c.id + AND f.policy_id = p.id + AND c.actor_id = agm.actor_id + AND p.actor_group_id = agm.group_id + """) + + # Step 4: Delete flow records where a membership couldn't be found + execute(""" + DELETE FROM flows + WHERE actor_group_membership_id IS NULL + """) + + # Step 5: Now that all rows are populated, make the column NOT NULL + execute(""" + ALTER TABLE flows + ALTER COLUMN actor_group_membership_id SET NOT NULL + """) + + # Step 6: Add an index on the new foreign key for performance + execute(""" + CREATE INDEX IF NOT EXISTS flows_actor_group_membership_id_index + ON flows USING BTREE (account_id, actor_group_membership_id, inserted_at DESC, id DESC) + """) + + # Step 7: Add the foreign key constraint + execute(""" + ALTER TABLE flows + ADD CONSTRAINT flows_actor_group_membership_id_fkey + FOREIGN KEY (actor_group_membership_id) + REFERENCES actor_group_memberships(id) + ON DELETE CASCADE + """) + end + + def down do + # Step 1: Drop the foreign key constraint + execute(""" + ALTER TABLE flows + DROP CONSTRAINT IF EXISTS flows_actor_group_membership_id_fkey + """) + + # Step 2: Drop the index + execute(""" + DROP INDEX IF EXISTS flows_actor_group_membership_id_index + """) + + # Step 3: Drop the column + execute(""" + ALTER TABLE flows + DROP COLUMN IF EXISTS actor_group_membership_id + """) + + # Note: The data deleted in the 'up' migration is not restored. + end +end diff --git a/elixir/apps/domain/priv/repo/seeds.exs b/elixir/apps/domain/priv/repo/seeds.exs index 5d0c3b27c..c158e162d 100644 --- a/elixir/apps/domain/priv/repo/seeds.exs +++ b/elixir/apps/domain/priv/repo/seeds.exs @@ -1048,7 +1048,7 @@ defmodule Domain.Repo.Seeds do IO.puts(" #{search_domain_resource.address} - DNS - gateways: #{gateway_name}") IO.puts("") - {:ok, _} = + {:ok, policy} = Policies.create_policy( %{ name: "All Access To Google", @@ -1175,11 +1175,12 @@ defmodule Domain.Repo.Seeds do IO.puts("") - {:ok, _resource, _flow, _expires_at} = - Flows.authorize_flow( + {:ok, _flow} = + Flows.create_flow( user_iphone, gateway1, cidr_resource.id, + policy, unprivileged_subject ) end diff --git a/elixir/apps/domain/test/domain/actors_test.exs b/elixir/apps/domain/test/domain/actors_test.exs index 5da0422eb..b2d0859a9 100644 --- a/elixir/apps/domain/test/domain/actors_test.exs +++ b/elixir/apps/domain/test/domain/actors_test.exs @@ -4,7 +4,6 @@ defmodule Domain.ActorsTest do alias Domain.Auth alias Domain.Clients alias Domain.Actors - alias Domain.Events describe "fetch_groups_count_grouped_by_provider_id/1" do test "returns empty map when there are no groups" do @@ -1169,28 +1168,9 @@ defmodule Domain.ActorsTest do provider: provider, group1: group1, group2: group2, - actor1: actor1, identity1: identity1, identity2: identity2 } do - policy = Fixtures.Policies.create_policy(account: account, actor_group: group1) - - client = - Fixtures.Clients.create_client( - account: account, - actor: actor1, - identity: identity1 - ) - - flow = - Fixtures.Flows.create_flow( - account: account, - actor_group: group1, - client: client, - resource_id: policy.resource_id, - policy: policy - ) - Fixtures.Actors.create_membership( account: account, group: group1, @@ -1233,27 +1213,6 @@ defmodule Domain.ActorsTest do assert Repo.aggregate(Actors.Membership, :count) == 0 assert Repo.aggregate(Actors.Membership.Query.all(), :count) == 0 - - # TODO: WAL - # These tests will be made redundant soon - Events.Hooks.ActorGroupMemberships.on_delete(%{ - "account_id" => account.id, - "actor_id" => identity1.actor_id, - "group_id" => group1.id - }) - - Events.Hooks.ActorGroupMemberships.on_delete(%{ - "account_id" => account.id, - "actor_id" => identity2.actor_id, - "group_id" => group2.id - }) - - :ok = Domain.PubSub.Flow.subscribe(flow.id) - - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} end test "deletes memberships of removed groups", %{ @@ -3036,30 +2995,6 @@ defmodule Domain.ActorsTest do assert token.deleted_at end - test "expires actor flows" do - account = Fixtures.Accounts.create_account() - actor = Fixtures.Actors.create_actor(account: account, type: :account_admin_user) - identity = Fixtures.Auth.create_identity(account: account, actor: actor) - subject = Fixtures.Auth.create_subject(identity: identity) - client = Fixtures.Clients.create_client(account: account, identity: identity) - - flow = - Fixtures.Flows.create_flow( - account: account, - subject: subject, - client: client - ) - - :ok = Domain.PubSub.Flow.subscribe(flow.id) - - assert {:ok, _actor} = disable_actor(actor, subject) - - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - end - test "returns error when trying to disable the last admin actor" do account = Fixtures.Accounts.create_account() actor = Fixtures.Actors.create_actor(account: account, type: :account_admin_user) @@ -3284,31 +3219,6 @@ defmodule Domain.ActorsTest do assert Repo.aggregate(Actors.Membership, :count) == 0 end - test "expires actor flows", %{ - account: account, - actor: actor, - identity: identity, - subject: subject - } do - client = Fixtures.Clients.create_client(account: account, identity: identity) - - flow = - Fixtures.Flows.create_flow( - account: account, - subject: subject, - client: client - ) - - :ok = Domain.PubSub.Flow.subscribe(flow.id) - - assert {:ok, _actor} = delete_actor(actor, subject) - - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - end - test "returns error when trying to delete the last admin actor", %{ actor: actor, subject: subject diff --git a/elixir/apps/domain/test/domain/auth/adapters/google_workspace/jobs/sync_directory_test.exs b/elixir/apps/domain/test/domain/auth/adapters/google_workspace/jobs/sync_directory_test.exs index b376387e7..c36737295 100644 --- a/elixir/apps/domain/test/domain/auth/adapters/google_workspace/jobs/sync_directory_test.exs +++ b/elixir/apps/domain/test/domain/auth/adapters/google_workspace/jobs/sync_directory_test.exs @@ -1,6 +1,6 @@ defmodule Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectoryTest do use Domain.DataCase, async: true - alias Domain.{Auth, Actors, Events, PubSub} + alias Domain.{Auth, Actors} alias Domain.Mocks.GoogleWorkspaceDirectory import Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectory @@ -575,20 +575,6 @@ defmodule Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectoryTest do identity: deleted_identity ) - deleted_identity_client = - Fixtures.Clients.create_client( - account: account, - actor: other_actor, - identity: deleted_identity - ) - - deleted_identity_flow = - Fixtures.Flows.create_flow( - account: account, - client: deleted_identity_client, - token_id: deleted_identity_token.id - ) - group = Fixtures.Actors.create_group( account: account, @@ -610,34 +596,12 @@ defmodule Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectoryTest do provider_identifier: "OU:OU_ID1" ) - policy = Fixtures.Policies.create_policy(account: account, actor_group: group) - - deleted_policy = - Fixtures.Policies.create_policy(account: account, actor_group: deleted_group) - - deleted_group_flow = - Fixtures.Flows.create_flow( - account: account, - actor_group: deleted_group, - resource_id: deleted_policy.resource_id, - policy: deleted_policy - ) - Fixtures.Actors.create_membership(account: account, actor: actor) Fixtures.Actors.create_membership(account: account, actor: actor, group: group) deleted_membership = Fixtures.Actors.create_membership(account: account, group: group) Fixtures.Actors.create_membership(account: account, actor: actor, group: deleted_group) Fixtures.Actors.create_membership(account: account, actor: actor, group: org_unit) - :ok = PubSub.Flow.subscribe(deleted_group_flow.id) - :ok = PubSub.Flow.subscribe(deleted_identity_flow.id) - :ok = PubSub.Actor.Memberships.subscribe(actor.id) - :ok = PubSub.Actor.Memberships.subscribe(other_actor.id) - :ok = PubSub.Actor.Memberships.subscribe(deleted_membership.actor_id) - :ok = PubSub.Actor.Policies.subscribe(actor.id) - :ok = PubSub.Actor.Policies.subscribe(other_actor.id) - :ok = PubSub.ActorGroup.Policies.subscribe(deleted_group.id) - GoogleWorkspaceDirectory.override_endpoint_url("http://localhost:#{bypass.port}/") GoogleWorkspaceDirectory.mock_groups_list_endpoint( @@ -706,18 +670,6 @@ defmodule Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectoryTest do # Created membership for a member of existing group assert Repo.get_by(Domain.Actors.Membership, actor_id: other_actor.id, group_id: group.id) - # Broadcasts allow_access for it - policy_id = policy.id - group_id = group.id - resource_id = policy.resource_id - - Events.Hooks.ActorGroupMemberships.on_insert(%{ - "actor_id" => actor.id, - "group_id" => group.id - }) - - assert_receive {:allow_access, ^policy_id, ^group_id, ^resource_id} - # Deletes membership that is not found on IdP end refute Repo.get_by(Domain.Actors.Membership, actor_id: deleted_membership.actor_id, @@ -727,47 +679,6 @@ defmodule Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectoryTest do # Signs out users which identity has been deleted deleted_identity_token = Repo.reload(deleted_identity_token) assert deleted_identity_token.deleted_at - - # Deleted group deletes all policies and broadcasts reject access events for them - policy_id = deleted_policy.id - group_id = deleted_group.id - resource_id = deleted_policy.resource_id - - # Simulate WAL events - Events.Hooks.ActorGroupMemberships.on_delete(%{ - "account_id" => deleted_identity.account_id, - "actor_id" => deleted_identity.actor_id, - "group_id" => deleted_group.id - }) - - Events.Hooks.Policies.on_delete(%{ - "id" => policy_id, - "actor_group_id" => group_id, - "resource_id" => resource_id, - "account_id" => deleted_identity.account_id - }) - - assert_receive {:reject_access, ^policy_id, ^group_id, ^resource_id} - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - # Deleted policies expire all flows authorized by them - flow_id = deleted_group_flow.id - client_id = deleted_group_flow.client_id - resource_id = deleted_group_flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - - # Expires flows for signed out user - flow_id = deleted_identity_flow.id - client_id = deleted_identity_flow.client_id - resource_id = deleted_identity_flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - - # Should not do anything else - refute_received {:allow_access, _policy_id, _group_id, _resource_id} - refute_received {:reject_access, _policy_id, _group_id, _resource_id} end test "resurrects deleted identities that reappear on the next sync", %{ diff --git a/elixir/apps/domain/test/domain/auth/adapters/jumpcloud/jobs/sync_directory_test.exs b/elixir/apps/domain/test/domain/auth/adapters/jumpcloud/jobs/sync_directory_test.exs index df00b99d1..b99c1527a 100644 --- a/elixir/apps/domain/test/domain/auth/adapters/jumpcloud/jobs/sync_directory_test.exs +++ b/elixir/apps/domain/test/domain/auth/adapters/jumpcloud/jobs/sync_directory_test.exs @@ -1,6 +1,6 @@ defmodule Domain.Auth.Adapters.JumpCloud.Jobs.SyncDirectoryTest do use Domain.DataCase, async: true - alias Domain.{Auth, Actors, Events, PubSub} + alias Domain.{Auth, Actors} alias Domain.Mocks.WorkOSDirectory import Domain.Auth.Adapters.JumpCloud.Jobs.SyncDirectory @@ -363,20 +363,6 @@ defmodule Domain.Auth.Adapters.JumpCloud.Jobs.SyncDirectoryTest do identity: deleted_identity ) - deleted_identity_client = - Fixtures.Clients.create_client( - account: account, - actor: other_actor, - identity: deleted_identity - ) - - deleted_identity_flow = - Fixtures.Flows.create_flow( - account: account, - client: deleted_identity_client, - token_id: deleted_identity_token.id - ) - group = Fixtures.Actors.create_group( account: account, @@ -398,33 +384,10 @@ defmodule Domain.Auth.Adapters.JumpCloud.Jobs.SyncDirectoryTest do provider_identifier: "G:DELETED_GROUP_ID!" ) - policy = Fixtures.Policies.create_policy(account: account, actor_group: group) - - deleted_policy = - Fixtures.Policies.create_policy(account: account, actor_group: deleted_group) - - deleted_group_flow = - Fixtures.Flows.create_flow( - account: account, - actor_group: deleted_group, - resource_id: deleted_policy.resource_id, - policy: deleted_policy - ) - Fixtures.Actors.create_membership(account: account, actor: actor) Fixtures.Actors.create_membership(account: account, actor: actor, group: group) - deleted_membership = Fixtures.Actors.create_membership(account: account, group: group) Fixtures.Actors.create_membership(account: account, actor: actor, group: deleted_group) - :ok = PubSub.Flow.subscribe(deleted_identity_flow.id) - :ok = PubSub.Flow.subscribe(deleted_group_flow.id) - :ok = PubSub.Actor.Memberships.subscribe(actor.id) - :ok = PubSub.Actor.Memberships.subscribe(other_actor.id) - :ok = PubSub.Actor.Memberships.subscribe(deleted_membership.actor_id) - :ok = PubSub.Actor.Policies.subscribe(actor.id) - :ok = PubSub.Actor.Policies.subscribe(other_actor.id) - :ok = PubSub.ActorGroup.Policies.subscribe(deleted_group.id) - WorkOSDirectory.override_base_url("http://localhost:#{bypass.port}") WorkOSDirectory.mock_list_directories_endpoint(bypass) WorkOSDirectory.mock_list_users_endpoint(bypass, users) @@ -473,65 +436,12 @@ defmodule Domain.Auth.Adapters.JumpCloud.Jobs.SyncDirectoryTest do group_id: group.id ) - # Broadcasts allow_access for it - policy_id = policy.id - group_id = group.id - resource_id = policy.resource_id - - Events.Hooks.ActorGroupMemberships.on_insert(%{ - "actor_id" => actor.id, - "group_id" => group.id - }) - - assert_receive {:allow_access, ^policy_id, ^group_id, ^resource_id} - # Deletes membership that is not found on IdP end refute Repo.get_by(Domain.Actors.Membership, group_id: deleted_group.id) # Signs out users which identity has been deleted deleted_identity_token = Repo.reload(deleted_identity_token) assert deleted_identity_token.deleted_at - - # Deleted group deletes all policies and broadcasts reject access events for them - policy_id = deleted_policy.id - group_id = deleted_group.id - resource_id = deleted_policy.resource_id - - # Simulate the WAL events - Events.Hooks.ActorGroupMemberships.on_delete(%{ - "account_id" => deleted_group.account_id, - "actor_id" => actor.id, - "group_id" => deleted_group.id - }) - - Events.Hooks.Policies.on_delete(%{ - "id" => policy_id, - "account_id" => deleted_policy.account_id, - "actor_group_id" => group_id, - "resource_id" => resource_id - }) - - assert_receive {:reject_access, ^policy_id, ^group_id, ^resource_id} - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - # Deleted policies expire all flows authorized by them - flow_id = deleted_group_flow.id - client_id = deleted_group_flow.client_id - resource_id = deleted_group_flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - - # Expires flows for signed out user - flow_id = deleted_identity_flow.id - client_id = deleted_identity_flow.client_id - resource_id = deleted_identity_flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - - # Should not do anything else - refute_received {:allow_access, _policy_id, _group_id, _resource_id} - refute_received {:reject_access, _policy_id, _group_id, _resource_id} end test "resurrects deleted identities that reappear on the next sync", %{ diff --git a/elixir/apps/domain/test/domain/auth/adapters/microsoft_entra/jobs/sync_directory_test.exs b/elixir/apps/domain/test/domain/auth/adapters/microsoft_entra/jobs/sync_directory_test.exs index 7447fe8a9..274915530 100644 --- a/elixir/apps/domain/test/domain/auth/adapters/microsoft_entra/jobs/sync_directory_test.exs +++ b/elixir/apps/domain/test/domain/auth/adapters/microsoft_entra/jobs/sync_directory_test.exs @@ -1,6 +1,6 @@ defmodule Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectoryTest do use Domain.DataCase, async: true - alias Domain.{Auth, Actors, Events, PubSub} + alias Domain.{Auth, Actors} alias Domain.Mocks.MicrosoftEntraDirectory import Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectory @@ -411,20 +411,6 @@ defmodule Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectoryTest do identity: deleted_identity ) - deleted_identity_client = - Fixtures.Clients.create_client( - account: account, - actor: other_actor, - identity: deleted_identity - ) - - deleted_identity_flow = - Fixtures.Flows.create_flow( - account: account, - client: deleted_identity_client, - token_id: deleted_identity_token.id - ) - group = Fixtures.Actors.create_group( account: account, @@ -446,33 +432,11 @@ defmodule Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectoryTest do provider_identifier: "G:DELETED_GROUP_ID!" ) - policy = Fixtures.Policies.create_policy(account: account, actor_group: group) - - deleted_policy = - Fixtures.Policies.create_policy(account: account, actor_group: deleted_group) - - deleted_group_flow = - Fixtures.Flows.create_flow( - account: account, - actor_group: deleted_group, - resource_id: deleted_policy.resource_id, - policy: deleted_policy - ) - Fixtures.Actors.create_membership(account: account, actor: actor) Fixtures.Actors.create_membership(account: account, actor: actor, group: group) deleted_membership = Fixtures.Actors.create_membership(account: account, group: group) Fixtures.Actors.create_membership(account: account, actor: actor, group: deleted_group) - :ok = PubSub.Flow.subscribe(deleted_identity_flow.id) - :ok = PubSub.Flow.subscribe(deleted_group_flow.id) - :ok = PubSub.Actor.Memberships.subscribe(actor.id) - :ok = PubSub.Actor.Memberships.subscribe(other_actor.id) - :ok = PubSub.Actor.Memberships.subscribe(deleted_membership.actor_id) - :ok = PubSub.Actor.Policies.subscribe(actor.id) - :ok = PubSub.Actor.Policies.subscribe(other_actor.id) - :ok = PubSub.ActorGroup.Policies.subscribe(deleted_group.id) - MicrosoftEntraDirectory.mock_groups_list_endpoint( bypass, 200, @@ -536,18 +500,6 @@ defmodule Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectoryTest do # Created membership for a member of existing group assert Repo.get_by(Domain.Actors.Membership, actor_id: other_actor.id, group_id: group.id) - # Broadcasts allow_access for it - policy_id = policy.id - group_id = group.id - resource_id = policy.resource_id - - Events.Hooks.ActorGroupMemberships.on_insert(%{ - "actor_id" => actor.id, - "group_id" => group.id - }) - - assert_receive {:allow_access, ^policy_id, ^group_id, ^resource_id} - # Deletes membership that is not found on IdP end refute Repo.get_by( Domain.Actors.Membership, @@ -558,47 +510,6 @@ defmodule Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectoryTest do # Signs out users which identity has been deleted deleted_identity_token = Repo.reload(deleted_identity_token) assert deleted_identity_token.deleted_at - - # Deleted group deletes all policies and broadcasts reject access events for them - policy_id = deleted_policy.id - group_id = deleted_group.id - resource_id = deleted_policy.resource_id - - # Simulate WAL events - Events.Hooks.ActorGroupMemberships.on_delete(%{ - "account_id" => deleted_identity.account_id, - "actor_id" => deleted_identity.actor_id, - "group_id" => deleted_group.id - }) - - Events.Hooks.Policies.on_delete(%{ - "id" => policy_id, - "account_id" => deleted_policy.account_id, - "actor_group_id" => group_id, - "resource_id" => resource_id - }) - - assert_receive {:reject_access, ^policy_id, ^group_id, ^resource_id} - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - # Deleted policies expire all flows authorized by them - flow_id = deleted_group_flow.id - client_id = deleted_group_flow.client_id - resource_id = deleted_group_flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - - # Expires flows for signed out user - flow_id = deleted_identity_flow.id - client_id = deleted_identity_flow.client_id - resource_id = deleted_identity_flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - - # Should not do anything else - refute_received {:allow_access, _policy_id, _group_id, _resource_id} - refute_received {:reject_access, _policy_id, _group_id, _resource_id} end test "stops the sync retries on 401 error on the provider", %{ diff --git a/elixir/apps/domain/test/domain/auth/adapters/okta/jobs/sync_directory_test.exs b/elixir/apps/domain/test/domain/auth/adapters/okta/jobs/sync_directory_test.exs index 6971cdb23..e7dc275e9 100644 --- a/elixir/apps/domain/test/domain/auth/adapters/okta/jobs/sync_directory_test.exs +++ b/elixir/apps/domain/test/domain/auth/adapters/okta/jobs/sync_directory_test.exs @@ -1,6 +1,6 @@ defmodule Domain.Auth.Adapters.Okta.Jobs.SyncDirectoryTest do use Domain.DataCase, async: true - alias Domain.{Auth, Actors, Events, PubSub} + alias Domain.{Auth, Actors} alias Domain.Mocks.OktaDirectory import Domain.Auth.Adapters.Okta.Jobs.SyncDirectory @@ -655,20 +655,6 @@ defmodule Domain.Auth.Adapters.Okta.Jobs.SyncDirectoryTest do identity: deleted_identity ) - deleted_identity_client = - Fixtures.Clients.create_client( - account: account, - actor: other_actor, - identity: deleted_identity - ) - - deleted_identity_flow = - Fixtures.Flows.create_flow( - account: account, - client: deleted_identity_client, - token_id: deleted_identity_token.id - ) - group = Fixtures.Actors.create_group( account: account, @@ -690,33 +676,11 @@ defmodule Domain.Auth.Adapters.Okta.Jobs.SyncDirectoryTest do provider_identifier: "G:DELETED_GROUP_ID!" ) - policy = Fixtures.Policies.create_policy(account: account, actor_group: group) - - deleted_policy = - Fixtures.Policies.create_policy(account: account, actor_group: deleted_group) - - deleted_group_flow = - Fixtures.Flows.create_flow( - account: account, - actor_group: deleted_group, - resource_id: deleted_policy.resource_id, - policy: deleted_policy - ) - Fixtures.Actors.create_membership(account: account, actor: actor) Fixtures.Actors.create_membership(account: account, actor: actor, group: group) deleted_membership = Fixtures.Actors.create_membership(account: account, group: group) Fixtures.Actors.create_membership(account: account, actor: actor, group: deleted_group) - :ok = PubSub.Flow.subscribe(deleted_identity_flow.id) - :ok = PubSub.Flow.subscribe(deleted_group_flow.id) - :ok = PubSub.Actor.Memberships.subscribe(actor.id) - :ok = PubSub.Actor.Memberships.subscribe(other_actor.id) - :ok = PubSub.Actor.Memberships.subscribe(deleted_membership.actor_id) - :ok = PubSub.Actor.Policies.subscribe(actor.id) - :ok = PubSub.Actor.Policies.subscribe(other_actor.id) - :ok = PubSub.ActorGroup.Policies.subscribe(deleted_group.id) - OktaDirectory.mock_groups_list_endpoint(bypass, 200, Jason.encode!(groups)) OktaDirectory.mock_users_list_endpoint(bypass, 200, Jason.encode!(users)) @@ -771,18 +735,6 @@ defmodule Domain.Auth.Adapters.Okta.Jobs.SyncDirectoryTest do # Creates membership for a member of existing group assert Repo.get_by(Domain.Actors.Membership, actor_id: other_actor.id, group_id: group.id) - # Broadcasts allow_access for it - policy_id = policy.id - group_id = group.id - resource_id = policy.resource_id - - Events.Hooks.ActorGroupMemberships.on_insert(%{ - "actor_id" => other_actor.id, - "group_id" => group.id - }) - - assert_receive {:allow_access, ^policy_id, ^group_id, ^resource_id} - # Deletes membership that is not found on IdP end refute Repo.get_by(Domain.Actors.Membership, actor_id: deleted_membership.actor_id, @@ -792,48 +744,6 @@ defmodule Domain.Auth.Adapters.Okta.Jobs.SyncDirectoryTest do # Signs out users which identity has been deleted deleted_identity_token = Repo.reload(deleted_identity_token) assert deleted_identity_token.deleted_at - - # Deleted group deletes all policies and broadcasts reject access events for them - policy_id = deleted_policy.id - group_id = deleted_group.id - resource_id = deleted_policy.resource_id - account_id = deleted_policy.account_id - - # Simulate WAL events - Events.Hooks.ActorGroupMemberships.on_delete(%{ - "account_id" => deleted_membership.account_id, - "actor_id" => actor.id, - "group_id" => deleted_group.id - }) - - Events.Hooks.Policies.on_delete(%{ - "id" => policy_id, - "actor_group_id" => group_id, - "resource_id" => resource_id, - "account_id" => account_id - }) - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - assert_receive {:reject_access, ^policy_id, ^group_id, ^resource_id} - - # Deleted policies expire all flows authorized by them - flow_id = deleted_group_flow.id - client_id = deleted_group_flow.client_id - resource_id = deleted_group_flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - - # Expires flows for signed out user - flow_id = deleted_identity_flow.id - client_id = deleted_identity_flow.client_id - resource_id = deleted_identity_flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - - # Should not do anything else - refute_received {:allow_access, _policy_id, _group_id, _resource_id} - refute_received {:reject_access, _policy_id, _group_id, _resource_id} end test "resurrects deleted identities that reappear on the next sync", %{ diff --git a/elixir/apps/domain/test/domain/auth_test.exs b/elixir/apps/domain/test/domain/auth_test.exs index a7193da75..86941b266 100644 --- a/elixir/apps/domain/test/domain/auth_test.exs +++ b/elixir/apps/domain/test/domain/auth_test.exs @@ -1054,31 +1054,6 @@ defmodule Domain.AuthTest do assert token.deleted_at end - test "expires provider flows", %{ - account: account, - provider: provider, - identity: identity, - subject: subject - } do - client = Fixtures.Clients.create_client(account: account, identity: identity) - - flow = - Fixtures.Flows.create_flow( - account: account, - subject: subject, - client: client - ) - - :ok = Domain.PubSub.Flow.subscribe(flow.id) - - flow_id = flow.id - client_id = client.id - resource_id = flow.resource_id - - assert {:ok, _provider} = disable_provider(provider, subject) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - end - test "returns error when trying to disable the last provider", %{ subject: subject, provider: provider @@ -1279,31 +1254,6 @@ defmodule Domain.AuthTest do assert actor_group.deleted_at end - test "expires provider flows", %{ - account: account, - provider: provider, - identity: identity, - subject: subject - } do - client = Fixtures.Clients.create_client(account: account, identity: identity) - - flow = - Fixtures.Flows.create_flow( - account: account, - subject: subject, - client: client - ) - - :ok = Domain.PubSub.Flow.subscribe(flow.id) - - flow_id = flow.id - client_id = client.id - resource_id = flow.resource_id - - assert {:ok, _provider} = delete_provider(provider, subject) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - end - test "returns error when trying to delete the last provider", %{ subject: subject, provider: provider @@ -1878,22 +1828,6 @@ defmodule Domain.AuthTest do identity: deleted_identity ) - deleted_identity_client = - Fixtures.Clients.create_client( - account: account, - actor: deleted_identity_actor, - identity: deleted_identity - ) - - deleted_identity_flow = - Fixtures.Flows.create_flow( - account: account, - client: deleted_identity_client, - token_id: deleted_identity_token.id - ) - - :ok = Domain.PubSub.Flow.subscribe(deleted_identity_flow.id) - for n <- 1..4 do Fixtures.Auth.create_identity( account: account, @@ -1958,12 +1892,6 @@ defmodule Domain.AuthTest do # Signs out users which identity has been deleted deleted_identity_token = Repo.reload(deleted_identity_token) assert deleted_identity_token.deleted_at - - # Expires flows for signed out user - flow_id = deleted_identity_flow.id - client_id = deleted_identity_flow.client_id - resource_id = deleted_identity_flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} end test "circuit breaker prevents mass deletions of identities", %{ @@ -2789,33 +2717,6 @@ defmodule Domain.AuthTest do assert token.deleted_at end - test "expires all flows created using deleted tokens", %{ - account: account, - actor: actor, - identity: identity, - subject: subject - } do - client = Fixtures.Clients.create_client(account: account, identity: identity) - - flow = - Fixtures.Flows.create_flow( - account: account, - identity: identity, - actor: actor, - subject: subject, - client: client - ) - - :ok = Domain.PubSub.Flow.subscribe(flow.id) - - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - - assert delete_identities_for(actor, subject) == :ok - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - end - test "does not remove identities that belong to another actor", %{ account: account, provider: provider, diff --git a/elixir/apps/domain/test/domain/clients_test.exs b/elixir/apps/domain/test/domain/clients_test.exs index be184c264..ebfa8c38b 100644 --- a/elixir/apps/domain/test/domain/clients_test.exs +++ b/elixir/apps/domain/test/domain/clients_test.exs @@ -1,7 +1,7 @@ defmodule Domain.ClientsTest do use Domain.DataCase, async: true import Domain.Clients - alias Domain.{Clients, PubSub} + alias Domain.Clients setup do account = Fixtures.Accounts.create_account() @@ -119,8 +119,6 @@ defmodule Domain.ClientsTest do {:ok, _} = Clients.Presence.Account.track(client.account_id, client.id) {:ok, _} = Clients.Presence.Actor.track(client.actor_id, client.id) - :ok = PubSub.Client.subscribe(client.id) - :ok = PubSub.Account.Clients.subscribe(client.account_id) assert {:ok, client} = fetch_client_by_id(client.id, subject, preload: [:online?]) assert client.online? == true @@ -228,8 +226,7 @@ defmodule Domain.ClientsTest do {:ok, _} = Clients.Presence.Account.track(client.account_id, client.id) {:ok, _} = Clients.Presence.Actor.track(client.actor_id, client.id) - :ok = PubSub.Client.subscribe(client.id) - :ok = PubSub.Account.Clients.subscribe(client.account_id) + assert client = fetch_client_by_id!(client.id, preload: [:online?]) assert client.online? == true end @@ -290,8 +287,7 @@ defmodule Domain.ClientsTest do {:ok, _} = Clients.Presence.Account.track(client.account_id, client.id) {:ok, _} = Clients.Presence.Actor.track(client.actor_id, client.id) - :ok = PubSub.Client.subscribe(client.id) - :ok = PubSub.Account.Clients.subscribe(client.account_id) + assert {:ok, [client], _metadata} = list_clients(subject, preload: [:online?]) assert client.online? == true end @@ -1077,32 +1073,6 @@ defmodule Domain.ClientsTest do assert is_nil(client.verified_by_subject) end - test "expires flows for the unverified client", %{ - account: account, - admin_actor: actor, - admin_subject: subject - } do - client = Fixtures.Clients.create_client(actor: actor) - - flow = - Fixtures.Flows.create_flow( - account: account, - actor: actor, - client: client, - subject: subject - ) - - :ok = Domain.PubSub.Flow.subscribe(flow.id) - - assert {:ok, client} = verify_client(client, subject) - assert {:ok, _client} = remove_client_verification(client, subject) - - flow_id = flow.id - client_id = client.id - resource_id = flow.resource_id - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - end - test "returns error when subject has no permission to verify clients", %{ admin_actor: actor, admin_subject: subject diff --git a/elixir/apps/domain/test/domain/events/hooks/accounts_test.exs b/elixir/apps/domain/test/domain/events/hooks/accounts_test.exs index e172a8c37..8f584d6e6 100644 --- a/elixir/apps/domain/test/domain/events/hooks/accounts_test.exs +++ b/elixir/apps/domain/test/domain/events/hooks/accounts_test.exs @@ -1,92 +1,84 @@ defmodule Domain.Events.Hooks.AccountsTest do use Domain.DataCase, async: true + alias Domain.Accounts import Domain.Events.Hooks.Accounts - setup do - %{old_data: %{}, data: %{}} - end - describe "insert/1" do - test "returns :ok", %{data: data} do - assert :ok == on_insert(data) + test "returns :ok" do + assert :ok == on_insert(%{}) end end describe "update/2" do - test "disconnects gateways if slug changes" do - account = Fixtures.Accounts.create_account() - gateway = Fixtures.Gateways.create_gateway(account: account) - :ok = Domain.Gateways.Presence.connect(gateway) + test "sends delete when account is disabled" do + account_id = "00000000-0000-0000-0000-000000000001" - old_data = %{"slug" => "old"} - data = %{"slug" => "new", "id" => account.id} - - assert :ok == on_update(old_data, data) - - assert_receive "disconnect" - end - - test "sends :config_changed if config changes" do - account = Fixtures.Accounts.create_account() - gateway = Fixtures.Gateways.create_gateway(account: account) - - :ok = Domain.PubSub.Account.subscribe(account.id) - :ok = Domain.Gateways.Presence.connect(gateway) + :ok = Domain.PubSub.Account.subscribe(account_id) old_data = %{ - "id" => account.id, - "config" => %{"search_domain" => "old_value", "clients_upstream_dns" => []} + "id" => account_id, + "disabled_at" => nil } data = %{ - "id" => account.id, - "config" => %{ - "search_domain" => "new_value", - "clients_upstream_dns" => [%{"protocol" => "ip_port", "address" => "8.8.8.8"}] - } + "id" => account_id, + "disabled_at" => "2023-10-01T00:00:00Z" } assert :ok == on_update(old_data, data) - assert_receive :config_changed - refute_receive "disconnect" + assert_receive {:deleted, %Accounts.Account{} = account} + + assert account.id == account_id end - test "does not send :config_changed if config does not change" do - account = Fixtures.Accounts.create_account() - - :ok = Domain.PubSub.Account.subscribe(account.id) + test "sends delete when soft-deleted" do + account_id = "00000000-0000-0000-0000-000000000002" + :ok = Domain.PubSub.Account.subscribe(account_id) old_data = %{ - "id" => account.id, - "config" => %{"search_domain" => "old_value", "clients_upstream_dns" => []} + "id" => account_id, + "deleted_at" => nil } data = %{ - "id" => account.id, - "config" => %{"search_domain" => "old_value", "clients_upstream_dns" => []} + "id" => account_id, + "deleted_at" => "2023-10-01T00:00:00Z" } assert :ok == on_update(old_data, data) - refute_receive :config_changed - end + assert_receive {:deleted, %Accounts.Account{} = account} - test "sends disconnect to clients if account is disabled" do - account_id = Fixtures.Accounts.create_account().id - - old_data = %{"id" => account_id, "disabled_at" => nil} - data = %{"id" => account_id, "disabled_at" => DateTime.utc_now()} - - :ok = Domain.PubSub.Account.Clients.subscribe(account_id) - - assert :ok == on_update(old_data, data) - - assert_receive "disconnect" + assert account.id == account_id end end describe "delete/1" do - test "returns :ok", %{data: data} do - assert :ok == on_delete(data) + test "delete broadcasts deleted account" do + account_id = "00000000-0000-0000-0000-000000000003" + :ok = Domain.PubSub.Account.subscribe(account_id) + + old_data = %{ + "id" => account_id, + "deleted_at" => "2023-10-01T00:00:00Z" + } + + assert :ok == on_delete(old_data) + assert_receive {:deleted, %Accounts.Account{} = account} + assert account.id == account_id + assert account.deleted_at == ~U[2023-10-01 00:00:00.000000Z] + end + + test "deletes associated flows when account is deleted" do + account = Fixtures.Accounts.create_account() + flow = Fixtures.Flows.create_flow(account: account) + + old_data = %{ + "id" => account.id, + "deleted_at" => "2023-10-01T00:00:00Z" + } + + assert :ok == on_delete(old_data) + assert Repo.get_by(Domain.Flows.Flow, id: flow.id) == nil end end end diff --git a/elixir/apps/domain/test/domain/events/hooks/actor_group_memberships_test.exs b/elixir/apps/domain/test/domain/events/hooks/actor_group_memberships_test.exs index 8c639112f..14a7fdb9e 100644 --- a/elixir/apps/domain/test/domain/events/hooks/actor_group_memberships_test.exs +++ b/elixir/apps/domain/test/domain/events/hooks/actor_group_memberships_test.exs @@ -1,37 +1,34 @@ defmodule Domain.Events.Hooks.ActorGroupMembershipsTest do use API.ChannelCase, async: true import Domain.Events.Hooks.ActorGroupMemberships + alias Domain.Actors alias Domain.PubSub - setup do - %{old_data: %{}, data: %{}} - end - describe "insert/1" do - test "returns :ok" do - actor_id = "#{Ecto.UUID.generate()}" - group_id = "#{Ecto.UUID.generate()}" + test "broadcasts membership" do + account_id = "00000000-0000-0000-0000-000000000001" + actor_id = "00000000-0000-0000-0000-000000000002" + group_id = "00000000-0000-0000-0000-000000000003" + + :ok = PubSub.Account.subscribe(account_id) data = %{ + "account_id" => account_id, "actor_id" => actor_id, "group_id" => group_id } - :ok = PubSub.Actor.Memberships.subscribe(actor_id) - assert :ok == on_insert(data) - - # TODO: WAL - # Remove this when direct broadcast is implement - Process.sleep(100) - - assert_receive {:create_membership, ^actor_id, ^group_id} + assert_receive {:created, %Actors.Membership{} = membership} + assert membership.account_id == account_id + assert membership.actor_id == actor_id + assert membership.group_id == group_id end end describe "update/2" do - test "returns :ok", %{old_data: old_data, data: data} do - assert :ok == on_update(old_data, data) + test "returns :ok" do + assert :ok == on_update(%{}, %{}) end end @@ -40,80 +37,53 @@ defmodule Domain.Events.Hooks.ActorGroupMembershipsTest do account = Fixtures.Accounts.create_account() actor_group = Fixtures.Actors.create_group(account: account) actor = Fixtures.Actors.create_actor(account: account, type: :account_admin_user) - Fixtures.Actors.create_membership(account: account, group: actor_group, actor: actor) - identity = Fixtures.Auth.create_identity(account: account, actor: actor) - subject = Fixtures.Auth.create_subject(identity: identity) - client = Fixtures.Clients.create_client(subject: subject) - resource = Fixtures.Resources.create_resource(account: account) - - policy = - Fixtures.Policies.create_policy( - account: account, - resource: resource, - actor_group: actor_group - ) - - {:ok, _reply, socket} = - API.Client.Socket - |> socket("client:#{client.id}", %{ - opentelemetry_ctx: OpenTelemetry.Ctx.new(), - opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"), - client: client, - subject: subject, - turn_salt: "test_salt" - }) - |> subscribe_and_join(API.Client.Channel, "client") + membership = + Fixtures.Actors.create_membership(account: account, group: actor_group, actor: actor) %{ account: account, actor_group: actor_group, actor: actor, - identity: identity, - subject: subject, - client: client, - resource: resource, - policy: policy, - socket: socket + membership: membership } end - test "returns :ok" do - actor_id = "#{Ecto.UUID.generate()}" - group_id = "#{Ecto.UUID.generate()}" + test "broadcasts deleted membership" do + account_id = "00000000-0000-0000-0000-000000000001" + :ok = PubSub.Account.subscribe(account_id) - data = %{ - "account_id" => "#{Ecto.UUID.generate()}", - "actor_id" => actor_id, - "group_id" => group_id + old_data = %{ + "id" => "00000000-0000-0000-0000-000000000000", + "account_id" => "00000000-0000-0000-0000-000000000001", + "actor_id" => "00000000-0000-0000-0000-000000000002", + "group_id" => "00000000-0000-0000-0000-000000000003" } - :ok = PubSub.Actor.Memberships.subscribe(actor_id) + assert :ok == on_delete(old_data) - assert :ok == on_delete(data) - assert_receive {:delete_membership, ^actor_id, ^group_id} + assert_receive {:deleted, %Actors.Membership{} = membership} + assert membership.id == "00000000-0000-0000-0000-000000000000" + assert membership.account_id == "00000000-0000-0000-0000-000000000001" + assert membership.actor_id == "00000000-0000-0000-0000-000000000002" + assert membership.group_id == "00000000-0000-0000-0000-000000000003" end - test "client channel pushes \"resource_deleted\" when affected membership is deleted", %{ - actor: actor, - subject: subject, - actor_group: actor_group - } do - assert_push "init", %{} - # TODO: WAL - # This is needed because the :reject_access received in the client channel re-fetches allowed resources for this client. - # Remove this when that's cleaned up. - {:ok, _actor} = Domain.Actors.update_actor(actor, %{memberships: []}, subject) + test "deletes flows for membership", %{account: account, membership: membership} do + flow = Fixtures.Flows.create_flow(account: account, actor_group_membership: membership) + unrelated_flow = Fixtures.Flows.create_flow(account: account) - assert :ok = - on_delete(%{ - "account_id" => actor.account_id, - "actor_id" => actor.id, - "group_id" => actor_group.id - }) + old_data = %{ + "id" => membership.id, + "account_id" => membership.account_id, + "actor_id" => membership.actor_id, + "group_id" => membership.group_id + } - assert_push "resource_deleted", _payload - refute_push "resource_created_or_updated", _payload + assert ^flow = Repo.get_by(Domain.Flows.Flow, actor_group_membership_id: membership.id) + assert :ok == on_delete(old_data) + assert nil == Repo.get_by(Domain.Flows.Flow, actor_group_membership_id: membership.id) + assert ^unrelated_flow = Repo.get_by(Domain.Flows.Flow, id: unrelated_flow.id) end end end diff --git a/elixir/apps/domain/test/domain/events/hooks/actor_groups_test.exs b/elixir/apps/domain/test/domain/events/hooks/actor_groups_test.exs deleted file mode 100644 index 7cb6c71f6..000000000 --- a/elixir/apps/domain/test/domain/events/hooks/actor_groups_test.exs +++ /dev/null @@ -1,26 +0,0 @@ -defmodule Domain.Events.Hooks.ActorGroupsTest do - use ExUnit.Case, async: true - import Domain.Events.Hooks.ActorGroups - - setup do - %{old_data: %{}, data: %{}} - end - - describe "insert/1" do - test "returns :ok", %{data: data} do - assert :ok == on_insert(data) - end - end - - describe "update/2" do - test "returns :ok", %{old_data: old_data, data: data} do - assert :ok == on_update(old_data, data) - end - end - - describe "delete/1" do - test "returns :ok", %{data: data} do - assert :ok == on_delete(data) - end - end -end diff --git a/elixir/apps/domain/test/domain/events/hooks/actors_test.exs b/elixir/apps/domain/test/domain/events/hooks/actors_test.exs deleted file mode 100644 index 635aa8da2..000000000 --- a/elixir/apps/domain/test/domain/events/hooks/actors_test.exs +++ /dev/null @@ -1,26 +0,0 @@ -defmodule Domain.Events.Hooks.ActorsTest do - use ExUnit.Case, async: true - import Domain.Events.Hooks.Actors - - setup do - %{old_data: %{}, data: %{}} - end - - describe "insert/1" do - test "returns :ok", %{data: data} do - assert :ok == on_insert(data) - end - end - - describe "update/2" do - test "returns :ok", %{old_data: old_data, data: data} do - assert :ok == on_update(old_data, data) - end - end - - describe "delete/1" do - test "returns :ok", %{data: data} do - assert :ok == on_delete(data) - end - end -end diff --git a/elixir/apps/domain/test/domain/events/hooks/auth_identities_test.exs b/elixir/apps/domain/test/domain/events/hooks/auth_identities_test.exs deleted file mode 100644 index 60a7571b3..000000000 --- a/elixir/apps/domain/test/domain/events/hooks/auth_identities_test.exs +++ /dev/null @@ -1,26 +0,0 @@ -defmodule Domain.Events.Hooks.AuthIdentitiesTest do - use ExUnit.Case, async: true - import Domain.Events.Hooks.AuthIdentities - - setup do - %{old_data: %{}, data: %{}} - end - - describe "insert/1" do - test "returns :ok", %{data: data} do - assert :ok == on_insert(data) - end - end - - describe "update/2" do - test "returns :ok", %{old_data: old_data, data: data} do - assert :ok == on_update(old_data, data) - end - end - - describe "delete/1" do - test "returns :ok", %{data: data} do - assert :ok == on_delete(data) - end - end -end diff --git a/elixir/apps/domain/test/domain/events/hooks/auth_providers_test.exs b/elixir/apps/domain/test/domain/events/hooks/auth_providers_test.exs deleted file mode 100644 index e75863c46..000000000 --- a/elixir/apps/domain/test/domain/events/hooks/auth_providers_test.exs +++ /dev/null @@ -1,26 +0,0 @@ -defmodule Domain.Events.Hooks.AuthProvidersTest do - use ExUnit.Case, async: true - import Domain.Events.Hooks.AuthProviders - - setup do - %{old_data: %{}, data: %{}} - end - - describe "insert/1" do - test "returns :ok", %{data: data} do - assert :ok == on_insert(data) - end - end - - describe "update/2" do - test "returns :ok", %{old_data: old_data, data: data} do - assert :ok == on_update(old_data, data) - end - end - - describe "delete/1" do - test "returns :ok", %{data: data} do - assert :ok == on_delete(data) - end - end -end diff --git a/elixir/apps/domain/test/domain/events/hooks/clients_test.exs b/elixir/apps/domain/test/domain/events/hooks/clients_test.exs index b6402ec9a..2f88af771 100644 --- a/elixir/apps/domain/test/domain/events/hooks/clients_test.exs +++ b/elixir/apps/domain/test/domain/events/hooks/clients_test.exs @@ -3,81 +3,90 @@ defmodule Domain.Events.Hooks.ClientsTest do import Domain.Events.Hooks.Clients alias Domain.{Clients, PubSub} - setup do - %{old_data: %{}, data: %{}} - end - describe "insert/1" do - test "returns :ok", %{data: data} do - assert :ok == on_insert(data) + test "returns :ok" do + assert :ok == on_insert(%{}) end end describe "update/2" do - test "soft-delete broadcasts disconnect" do + test "soft-delete broadcasts deleted client" do client = Fixtures.Clients.create_client() - :ok = Clients.Presence.connect(client) + :ok = PubSub.Account.subscribe(client.account_id) - old_data = %{"id" => client.id, "deleted_at" => nil} - data = %{"id" => client.id, "deleted_at" => DateTime.utc_now()} + old_data = %{"id" => client.id, "deleted_at" => nil, "account_id" => client.account_id} + + data = %{ + "id" => client.id, + "deleted_at" => DateTime.utc_now(), + "account_id" => client.account_id + } assert :ok == on_update(old_data, data) - - assert_receive "disconnect" - refute_receive :updated + assert_receive {:deleted, %Clients.Client{} = deleted_client} + assert deleted_client.id == client.id end - test "update broadcasts :update" do - client = Fixtures.Clients.create_client() - :ok = Clients.Presence.connect(client) + test "update broadcasts updated client" do + account = Fixtures.Accounts.create_account() + client = Fixtures.Clients.create_client(account: account) + :ok = PubSub.Account.subscribe(client.account_id) - old_data = %{"id" => client.id, "name" => "Old Client"} - data = %{"id" => client.id, "name" => "Updated Client"} + old_data = %{"id" => client.id, "name" => "Old Name", "account_id" => client.account_id} + data = %{"id" => client.id, "name" => "New Name", "account_id" => client.account_id} assert :ok == on_update(old_data, data) - assert_receive {:updated, %Clients.Client{} = updated_client} - assert updated_client.id == client.id - refute_receive "disconnect" + assert_receive {:updated, %Clients.Client{} = old_client, %Clients.Client{} = new_client} + assert old_client.name == "Old Name" + assert new_client.name == "New Name" + assert new_client.id == client.id + end + + test "update unverifies client and deletes associated flows" do + account = Fixtures.Accounts.create_account() + client = Fixtures.Clients.create_client(account: account, verified_at: DateTime.utc_now()) + :ok = PubSub.Account.subscribe(client.account_id) + + old_data = %{ + "id" => client.id, + "verified_at" => "2023-10-01T00:00:00Z", + "account_id" => client.account_id + } + + data = %{"id" => client.id, "verified_at" => nil, "account_id" => client.account_id} + + assert flow = Fixtures.Flows.create_flow(client: client, account: account) + assert :ok == on_update(old_data, data) + assert_receive {:updated, %Clients.Client{}, %Clients.Client{} = new_client} + assert is_nil(new_client.verified_at) + assert new_client.id == client.id + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) end end describe "delete/1" do - test "broadcasts disconnect" do - client = Fixtures.Clients.create_client() - :ok = Clients.Presence.connect(client) + test "broadcasts deleted client" do + account = Fixtures.Accounts.create_account() + client = Fixtures.Clients.create_client(account: account) + :ok = PubSub.Account.subscribe(client.account_id) - old_data = %{"id" => client.id} + old_data = %{"id" => client.id, "account_id" => client.account_id} assert :ok == on_delete(old_data) - - assert_receive "disconnect" - refute_receive :updated + assert_receive {:deleted, %Clients.Client{} = deleted_client} + assert deleted_client.id == client.id end - end - describe "connect/1" do - test "tracks client presence and subscribes to topics" do - client = Fixtures.Clients.create_client() - assert :ok == Clients.Presence.connect(client) + test "deletes associated flows" do + account = Fixtures.Accounts.create_account() + client = Fixtures.Clients.create_client(account: account) - assert Clients.Presence.Account.get(client.account_id, client.id) - assert Clients.Presence.Actor.get(client.actor_id, client.id) + old_data = %{"id" => client.id, "account_id" => client.account_id} - PubSub.Account.Clients.broadcast(client.account_id, :test_event) - - assert_receive :test_event - end - end - - describe "broadcast/2" do - test "broadcasts payload to client topic" do - client = Fixtures.Clients.create_client() - :ok = Clients.Presence.connect(client) - - assert :ok == PubSub.Client.broadcast(client.id, :updated) - - assert_receive :updated + assert flow = Fixtures.Flows.create_flow(client: client, account: account) + assert :ok == on_delete(old_data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) end end end diff --git a/elixir/apps/domain/test/domain/events/hooks/flows_test.exs b/elixir/apps/domain/test/domain/events/hooks/flows_test.exs new file mode 100644 index 000000000..f981c2aac --- /dev/null +++ b/elixir/apps/domain/test/domain/events/hooks/flows_test.exs @@ -0,0 +1,49 @@ +defmodule Domain.Events.Hooks.FlowsTest do + use Domain.DataCase, async: true + import Domain.Events.Hooks.Flows + alias Domain.Flows + + describe "insert/1" do + test "returns :ok" do + assert :ok == on_insert(%{}) + end + end + + describe "update/2" do + test "returns :ok" do + assert :ok == on_update(%{}, %{}) + end + end + + describe "delete/1" do + test "delete broadcasts deleted flow" do + :ok = Domain.PubSub.Account.subscribe("00000000-0000-0000-0000-000000000000") + + old_data = %{ + "id" => "00000000-0000-0000-0000-000000000001", + "account_id" => "00000000-0000-0000-0000-000000000000", + "client_id" => "00000000-0000-0000-0000-000000000002", + "gateway_id" => "00000000-0000-0000-0000-000000000003", + "resource_id" => "00000000-0000-0000-0000-000000000004", + "token_id" => "00000000-0000-0000-0000-000000000005", + "actor_group_membership_id" => "00000000-0000-0000-0000-000000000006", + "policy_id" => "00000000-0000-0000-0000-000000000007", + "inserted_at" => "2023-01-01T00:00:00Z" + } + + assert :ok == on_delete(old_data) + + assert_receive {:deleted, %Flows.Flow{} = flow} + + assert flow.id == "00000000-0000-0000-0000-000000000001" + assert flow.account_id == "00000000-0000-0000-0000-000000000000" + assert flow.client_id == "00000000-0000-0000-0000-000000000002" + assert flow.gateway_id == "00000000-0000-0000-0000-000000000003" + assert flow.resource_id == "00000000-0000-0000-0000-000000000004" + assert flow.token_id == "00000000-0000-0000-0000-000000000005" + assert flow.actor_group_membership_id == "00000000-0000-0000-0000-000000000006" + assert flow.policy_id == "00000000-0000-0000-0000-000000000007" + assert flow.inserted_at == ~U[2023-01-01 00:00:00.000000Z] + end + end +end diff --git a/elixir/apps/domain/test/domain/events/hooks/gateway_groups_test.exs b/elixir/apps/domain/test/domain/events/hooks/gateway_groups_test.exs index 4738957aa..72cd98bfc 100644 --- a/elixir/apps/domain/test/domain/events/hooks/gateway_groups_test.exs +++ b/elixir/apps/domain/test/domain/events/hooks/gateway_groups_test.exs @@ -1,26 +1,50 @@ defmodule Domain.Events.Hooks.GatewayGroupsTest do use ExUnit.Case, async: true import Domain.Events.Hooks.GatewayGroups - - setup do - %{old_data: %{}, data: %{}} - end + alias Domain.Gateways describe "insert/1" do - test "returns :ok", %{data: data} do - assert :ok == on_insert(data) + test "returns :ok" do + assert :ok == on_insert(%{}) end end describe "update/2" do - test "returns :ok", %{old_data: old_data, data: data} do + test "returns :ok for soft-deleted gateway group" do + # Deleting a gateway group will delete the associated gateways which + # handles all side effects we need to handle, including removing any + # resources from the client's resource list. + assert :ok = on_delete(%{}) + end + + test "broadcasts updated gateway group" do + account_id = "00000000-0000-0000-0000-000000000000" + + :ok = Domain.PubSub.Account.subscribe(account_id) + + old_data = %{ + "id" => "00000000-0000-0000-0000-000000000001", + "account_id" => account_id, + "name" => "Old Gateway Group", + "deleted_at" => nil + } + + data = Map.put(old_data, "name", "Updated Gateway Group") + assert :ok == on_update(old_data, data) + + assert_receive {:updated, %Gateways.Group{} = old_group, %Gateways.Group{} = new_group} + assert old_group.id == old_data["id"] + assert new_group.name == data["name"] end end describe "delete/1" do - test "returns :ok", %{data: data} do - assert :ok == on_delete(data) + test "returns :ok" do + # Deleting a gateway group will delete the associated gateways which + # handles all side effects we need to handle, including removing any + # resources from the client's resource list. + assert :ok = on_delete(%{}) end end end diff --git a/elixir/apps/domain/test/domain/events/hooks/gateways_test.exs b/elixir/apps/domain/test/domain/events/hooks/gateways_test.exs index 4adcbe0df..05a4d5a41 100644 --- a/elixir/apps/domain/test/domain/events/hooks/gateways_test.exs +++ b/elixir/apps/domain/test/domain/events/hooks/gateways_test.exs @@ -2,52 +2,74 @@ defmodule Domain.Events.Hooks.GatewaysTest do use Domain.DataCase, async: true import Domain.Events.Hooks.Gateways - setup do - %{old_data: %{}, data: %{}} - end - describe "insert/1" do - test "returns :ok", %{data: data} do - assert :ok == on_insert(data) + test "returns :ok" do + assert :ok == on_insert(%{}) end end describe "update/2" do - test "soft-delete broadcasts disconnect" do - gateway = Fixtures.Gateways.create_gateway() + test "soft-delete broadcasts deleted gateway" do + account = Fixtures.Accounts.create_account() + gateway = Fixtures.Gateways.create_gateway(account: account) - old_data = %{"id" => gateway.id, "deleted_at" => nil} - data = %{"id" => gateway.id, "deleted_at" => "2023-10-01T00:00:00Z"} + :ok = Domain.PubSub.Account.subscribe(account.id) - :ok = Domain.Gateways.Presence.connect(gateway) - :ok = on_update(old_data, data) + old_data = %{"id" => gateway.id, "deleted_at" => nil, "account_id" => account.id} + data = Map.put(old_data, "deleted_at", "2023-01-01T00:00:00Z") - assert_receive "disconnect" + assert :ok = on_update(old_data, data) + + assert_receive {:deleted, %Domain.Gateways.Gateway{} = deleted_gateway} + assert deleted_gateway.id == gateway.id end - test "regular update does not broadcast disconnect" do - gateway = Fixtures.Gateways.create_gateway() + test "soft-delete deletes flows" do + account = Fixtures.Accounts.create_account() + gateway = Fixtures.Gateways.create_gateway(account: account) - old_data = %{"id" => gateway.id} - data = %{"id" => gateway.id, "name" => "New Gateway Name"} + old_data = %{"id" => gateway.id, "deleted_at" => nil, "account_id" => account.id} + data = Map.put(old_data, "deleted_at", "2023-01-01T00:00:00Z") - :ok = Domain.Gateways.Presence.connect(gateway) - :ok = on_update(old_data, data) + assert flow = Fixtures.Flows.create_flow(gateway: gateway, account: account) + assert :ok = on_update(old_data, data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) + end - refute_receive "disconnect" + test "update returns :ok" do + assert :ok = on_update(%{}, %{}) end end describe "delete/1" do - test "delete broadcasts disconnect" do - gateway = Fixtures.Gateways.create_gateway() + test "delete broadcasts deleted gateway" do + account = Fixtures.Accounts.create_account() + gateway = Fixtures.Gateways.create_gateway(account: account) - old_data = %{"id" => gateway.id} + :ok = Domain.PubSub.Account.subscribe(account.id) - :ok = Domain.Gateways.Presence.connect(gateway) - :ok = on_delete(old_data) + old_data = %{ + "id" => gateway.id, + "account_id" => account.id, + "name" => "Test Gateway", + "deleted_at" => nil + } - assert_receive "disconnect" + assert :ok = on_delete(old_data) + + assert_receive {:deleted, %Domain.Gateways.Gateway{} = deleted_gateway} + assert deleted_gateway.id == gateway.id + end + + test "deletes flows" do + account = Fixtures.Accounts.create_account() + gateway = Fixtures.Gateways.create_gateway(account: account) + + old_data = %{"id" => gateway.id, "account_id" => account.id, "deleted_at" => nil} + + assert flow = Fixtures.Flows.create_flow(gateway: gateway, account: account) + assert :ok = on_delete(old_data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) end end end diff --git a/elixir/apps/domain/test/domain/events/hooks/policies_test.exs b/elixir/apps/domain/test/domain/events/hooks/policies_test.exs index 2654571b1..fa0fc4423 100644 --- a/elixir/apps/domain/test/domain/events/hooks/policies_test.exs +++ b/elixir/apps/domain/test/domain/events/hooks/policies_test.exs @@ -1,277 +1,280 @@ defmodule Domain.Events.Hooks.PoliciesTest do use Domain.DataCase, async: true import Domain.Events.Hooks.Policies - alias Domain.PubSub + alias Domain.{Policies, PubSub} describe "insert/1" do - test "broadcasts :create_policy and :allow_access" do - policy_id = "policy-123" - account_id = "account-456" - actor_group_id = "group-456" - resource_id = "resource-789" + test "broadcasts created policy" do + account = Fixtures.Accounts.create_account() + policy = Fixtures.Policies.create_policy(account: account) + :ok = PubSub.Account.subscribe(account.id) data = %{ - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id + "id" => policy.id, + "account_id" => account.id, + "actor_group_id" => policy.actor_group_id, + "resource_id" => policy.resource_id, + "disabled_at" => nil, + "deleted_at" => nil } - :ok = PubSub.Policy.subscribe(policy_id) - :ok = PubSub.Account.Policies.subscribe(account_id) - :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id) - assert :ok == on_insert(data) - assert_receive {:create_policy, ^policy_id} - assert_receive {:create_policy, ^policy_id} - assert_receive {:allow_access, ^policy_id, ^actor_group_id, ^resource_id} + assert_receive {:created, %Policies.Policy{} = policy} + + assert policy.id == data["id"] + assert policy.account_id == data["account_id"] + assert policy.actor_group_id == data["actor_group_id"] + assert policy.resource_id == data["resource_id"] end end describe "update/2" do - test "enable: broadcasts :enable_policy and :allow_access" do - policy_id = "policy-123" - account_id = "account-456" - actor_group_id = "group-456" - resource_id = "resource-789" + test "disable policy broadcasts deleted policy and deletes flows" do + account = Fixtures.Accounts.create_account() + resource = Fixtures.Resources.create_resource(account: account) + policy = Fixtures.Policies.create_policy(account: account, resource: resource) + :ok = PubSub.Account.subscribe(account.id) old_data = %{ - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id, - "disabled_at" => "2023-10-01T00:00:00Z" - } - - data = Map.put(old_data, "disabled_at", nil) - - :ok = Domain.PubSub.Policy.subscribe(policy_id) - :ok = Domain.PubSub.Account.Policies.subscribe(account_id) - :ok = Domain.PubSub.ActorGroup.Policies.subscribe(actor_group_id) - - assert :ok == on_update(old_data, data) - assert_receive {:enable_policy, ^policy_id} - assert_receive {:enable_policy, ^policy_id} - assert_receive {:allow_access, ^policy_id, ^actor_group_id, ^resource_id} - end - - test "disable: broadcasts :disable_policy and :reject_access" do - flow = Fixtures.Flows.create_flow() - flow_id = flow.id - client_id = flow.client_id - policy_id = flow.policy_id - account_id = flow.account_id - actor_group_id = "group-456" - resource_id = flow.resource_id - - old_data = %{ - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id, - "disabled_at" => nil + "id" => policy.id, + "account_id" => account.id, + "actor_group_id" => policy.actor_group_id, + "resource_id" => policy.resource_id, + "disabled_at" => nil, + "deleted_at" => nil } data = Map.put(old_data, "disabled_at", "2023-10-01T00:00:00Z") - :ok = PubSub.Flow.subscribe(flow_id) - :ok = PubSub.Policy.subscribe(policy_id) - :ok = PubSub.Account.Policies.subscribe(account_id) - :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id) + # Create a flow that should be deleted + flow = Fixtures.Flows.create_flow(policy: policy, resource: resource, account: account) assert :ok == on_update(old_data, data) - assert_receive {:disable_policy, ^policy_id} - assert_receive {:disable_policy, ^policy_id} - assert_receive {:reject_access, ^policy_id, ^actor_group_id, ^resource_id} + assert_receive {:deleted, %Policies.Policy{} = broadcasted_policy} - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) + assert broadcasted_policy.id == data["id"] + assert broadcasted_policy.account_id == data["account_id"] - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + # Verify flow was deleted + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) end - test "soft-delete: broadcasts :delete_policy and :reject_access" do - flow = Fixtures.Flows.create_flow() - flow_id = flow.id - client_id = flow.client_id - policy_id = flow.policy_id - account_id = flow.account_id - actor_group_id = "group-456" - resource_id = flow.resource_id + test "enable policy broadcasts created policy" do + account = Fixtures.Accounts.create_account() + policy = Fixtures.Policies.create_policy(account: account) + :ok = PubSub.Account.subscribe(account.id) old_data = %{ - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id, + "id" => policy.id, + "account_id" => account.id, + "actor_group_id" => policy.actor_group_id, + "resource_id" => policy.resource_id, + "disabled_at" => "2023-09-01T00:00:00Z", + "deleted_at" => nil + } + + data = Map.put(old_data, "disabled_at", nil) + + assert :ok == on_update(old_data, data) + assert_receive {:created, %Policies.Policy{} = policy} + + assert policy.id == data["id"] + assert policy.account_id == data["account_id"] + assert policy.actor_group_id == data["actor_group_id"] + assert policy.resource_id == data["resource_id"] + end + + test "soft-delete broadcasts deleted policy" do + account = Fixtures.Accounts.create_account() + policy = Fixtures.Policies.create_policy(account: account) + :ok = PubSub.Account.subscribe(account.id) + + old_data = %{ + "id" => policy.id, + "account_id" => account.id, + "actor_group_id" => policy.actor_group_id, + "resource_id" => policy.resource_id, + "disabled_at" => nil, "deleted_at" => nil } data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z") - :ok = PubSub.Flow.subscribe(flow_id) - :ok = PubSub.Policy.subscribe(policy_id) - :ok = PubSub.Account.Policies.subscribe(account_id) - :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id) - assert :ok == on_update(old_data, data) - assert_receive {:delete_policy, ^policy_id} - assert_receive {:delete_policy, ^policy_id} - assert_receive {:reject_access, ^policy_id, ^actor_group_id, ^resource_id} + assert_receive {:deleted, %Policies.Policy{} = policy} - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + assert policy.id == old_data["id"] + assert policy.account_id == old_data["account_id"] + assert policy.actor_group_id == old_data["actor_group_id"] + assert policy.resource_id == old_data["resource_id"] end - test "breaking update: broadcasts :delete_policy, :reject_access, :create_policy, :allow_access" do - flow = Fixtures.Flows.create_flow() - flow_id = flow.id - client_id = flow.client_id - policy_id = flow.policy_id - account_id = flow.account_id - actor_group_id = "group-456" - resource_id = flow.resource_id + test "soft-delete deletes flows" do + account = Fixtures.Accounts.create_account() + resource = Fixtures.Resources.create_resource(account: account) + + policy = + Fixtures.Policies.create_policy( + account: account, + resource: resource + ) old_data = %{ - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id, - "conditions" => [] + "id" => policy.id, + "account_id" => account.id, + "actor_group_id" => policy.actor_group_id, + "resource_id" => resource.id, + "deleted_at" => nil } - data = Map.put(old_data, "resource_id", "new-resource-123") + data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z") - :ok = PubSub.Flow.subscribe(flow_id) - :ok = PubSub.Policy.subscribe(policy_id) - :ok = PubSub.Account.Policies.subscribe(account_id) - :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id) + assert flow = + Fixtures.Flows.create_flow( + policy: policy, + resource: resource, + account: account + ) - assert :ok == on_update(old_data, data) - - # TODO: WAL - # Remove this when side effects are directly broadcasted - Process.sleep(100) - - assert_receive {:delete_policy, ^policy_id} - assert_receive {:delete_policy, ^policy_id} - assert_receive {:reject_access, ^policy_id, ^actor_group_id, ^resource_id} - - assert_receive {:create_policy, ^policy_id} - assert_receive {:create_policy, ^policy_id} - assert_receive {:allow_access, ^policy_id, ^actor_group_id, "new-resource-123"} - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + assert :ok = on_update(old_data, data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) end - test "breaking update: disabled policy has no side-effects" do - flow = Fixtures.Flows.create_flow() - flow_id = flow.id - client_id = flow.client_id - policy_id = flow.policy_id - account_id = flow.account_id - actor_group_id = "group-456" - resource_id = flow.resource_id + test "non-breaking update broadcasts updated policy" do + account = Fixtures.Accounts.create_account() + policy = Fixtures.Policies.create_policy(account: account) + :ok = PubSub.Account.subscribe(account.id) old_data = %{ - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id, - "disabled_at" => "2023-10-01T00:00:00Z" + "id" => policy.id, + "description" => "Old description", + "account_id" => account.id, + "actor_group_id" => policy.actor_group_id, + "resource_id" => policy.resource_id, + "disabled_at" => nil, + "deleted_at" => nil } - data = Map.put(old_data, "resource_id", "new-resource-123") - - :ok = PubSub.Policy.subscribe(policy_id) - :ok = PubSub.Account.Policies.subscribe(account_id) - :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id) + data = Map.put(old_data, "description", "Updated description") assert :ok == on_update(old_data, data) - - refute_receive {:delete_policy, ^policy_id} - refute_receive {:reject_access, ^policy_id, ^actor_group_id, ^resource_id} - refute_receive {:create_policy, ^policy_id} - refute_receive {:allow_access, ^policy_id, ^actor_group_id, "new-resource-123"} - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - refute_receive {:expire_flow, ^flow_id, ^client_id, "new-resource-123"} + assert_receive {:updated, %Policies.Policy{} = old_policy, %Policies.Policy{} = new_policy} + assert old_policy.id == old_data["id"] + assert new_policy.description == data["description"] + assert new_policy.account_id == old_data["account_id"] + assert new_policy.actor_group_id == old_data["actor_group_id"] + assert new_policy.resource_id == old_data["resource_id"] end - test "non-breaking-update: broadcasts :update_policy" do - policy_id = "policy-123" - account_id = "account-456" - actor_group_id = "group-456" - resource_id = "resource-789" + test "breaking update deletes flows" do + account = Fixtures.Accounts.create_account() + policy = Fixtures.Policies.create_policy(account: account) old_data = %{ - "description" => "Old Policy", - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id, - "disabled_at" => "2023-10-01T00:00:00Z" + "id" => policy.id, + "account_id" => account.id, + "actor_group_id" => policy.actor_group_id, + "resource_id" => policy.resource_id, + "deleted_at" => nil } - data = Map.put(old_data, "resource_id", "new-resource-123") + data = Map.put(old_data, "resource_id", "00000000-0000-0000-0000-000000000001") - :ok = PubSub.Policy.subscribe(policy_id) - :ok = PubSub.Account.Policies.subscribe(account_id) - :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id) + assert flow = + Fixtures.Flows.create_flow( + policy: policy, + account: account + ) - assert :ok == on_update(old_data, data) + assert :ok = on_update(old_data, data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) + end - assert_receive {:update_policy, ^policy_id} - assert_receive {:update_policy, ^policy_id} + test "breaking update on actor_group_id deletes flows" do + account = Fixtures.Accounts.create_account() + policy = Fixtures.Policies.create_policy(account: account) + + old_data = %{ + "id" => policy.id, + "account_id" => account.id, + "actor_group_id" => policy.actor_group_id, + "resource_id" => policy.resource_id, + "deleted_at" => nil + } + + data = Map.put(old_data, "actor_group_id", "00000000-0000-0000-0000-000000000001") + + assert flow = Fixtures.Flows.create_flow(policy: policy, account: account) + + assert :ok = on_update(old_data, data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) + end + + test "breaking update on conditions deletes flows" do + account = Fixtures.Accounts.create_account() + policy = Fixtures.Policies.create_policy(account: account) + + old_data = %{ + "id" => policy.id, + "account_id" => account.id, + "actor_group_id" => policy.actor_group_id, + "resource_id" => policy.resource_id, + "conditions" => [ + %{"property" => "remote_ip", "operator" => "is_in", "values" => ["10.0.0.1"]} + ], + "deleted_at" => nil + } + + data = + Map.put(old_data, "conditions", [ + %{"property" => "remote_ip", "operator" => "is_in", "values" => ["10.0.0.2"]} + ]) + + assert flow = Fixtures.Flows.create_flow(policy: policy, account: account) + + assert :ok = on_update(old_data, data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) end end describe "delete/1" do - test "broadcasts :delete_policy and :reject_access" do - flow = Fixtures.Flows.create_flow() - flow_id = flow.id - client_id = flow.client_id - policy_id = flow.policy_id - account_id = flow.account_id - actor_group_id = "group-456" - resource_id = flow.resource_id + test "broadcasts deleted policy" do + account = Fixtures.Accounts.create_account() + policy = Fixtures.Policies.create_policy(account: account) + :ok = PubSub.Account.subscribe(account.id) old_data = %{ - "id" => policy_id, - "account_id" => account_id, - "actor_group_id" => actor_group_id, - "resource_id" => resource_id, + "id" => policy.id, + "account_id" => account.id, + "actor_group_id" => policy.actor_group_id, + "resource_id" => policy.resource_id + } + + assert :ok == on_delete(old_data) + assert_receive {:deleted, %Policies.Policy{} = policy} + + assert policy.id == old_data["id"] + assert policy.account_id == old_data["account_id"] + assert policy.actor_group_id == old_data["actor_group_id"] + assert policy.resource_id == old_data["resource_id"] + end + + test "deletes flows" do + account = Fixtures.Accounts.create_account() + policy = Fixtures.Policies.create_policy(account: account) + + old_data = %{ + "id" => policy.id, + "account_id" => account.id, + "actor_group_id" => policy.actor_group_id, + "resource_id" => policy.resource_id, "deleted_at" => nil } - data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z") + assert flow = Fixtures.Flows.create_flow(policy: policy, account: account) - :ok = PubSub.Flow.subscribe(flow_id) - :ok = PubSub.Policy.subscribe(policy_id) - :ok = PubSub.Account.Policies.subscribe(account_id) - :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id) - - assert :ok == on_update(old_data, data) - assert_receive {:delete_policy, ^policy_id} - assert_receive {:delete_policy, ^policy_id} - assert_receive {:reject_access, ^policy_id, ^actor_group_id, ^resource_id} - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + assert :ok = on_delete(old_data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) end end end diff --git a/elixir/apps/domain/test/domain/events/hooks/relay_groups_test.exs b/elixir/apps/domain/test/domain/events/hooks/relay_groups_test.exs deleted file mode 100644 index 652c8517d..000000000 --- a/elixir/apps/domain/test/domain/events/hooks/relay_groups_test.exs +++ /dev/null @@ -1,26 +0,0 @@ -defmodule Domain.Events.Hooks.RelayGroupsTest do - use ExUnit.Case, async: true - import Domain.Events.Hooks.RelayGroups - - setup do - %{old_data: %{}, data: %{}} - end - - describe "insert/1" do - test "returns :ok", %{data: data} do - assert :ok == on_insert(data) - end - end - - describe "update/2" do - test "returns :ok", %{old_data: old_data, data: data} do - assert :ok == on_update(old_data, data) - end - end - - describe "delete/1" do - test "returns :ok", %{data: data} do - assert :ok == on_delete(data) - end - end -end diff --git a/elixir/apps/domain/test/domain/events/hooks/relays_test.exs b/elixir/apps/domain/test/domain/events/hooks/relays_test.exs deleted file mode 100644 index 4962339df..000000000 --- a/elixir/apps/domain/test/domain/events/hooks/relays_test.exs +++ /dev/null @@ -1,26 +0,0 @@ -defmodule Domain.Events.Hooks.RelaysTest do - use ExUnit.Case, async: true - import Domain.Events.Hooks.Relays - - setup do - %{old_data: %{}, data: %{}} - end - - describe "insert/1" do - test "returns :ok", %{data: data} do - assert :ok == on_insert(data) - end - end - - describe "update/2" do - test "returns :ok", %{old_data: old_data, data: data} do - assert :ok == on_update(old_data, data) - end - end - - describe "delete/1" do - test "returns :ok", %{data: data} do - assert :ok == on_delete(data) - end - end -end diff --git a/elixir/apps/domain/test/domain/events/hooks/resource_connections_test.exs b/elixir/apps/domain/test/domain/events/hooks/resource_connections_test.exs index deb79ac32..19bbdb342 100644 --- a/elixir/apps/domain/test/domain/events/hooks/resource_connections_test.exs +++ b/elixir/apps/domain/test/domain/events/hooks/resource_connections_test.exs @@ -2,35 +2,57 @@ defmodule Domain.Events.Hooks.ResourceConnectionsTest do use Domain.DataCase, async: true import Domain.Events.Hooks.ResourceConnections - setup do - %{old_data: %{}, data: %{}} - end - describe "insert/1" do - test "returns :ok", %{data: data} do + test "broadcasts created resource connection" do + account = Fixtures.Accounts.create_account() + resource = Fixtures.Resources.create_resource(account: account) + gateway_group = Fixtures.Gateways.create_group(account: account) + + :ok = Domain.PubSub.Account.subscribe(account.id) + + data = %{ + "account_id" => account.id, + "resource_id" => resource.id, + "gateway_group_id" => gateway_group.id, + "deleted_at" => nil + } + assert :ok == on_insert(data) + + assert_receive {:created, %Domain.Resources.Connection{} = connection} + assert connection.account_id == data["account_id"] + assert connection.resource_id == data["resource_id"] + assert connection.gateway_group_id == data["gateway_group_id"] end end describe "update/2" do - test "returns :ok", %{old_data: old_data, data: data} do - assert :ok == on_update(old_data, data) + test "returns :ok" do + assert :ok = on_update(%{}, %{}) end end describe "delete/1" do - test "returns :ok" do - flow = Fixtures.Flows.create_flow() - :ok = Domain.PubSub.Flow.subscribe(flow.id) + test "broadcasts deleted connection" do + account = Fixtures.Accounts.create_account() + resource = Fixtures.Resources.create_resource(account: account) + gateway_group = Fixtures.Gateways.create_group(account: account) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id + :ok = Domain.PubSub.Account.subscribe(account.id) - assert :ok == - on_delete(%{"account_id" => flow.account_id, "resource_id" => flow.resource_id}) + old_data = %{ + "account_id" => account.id, + "resource_id" => resource.id, + "gateway_group_id" => gateway_group.id, + "deleted_at" => nil + } - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + assert :ok == on_delete(old_data) + + assert_receive {:deleted, %Domain.Resources.Connection{} = deleted_connection} + assert deleted_connection.account_id == old_data["account_id"] + assert deleted_connection.resource_id == old_data["resource_id"] + assert deleted_connection.gateway_group_id == old_data["gateway_group_id"] end end end diff --git a/elixir/apps/domain/test/domain/events/hooks/resources_test.exs b/elixir/apps/domain/test/domain/events/hooks/resources_test.exs index 26eb93575..58b5af71a 100644 --- a/elixir/apps/domain/test/domain/events/hooks/resources_test.exs +++ b/elixir/apps/domain/test/domain/events/hooks/resources_test.exs @@ -4,207 +4,202 @@ defmodule Domain.Events.Hooks.ResourcesTest do alias Domain.PubSub describe "insert/1" do - test "broadcasts :create_resource to subscribed" do - resource_id = "test_resource" - account_id = "test_account" - :ok = PubSub.Resource.subscribe(resource_id) - :ok = PubSub.Account.Resources.subscribe(account_id) + test "broadcasts created resource" do + account = Fixtures.Accounts.create_account() + filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}] + resource = Fixtures.Resources.create_resource(account: account, filters: filters) - data = %{"id" => resource_id, "account_id" => account_id} + :ok = PubSub.Account.subscribe(account.id) + + data = %{ + "id" => resource.id, + "account_id" => account.id, + "address_description" => resource.address_description, + "type" => resource.type, + "address" => resource.address, + "filters" => filters, + "ip_stack" => resource.ip_stack, + "deleted_at" => nil + } assert :ok == on_insert(data) - - # we expect two - once for the resource subscription, and once for the account - assert_receive {:create_resource, ^resource_id} - assert_receive {:create_resource, ^resource_id} - - :ok = PubSub.Resource.unsubscribe(resource_id) - - assert :ok = on_insert(data) - assert_receive {:create_resource, ^resource_id} - refute_receive {:create_resource, ^resource_id} + assert_receive {:created, %Domain.Resources.Resource{} = created_resource} + assert created_resource.id == resource.id + assert created_resource.account_id == resource.account_id + assert created_resource.type == resource.type + assert created_resource.address == resource.address + assert created_resource.filters == resource.filters + assert created_resource.ip_stack == resource.ip_stack + assert created_resource.address_description == resource.address_description end end describe "update/2" do - setup do - flow = Fixtures.Flows.create_flow() + test "soft-delete broadcasts deleted resource" do + account = Fixtures.Accounts.create_account() + filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}] + resource = Fixtures.Resources.create_resource(account: account, filters: filters) + + :ok = PubSub.Account.subscribe(account.id) old_data = %{ - "type" => "dns", - "address" => "1.2.3.4", - "filters" => [], - "ip_stack" => "dual", - "id" => flow.resource_id, - "account_id" => flow.account_id + "id" => resource.id, + "account_id" => account.id, + "address_description" => resource.address_description, + "type" => resource.type, + "address" => resource.address, + "filters" => filters, + "ip_stack" => resource.ip_stack, + "deleted_at" => nil } - %{flow: flow, old_data: old_data} - end - - test "broadcasts :delete_resource to subscribed for soft-deletions" do - resource_id = "test_resource" - account_id = "test_account" - :ok = PubSub.Resource.subscribe(resource_id) - :ok = PubSub.Account.Resources.subscribe(account_id) - - old_data = %{"id" => resource_id, "account_id" => account_id, "deleted_at" => nil} - - data = %{ - "id" => resource_id, - "account_id" => account_id, - "deleted_at" => DateTime.utc_now() - } + data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z") assert :ok == on_update(old_data, data) - assert_receive {:delete_resource, ^resource_id} - assert_receive {:delete_resource, ^resource_id} - :ok = PubSub.Resource.unsubscribe(resource_id) + assert_receive {:deleted, %Domain.Resources.Resource{} = deleted_resource} - assert :ok = on_update(old_data, data) - assert_receive {:delete_resource, ^resource_id} - refute_receive {:delete_resource, ^resource_id} + assert deleted_resource.id == resource.id + assert deleted_resource.account_id == resource.account_id + assert deleted_resource.type == resource.type + assert deleted_resource.address == resource.address + assert deleted_resource.filters == resource.filters + assert deleted_resource.ip_stack == resource.ip_stack + assert deleted_resource.address_description == resource.address_description end - test "expires flows when resource type changes", %{flow: flow, old_data: old_data} do - :ok = PubSub.Flow.subscribe(flow.id) - :ok = PubSub.Resource.subscribe(flow.resource_id) - :ok = PubSub.Account.Resources.subscribe(flow.account_id) + test "soft-delete deletes flows" do + account = Fixtures.Accounts.create_account() + filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}] + resource = Fixtures.Resources.create_resource(account: account, filters: filters) + + old_data = %{ + "id" => resource.id, + "account_id" => account.id, + "address_description" => resource.address_description, + "type" => resource.type, + "address" => resource.address, + "filters" => filters, + "ip_stack" => resource.ip_stack, + "deleted_at" => nil + } + + data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z") + + assert flow = Fixtures.Flows.create_flow(resource: resource, account: account) + assert :ok = on_update(old_data, data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) + end + + test "regular update broadcasts updated resource" do + account = Fixtures.Accounts.create_account() + filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}] + resource = Fixtures.Resources.create_resource(account: account, filters: filters) + + :ok = PubSub.Account.subscribe(account.id) + + old_data = %{ + "id" => resource.id, + "account_id" => account.id, + "address_description" => resource.address_description, + "type" => resource.type, + "address" => resource.address, + "filters" => filters, + "ip_stack" => resource.ip_stack, + "deleted_at" => nil + } + + data = Map.put(old_data, "address", "new-address.example.com") + + assert :ok == on_update(old_data, data) + + assert_receive {:updated, %Domain.Resources.Resource{}, + %Domain.Resources.Resource{} = updated_resource} + + assert updated_resource.id == resource.id + assert updated_resource.account_id == resource.account_id + assert updated_resource.type == resource.type + assert updated_resource.address == "new-address.example.com" + assert updated_resource.filters == resource.filters + assert updated_resource.ip_stack == resource.ip_stack + assert updated_resource.address_description == resource.address_description + end + + test "breaking update deletes flows" do + account = Fixtures.Accounts.create_account() + filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}] + resource = Fixtures.Resources.create_resource(account: account, filters: filters) + + old_data = %{ + "id" => resource.id, + "account_id" => account.id, + "address_description" => resource.address_description, + "type" => "dns", + "address" => resource.address, + "filters" => filters, + "ip_stack" => resource.ip_stack, + "deleted_at" => nil + } data = Map.put(old_data, "type", "cidr") - assert :ok == on_update(old_data, data) - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - assert_receive {:delete_resource, ^resource_id} - assert_receive {:delete_resource, ^resource_id} - assert_receive {:create_resource, ^resource_id} - assert_receive {:create_resource, ^resource_id} - end - - test "expires flows when resource address changes", %{flow: flow, old_data: old_data} do - :ok = PubSub.Flow.subscribe(flow.id) - :ok = PubSub.Resource.subscribe(flow.resource_id) - :ok = PubSub.Account.Resources.subscribe(flow.account_id) - - data = Map.put(old_data, "address", "4.3.2.1") - - assert :ok == on_update(old_data, data) - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - assert_receive {:delete_resource, ^resource_id} - assert_receive {:delete_resource, ^resource_id} - assert_receive {:create_resource, ^resource_id} - assert_receive {:create_resource, ^resource_id} - end - - test "expires flows when resource filters change", %{flow: flow, old_data: old_data} do - :ok = PubSub.Flow.subscribe(flow.id) - :ok = PubSub.Resource.subscribe(flow.resource_id) - :ok = PubSub.Account.Resources.subscribe(flow.account_id) - - data = Map.put(old_data, "filters", ["new_filter"]) - - assert :ok == on_update(old_data, data) - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - assert_receive {:delete_resource, ^resource_id} - assert_receive {:delete_resource, ^resource_id} - assert_receive {:create_resource, ^resource_id} - assert_receive {:create_resource, ^resource_id} - end - - test "expires flows when resource ip_stack changes", %{flow: flow, old_data: old_data} do - :ok = PubSub.Flow.subscribe(flow.id) - :ok = PubSub.Resource.subscribe(flow.resource_id) - :ok = PubSub.Account.Resources.subscribe(flow.account_id) - - data = Map.put(old_data, "ip_stack", "ipv4_only") - - assert :ok == on_update(old_data, data) - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - assert_receive {:delete_resource, ^resource_id} - assert_receive {:delete_resource, ^resource_id} - assert_receive {:create_resource, ^resource_id} - assert_receive {:create_resource, ^resource_id} - end - - test "broadcasts update for non-addressability change", %{flow: flow, old_data: old_data} do - :ok = PubSub.Resource.subscribe(flow.resource_id) - :ok = PubSub.Account.Resources.subscribe(flow.account_id) - - data = Map.put(old_data, "name", "New Name") - - assert :ok == on_update(old_data, data) - - # TODO: WAL - # Remove this after direct broadcast - Process.sleep(100) - - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - - refute_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - assert_receive {:update_resource, ^resource_id} - assert_receive {:update_resource, ^resource_id} - refute_receive {:delete_resource, ^resource_id} - refute_receive {:create_resource, ^resource_id} + assert flow = Fixtures.Flows.create_flow(resource: resource, account: account) + assert :ok = on_update(old_data, data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) end end describe "delete/1" do - test "broadcasts :delete_resource to subscribed" do - resource_id = "test_resource" - account_id = "test_account" - :ok = PubSub.Resource.subscribe(resource_id) - :ok = PubSub.Account.Resources.subscribe(account_id) + test "broadcasts deleted resource" do + account = Fixtures.Accounts.create_account() + filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}] + resource = Fixtures.Resources.create_resource(account: account, filters: filters) - old_data = %{"id" => resource_id, "account_id" => account_id} + :ok = PubSub.Account.subscribe(account.id) + + old_data = %{ + "id" => resource.id, + "account_id" => account.id, + "address_description" => resource.address_description, + "type" => resource.type, + "address" => resource.address, + "filters" => filters, + "ip_stack" => resource.ip_stack, + "deleted_at" => nil + } assert :ok == on_delete(old_data) - assert_receive {:delete_resource, ^resource_id} - assert_receive {:delete_resource, ^resource_id} - :ok = PubSub.Resource.unsubscribe(resource_id) + assert_receive {:deleted, %Domain.Resources.Resource{} = deleted_resource} + assert deleted_resource.id == resource.id + assert deleted_resource.account_id == resource.account_id + assert deleted_resource.type == resource.type + assert deleted_resource.address == resource.address + assert deleted_resource.filters == resource.filters + assert deleted_resource.ip_stack == resource.ip_stack + assert deleted_resource.address_description == resource.address_description + end + + test "deletes flows" do + account = Fixtures.Accounts.create_account() + filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}] + resource = Fixtures.Resources.create_resource(account: account, filters: filters) + + old_data = %{ + "id" => resource.id, + "account_id" => account.id, + "address_description" => resource.address_description, + "type" => resource.type, + "address" => resource.address, + "filters" => filters, + "ip_stack" => resource.ip_stack, + "deleted_at" => nil + } + + assert flow = Fixtures.Flows.create_flow(resource: resource, account: account) assert :ok = on_delete(old_data) - assert_receive {:delete_resource, ^resource_id} - refute_receive {:delete_resource, ^resource_id} + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) end end end diff --git a/elixir/apps/domain/test/domain/events/hooks/tokens_test.exs b/elixir/apps/domain/test/domain/events/hooks/tokens_test.exs index ef6a08ba8..27ee987f5 100644 --- a/elixir/apps/domain/test/domain/events/hooks/tokens_test.exs +++ b/elixir/apps/domain/test/domain/events/hooks/tokens_test.exs @@ -1,62 +1,98 @@ defmodule Domain.Events.Hooks.TokensTest do - use ExUnit.Case, async: true + use Domain.DataCase, async: true import Domain.Events.Hooks.Tokens - setup do - %{old_data: %{}, data: %{}} - end - describe "insert/1" do - test "returns :ok", %{data: data} do - assert :ok == on_insert(data) + test "returns :ok" do + assert :ok == on_insert(%{}) end end describe "update/2" do - test "does not broadcast for email token updates" do - token_id = "token-id-123" - topic = "sessions:#{token_id}" - old_data = %{"id" => token_id, "type" => "email"} - - :ok = Domain.PubSub.subscribe("sessions:#{token_id}") - - assert :ok = on_update(old_data, %{}) - - refute_receive %Phoenix.Socket.Broadcast{ - topic: ^topic, - event: "disconnect" - } + test "returns :ok for email token updates" do + assert :ok = on_update(%{"type" => "email"}, %{"type" => "email"}) end - test "broadcasts disconnect for soft-deletions" do - token_id = "token-id-123" - topic = "sessions:#{token_id}" - old_data = %{"id" => token_id, "deleted_at" => nil} - data = %{"id" => token_id, "deleted_at" => DateTime.utc_now()} - :ok = Domain.PubSub.subscribe("sessions:#{token_id}") + test "soft-delete broadcasts deleted token" do + account = Fixtures.Accounts.create_account() + token = Fixtures.Tokens.create_token(account: account) + :ok = Domain.PubSub.Account.subscribe(account.id) - assert :ok = on_update(old_data, data) - - assert_receive %Phoenix.Socket.Broadcast{ - topic: ^topic, - event: "disconnect" + old_data = %{ + "id" => token.id, + "account_id" => account.id, + "type" => token.type, + "deleted_at" => nil } + + data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z") + + assert :ok == on_update(old_data, data) + assert_receive {:deleted, %Domain.Tokens.Token{} = deleted_token} + assert deleted_token.id == old_data["id"] + assert deleted_token.account_id == old_data["account_id"] + assert deleted_token.type == old_data["type"] + end + + test "soft-delete deletes flows" do + account = Fixtures.Accounts.create_account() + token = Fixtures.Tokens.create_token(account: account) + + old_data = %{ + "id" => token.id, + "account_id" => account.id, + "type" => token.type, + "deleted_at" => nil + } + + data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z") + + assert flow = Fixtures.Flows.create_flow(account: account, token: token) + assert flow.token_id == token.id + assert :ok = on_update(old_data, data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) + end + + test "regular update returns :ok" do + assert :ok = on_update(%{}, %{}) end end describe "delete/1" do - test "broadcasts disconnect for deletions" do - token_id = "token-id-123" - topic = "sessions:#{token_id}" - old_data = %{"id" => token_id} - :ok = Domain.PubSub.subscribe("sessions:#{token_id}") + test "broadcasts deleted token" do + account = Fixtures.Accounts.create_account() + token = Fixtures.Tokens.create_token(account: account) + :ok = Domain.PubSub.Account.subscribe(account.id) - assert :ok = on_delete(old_data) - - assert_receive %Phoenix.Socket.Broadcast{ - topic: ^topic, - event: "disconnect" + old_data = %{ + "id" => token.id, + "account_id" => account.id, + "type" => token.type, + "deleted_at" => nil } + + assert :ok == on_delete(old_data) + + assert_receive {:deleted, %Domain.Tokens.Token{} = deleted_token} + assert deleted_token.id == old_data["id"] + assert deleted_token.account_id == old_data["account_id"] + assert deleted_token.type == old_data["type"] + end + + test "deletes flows" do + account = Fixtures.Accounts.create_account() + token = Fixtures.Tokens.create_token(account: account) + + old_data = %{ + "id" => token.id, + "account_id" => account.id, + "type" => token.type, + "deleted_at" => nil + } + + assert flow = Fixtures.Flows.create_flow(account: account, token: token) + assert :ok = on_delete(old_data) + refute Repo.get_by(Domain.Flows.Flow, id: flow.id) end end end diff --git a/elixir/apps/domain/test/domain/events/replication_connection_test.exs b/elixir/apps/domain/test/domain/events/replication_connection_test.exs index fdbc63b17..99e6a171f 100644 --- a/elixir/apps/domain/test/domain/events/replication_connection_test.exs +++ b/elixir/apps/domain/test/domain/events/replication_connection_test.exs @@ -1,5 +1,5 @@ defmodule Domain.Events.ReplicationConnectionTest do - use ExUnit.Case, async: true + use Domain.DataCase, async: true import ExUnit.CaptureLog alias Domain.Events.ReplicationConnection @@ -15,7 +15,12 @@ defmodule Domain.Events.ReplicationConnectionTest do describe "on_write/6 for inserts" do test "logs warning for unknown table" do table = "unknown_table" - data = %{"id" => Ecto.UUID.generate(), "name" => "test"} + + data = %{ + "account_id" => Ecto.UUID.generate(), + "id" => Ecto.UUID.generate(), + "name" => "test" + } log_output = capture_log(fn -> @@ -29,7 +34,11 @@ defmodule Domain.Events.ReplicationConnectionTest do test "handles known tables without errors", %{tables: tables} do for table <- tables do - data = %{"id" => Ecto.UUID.generate(), "table" => table} + data = %{ + "account_id" => Ecto.UUID.generate(), + "id" => Ecto.UUID.generate(), + "table" => table + } # The actual hook call might fail if the hook modules aren't available, # but we can test that our routing logic works @@ -51,6 +60,7 @@ defmodule Domain.Events.ReplicationConnectionTest do capture_log(fn -> try do ReplicationConnection.on_write(%{}, 0, :insert, table, nil, %{ + "account_id" => Ecto.UUID.generate(), "id" => Ecto.UUID.generate() }) rescue @@ -68,8 +78,14 @@ defmodule Domain.Events.ReplicationConnectionTest do describe "on_write/6 for updates" do test "logs warning for unknown table" do table = "unknown_table" - old_data = %{"id" => Ecto.UUID.generate(), "name" => "old"} - data = %{"id" => old_data["id"], "name" => "new"} + + old_data = %{ + "account_id" => Ecto.UUID.generate(), + "id" => Ecto.UUID.generate(), + "name" => "old" + } + + data = %{"account_id" => Ecto.UUID.generate(), "id" => old_data["id"], "name" => "new"} log_output = capture_log(fn -> @@ -82,8 +98,13 @@ defmodule Domain.Events.ReplicationConnectionTest do end test "handles known tables", %{tables: tables} do - old_data = %{"id" => Ecto.UUID.generate(), "name" => "old name"} - data = %{"id" => old_data["id"], "name" => "new name"} + old_data = %{ + "account_id" => Ecto.UUID.generate(), + "id" => Ecto.UUID.generate(), + "name" => "old name" + } + + data = %{"account_id" => Ecto.UUID.generate(), "id" => old_data["id"], "name" => "new name"} for table <- tables do try do @@ -101,7 +122,12 @@ defmodule Domain.Events.ReplicationConnectionTest do describe "on_write/6 for deletes" do test "logs warning for unknown table" do table = "unknown_table" - old_data = %{"id" => Ecto.UUID.generate(), "name" => "deleted"} + + old_data = %{ + "account_id" => Ecto.UUID.generate(), + "id" => Ecto.UUID.generate(), + "name" => "deleted" + } log_output = capture_log(fn -> @@ -114,7 +140,11 @@ defmodule Domain.Events.ReplicationConnectionTest do end test "handles known tables", %{tables: tables} do - old_data = %{"id" => Ecto.UUID.generate(), "name" => "deleted item"} + old_data = %{ + "account_id" => Ecto.UUID.generate(), + "id" => Ecto.UUID.generate(), + "name" => "deleted item" + } for table <- tables do try do @@ -139,7 +169,11 @@ defmodule Domain.Events.ReplicationConnectionTest do # Insert try do - result = ReplicationConnection.on_write(state, 1, :insert, table, nil, %{"id" => "123"}) + result = + ReplicationConnection.on_write(state, 1, :insert, table, nil, %{ + "id" => "00000000-0000-0000-0000-000000000001" + }) + assert result == state rescue FunctionClauseError -> :ok @@ -148,10 +182,17 @@ defmodule Domain.Events.ReplicationConnectionTest do # Update try do result = - ReplicationConnection.on_write(state, 2, :update, table, %{"id" => "123"}, %{ - "id" => "123", - "updated" => true - }) + ReplicationConnection.on_write( + state, + 2, + :update, + table, + %{"id" => "00000000-0000-0000-0000-000000000001"}, + %{ + "id" => "00000000-0000-0000-0000-000000000001", + "updated" => true + } + ) assert result == state rescue @@ -160,7 +201,16 @@ defmodule Domain.Events.ReplicationConnectionTest do # Delete try do - result = ReplicationConnection.on_write(state, 3, :delete, table, %{"id" => "123"}, nil) + result = + ReplicationConnection.on_write( + state, + 3, + :delete, + table, + %{"id" => "00000000-0000-0000-0000-000000000001"}, + nil + ) + assert result == state rescue FunctionClauseError -> :ok @@ -223,11 +273,8 @@ defmodule Domain.Events.ReplicationConnectionTest do tables_to_hooks = %{ "accounts" => Domain.Events.Hooks.Accounts, "actor_group_memberships" => Domain.Events.Hooks.ActorGroupMemberships, - "actor_groups" => Domain.Events.Hooks.ActorGroups, - "actors" => Domain.Events.Hooks.Actors, - "auth_identities" => Domain.Events.Hooks.AuthIdentities, - "auth_providers" => Domain.Events.Hooks.AuthProviders, "clients" => Domain.Events.Hooks.Clients, + "flows" => Domain.Events.Hooks.Flows, "gateway_groups" => Domain.Events.Hooks.GatewayGroups, "gateways" => Domain.Events.Hooks.Gateways, "policies" => Domain.Events.Hooks.Policies, @@ -241,11 +288,8 @@ defmodule Domain.Events.ReplicationConnectionTest do [ "accounts", "actor_group_memberships", - "actor_groups", - "actors", - "auth_identities", - "auth_providers", "clients", + "flows", "gateway_groups", "gateways", "policies", diff --git a/elixir/apps/domain/test/domain/flows_test.exs b/elixir/apps/domain/test/domain/flows_test.exs index 5ca338ad5..52af61573 100644 --- a/elixir/apps/domain/test/domain/flows_test.exs +++ b/elixir/apps/domain/test/domain/flows_test.exs @@ -9,7 +9,9 @@ defmodule Domain.FlowsTest do actor_group = Fixtures.Actors.create_group(account: account) actor = Fixtures.Actors.create_actor(type: :account_admin_user, account: account) - Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) + + membership = + Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) Domain.Config.put_env_override(:outbound_email_adapter_configured?, true) provider = Fixtures.Auth.create_email_provider(account: account) @@ -41,6 +43,7 @@ defmodule Domain.FlowsTest do actor_group: actor_group, actor: actor, provider: provider, + membership: membership, identity: identity, subject: subject, client: client, @@ -51,200 +54,200 @@ defmodule Domain.FlowsTest do } end - describe "authorize_flow/4" do - test "returns error when resource does not exist", %{ - client: client, - gateway: gateway, - subject: subject - } do - resource_id = Ecto.UUID.generate() - assert authorize_flow(client, gateway, resource_id, subject) == {:error, :not_found} - end + describe "create_flow/4" do + # test "returns error when resource does not exist", %{ + # client: client, + # gateway: gateway, + # subject: subject + # } do + # resource_id = Ecto.UUID.generate() + # assert authorize_flow(client, gateway, resource_id, subject) == {:error, :not_found} + # end + # + # test "returns error when UUID is invalid", %{ + # client: client, + # gateway: gateway, + # subject: subject + # } do + # assert authorize_flow(client, gateway, "foo", subject) == {:error, :not_found} + # end + # + # test "returns authorized resource", %{ + # client: client, + # gateway: gateway, + # resource: resource, + # policy: policy, + # subject: subject + # } do + # assert {:ok, fetched_resource, _flow, _expires_at} = + # authorize_flow(client, gateway, resource.id, subject) + # + # assert fetched_resource.id == resource.id + # assert hd(fetched_resource.authorized_by_policies).id == policy.id + # end + # + # test "returns error when some conditions are not satisfied", %{ + # account: account, + # actor_group: actor_group, + # client: client, + # gateway_group: gateway_group, + # gateway: gateway, + # subject: subject + # } do + # resource = + # Fixtures.Resources.create_resource( + # account: account, + # connections: [%{gateway_group_id: gateway_group.id}] + # ) + # + # Fixtures.Policies.create_policy( + # account: account, + # actor_group: actor_group, + # resource: resource, + # conditions: [ + # %{ + # property: :remote_ip_location_region, + # operator: :is_in, + # values: ["AU"] + # }, + # %{ + # property: :remote_ip_location_region, + # operator: :is_not_in, + # values: [client.last_seen_remote_ip_location_region] + # }, + # %{ + # property: :remote_ip, + # operator: :is_in_cidr, + # values: ["0.0.0.0/0", "0::/0"] + # } + # ] + # ) + # + # assert authorize_flow(client, gateway, resource.id, subject) == + # {:error, {:forbidden, violated_properties: [:remote_ip_location_region]}} + # end + # + # test "returns error when all conditions are not satisfied", %{ + # account: account, + # actor_group: actor_group, + # client: client, + # gateway_group: gateway_group, + # gateway: gateway, + # subject: subject + # } do + # resource = + # Fixtures.Resources.create_resource( + # account: account, + # connections: [%{gateway_group_id: gateway_group.id}] + # ) + # + # Fixtures.Policies.create_policy( + # account: account, + # actor_group: actor_group, + # resource: resource, + # conditions: [ + # %{ + # property: :remote_ip_location_region, + # operator: :is_in, + # values: ["AU"] + # }, + # %{ + # property: :remote_ip_location_region, + # operator: :is_not_in, + # values: [client.last_seen_remote_ip_location_region] + # } + # ] + # ) + # + # assert authorize_flow(client, gateway, resource.id, subject) == + # {:error, {:forbidden, violated_properties: [:remote_ip_location_region]}} + # end + # + # test "creates a flow when the only policy conditions are satisfied", %{ + # account: account, + # actor: actor, + # resource: resource, + # client: client, + # policy: policy, + # gateway: gateway, + # subject: subject + # } do + # actor_group2 = Fixtures.Actors.create_group(account: account) + # Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group2) + # + # time = Time.utc_now() + # midnight = Time.from_iso8601!("23:59:59.999999") + # + # date = Date.utc_today() + # day_of_week = Enum.at(~w[M T W R F S U], Date.day_of_week(date) - 1) + # + # Fixtures.Policies.create_policy( + # account: account, + # actor_group: actor_group2, + # resource: resource, + # conditions: [ + # %{ + # property: :remote_ip_location_region, + # operator: :is_not_in, + # values: [client.last_seen_remote_ip_location_region] + # }, + # %{ + # property: :current_utc_datetime, + # operator: :is_in_day_of_week_time_ranges, + # values: [ + # "#{day_of_week}/#{time}-#{midnight}/UTC" + # ] + # } + # ] + # ) + # + # assert {:ok, _fetched_resource, flow, expires_at} = + # authorize_flow(client, gateway, resource.id, subject) + # + # assert flow.policy_id == policy.id + # assert DateTime.diff(expires_at, DateTime.new!(date, midnight)) < 5 + # end + # + # test "creates a flow when all conditions for at least one of the policies are satisfied", %{ + # account: account, + # actor_group: actor_group, + # client: client, + # gateway_group: gateway_group, + # gateway: gateway, + # subject: subject + # } do + # resource = + # Fixtures.Resources.create_resource( + # account: account, + # connections: [%{gateway_group_id: gateway_group.id}] + # ) + # + # Fixtures.Policies.create_policy( + # account: account, + # actor_group: actor_group, + # resource: resource, + # conditions: [ + # %{ + # property: :remote_ip_location_region, + # operator: :is_in, + # values: [client.last_seen_remote_ip_location_region] + # }, + # %{ + # property: :remote_ip, + # operator: :is_in_cidr, + # values: ["0.0.0.0/0", "0::/0"] + # } + # ] + # ) + # + # assert {:ok, _fetched_resource, flow, expires_at} = + # authorize_flow(client, gateway, resource.id, subject) + # + # assert flow.resource_id == resource.id + # assert expires_at == subject.expires_at + # end - test "returns error when UUID is invalid", %{ - client: client, - gateway: gateway, - subject: subject - } do - assert authorize_flow(client, gateway, "foo", subject) == {:error, :not_found} - end - - test "returns authorized resource", %{ - client: client, - gateway: gateway, - resource: resource, - policy: policy, - subject: subject - } do - assert {:ok, fetched_resource, _flow, _expires_at} = - authorize_flow(client, gateway, resource.id, subject) - - assert fetched_resource.id == resource.id - assert hd(fetched_resource.authorized_by_policies).id == policy.id - end - - test "returns error when some conditions are not satisfied", %{ - account: account, - actor_group: actor_group, - client: client, - gateway_group: gateway_group, - gateway: gateway, - subject: subject - } do - resource = - Fixtures.Resources.create_resource( - account: account, - connections: [%{gateway_group_id: gateway_group.id}] - ) - - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource, - conditions: [ - %{ - property: :remote_ip_location_region, - operator: :is_in, - values: ["AU"] - }, - %{ - property: :remote_ip_location_region, - operator: :is_not_in, - values: [client.last_seen_remote_ip_location_region] - }, - %{ - property: :remote_ip, - operator: :is_in_cidr, - values: ["0.0.0.0/0", "0::/0"] - } - ] - ) - - assert authorize_flow(client, gateway, resource.id, subject) == - {:error, {:forbidden, violated_properties: [:remote_ip_location_region]}} - end - - test "returns error when all conditions are not satisfied", %{ - account: account, - actor_group: actor_group, - client: client, - gateway_group: gateway_group, - gateway: gateway, - subject: subject - } do - resource = - Fixtures.Resources.create_resource( - account: account, - connections: [%{gateway_group_id: gateway_group.id}] - ) - - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource, - conditions: [ - %{ - property: :remote_ip_location_region, - operator: :is_in, - values: ["AU"] - }, - %{ - property: :remote_ip_location_region, - operator: :is_not_in, - values: [client.last_seen_remote_ip_location_region] - } - ] - ) - - assert authorize_flow(client, gateway, resource.id, subject) == - {:error, {:forbidden, violated_properties: [:remote_ip_location_region]}} - end - - test "creates a flow when the only policy conditions are satisfied", %{ - account: account, - actor: actor, - resource: resource, - client: client, - policy: policy, - gateway: gateway, - subject: subject - } do - actor_group2 = Fixtures.Actors.create_group(account: account) - Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group2) - - time = Time.utc_now() - midnight = Time.from_iso8601!("23:59:59.999999") - - date = Date.utc_today() - day_of_week = Enum.at(~w[M T W R F S U], Date.day_of_week(date) - 1) - - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group2, - resource: resource, - conditions: [ - %{ - property: :remote_ip_location_region, - operator: :is_not_in, - values: [client.last_seen_remote_ip_location_region] - }, - %{ - property: :current_utc_datetime, - operator: :is_in_day_of_week_time_ranges, - values: [ - "#{day_of_week}/#{time}-#{midnight}/UTC" - ] - } - ] - ) - - assert {:ok, _fetched_resource, flow, expires_at} = - authorize_flow(client, gateway, resource.id, subject) - - assert flow.policy_id == policy.id - assert DateTime.diff(expires_at, DateTime.new!(date, midnight)) < 5 - end - - test "creates a flow when all conditions for at least one of the policies are satisfied", %{ - account: account, - actor_group: actor_group, - client: client, - gateway_group: gateway_group, - gateway: gateway, - subject: subject - } do - resource = - Fixtures.Resources.create_resource( - account: account, - connections: [%{gateway_group_id: gateway_group.id}] - ) - - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource, - conditions: [ - %{ - property: :remote_ip_location_region, - operator: :is_in, - values: [client.last_seen_remote_ip_location_region] - }, - %{ - property: :remote_ip, - operator: :is_in_cidr, - values: ["0.0.0.0/0", "0::/0"] - } - ] - ) - - assert {:ok, _fetched_resource, flow, expires_at} = - authorize_flow(client, gateway, resource.id, subject) - - assert flow.resource_id == resource.id - assert expires_at == subject.expires_at - end - - test "creates a network flow for users", %{ + test "creates a new flow for users", %{ account: account, gateway: gateway, resource: resource, @@ -257,8 +260,8 @@ defmodule Domain.FlowsTest do Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) - assert {:ok, _fetched_resource, %Flows.Flow{} = flow, expires_at} = - authorize_flow(client, gateway, resource.id, subject) + assert {:ok, %Flows.Flow{} = flow} = + create_flow(client, gateway, resource.id, policy, subject) assert flow.policy_id == policy.id assert flow.client_id == client.id @@ -268,10 +271,9 @@ defmodule Domain.FlowsTest do assert flow.client_remote_ip.address == subject.context.remote_ip assert flow.client_user_agent == subject.context.user_agent assert flow.gateway_remote_ip == gateway.last_seen_remote_ip - assert expires_at == subject.expires_at end - test "creates a network flow for service accounts", %{ + test "creates a new flow for service accounts", %{ account: account, actor_group: actor_group, gateway: gateway, @@ -286,8 +288,8 @@ defmodule Domain.FlowsTest do client = Fixtures.Clients.create_client(account: account, actor: actor, identity: identity) - assert {:ok, _fetched_resource, %Flows.Flow{} = flow, expires_at} = - authorize_flow(client, gateway, resource.id, subject) + assert {:ok, %Flows.Flow{} = flow} = + create_flow(client, gateway, resource.id, policy, subject) assert flow.policy_id == policy.id assert flow.client_id == client.id @@ -297,94 +299,37 @@ defmodule Domain.FlowsTest do assert flow.client_remote_ip.address == subject.context.remote_ip assert flow.client_user_agent == subject.context.user_agent assert flow.gateway_remote_ip == gateway.last_seen_remote_ip - assert expires_at == subject.expires_at end - test "does not return authorized access to deleted resources", %{ - client: client, - gateway: gateway, - resource: resource, - subject: subject - } do - {:ok, resource} = Domain.Resources.delete_resource(resource, subject) - - assert authorize_flow(client, gateway, resource.id, subject) == {:error, :not_found} - end - - test "does not authorize access to resources in other accounts", %{ - client: client, - gateway: gateway, - subject: subject - } do - resource = Fixtures.Resources.create_resource() - assert authorize_flow(client, gateway, resource.id, subject) == {:error, :not_found} - end - - test "returns error on account_id mismatch", %{ - client: client, - gateway: gateway, - resource: resource, - subject: subject - } do - other_subject = Fixtures.Auth.create_subject() - other_client = Fixtures.Clients.create_client() - other_gateway = Fixtures.Gateways.create_gateway() - - assert_raise FunctionClauseError, fn -> - assert authorize_flow(client, gateway, resource.id, other_subject) - end - - assert_raise FunctionClauseError, fn -> - assert authorize_flow(client, other_gateway, resource.id, subject) - end - - assert_raise FunctionClauseError, fn -> - assert authorize_flow(other_client, gateway, resource.id, subject) - end - end - - test "returns error when subject has no permission to create flows", %{ - client: client, - gateway: gateway, - resource: resource, - subject: subject - } do - subject = Fixtures.Auth.remove_permissions(subject) - - assert authorize_flow(client, gateway, resource.id, subject) == - {:error, - {:unauthorized, - reason: :missing_permissions, - missing_permissions: [ - Flows.Authorizer.create_flows_permission() - ]}} - - subject = Fixtures.Auth.add_permission(subject, Flows.Authorizer.create_flows_permission()) - - assert authorize_flow(client, gateway, resource.id, subject) == - {:error, - {:unauthorized, - reason: :missing_permissions, - missing_permissions: [ - Domain.Resources.Authorizer.view_available_resources_permission() - ]}} - end - - test "preloads assocs", %{ - client: client, - gateway: gateway, - resource: resource, - subject: subject - } do - assert {:ok, resource, _flow, _expires_at} = - authorize_flow(client, gateway, resource.id, subject, preload: :connections) - - assert Ecto.assoc_loaded?(resource.connections) - assert Ecto.assoc_loaded?(resource.connections) - assert Ecto.assoc_loaded?(resource.connections) - assert Ecto.assoc_loaded?(resource.connections) - assert length(resource.connections) == 1 - end + # TODO: Rename Flows + # Authorization a policy authorization ("flow") makes no sense + # test "returns error when subject has no permission to create flows", %{ + # client: client, + # gateway: gateway, + # resource: resource, + # policy: policy, + # subject: subject + # } do + # subject = Fixtures.Auth.remove_permissions(subject) + # + # assert create_flow(client, gateway, resource.id, policy, subject) == + # {:error, + # {:unauthorized, + # reason: :missing_permissions, + # missing_permissions: [ + # Flows.Authorizer.create_flows_permission() + # ]}} + # + # subject = Fixtures.Auth.add_permission(subject, Flows.Authorizer.create_flows_permission()) + # + # assert create_flow(client, gateway, resource.id, policy, subject) == + # {:error, + # {:unauthorized, + # reason: :missing_permissions, + # missing_permissions: [ + # Domain.Resources.Authorizer.view_available_resources_permission() + # ]}} + # end end describe "fetch_flow_by_id/3" do @@ -401,6 +346,7 @@ defmodule Domain.FlowsTest do client: client, gateway: gateway, resource: resource, + membership: membership, policy: policy, subject: subject } do @@ -409,6 +355,7 @@ defmodule Domain.FlowsTest do account: account, subject: subject, client: client, + actor_group_membership: membership, policy: policy, resource: resource, gateway: gateway @@ -438,6 +385,7 @@ defmodule Domain.FlowsTest do client: client, gateway: gateway, resource: resource, + membership: membership, policy: policy, subject: subject } do @@ -446,6 +394,7 @@ defmodule Domain.FlowsTest do account: account, subject: subject, client: client, + actor_group_membership: membership, policy: policy, resource: resource, gateway: gateway @@ -470,6 +419,72 @@ defmodule Domain.FlowsTest do end end + describe "all_gateway_flows_for_cache!/1" do + test "returns the later of two client_id/resource_id pair", %{ + account: account, + client: client, + gateway: gateway, + membership: membership, + resource: resource, + policy: policy, + subject: subject + } do + flow1 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: client, + actor_group_membership: membership, + policy: policy, + resource: resource, + gateway: gateway + ) + + flow2 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: client, + actor_group_membership: membership, + policy: policy, + resource: resource, + gateway: gateway + ) + + assert flow1.client_id == flow2.client_id + assert flow1.resource_id == flow2.resource_id + + assert DateTime.compare(flow2.inserted_at, flow1.inserted_at) == :gt + + assert [{{flow2.client_id, flow2.resource_id}, flow2.inserted_at}] == + Flows.all_gateway_flows_for_cache!(gateway) + end + + test "returns flow when only one unique one exists", %{ + account: account, + client: client, + gateway: gateway, + membership: membership, + resource: resource, + policy: policy, + subject: subject + } do + flow = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: client, + actor_group_membership: membership, + policy: policy, + resource: resource, + gateway: gateway + ) + + assert [{{flow.client_id, flow.resource_id}, flow.inserted_at}] == + Flows.all_gateway_flows_for_cache!(gateway) + end + end + describe "list_flows_for/3" do test "returns empty list when there are no flows", %{ actor: actor, @@ -508,6 +523,7 @@ defmodule Domain.FlowsTest do actor: actor, client: client, gateway: gateway, + membership: membership, resource: resource, policy: policy, subject: subject @@ -517,6 +533,7 @@ defmodule Domain.FlowsTest do account: account, subject: subject, client: client, + actor_group_membership: membership, policy: policy, resource: resource, gateway: gateway @@ -572,12 +589,13 @@ defmodule Domain.FlowsTest do end end - describe "expire_flows_for/1" do + describe "delete_flows_for/1" do setup %{ account: account, client: client, gateway: gateway, resource: resource, + membership: membership, policy: policy, subject: subject } do @@ -588,6 +606,7 @@ defmodule Domain.FlowsTest do account: account, subject: subject, client: client, + actor_group_membership: membership, policy: policy, resource: resource, gateway: gateway @@ -596,186 +615,304 @@ defmodule Domain.FlowsTest do %{flow: flow} end - test "expires flows for policy actor group", %{ - flow: flow, - actor_group: actor_group + test "deletes flows for account", %{ + account: account } do - :ok = Domain.PubSub.Flow.subscribe(flow.id) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert :ok = expire_flows_for(actor_group) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + assert {1, nil} = delete_flows_for(account) end - test "expires flows for client identity", %{ - flow: flow, - identity: identity + test "deletes flows for membership", %{ + membership: membership } do - :ok = Domain.PubSub.Flow.subscribe(flow.id) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert :ok = expire_flows_for(identity) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + assert {1, nil} = delete_flows_for(membership) end - test "expires flows for client", %{ - flow: flow, + test "deletes flows for client", %{ client: client } do - :ok = Domain.PubSub.Flow.subscribe(flow.id) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert :ok = expire_flows_for(client) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + assert {1, nil} = delete_flows_for(client) + end + + test "deletes flows for gateway", %{ + gateway: gateway + } do + assert {1, nil} = delete_flows_for(gateway) + end + + test "deletes flows for policy", %{ + policy: policy + } do + assert {1, nil} = delete_flows_for(policy) + end + + test "deletes flows for resource", %{ + resource: resource + } do + assert {1, nil} = delete_flows_for(resource) + end + + test "deletes flows for token", %{ + subject: subject + } do + {:ok, token} = Domain.Tokens.fetch_token_by_id(subject.token_id, subject) + + assert {1, nil} = delete_flows_for(token) end end - describe "expire_flows_for/2" do + describe "delete_stale_flows_on_connect/2" do setup %{ + account: account + } do + # Create additional resources for testing + resource2 = Fixtures.Resources.create_resource(account: account) + resource3 = Fixtures.Resources.create_resource(account: account) + + %{ + resource2: resource2, + resource3: resource3 + } + end + + test "deletes flows for resources not in authorized list", %{ account: account, client: client, gateway: gateway, - resource: resource, + resource: resource1, + resource2: resource2, + resource3: resource3, + membership: membership, policy: policy, subject: subject } do - subject = %{subject | expires_at: DateTime.utc_now() |> DateTime.add(1, :day)} - - flow = + # Create flows for multiple resources + flow1 = Fixtures.Flows.create_flow( account: account, subject: subject, client: client, + actor_group_membership: membership, policy: policy, - resource: resource, + resource: resource1, gateway: gateway ) - %{flow: flow} + flow2 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: client, + actor_group_membership: membership, + policy: policy, + resource: resource2, + gateway: gateway + ) + + flow3 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: client, + actor_group_membership: membership, + policy: policy, + resource: resource3, + gateway: gateway + ) + + # Only authorize resource1 and resource2, resource3 should be deleted + authorized_resources = [resource1, resource2] + + assert {1, nil} = delete_stale_flows_on_connect(client, authorized_resources) + + # Verify flow3 was deleted but flow1 and flow2 remain + assert {:ok, ^flow1} = fetch_flow_by_id(flow1.id, subject) + assert {:ok, ^flow2} = fetch_flow_by_id(flow2.id, subject) + assert {:error, :not_found} = fetch_flow_by_id(flow3.id, subject) end - test "expires flows for actor id and policy actor group id", %{ - flow: flow, - actor: actor, - policy: policy - } do - :ok = Domain.PubSub.Flow.subscribe(flow.id) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - - assert :ok = expire_flows_for(actor.account_id, actor.id, policy.actor_group_id) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - end - - test "expires flows for actor", %{ - flow: flow, - actor: actor, + test "deletes no flows when all resources are authorized", %{ + account: account, + client: client, + gateway: gateway, + resource: resource1, + resource2: resource2, + membership: membership, + policy: policy, subject: subject } do - :ok = Domain.PubSub.Flow.subscribe(flow.id) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert :ok = expire_flows_for(actor, subject) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + flow1 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: client, + actor_group_membership: membership, + policy: policy, + resource: resource1, + gateway: gateway + ) + + flow2 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: client, + actor_group_membership: membership, + policy: policy, + resource: resource2, + gateway: gateway + ) + + # Authorize all resources + authorized_resources = [resource1, resource2] + + assert {0, nil} = delete_stale_flows_on_connect(client, authorized_resources) + + # Verify both flows still exist + assert {:ok, ^flow1} = fetch_flow_by_id(flow1.id, subject) + assert {:ok, ^flow2} = fetch_flow_by_id(flow2.id, subject) end - test "expires flows for policy", %{ - flow: flow, - policy: policy - } do - :ok = Domain.PubSub.Flow.subscribe(flow.id) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert :ok = expire_flows_for_policy_id(policy.account_id, policy.id) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - end - - test "expires flows for resource", %{ - flow: flow, - resource: resource, + test "deletes all flows when authorized list is empty", %{ + account: account, + client: client, + gateway: gateway, + resource: resource1, + resource2: resource2, + membership: membership, + policy: policy, subject: subject } do - :ok = Domain.PubSub.Flow.subscribe(flow.id) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert :ok = expire_flows_for(resource, subject) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + flow1 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: client, + actor_group_membership: membership, + policy: policy, + resource: resource1, + gateway: gateway + ) + + flow2 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: client, + actor_group_membership: membership, + policy: policy, + resource: resource2, + gateway: gateway + ) + + # Empty authorized list - all flows should be deleted + authorized_resources = [] + + assert {2, nil} = delete_stale_flows_on_connect(client, authorized_resources) + + # Verify both flows were deleted + assert {:error, :not_found} = fetch_flow_by_id(flow1.id, subject) + assert {:error, :not_found} = fetch_flow_by_id(flow2.id, subject) end - test "expires flows for policy actor group", %{ - flow: flow, - actor_group: actor_group, + test "only affects flows for the specified client", %{ + account: account, + client: client, + gateway: gateway, + resource: resource1, + resource2: resource2, + membership: membership, + policy: policy, subject: subject } do - :ok = Domain.PubSub.Flow.subscribe(flow.id) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert :ok = expire_flows_for(actor_group, subject) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + # Create another client + other_client = Fixtures.Clients.create_client(account: account) + + # Create flows for both clients with the same resources + flow1 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: client, + actor_group_membership: membership, + policy: policy, + resource: resource1, + gateway: gateway + ) + + flow2 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: other_client, + actor_group_membership: membership, + policy: policy, + resource: resource1, + gateway: gateway + ) + + # Only authorize resource2 for the first client (resource1 should be deleted) + authorized_resources = [resource2] + + assert {1, nil} = delete_stale_flows_on_connect(client, authorized_resources) + + # Verify only the first client's flow was deleted + assert {:error, :not_found} = fetch_flow_by_id(flow1.id, subject) + assert {:ok, ^flow2} = fetch_flow_by_id(flow2.id, subject) end - test "expires flows for client identity", %{ - flow: flow, - identity: identity, + test "only affects flows for the specified account", %{ + account: account, + client: client, + gateway: gateway, + resource: resource1, + membership: membership, + policy: policy, subject: subject } do - :ok = Domain.PubSub.Flow.subscribe(flow.id) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert :ok = expire_flows_for(identity, subject) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + # Create flows in the current account + flow1 = + Fixtures.Flows.create_flow( + account: account, + subject: subject, + client: client, + actor_group_membership: membership, + policy: policy, + resource: resource1, + gateway: gateway + ) + + # Create flows in a different account + Fixtures.Flows.create_flow() + + # Empty authorized list for current account + authorized_resources = [] + + assert {1, nil} = delete_stale_flows_on_connect(client, authorized_resources) + + # Verify only the current account's flow was deleted + assert {:error, :not_found} = fetch_flow_by_id(flow1.id, subject) + # We can't easily verify flow2 still exists since it's in another account, + # but the fact that only 1 flow was deleted confirms account isolation end - test "expires flows for client identity provider", %{ - flow: flow, - provider: provider, - subject: subject + test "handles case when no flows exist for client", %{ + client: client } do - :ok = Domain.PubSub.Flow.subscribe(flow.id) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - assert :ok = expire_flows_for(provider, subject) - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} + # Try to delete stale flows for a client with no flows + authorized_resources = [] + + assert {0, nil} = delete_stale_flows_on_connect(client, authorized_resources) end - test "returns error when subject has no permission to expire flows", %{ - resource: resource, - subject: subject - } do - subject = Fixtures.Auth.remove_permissions(subject) - - assert expire_flows_for(resource, subject) == - {:error, - {:unauthorized, - reason: :missing_permissions, - missing_permissions: [Authorizer.create_flows_permission()]}} - end - - test "does not do anything on state conflict", %{ - resource: resource, - actor_group: actor_group, - subject: subject - } do - assert :ok = expire_flows_for(resource, subject) - assert :ok = expire_flows_for(actor_group, subject) - assert :ok = expire_flows_for(resource, subject) - end - - test "does not expire flows outside of account", %{ + test "handles case when client has no flows but resources are provided", %{ + account: account, resource: resource } do - subject = Fixtures.Auth.create_subject() - assert :ok = expire_flows_for(resource, subject) + # Create a client with no flows + client_with_no_flows = Fixtures.Clients.create_client(account: account) + authorized_resources = [resource] + + assert {0, nil} = delete_stale_flows_on_connect(client_with_no_flows, authorized_resources) end end end diff --git a/elixir/apps/domain/test/domain/notifications/jobs/outdated_gateways_test.exs b/elixir/apps/domain/test/domain/notifications/jobs/outdated_gateways_test.exs index 8428bb37c..77826be97 100644 --- a/elixir/apps/domain/test/domain/notifications/jobs/outdated_gateways_test.exs +++ b/elixir/apps/domain/test/domain/notifications/jobs/outdated_gateways_test.exs @@ -1,7 +1,7 @@ defmodule Domain.Notifications.Jobs.OutdatedGatewaysTest do use Domain.DataCase, async: true import Domain.Notifications.Jobs.OutdatedGateways - alias Domain.{ComponentVersions, Gateways, PubSub} + alias Domain.{ComponentVersions, Gateways} describe "execute/1" do setup do @@ -44,7 +44,6 @@ defmodule Domain.Notifications.Jobs.OutdatedGatewaysTest do :ok = Gateways.Presence.Group.subscribe(gateway_group.id) {:ok, _} = Gateways.Presence.Group.track(gateway.group_id, gateway.id) {:ok, _} = Gateways.Presence.Account.track(gateway.account_id, gateway.id) - :ok = PubSub.Gateway.subscribe(gateway.id) assert_receive %Phoenix.Socket.Broadcast{topic: "presences:group_gateways:" <> _} assert execute(%{}) == :ok @@ -70,7 +69,7 @@ defmodule Domain.Notifications.Jobs.OutdatedGatewaysTest do :ok = Gateways.Presence.Group.subscribe(gateway_group.id) {:ok, _} = Gateways.Presence.Group.track(gateway.group_id, gateway.id) {:ok, _} = Gateways.Presence.Account.track(gateway.account_id, gateway.id) - :ok = PubSub.Gateway.subscribe(gateway.id) + assert_receive %Phoenix.Socket.Broadcast{topic: "presences:group_gateways:" <> _} assert execute(%{}) == :ok diff --git a/elixir/apps/domain/test/domain/resources_test.exs b/elixir/apps/domain/test/domain/resources_test.exs index c36bd0a6e..ed640f7ff 100644 --- a/elixir/apps/domain/test/domain/resources_test.exs +++ b/elixir/apps/domain/test/domain/resources_test.exs @@ -3,7 +3,6 @@ defmodule Domain.ResourcesTest do import Domain.Resources alias Domain.Resources alias Domain.Actors - alias Domain.Events setup do account = Fixtures.Accounts.create_account() @@ -222,222 +221,6 @@ defmodule Domain.ResourcesTest do end end - describe "fetch_and_authorize_resource_by_id/3" do - test "returns error when resource does not exist", %{subject: subject} do - assert fetch_and_authorize_resource_by_id(Ecto.UUID.generate(), subject) == - {:error, :not_found} - end - - test "returns error when UUID is invalid", %{subject: subject} do - assert fetch_and_authorize_resource_by_id("foo", subject) == {:error, :not_found} - end - - test "returns authorized resource for account admin", %{ - account: account, - actor: actor, - subject: subject - } do - resource = Fixtures.Resources.create_resource(account: account) - actor_group = Fixtures.Actors.create_group(account: account) - Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) - - assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found} - - policy = - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource - ) - - assert {:ok, fetched_resource} = fetch_and_authorize_resource_by_id(resource.id, subject) - assert fetched_resource.id == resource.id - assert Enum.map(fetched_resource.authorized_by_policies, & &1.id) == [policy.id] - end - - test "returns authorized resource for account user", %{ - account: account - } do - actor_group = Fixtures.Actors.create_group(account: account) - actor = Fixtures.Actors.create_actor(type: :account_user, account: account) - Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) - - identity = Fixtures.Auth.create_identity(account: account, actor: actor) - subject = Fixtures.Auth.create_subject(identity: identity) - - resource = Fixtures.Resources.create_resource(account: account) - - assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found} - - policy = - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource - ) - - assert {:ok, fetched_resource} = fetch_and_authorize_resource_by_id(resource.id, subject) - assert fetched_resource.id == resource.id - assert Enum.map(fetched_resource.authorized_by_policies, & &1.id) == [policy.id] - end - - test "returns authorized resource using one of multiple policies for account user", %{ - account: account - } do - actor = Fixtures.Actors.create_actor(type: :account_user, account: account) - identity = Fixtures.Auth.create_identity(account: account, actor: actor) - subject = Fixtures.Auth.create_subject(identity: identity) - resource = Fixtures.Resources.create_resource(account: account) - - actor_group1 = Fixtures.Actors.create_group(account: account) - Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group1) - - policy1 = - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group1, - resource: resource - ) - - actor_group2 = Fixtures.Actors.create_group(account: account) - Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group2) - - policy2 = - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group2, - resource: resource - ) - - assert {:ok, fetched_resource} = fetch_and_authorize_resource_by_id(resource.id, subject) - assert fetched_resource.id == resource.id - - authorized_by_policy_ids = Enum.map(fetched_resource.authorized_by_policies, & &1.id) - policy_ids = [policy1.id, policy2.id] - assert Enum.sort(authorized_by_policy_ids) == Enum.sort(policy_ids) - end - - test "does not return deleted resources", %{account: account, actor: actor, subject: subject} do - {:ok, resource} = - Fixtures.Resources.create_resource(account: account) - |> delete_resource(subject) - - actor_group = Fixtures.Actors.create_group(account: account) - Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) - - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource - ) - - assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found} - end - - test "does not authorize using deleted policies", %{ - account: account, - actor: actor, - subject: subject - } do - resource = Fixtures.Resources.create_resource(account: account) - actor_group = Fixtures.Actors.create_group(account: account) - Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) - - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource - ) - |> Fixtures.Policies.delete_policy() - - assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found} - end - - test "does not authorize using deleted group membership", %{ - account: account, - subject: subject - } do - resource = Fixtures.Resources.create_resource(account: account) - actor_group = Fixtures.Actors.create_group(account: account) - - # memberships are not soft deleted - # Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) - - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource - ) - - assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found} - end - - test "does not authorize using disabled policies", %{ - account: account, - actor: actor, - subject: subject - } do - resource = Fixtures.Resources.create_resource(account: account) - actor_group = Fixtures.Actors.create_group(account: account) - Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) - - policy = - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource - ) - - {:ok, _policy} = Domain.Policies.disable_policy(policy, subject) - - assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found} - end - - test "does not return resources in other accounts", %{subject: subject} do - resource = Fixtures.Resources.create_resource() - assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found} - end - - test "returns error when subject has no permission to view resources", %{subject: subject} do - subject = Fixtures.Auth.remove_permissions(subject) - - assert fetch_and_authorize_resource_by_id(Ecto.UUID.generate(), subject) == - {:error, - {:unauthorized, - reason: :missing_permissions, - missing_permissions: [Resources.Authorizer.view_available_resources_permission()]}} - end - - test "associations are preloaded when opts given", %{ - account: account, - actor: actor, - subject: subject - } do - actor_group = Fixtures.Actors.create_group(account: account) - Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group) - - gateway_group = Fixtures.Gateways.create_group(account: account) - - resource = - Fixtures.Resources.create_resource( - account: account, - connections: [%{gateway_group_id: gateway_group.id}] - ) - - Fixtures.Policies.create_policy( - account: account, - actor_group: actor_group, - resource: resource - ) - - assert {:ok, resource} = - fetch_and_authorize_resource_by_id(resource.id, subject, preload: :connections) - - assert Ecto.assoc_loaded?(resource.connections) - assert length(resource.connections) == 1 - end - end - describe "all_authorized_resources/1" do test "returns empty list when there are no resources", %{subject: subject} do assert {:ok, []} = all_authorized_resources(subject) @@ -1525,23 +1308,6 @@ defmodule Domain.ResourcesTest do assert resource.address_description == attrs["address_description"] end - test "does not expire flows when connections are not updated", %{ - account: account, - resource: resource, - subject: subject - } do - flow = Fixtures.Flows.create_flow(account: account, resource: resource, subject: subject) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - - :ok = Domain.PubSub.Flow.subscribe(flow_id) - - attrs = %{"name" => "foo"} - assert {:ok, _resource} = update_resource(resource, attrs, subject) - refute_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - end - test "allows to update connections", %{account: account, resource: resource, subject: subject} do group = Fixtures.Gateways.create_group(account: account, subject: subject) gateway1 = Fixtures.Gateways.create_gateway(account: account, group: group) @@ -1553,13 +1319,6 @@ defmodule Domain.ResourcesTest do gateway2 = Fixtures.Gateways.create_gateway(account: account) - flow = Fixtures.Flows.create_flow(account: account, resource: resource, subject: subject) - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - - :ok = Domain.PubSub.Flow.subscribe(flow_id) - attrs = %{ "connections" => [ %{gateway_group_id: gateway1.group_id}, @@ -1575,15 +1334,6 @@ defmodule Domain.ResourcesTest do assert {:ok, resource} = update_resource(resource, attrs, subject) gateway_group_ids = Enum.map(resource.connections, & &1.gateway_group_id) assert gateway_group_ids == [gateway2.group_id] - - # TODO: WAL - # Remove this when directly broadcasting flow removals - Events.Hooks.ResourceConnections.on_delete(%{ - "account_id" => resource.account_id, - "resource_id" => resource.id - }) - - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} end test "does not allow to remove all connections", %{resource: resource, subject: subject} do diff --git a/elixir/apps/domain/test/domain/schema_helpers_test.exs b/elixir/apps/domain/test/domain/schema_helpers_test.exs index b7ea75d8b..aed328237 100644 --- a/elixir/apps/domain/test/domain/schema_helpers_test.exs +++ b/elixir/apps/domain/test/domain/schema_helpers_test.exs @@ -7,21 +7,15 @@ defmodule Domain.SchemaHelpersTest do defmodule NestedEmbedSchema do use Ecto.Schema - import Ecto.Changeset @primary_key false embedded_schema do field :nested_field, :string end - - def changeset(struct, params) do - cast(struct, params, [:nested_field]) - end end defmodule EmbeddedSchema do use Ecto.Schema - import Ecto.Changeset @primary_key false embedded_schema do @@ -29,26 +23,34 @@ defmodule Domain.SchemaHelpersTest do field :sub_field2, :string embeds_one :nested_item, NestedEmbedSchema end - - def changeset(struct, params) do - struct - |> cast(params, [:sub_field1, :sub_field2]) - |> cast_embed(:nested_item) - end end defmodule ListItemSchema do use Ecto.Schema - import Ecto.Changeset @primary_key false embedded_schema do field :list_field1, :string field :list_field2, :string, default: "default_value" end + end - def changeset(struct, params) do - cast(struct, params, [:list_field1, :list_field2]) + defmodule DateTimeSchema do + use Ecto.Schema + + @primary_key false + embedded_schema do + field :datetime_field, :utc_datetime_usec + end + end + + defmodule NestedEnumSchema do + use Ecto.Schema + + @primary_key false + embedded_schema do + field :enum_field, Ecto.Enum, values: ~w[option1 option2 option3]a + field :values, {:array, :string} end end @@ -183,5 +185,144 @@ defmodule Domain.SchemaHelpersTest do assert %RootSchema{field1: "Empty Embedded", embedded_item: item} = result assert %EmbeddedSchema{sub_field1: nil, sub_field2: nil} = item end + + test "correctly casts ISO 8601 datetime string" do + params = %{ + "datetime_field" => "2023-12-25T10:30:45Z" + } + + result = SchemaHelpers.struct_from_params(DateTimeSchema, params) + + assert %DateTimeSchema{datetime_field: datetime} = result + assert %DateTime{} = datetime + assert datetime.year == 2023 + assert datetime.month == 12 + assert datetime.day == 25 + assert datetime.hour == 10 + assert datetime.minute == 30 + assert datetime.second == 45 + assert datetime.time_zone == "Etc/UTC" + end + + test "correctly casts ISO 8601 datetime string with microseconds" do + params = %{ + "datetime_field" => "2023-12-25T10:30:45.123456Z" + } + + result = SchemaHelpers.struct_from_params(DateTimeSchema, params) + + assert %DateTimeSchema{datetime_field: datetime} = result + assert %DateTime{} = datetime + assert datetime.microsecond == {123_456, 6} + end + + test "correctly casts ISO 8601 datetime string with timezone offset" do + params = %{ + "datetime_field" => "2023-12-25T10:30:45+02:00" + } + + result = SchemaHelpers.struct_from_params(DateTimeSchema, params) + + assert %DateTimeSchema{datetime_field: datetime} = result + assert %DateTime{} = datetime + # Should be converted to UTC + assert datetime.time_zone == "Etc/UTC" + # Should be adjusted for timezone (10:30 +02:00 = 08:30 UTC) + assert datetime.hour == 8 + assert datetime.minute == 30 + end + + test "correctly casts ISO 8601 datetime string with negative timezone offset" do + params = %{ + "datetime_field" => "2023-12-25T10:30:45-05:00" + } + + result = SchemaHelpers.struct_from_params(DateTimeSchema, params) + + assert %DateTimeSchema{datetime_field: datetime} = result + assert %DateTime{} = datetime + # Should be converted to UTC + assert datetime.time_zone == "Etc/UTC" + # Should be adjusted for timezone (10:30 -05:00 = 15:30 UTC) + assert datetime.hour == 15 + assert datetime.minute == 30 + end + + test "handles nil datetime field" do + params = %{ + "datetime_field" => nil + } + + result = SchemaHelpers.struct_from_params(DateTimeSchema, params) + + assert %DateTimeSchema{datetime_field: nil} = result + end + + test "handles missing datetime field" do + params = %{} + + result = SchemaHelpers.struct_from_params(DateTimeSchema, params) + + assert %DateTimeSchema{datetime_field: nil} = result + end + + test "handles DateTime struct input" do + datetime = DateTime.utc_now() + + params = %{ + "datetime_field" => datetime + } + + result = SchemaHelpers.struct_from_params(DateTimeSchema, params) + + assert %DateTimeSchema{datetime_field: ^datetime} = result + end + + test "handles invalid datetime string gracefully" do + params = %{ + "datetime_field" => "invalid-datetime" + } + + result = SchemaHelpers.struct_from_params(DateTimeSchema, params) + + # Ecto casting should handle invalid datetime strings by setting to nil + # or keeping the original value depending on changeset validation + assert %DateTimeSchema{} = result + end + + test "correctly casts datetime string without 'Z' suffix" do + params = %{ + "datetime_field" => "2023-12-25T10:30:45" + } + + result = SchemaHelpers.struct_from_params(DateTimeSchema, params) + + assert %DateTimeSchema{datetime_field: datetime} = result + # Should still parse as UTC when no timezone is specified + assert %DateTime{} = datetime + assert datetime.year == 2023 + assert datetime.month == 12 + assert datetime.day == 25 + end + + test "correctly casts NaiveDateTime to DateTime" do + naive_datetime = ~N[2023-12-25 10:30:45] + + params = %{ + "datetime_field" => naive_datetime + } + + result = SchemaHelpers.struct_from_params(DateTimeSchema, params) + + assert %DateTimeSchema{datetime_field: datetime} = result + assert %DateTime{} = datetime + assert datetime.year == 2023 + assert datetime.month == 12 + assert datetime.day == 25 + assert datetime.hour == 10 + assert datetime.minute == 30 + assert datetime.second == 45 + assert datetime.time_zone == "Etc/UTC" + end end end diff --git a/elixir/apps/domain/test/domain/tokens_test.exs b/elixir/apps/domain/test/domain/tokens_test.exs index 77bba4cf3..b2d1dc1d0 100644 --- a/elixir/apps/domain/test/domain/tokens_test.exs +++ b/elixir/apps/domain/test/domain/tokens_test.exs @@ -498,7 +498,7 @@ defmodule Domain.TokensTest do } do other_token = Fixtures.Tokens.create_token(account: account, identity: identity) - assert {:ok, deleted_token} = delete_token_for(subject) + assert {:ok, [deleted_token]} = delete_token_for(subject) assert deleted_token.id == subject.token_id assert Repo.get(Tokens.Token, subject.token_id).deleted_at @@ -506,29 +506,6 @@ defmodule Domain.TokensTest do refute Repo.get(Tokens.Token, other_token.id).deleted_at end - test "expires flows for given subject", %{ - account: account, - identity: identity, - subject: subject - } do - flow = - Fixtures.Flows.create_flow( - account: account, - identity: identity, - subject: subject - ) - - :ok = Domain.PubSub.Flow.subscribe(flow.id) - - assert {:ok, _token} = delete_token_for(subject) - - flow_id = flow.id - client_id = flow.client_id - resource_id = flow.resource_id - - assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id} - end - test "does not delete tokens for other actors", %{account: account, subject: subject} do token = Fixtures.Tokens.create_token(account: account) diff --git a/elixir/apps/domain/test/support/fixtures/flows.ex b/elixir/apps/domain/test/support/fixtures/flows.ex index 324fe7e41..e3c1702f1 100644 --- a/elixir/apps/domain/test/support/fixtures/flows.ex +++ b/elixir/apps/domain/test/support/fixtures/flows.ex @@ -52,11 +52,15 @@ defmodule Domain.Fixtures.Flows do |> Fixtures.Resources.create_resource() end) - {actor_group_id, attrs} = - pop_assoc_fixture_id(attrs, :actor_group, fn assoc_attrs -> + {membership, attrs} = + pop_assoc_fixture(attrs, :actor_group_membership, fn assoc_attrs -> assoc_attrs - |> Enum.into(%{account: account, subject: subject}) - |> Fixtures.Actors.create_group() + |> Enum.into(%{ + account: account, + subject: subject, + actor_id: client.actor_id + }) + |> Fixtures.Actors.create_membership() end) {policy_id, attrs} = @@ -64,14 +68,17 @@ defmodule Domain.Fixtures.Flows do assoc_attrs |> Enum.into(%{ account: account, - actor_group_id: actor_group_id, + actor_group_id: membership.group_id, resource_id: resource_id, subject: subject }) |> Fixtures.Policies.create_policy() end) - {token_id, _attrs} = Map.pop(attrs, :token_id, subject.token_id) + {token_id, _attrs} = + pop_assoc_fixture_id(attrs, :token, fn _assoc_attrs -> + %{id: subject.token_id} + end) Flows.Flow.Changeset.create(%{ token_id: token_id, @@ -79,6 +86,7 @@ defmodule Domain.Fixtures.Flows do client_id: client.id, gateway_id: gateway.id, resource_id: resource_id, + actor_group_membership_id: membership.id, account_id: account.id, client_remote_ip: client.last_seen_remote_ip, client_user_agent: client.last_seen_user_agent, diff --git a/elixir/apps/web/lib/web.ex b/elixir/apps/web/lib/web.ex index d89db5447..d70fa1eef 100644 --- a/elixir/apps/web/lib/web.ex +++ b/elixir/apps/web/lib/web.ex @@ -67,12 +67,6 @@ defmodule Web do {:noreply, socket} end end - - # ignore "disconnect" message that is broadcasted for some pages - # because of subscription for relay/gateway group events - def handle_info("disconnect", socket) do - {:noreply, socket} - end end end diff --git a/elixir/apps/web/lib/web/live/actors/show.ex b/elixir/apps/web/lib/web/live/actors/show.ex index cd95340c2..eda3f910d 100644 --- a/elixir/apps/web/lib/web/live/actors/show.ex +++ b/elixir/apps/web/lib/web/live/actors/show.ex @@ -630,6 +630,8 @@ defmodule Web.Actors.Show do {:noreply, reload_live_table!(socket, "clients")} end + def handle_info(_message, socket), do: {:noreply, socket} + def handle_event(event, params, socket) when event in ["paginate", "order_by", "filter"], do: handle_live_table_event(event, params, socket) diff --git a/elixir/apps/web/lib/web/live/policies/index.ex b/elixir/apps/web/lib/web/live/policies/index.ex index cc82c8e47..35d6109bd 100644 --- a/elixir/apps/web/lib/web/live/policies/index.ex +++ b/elixir/apps/web/lib/web/live/policies/index.ex @@ -4,7 +4,7 @@ defmodule Web.Policies.Index do def mount(_params, _session, socket) do if connected?(socket) do - :ok = PubSub.Account.Policies.subscribe(socket.assigns.account.id) + :ok = PubSub.Account.subscribe(socket.assigns.account.id) end socket = @@ -123,7 +123,15 @@ defmodule Web.Policies.Index do when event in ["paginate", "order_by", "filter", "reload"], do: handle_live_table_event(event, params, socket) - def handle_info({_action, _policy_id}, socket) do + def handle_info({_action, %Policies.Policy{}, %Policies.Policy{}}, socket) do {:noreply, assign(socket, stale: true)} end + + def handle_info({_action, %Policies.Policy{}}, socket) do + {:noreply, assign(socket, stale: true)} + end + + def handle_info(_, socket) do + {:noreply, socket} + end end diff --git a/elixir/apps/web/lib/web/live/policies/show.ex b/elixir/apps/web/lib/web/live/policies/show.ex index 236d16715..7aefd403a 100644 --- a/elixir/apps/web/lib/web/live/policies/show.ex +++ b/elixir/apps/web/lib/web/live/policies/show.ex @@ -16,7 +16,7 @@ defmodule Web.Policies.Show do providers = Auth.all_active_providers_for_account!(socket.assigns.account) if connected?(socket) do - :ok = PubSub.Policy.subscribe(policy.id) + :ok = PubSub.Account.subscribe(policy.account_id) end socket = @@ -302,7 +302,12 @@ defmodule Web.Policies.Show do """ end - def handle_info({_action, _policy_id}, socket) do + # TODO: Do we really want to update the view in place? + def handle_info( + {_action, _old_policy, %Policies.Policy{id: policy_id}}, + %{assigns: %{policy: %{id: id}}} = socket + ) + when policy_id == id do {:ok, policy} = Policies.fetch_policy_by_id_or_persistent_id( socket.assigns.policy.id, @@ -318,6 +323,10 @@ defmodule Web.Policies.Show do {:noreply, assign(socket, policy: policy)} end + def handle_info(_, socket) do + {:noreply, socket} + end + def handle_event(event, params, socket) when event in ["paginate", "order_by", "filter"], do: handle_live_table_event(event, params, socket) diff --git a/elixir/apps/web/lib/web/live/resources/index.ex b/elixir/apps/web/lib/web/live/resources/index.ex index 50859c609..6dce207b5 100644 --- a/elixir/apps/web/lib/web/live/resources/index.ex +++ b/elixir/apps/web/lib/web/live/resources/index.ex @@ -5,7 +5,7 @@ defmodule Web.Resources.Index do def mount(_params, _session, socket) do if connected?(socket) do - :ok = PubSub.Account.Resources.subscribe(socket.assigns.account.id) + :ok = PubSub.Account.subscribe(socket.assigns.account.id) end socket = @@ -170,7 +170,15 @@ defmodule Web.Resources.Index do when event in ["paginate", "order_by", "filter", "reload"], do: handle_live_table_event(event, params, socket) - def handle_info({_action, _resource_id}, socket) do + def handle_info({_action, %Resources.Resource{}, %Resources.Resource{}}, socket) do {:noreply, assign(socket, stale: true)} end + + def handle_info({_action, %Resources.Resource{}}, socket) do + {:noreply, assign(socket, stale: true)} + end + + def handle_info(_, socket) do + {:noreply, socket} + end end diff --git a/elixir/apps/web/lib/web/live/resources/show.ex b/elixir/apps/web/lib/web/live/resources/show.ex index b12afd45a..ef687838b 100644 --- a/elixir/apps/web/lib/web/live/resources/show.ex +++ b/elixir/apps/web/lib/web/live/resources/show.ex @@ -9,7 +9,7 @@ defmodule Web.Resources.Show do {:ok, actor_groups_peek} <- Resources.peek_resource_actor_groups([resource], 3, socket.assigns.subject) do if connected?(socket) do - :ok = PubSub.Resource.subscribe(resource.id) + :ok = PubSub.Account.subscribe(resource.account_id) end socket = @@ -399,7 +399,12 @@ defmodule Web.Resources.Show do """ end - def handle_info({_action, _resource_id}, socket) do + # TODO: Do we really want to update the view in place? + def handle_info( + {_action, _old_resource, %Resources.Resource{id: resource_id}}, + %{assigns: %{resource: %{id: id}}} = socket + ) + when resource_id == id do {:ok, resource} = Resources.fetch_resource_by_id(socket.assigns.resource.id, socket.assigns.subject, preload: [ @@ -413,6 +418,10 @@ defmodule Web.Resources.Show do {:noreply, assign(socket, resource: resource)} end + def handle_info(_, socket) do + {:noreply, socket} + end + def handle_event(event, params, socket) when event in ["paginate", "order_by", "filter"], do: handle_live_table_event(event, params, socket) diff --git a/elixir/apps/web/test/web/acceptance/auth_test.exs b/elixir/apps/web/test/web/acceptance/auth_test.exs index 9fea400c3..b4084f0b9 100644 --- a/elixir/apps/web/test/web/acceptance/auth_test.exs +++ b/elixir/apps/web/test/web/acceptance/auth_test.exs @@ -92,7 +92,17 @@ defmodule Web.Acceptance.AuthTest do for token <- tokens do assert %DateTime{} = token.deleted_at - Domain.Events.Hooks.Tokens.on_delete(%{"id" => token.id}) + + Domain.Events.Hooks.Tokens.on_delete(%{ + "remaining_attempts" => token.remaining_attempts, + "actor_id" => token.actor_id, + "name" => token.name, + "type" => "#{token.type}", + "account_id" => token.account_id, + "id" => token.id, + "identity_id" => token.identity_id, + "expires_at" => "#{token.expires_at}" + }) end wait_for( diff --git a/elixir/apps/web/test/web/live/actors/show_test.exs b/elixir/apps/web/test/web/live/actors/show_test.exs index 525be8862..8f59ccd49 100644 --- a/elixir/apps/web/test/web/live/actors/show_test.exs +++ b/elixir/apps/web/test/web/live/actors/show_test.exs @@ -184,7 +184,7 @@ defmodule Web.Live.Actors.ShowTest do "#{flow.gateway.group.name}-#{flow.gateway.name} #{flow.gateway.last_seen_remote_ip}" end - test "renders flows even for deleted policy assocs", %{ + test "does not render flows for deleted policy assocs", %{ conn: conn } do account = Fixtures.Accounts.create_account() @@ -207,21 +207,11 @@ defmodule Web.Live.Actors.ShowTest do |> authorize_conn(identity) |> live(~p"/#{account}/actors/#{actor}") - [row] = - lv - |> element("#flows") - |> render() - |> table_to_map() - - assert row["authorized"] - assert row["policy"] =~ flow.policy.actor_group.name - assert row["policy"] =~ flow.policy.resource.name - - assert row["client"] == - "#{flow.client.name} #{client.last_seen_remote_ip}" - - assert row["gateway"] == - "#{flow.gateway.group.name}-#{flow.gateway.name} #{flow.gateway.last_seen_remote_ip}" + assert [] == + lv + |> element("#flows") + |> render() + |> table_to_map() end test "renders groups table", %{ diff --git a/elixir/apps/web/test/web/live/clients/show_test.exs b/elixir/apps/web/test/web/live/clients/show_test.exs index 08583b797..63715fe37 100644 --- a/elixir/apps/web/test/web/live/clients/show_test.exs +++ b/elixir/apps/web/test/web/live/clients/show_test.exs @@ -248,7 +248,7 @@ defmodule Web.Live.Clients.ShowTest do "#{flow.gateway.group.name}-#{flow.gateway.name} #{flow.gateway.last_seen_remote_ip}" end - test "renders flows even for deleted policy assocs", %{ + test "does not render flows for deleted policy assocs", %{ account: account, identity: identity, client: client, @@ -269,19 +269,11 @@ defmodule Web.Live.Clients.ShowTest do |> authorize_conn(identity) |> live(~p"/#{account}/clients/#{client}") - [row] = - lv - |> element("#flows") - |> render() - |> table_to_map() - - assert row["authorized"] - assert row["remote ip"] == to_string(client.last_seen_remote_ip) - assert row["policy"] =~ flow.policy.actor_group.name - assert row["policy"] =~ flow.policy.resource.name - - assert row["gateway"] == - "#{flow.gateway.group.name}-#{flow.gateway.name} #{flow.gateway.last_seen_remote_ip}" + assert [] == + lv + |> element("#flows") + |> render() + |> table_to_map() end test "allows editing clients", %{ diff --git a/elixir/apps/web/test/web/live/resources/edit_test.exs b/elixir/apps/web/test/web/live/resources/edit_test.exs index 76740a5f9..9d8624594 100644 --- a/elixir/apps/web/test/web/live/resources/edit_test.exs +++ b/elixir/apps/web/test/web/live/resources/edit_test.exs @@ -1,6 +1,6 @@ defmodule Web.Live.Resources.EditTest do use Web.ConnCase, async: true - alias Domain.{Events, PubSub} + alias Domain.{Events, PubSub, Resources} setup do account = Fixtures.Accounts.create_account() @@ -229,7 +229,7 @@ defmodule Web.Live.Resources.EditTest do } } - :ok = PubSub.Account.Resources.subscribe(account.id) + :ok = PubSub.Account.subscribe(account.id) {:ok, lv, _html} = conn @@ -240,7 +240,7 @@ defmodule Web.Live.Resources.EditTest do "account_id" => resource.account_id, "type" => resource.type, "address" => resource.address, - "filters" => resource.filters, + "filters" => [%{"protocol" => "icmp", "ports" => ["80"]}], "ip_stack" => resource.ip_stack } @@ -253,9 +253,9 @@ defmodule Web.Live.Resources.EditTest do |> render_submit() |> follow_redirect(conn, ~p"/#{account}/resources") - assert_receive {:delete_resource, delete_msg_resource_id} - assert_receive {:create_resource, create_msg_resource_id} - assert delete_msg_resource_id == create_msg_resource_id + assert_receive {:updated, %Resources.Resource{}, %Resources.Resource{} = updated_resource} + + assert updated_resource.id == resource.id assert updated_resource = Repo.get_by(Domain.Resources.Resource, id: resource.id) assert updated_resource.name == attrs.name diff --git a/elixir/config/config.exs b/elixir/config/config.exs index 6d66a3c75..25bb8b8c0 100644 --- a/elixir/config/config.exs +++ b/elixir/config/config.exs @@ -99,13 +99,10 @@ config :domain, Domain.Events.ReplicationConnection, table_subscriptions: ~w[ accounts actor_group_memberships - actor_groups - actors - auth_identities - auth_providers clients - gateway_groups + flows gateways + gateway_groups policies resource_connections resources