refactor(portal): cache access state in channel pids (#9773)

When changes occur in the Firezone DB that trigger side effects, we need
some mechanism to broadcast and handle these.

Before, the system we used was:

- Each process subscribes to a myriad of topics related to data it wants
to receive. In some cases it would subscribe to new topics based on
received events from existing topics (I.e. flows in the gateway
channel), and sometimes in a loop. It would then need to be sure to
_unsubscribe_ from these topics
- Handle the side effect in the `after_commit` hook of the Ecto function
call after it completes
- Broadcast only a simply (thin) event message with a DB id
- In the receiver, use the id(s) to re-evaluate, or lookup one or many
records associated with the change
- After the lookup completes, `push` the relevant message(s) to the
LiveView, `client` pid, or `gateway` pid in their respective channel
processes

This system had a number of drawbacks ranging from scalability issues to
undesirable access bugs:

1. The `after_commit` callback, on each App node, is not globally
ordered. Since we broadcast a thin event schema and read from the DB to
hydrate each event, this meant we had a `read after write` problem in
our event architecture, leading to the potential for lost updates. Case
in point: if a policy is updated from `resource_id-1` to
`resource_id-2`, and then back to `resource_id-1`, it's possible that,
given the right amount of delay, the gateway channel will receive two
`reject_access` events for `resource_id-1`, as opposed to one for
`resource_id-1` and one for `resource_id-2`, leading to the potential
for unauthorized access.
1. It was very difficult to ensure that the correct topics were being
subscribed to and unsubscribed from, and the correct number of times,
leading to maintenance issues for other engineers.
1. We had a nasty N+1 query problem whenever memberships were added or
removed that resolved in essentially all access related to that
membership (so all Policies touching its actor group) to be
re-evaluated, and broadcasted. This meant that any bulk addition or
deletion of memberships would generate so many queries that they'd
timeout or consume the entire connection pool.
1. We had no durability for side-effect processing. In some places, we
were iterating over many returned records to send broadcasts.
Broadcasting is not a zero-time operation, each call takes a small
amount of CPU time to copy the message into the receiver's mailbox. If
we deployed while this was happening, the state update would be lost
forever. If this was a `reject_access` for a Gateway, the Gateway would
never remove access for that particular flow.
1. On each flow authorization, we needed to hit `us-east1` not only to
"authorize" the flow, but to log it as well. This incurs latency
especially for users in other parts of the world, which happens on
_each_ connection setup to a new resource.
1. Since we read and re-authorize access due to the thin events
broadcasted from side effects, we risk hitting thundering herd problems
(see the N+1 query problem above) where a single DB change could result
in all receivers hitting the DB at once to "hydrate" their
processing.ion
1. If an administrator modifies the DB directly, or, if we need to run a
DB migration that involves side effects, they'll be lost, because the
side effect triggers happened in `after_commit` hooks that are only
available when querying the DB through Ecto. Manually deleting (or
resurrecting) a policy, for example, would not have updated any
connected clients or gateways with the new state.


To fix all of the above, we move to the system introduced in this PR:

- All changes are now serialized (for free) by Postgres and broadcasted
as a single event stream
- The number of topics has been reduced to just one, the `account_id` of
an account. All receivers subscribe to this one topic for the lifetime
of their pid and then only filter the events they want to act upon,
ignoring all other messages
- The events themselves have been turned into "fat" structs based on the
schemas they present. By making them properly typed, we can apply things
like the existing Policy authorizer functions to them as if we had just
fetched them from the DB.
- All flow creation now happens in memory and doesn't not need to incur
a DB hit in `us-east1` to proceed.
- Since clients and gateways now track state in a push-based manner from
the DB, this means very few actual DB queries are needed to maintain
state in the channel procs, and it also means we can be smarter about
when to send `resource_deleted` and `resource_created_or_updated`
appropriately, since we can always diff between what the client _had_
access to, and what they _now_ have access to.
- All DB operations, whether they happen from the application code, a
`psql` prompt, or even via Google SQL Studio in the GCP console, will
trigger the _same_ side effects.
- We now use a replication consumer based off Postgres logical decoding
of the write-ahead log using a _durable slot_. This means that Postgres
will retain _all events_ until they are acknowledged, giving us the
ability to ensure at-least-once processing semantics for our system.
Today, the ACK is simply, "did we broadcast this event successfully".
But in the future, we can assert that replies are received before we
acknowledge the event as processed back to Postgres.



The tests in this PR have been updated to pass given the refactor.
However, since we are tracking more state now in the channel procs, it
would be a good idea to add more tests for those edge cases. That is
saved as a later PR because (1) this one is already huge, and (2) we
need to get this out to staging to smoke test everything anyhow.

Fixes: #9908 
Fixes: #9909 
Fixes: #9910
Fixes: #9900 
Related: #9501

elixir/apps/api/lib/api/gateway/views/flow.ex

@@ -0,0 +1,11 @@
+defmodule API.Gateway.Views.Flow do
+  def render_many(flows) do
+    flows
+    |> Enum.map(fn {{client_id, resource_id}, _expires_at} ->
+      %{
+        client_id: client_id,
+        resource_id: resource_id
+      }
+    end)
+  end
+end

elixir/apps/domain/lib/domain/actors.ex

@@ -38,6 +38,13 @@ defmodule Domain.Actors do
     end
   end
 
+  def fetch_membership_by_actor_id_and_group_id(actor_id, group_id) do
+    Membership.Query.all()
+    |> Membership.Query.by_actor_id(actor_id)
+    |> Membership.Query.by_group_id(group_id)
+    |> Repo.fetch(Membership.Query, [])
+  end
+
   def list_groups(%Auth.Subject{} = subject, opts \\ []) do
     with :ok <- Auth.ensure_has_permissions(subject, Authorizer.manage_actors_permission()) do
       Group.Query.not_deleted()

elixir/apps/domain/lib/domain/actors/membership.ex

@@ -1,10 +1,9 @@
 defmodule Domain.Actors.Membership do
   use Domain, :schema
 
-  @primary_key false
   schema "actor_group_memberships" do
-    belongs_to :group, Domain.Actors.Group, primary_key: true
-    belongs_to :actor, Domain.Actors.Actor, primary_key: true
+    belongs_to :group, Domain.Actors.Group
+    belongs_to :actor, Domain.Actors.Actor
 
     belongs_to :account, Domain.Accounts.Account
   end

elixir/apps/domain/lib/domain/clients.ex

@@ -1,7 +1,7 @@
 defmodule Domain.Clients do
   use Supervisor
   alias Domain.{Repo, Auth}
-  alias Domain.{Accounts, Actors, Flows}
+  alias Domain.{Accounts, Actors}
   alias Domain.Clients.{Client, Authorizer, Presence}
   require Ecto.Query
 
@@ -211,16 +211,6 @@ defmodule Domain.Clients do
         with: &Client.Changeset.remove_verification(&1),
         preload: [:online?]
       )
-      |> case do
-        # TODO: WAL
-        # Broadcast flow side effects directly
-        {:ok, client} ->
-          :ok = Flows.expire_flows_for(client)
-          {:ok, client}
-
-        {:error, reason} ->
-          {:error, reason}
-      end
     end
   end
 
@@ -252,6 +242,10 @@ defmodule Domain.Clients do
     end
   end
 
+  # TODO: Hard delete
+  # We don't necessarily want to delete associated tokens when deleting a client because
+  # that token could be a multi-owner token in the case of a headless client.
+  # Instead we need to introduce the concept of ephemeral clients/gateways and permanent ones.
   defp delete_clients(queryable, subject) do
     {_count, clients} =
       queryable

elixir/apps/domain/lib/domain/clients/presence.ex

@@ -9,8 +9,7 @@ defmodule Domain.Clients.Presence do
   def connect(%Client{} = client) do
     with {:ok, _} <- __MODULE__.Account.track(client.account_id, client.id),
          {:ok, _} <- __MODULE__.Actor.track(client.actor_id, client.id) do
-      :ok = PubSub.Client.subscribe(client.id)
-      :ok = PubSub.Account.Clients.subscribe(client.account_id)
+      :ok
     end
   end
 

elixir/apps/domain/lib/domain/config/definitions.ex

@@ -687,6 +687,11 @@ defmodule Domain.Config.Definitions do
   """
   defconfig(:feature_idp_sync_enabled, :boolean, default: true)
 
+  @doc """
+  Boolean flag to turn UI flow activities on/off for all accounts.
+  """
+  defconfig(:feature_flow_activities_enabled, :boolean, default: false)
+
   @doc """
   Boolean flag to turn Account relays admin functionality on/off for all accounts.
   """

elixir/apps/domain/lib/domain/events/hooks.ex

@@ -3,7 +3,7 @@ defmodule Domain.Events.Hooks do
     A simple behavior to define hooks needed for processing WAL events.
   """
 
-  @callback on_insert(data :: map()) :: :ok
-  @callback on_update(old_data :: map(), data :: map()) :: :ok
-  @callback on_delete(old_data :: map()) :: :ok
+  @callback on_insert(data :: map()) :: :ok | {:error, term()}
+  @callback on_update(old_data :: map(), data :: map()) :: :ok | {:error, term()}
+  @callback on_delete(old_data :: map()) :: :ok | {:error, term()}
 end

elixir/apps/domain/lib/domain/events/hooks/accounts.ex

@@ -1,6 +1,6 @@
 defmodule Domain.Events.Hooks.Accounts do
   @behaviour Domain.Events.Hooks
-  alias Domain.PubSub
+  alias Domain.{Accounts, PubSub, SchemaHelpers}
   require Logger
 
   @impl true
@@ -9,34 +9,40 @@ defmodule Domain.Events.Hooks.Accounts do
   # Account slug changed - disconnect gateways for updated init
 
   @impl true
-  def on_update(%{"slug" => old_slug}, %{"slug" => slug, "id" => account_id} = _data)
-      when old_slug != slug do
-    # Technically we could push a :slug_changed message to the Gateways here,
-    # but at the time of writing, disconnecting and reconnecting is safer to ensure
-    # all relevant state on the Gateway is updated correctly.
-    PubSub.Account.Gateways.disconnect(account_id)
-  end
 
-  # Account disabled - disconnect clients
-  @impl true
+  # Account disabled - process as a delete
   def on_update(
-        %{"disabled_at" => nil} = _old_data,
-        %{"disabled_at" => disabled_at, "id" => account_id} = _data
+        %{"disabled_at" => nil} = old_data,
+        %{"disabled_at" => disabled_at}
       )
       when not is_nil(disabled_at) do
-    PubSub.Account.Clients.disconnect(account_id)
+    on_delete(old_data)
   end
 
-  def on_update(%{"config" => old_config}, %{"config" => config, "id" => account_id}) do
-    if old_config != config do
-      PubSub.Account.broadcast(account_id, :config_changed)
-    else
-      :ok
-    end
+  # Account soft-deleted - process as a delete
+  def on_update(
+        %{"deleted_at" => nil} = old_data,
+        %{"deleted_at" => deleted_at}
+      )
+      when not is_nil(deleted_at) do
+    on_delete(old_data)
+  end
+
+  def on_update(old_data, data) do
+    old_account = SchemaHelpers.struct_from_params(Accounts.Account, old_data)
+    account = SchemaHelpers.struct_from_params(Accounts.Account, data)
+    PubSub.Account.broadcast(account.id, {:updated, old_account, account})
   end
 
   @impl true
-  def on_delete(_old_data) do
-    :ok
+
+  def on_delete(old_data) do
+    account = SchemaHelpers.struct_from_params(Accounts.Account, old_data)
+
+    # TODO: Hard delete
+    # This can be removed upon implementation of hard delete
+    Domain.Flows.delete_flows_for(account)
+
+    PubSub.Account.broadcast(account.id, {:deleted, account})
   end
 end

elixir/apps/domain/lib/domain/events/hooks/actor_group_memberships.ex

@@ -1,43 +1,24 @@
 defmodule Domain.Events.Hooks.ActorGroupMemberships do
   @behaviour Domain.Events.Hooks
-  alias Domain.{Flows, Policies, PubSub, Repo}
+  alias Domain.{Actors, SchemaHelpers, PubSub}
 
   @impl true
-  def on_insert(%{"actor_id" => actor_id, "group_id" => group_id} = _data) do
-    Task.start(fn ->
-      :ok = PubSub.Actor.Memberships.broadcast(actor_id, {:create_membership, actor_id, group_id})
-      broadcast_access(:allow, actor_id, group_id)
-    end)
-
-    :ok
+  def on_insert(data) do
+    membership = SchemaHelpers.struct_from_params(Actors.Membership, data)
+    PubSub.Account.broadcast(membership.account_id, {:created, membership})
   end
 
   @impl true
   def on_update(_old_data, _data), do: :ok
 
   @impl true
-  def on_delete(
-        %{"account_id" => account_id, "actor_id" => actor_id, "group_id" => group_id} = _old_data
-      ) do
-    Task.start(fn ->
-      :ok = PubSub.Actor.Memberships.broadcast(actor_id, {:delete_membership, actor_id, group_id})
-      broadcast_access(:reject, actor_id, group_id)
+  def on_delete(old_data) do
+    membership = SchemaHelpers.struct_from_params(Actors.Membership, old_data)
 
-      # TODO: WAL
-      # Broadcast flow side effects directly
-      :ok = Flows.expire_flows_for(account_id, actor_id, group_id)
-    end)
+    # TODO: Hard delete
+    # This can be removed upon implementation of hard delete
+    Domain.Flows.delete_flows_for(membership)
 
-    :ok
-  end
-
-  defp broadcast_access(action, actor_id, group_id) do
-    Policies.Policy.Query.not_deleted()
-    |> Policies.Policy.Query.by_actor_group_id(group_id)
-    |> Repo.all(checkout_timeout: 30_000)
-    |> Enum.each(fn policy ->
-      payload = {:"#{action}_access", policy.id, policy.actor_group_id, policy.resource_id}
-      :ok = PubSub.Actor.Policies.broadcast(actor_id, payload)
-    end)
+    PubSub.Account.broadcast(membership.account_id, {:deleted, membership})
   end
 end

elixir/apps/domain/lib/domain/events/hooks/actor_groups.ex

@@ -1,12 +0,0 @@
-defmodule Domain.Events.Hooks.ActorGroups do
-  @behaviour Domain.Events.Hooks
-
-  @impl true
-  def on_insert(_data), do: :ok
-
-  @impl true
-  def on_update(_old_data, _data), do: :ok
-
-  @impl true
-  def on_delete(_old_data), do: :ok
-end

elixir/apps/domain/lib/domain/events/hooks/actors.ex

@@ -1,12 +0,0 @@
-defmodule Domain.Events.Hooks.Actors do
-  @behaviour Domain.Events.Hooks
-
-  @impl true
-  def on_insert(_data), do: :ok
-
-  @impl true
-  def on_update(_old_data, _data), do: :ok
-
-  @impl true
-  def on_delete(_old_data), do: :ok
-end

elixir/apps/domain/lib/domain/events/hooks/auth_identities.ex

@@ -1,12 +0,0 @@
-defmodule Domain.Events.Hooks.AuthIdentities do
-  @behaviour Domain.Events.Hooks
-
-  @impl true
-  def on_insert(_data), do: :ok
-
-  @impl true
-  def on_update(_old_data, _data), do: :ok
-
-  @impl true
-  def on_delete(_old_data), do: :ok
-end

elixir/apps/domain/lib/domain/events/hooks/auth_providers.ex

@@ -1,12 +0,0 @@
-defmodule Domain.Events.Hooks.AuthProviders do
-  @behaviour Domain.Events.Hooks
-
-  @impl true
-  def on_insert(_data), do: :ok
-
-  @impl true
-  def on_update(_old_data, _data), do: :ok
-
-  @impl true
-  def on_delete(_old_data), do: :ok
-end

elixir/apps/domain/lib/domain/events/hooks/clients.ex

@@ -1,25 +1,41 @@
 defmodule Domain.Events.Hooks.Clients do
   @behaviour Domain.Events.Hooks
-  alias Domain.{Clients, PubSub, SchemaHelpers}
+  alias Domain.{Clients, SchemaHelpers, PubSub}
 
   @impl true
   def on_insert(_data), do: :ok
 
-  # Soft-delete
   @impl true
-  def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at} = _data)
+
+  # Soft-delete
+  def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at})
       when not is_nil(deleted_at) do
     on_delete(old_data)
   end
 
   # Regular update
-  def on_update(_old_data, data) do
+  def on_update(old_data, data) do
+    old_client = SchemaHelpers.struct_from_params(Clients.Client, old_data)
     client = SchemaHelpers.struct_from_params(Clients.Client, data)
-    PubSub.Client.broadcast(client.id, {:updated, client})
+
+    # Unverifying a client
+    # This is a special case - we need to delete associated flows when unverifying a client since
+    # it could affect connectivity if any policies are based on the verified status.
+    if not is_nil(old_client.verified_at) and is_nil(client.verified_at) do
+      Domain.Flows.delete_flows_for(client)
+    end
+
+    PubSub.Account.broadcast(client.account_id, {:updated, old_client, client})
   end
 
   @impl true
-  def on_delete(%{"id" => client_id} = _old_data) do
-    PubSub.Client.disconnect(client_id)
+  def on_delete(old_data) do
+    client = SchemaHelpers.struct_from_params(Clients.Client, old_data)
+
+    # TODO: Hard delete
+    # This can be removed upon implementation of hard delete
+    Domain.Flows.delete_flows_for(client)
+
+    PubSub.Account.broadcast(client.account_id, {:deleted, client})
   end
 end

elixir/apps/domain/lib/domain/events/hooks/flows.ex

@@ -0,0 +1,24 @@
+defmodule Domain.Events.Hooks.Flows do
+  @behaviour Domain.Events.Hooks
+  alias Domain.{Flows, PubSub, SchemaHelpers}
+
+  @impl true
+
+  # We don't react directly to flow creation events because connection setup
+  # is latency sensitive and we've already broadcasted the relevant message from
+  # client pid to gateway pid directly.
+  def on_insert(_data), do: :ok
+
+  @impl true
+
+  # Flows are never updated
+  def on_update(_old_data, _data), do: :ok
+
+  @impl true
+
+  # This will trigger reject_access for any subscribed gateways
+  def on_delete(old_data) do
+    flow = SchemaHelpers.struct_from_params(Flows.Flow, old_data)
+    PubSub.Account.broadcast(flow.account_id, {:deleted, flow})
+  end
+end

elixir/apps/domain/lib/domain/events/hooks/gateway_groups.ex

@@ -1,12 +1,31 @@
 defmodule Domain.Events.Hooks.GatewayGroups do
   @behaviour Domain.Events.Hooks
+  alias Domain.{Gateways, PubSub, SchemaHelpers}
 
   @impl true
   def on_insert(_data), do: :ok
 
+  # Soft-delete
   @impl true
-  def on_update(_old_data, _data), do: :ok
+  def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at})
+      when not is_nil(deleted_at) do
+    on_delete(old_data)
+  end
+
+  # Regular update
+  def on_update(old_data, data) do
+    old_gateway_group = SchemaHelpers.struct_from_params(Gateways.Group, old_data)
+    gateway_group = SchemaHelpers.struct_from_params(Gateways.Group, data)
+
+    PubSub.Account.broadcast(
+      gateway_group.account_id,
+      {:updated, old_gateway_group, gateway_group}
+    )
+  end
 
   @impl true
+
+  # Deleting a gateway group will delete the associated resource connection, where
+  # we handle removing it from the client's resource list.
   def on_delete(_old_data), do: :ok
 end

elixir/apps/domain/lib/domain/events/hooks/gateways.ex

@@ -1,13 +1,13 @@
 defmodule Domain.Events.Hooks.Gateways do
   @behaviour Domain.Events.Hooks
-  alias Domain.PubSub
+  alias Domain.{Gateways, PubSub, SchemaHelpers}
 
   @impl true
   def on_insert(_data), do: :ok
 
   # Soft-delete
   @impl true
-  def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at} = _data)
+  def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at})
       when not is_nil(deleted_at) do
     on_delete(old_data)
   end
@@ -16,7 +16,13 @@ defmodule Domain.Events.Hooks.Gateways do
   def on_update(_old_data, _data), do: :ok
 
   @impl true
-  def on_delete(%{"id" => gateway_id} = _old_data) do
-    PubSub.Gateway.disconnect(gateway_id)
+  def on_delete(old_data) do
+    gateway = SchemaHelpers.struct_from_params(Gateways.Gateway, old_data)
+
+    # TODO: Hard delete
+    # This can be removed upon implementation of hard delete
+    Domain.Flows.delete_flows_for(gateway)
+
+    PubSub.Account.broadcast(gateway.account_id, {:deleted, gateway})
   end
 end

elixir/apps/domain/lib/domain/events/hooks/policies.ex

@@ -1,175 +1,60 @@
 defmodule Domain.Events.Hooks.Policies do
   @behaviour Domain.Events.Hooks
-  alias Domain.{Flows, PubSub}
+  alias Domain.{Policies, PubSub, SchemaHelpers}
   require Logger
 
   @impl true
-  def on_insert(
-        %{
-          "id" => policy_id,
-          "account_id" => account_id,
-          "actor_group_id" => actor_group_id,
-          "resource_id" => resource_id
-        } =
-          _data
-      ) do
-    # TODO: WAL
-    # Creating a policy should broadcast directly to subscribed clients/gateways
-    payload = {:create_policy, policy_id}
-    :ok = PubSub.Policy.broadcast(policy_id, payload)
-    :ok = PubSub.Account.Policies.broadcast(account_id, payload)
-
-    payload = {:allow_access, policy_id, actor_group_id, resource_id}
-    :ok = PubSub.ActorGroup.Policies.broadcast(actor_group_id, payload)
+  def on_insert(data) do
+    policy = SchemaHelpers.struct_from_params(Policies.Policy, data)
+    PubSub.Account.broadcast(policy.account_id, {:created, policy})
   end
 
   @impl true
 
-  # Enable
-  def on_update(
-        %{"disabled_at" => disabled_at} = _old_data,
-        %{
-          "disabled_at" => nil,
-          "id" => policy_id,
-          "account_id" => account_id,
-          "actor_group_id" => actor_group_id,
-          "resource_id" => resource_id
-        } = _data
-      )
-      when not is_nil(disabled_at) do
-    # TODO: WAL
-    # Enabling a policy should broadcast directly to subscribed clients/gateways
-    payload = {:enable_policy, policy_id}
-    :ok = PubSub.Policy.broadcast(policy_id, payload)
-    :ok = PubSub.Account.Policies.broadcast(account_id, payload)
+  # Disable - process as delete
 
-    payload = {:allow_access, policy_id, actor_group_id, resource_id}
-    :ok = PubSub.ActorGroup.Policies.broadcast(actor_group_id, payload)
+  def on_update(%{"disabled_at" => nil}, %{"disabled_at" => disabled_at} = data)
+      when not is_nil(disabled_at) do
+    on_delete(data)
   end
 
-  # Disable
-  def on_update(
-        %{"disabled_at" => nil} = _old_data,
-        %{
-          "disabled_at" => disabled_at,
-          "id" => policy_id,
-          "account_id" => account_id,
-          "actor_group_id" => actor_group_id,
-          "resource_id" => resource_id
-        } = _data
-      )
+  # Enable - process as insert
+
+  def on_update(%{"disabled_at" => disabled_at}, %{"disabled_at" => nil} = data)
       when not is_nil(disabled_at) do
-    Task.start(fn ->
-      # TODO: WAL
-      # Disabling a policy should broadcast directly to the subscribed clients/gateways
-      payload = {:disable_policy, policy_id}
-      :ok = PubSub.Policy.broadcast(policy_id, payload)
-      :ok = PubSub.Account.Policies.broadcast(account_id, payload)
-
-      payload = {:reject_access, policy_id, actor_group_id, resource_id}
-      :ok = PubSub.ActorGroup.Policies.broadcast(actor_group_id, payload)
-
-      # TODO: WAL
-      # Broadcast flow side effects directly
-      :ok = Flows.expire_flows_for_policy_id(account_id, policy_id)
-    end)
-
-    :ok
+    on_insert(data)
   end
 
-  # Soft-delete
-  def on_update(
-        %{
-          "deleted_at" => nil
-        } = old_data,
-        %{"deleted_at" => deleted_at} = _data
-      )
+  # Soft-delete - process as delete
+  def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at})
       when not is_nil(deleted_at) do
     on_delete(old_data)
   end
 
-  # Breaking update - delete then create
-  def on_update(
-        %{
-          "id" => old_policy_id,
-          "account_id" => old_account_id,
-          "actor_group_id" => old_actor_group_id,
-          "resource_id" => old_resource_id,
-          "conditions" => old_conditions
-        } = _old_data,
-        %{
-          "id" => policy_id,
-          "account_id" => account_id,
-          "actor_group_id" => actor_group_id,
-          "resource_id" => resource_id,
-          "conditions" => conditions
-        } = data
-      )
-      when old_actor_group_id != actor_group_id or old_resource_id != resource_id or
-             old_conditions != conditions do
-    # Only act upon this if the policy is not deleted or disabled
-    if is_nil(data["deleted_at"]) and is_nil(data["disabled_at"]) do
-      Task.start(fn ->
-        # TODO: WAL
-        # Deleting a policy should broadcast directly to the subscribed clients/gateways
-        payload = {:delete_policy, old_policy_id}
-        :ok = PubSub.Policy.broadcast(old_policy_id, payload)
-        :ok = PubSub.Account.Policies.broadcast(old_account_id, payload)
+  # Regular update
+  def on_update(old_data, data) do
+    old_policy = SchemaHelpers.struct_from_params(Policies.Policy, old_data)
+    policy = SchemaHelpers.struct_from_params(Policies.Policy, data)
 
-        payload = {:reject_access, old_policy_id, old_actor_group_id, old_resource_id}
-        :ok = PubSub.ActorGroup.Policies.broadcast(old_actor_group_id, payload)
-
-        payload = {:create_policy, policy_id}
-        :ok = PubSub.Policy.broadcast(policy_id, payload)
-        :ok = PubSub.Account.Policies.broadcast(account_id, payload)
-
-        payload = {:allow_access, policy_id, actor_group_id, resource_id}
-        :ok = PubSub.ActorGroup.Policies.broadcast(actor_group_id, payload)
-
-        # TODO: WAL
-        # Broadcast flow side effects directly
-        :ok = Flows.expire_flows_for_policy_id(account_id, policy_id)
-      end)
-    else
-      Logger.warning("Breaking update ignored for policy as it is deleted or disabled",
-        policy_id: policy_id
-      )
+    # Breaking updates
+    # This is a special case - we need to delete related flows because connectivity has changed
+    if old_policy.conditions != policy.conditions or
+         old_policy.actor_group_id != policy.actor_group_id or
+         old_policy.resource_id != policy.resource_id do
+      Domain.Flows.delete_flows_for(policy)
     end
 
-    :ok
-  end
-
-  # Regular update - name, description, etc
-  def on_update(_old_data, %{"id" => policy_id, "account_id" => account_id} = _data) do
-    payload = {:update_policy, policy_id}
-    :ok = PubSub.Policy.broadcast(policy_id, payload)
-    :ok = PubSub.Account.Policies.broadcast(account_id, payload)
+    PubSub.Account.broadcast(policy.account_id, {:updated, old_policy, policy})
   end
 
   @impl true
-  def on_delete(
-        %{
-          "id" => policy_id,
-          "account_id" => account_id,
-          "actor_group_id" => actor_group_id,
-          "resource_id" => resource_id
-        } = _old_data
-      ) do
-    Task.start(fn ->
-      # TODO: WAL
-      # Deleting a policy should broadcast directly to the subscribed clients/gateways
-      payload = {:delete_policy, policy_id}
-      :ok = PubSub.Policy.broadcast(policy_id, payload)
-      :ok = PubSub.Account.Policies.broadcast(account_id, payload)
+  def on_delete(old_data) do
+    policy = SchemaHelpers.struct_from_params(Policies.Policy, old_data)
 
-      payload = {:reject_access, policy_id, actor_group_id, resource_id}
-      :ok = PubSub.ActorGroup.Policies.broadcast(actor_group_id, payload)
+    # TODO: Hard delete
+    # This can be removed upon implementation of hard delete
+    Domain.Flows.delete_flows_for(policy)
 
-      # TODO: WAL
-      # Broadcast flow side effects directly
-      :ok = Flows.expire_flows_for_policy_id(account_id, policy_id)
-    end)
-
-    :ok
+    PubSub.Account.broadcast(policy.account_id, {:deleted, policy})
   end
 end

elixir/apps/domain/lib/domain/events/hooks/relay_groups.ex

@@ -1,12 +0,0 @@
-defmodule Domain.Events.Hooks.RelayGroups do
-  @behaviour Domain.Events.Hooks
-
-  @impl true
-  def on_insert(_data), do: :ok
-
-  @impl true
-  def on_update(_old_data, _data), do: :ok
-
-  @impl true
-  def on_delete(_old_data), do: :ok
-end

elixir/apps/domain/lib/domain/events/hooks/relays.ex

@@ -1,12 +0,0 @@
-defmodule Domain.Events.Hooks.Relays do
-  @behaviour Domain.Events.Hooks
-
-  @impl true
-  def on_insert(_data), do: :ok
-
-  @impl true
-  def on_update(_old_data, _data), do: :ok
-
-  @impl true
-  def on_delete(_old_data), do: :ok
-end

elixir/apps/domain/lib/domain/events/hooks/resource_connections.ex

@@ -1,22 +1,19 @@
 defmodule Domain.Events.Hooks.ResourceConnections do
   @behaviour Domain.Events.Hooks
-  alias Domain.Flows
+  alias Domain.{SchemaHelpers, Resources, PubSub}
 
   @impl true
-  def on_insert(_data), do: :ok
+  def on_insert(data) do
+    connection = SchemaHelpers.struct_from_params(Resources.Connection, data)
+    PubSub.Account.broadcast(connection.account_id, {:created, connection})
+  end
 
   @impl true
   def on_update(_old_data, _data), do: :ok
 
   @impl true
-  def on_delete(%{"account_id" => account_id, "resource_id" => resource_id} = _old_data) do
-    # TODO: WAL
-    # Broadcast flow side effects directly
-    #  This hook is called when resources change sites.
-    Task.start(fn ->
-      :ok = Flows.expire_flows_for_resource_id(account_id, resource_id)
-    end)
-
-    :ok
+  def on_delete(old_data) do
+    connection = SchemaHelpers.struct_from_params(Resources.Connection, old_data)
+    PubSub.Account.broadcast(connection.account_id, {:deleted, connection})
   end
 end

elixir/apps/domain/lib/domain/events/hooks/resources.ex

@@ -1,71 +1,47 @@
 defmodule Domain.Events.Hooks.Resources do
   @behaviour Domain.Events.Hooks
-  alias Domain.{Flows, PubSub}
+  alias Domain.{SchemaHelpers, PubSub, Resources}
 
   @impl true
-  def on_insert(%{"id" => resource_id, "account_id" => account_id} = _data) do
-    payload = {:create_resource, resource_id}
-    PubSub.Resource.broadcast(resource_id, payload)
-    PubSub.Account.Resources.broadcast(account_id, payload)
+  def on_insert(data) do
+    resource = SchemaHelpers.struct_from_params(Resources.Resource, data)
+    PubSub.Account.broadcast(resource.account_id, {:created, resource})
   end
 
   @impl true
 
-  # Soft-delete
-  def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at} = _data)
+  # Soft-delete - process as delete
+  def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at})
       when not is_nil(deleted_at) do
     on_delete(old_data)
   end
 
-  # Breaking update - expire flows so that new flows are created
-  def on_update(
-        %{
-          "type" => old_type,
-          "address" => old_address,
-          "filters" => old_filters,
-          "ip_stack" => old_ip_stack
-        } = _old_data,
-        %{
-          "type" => type,
-          "address" => address,
-          "filters" => filters,
-          "ip_stack" => ip_stack,
-          "id" => resource_id,
-          "account_id" => account_id
-        } = _data
-      )
-      when old_type != type or
-             old_address != address or
-             old_filters != filters or
-             old_ip_stack != ip_stack do
-    # TODO: WAL
-    # Broadcast flow side effects directly
-    Task.start(fn ->
-      payload = {:delete_resource, resource_id}
-      PubSub.Resource.broadcast(resource_id, payload)
-      PubSub.Account.Resources.broadcast(account_id, payload)
+  # Regular update
+  def on_update(old_data, data) do
+    old_resource = SchemaHelpers.struct_from_params(Resources.Resource, old_data)
+    resource = SchemaHelpers.struct_from_params(Resources.Resource, data)
 
-      payload = {:create_resource, resource_id}
-      PubSub.Resource.broadcast(resource_id, payload)
-      PubSub.Account.Resources.broadcast(account_id, payload)
+    # Breaking updates
+    # This is a special case - we need to delete related flows because connectivity has changed
+    # Gateway _does_ handle resource filter changes so we don't need to delete flows
+    # for those changes
+    if old_resource.ip_stack != resource.ip_stack or
+         old_resource.type != resource.type or
+         old_resource.address != resource.address do
+      Domain.Flows.delete_flows_for(resource)
+    end
 
-      :ok = Flows.expire_flows_for_resource_id(account_id, resource_id)
-    end)
-
-    :ok
-  end
-
-  # Non-breaking update - for non-addressability changes - e.g. name, description, etc.
-  def on_update(_old_data, %{"id" => resource_id, "account_id" => account_id} = _data) do
-    payload = {:update_resource, resource_id}
-    PubSub.Resource.broadcast(resource_id, payload)
-    PubSub.Account.Resources.broadcast(account_id, payload)
+    PubSub.Account.broadcast(resource.account_id, {:updated, old_resource, resource})
   end
 
   @impl true
-  def on_delete(%{"id" => resource_id, "account_id" => account_id} = _old_data) do
-    payload = {:delete_resource, resource_id}
-    PubSub.Resource.broadcast(resource_id, payload)
-    PubSub.Account.Resources.broadcast(account_id, payload)
+  def on_delete(old_data) do
+    resource = SchemaHelpers.struct_from_params(Resources.Resource, old_data)
+
+    # TODO: Hard delete
+    # This can be removed upon implementation of hard delete
+    Domain.Flows.delete_flows_for(resource)
+
+    PubSub.Account.broadcast(resource.account_id, {:deleted, resource})
   end
 end

elixir/apps/domain/lib/domain/events/hooks/tokens.ex

@@ -1,6 +1,6 @@
 defmodule Domain.Events.Hooks.Tokens do
   @behaviour Domain.Events.Hooks
-  alias Domain.PubSub
+  alias Domain.{PubSub, Tokens, SchemaHelpers}
 
   @impl true
   def on_insert(_data), do: :ok
@@ -12,7 +12,7 @@ defmodule Domain.Events.Hooks.Tokens do
 
   def on_update(_old_data, %{"type" => "email"}), do: :ok
 
-  # Soft-delete
+  # Soft-delete - process as delete
   def on_update(%{"deleted_at" => nil} = old_data, %{"deleted_at" => deleted_at})
       when not is_nil(deleted_at) do
     on_delete(old_data)
@@ -22,7 +22,24 @@ defmodule Domain.Events.Hooks.Tokens do
   def on_update(_old_data, _new_data), do: :ok
 
   @impl true
-  def on_delete(%{"id" => token_id}) do
-    PubSub.Token.disconnect(token_id)
+  def on_delete(old_data) do
+    token = SchemaHelpers.struct_from_params(Tokens.Token, old_data)
+
+    # Disconnect all sockets using this token
+    disconnect_socket(token)
+
+    # TODO: Hard delete
+    # This can be removed upon implementation of hard delete
+    Domain.Flows.delete_flows_for(token)
+
+    PubSub.Account.broadcast(token.account_id, {:deleted, token})
+  end
+
+  # This is a special message that disconnects all sockets using this token,
+  # such as for LiveViews.
+  defp disconnect_socket(token) do
+    topic = Domain.Tokens.socket_id(token.id)
+    payload = %Phoenix.Socket.Broadcast{topic: topic, event: "disconnect"}
+    Domain.PubSub.broadcast(topic, payload)
   end
 end

elixir/apps/domain/lib/domain/events/replication_connection.ex

@@ -5,13 +5,10 @@ defmodule Domain.Events.ReplicationConnection do
   @tables_to_hooks %{
     "accounts" => Hooks.Accounts,
     "actor_group_memberships" => Hooks.ActorGroupMemberships,
-    "actor_groups" => Hooks.ActorGroups,
-    "actors" => Hooks.Actors,
-    "auth_identities" => Hooks.AuthIdentities,
-    "auth_providers" => Hooks.AuthProviders,
     "clients" => Hooks.Clients,
-    "gateway_groups" => Hooks.GatewayGroups,
+    "flows" => Hooks.Flows,
     "gateways" => Hooks.Gateways,
+    "gateway_groups" => Hooks.GatewayGroups,
     "policies" => Hooks.Policies,
     "resource_connections" => Hooks.ResourceConnections,
     "resources" => Hooks.Resources,
@@ -21,9 +18,9 @@ defmodule Domain.Events.ReplicationConnection do
   def on_write(state, _lsn, op, table, old_data, data) do
     if hook = Map.get(@tables_to_hooks, table) do
       case op do
-        :insert -> hook.on_insert(data)
-        :update -> hook.on_update(old_data, data)
-        :delete -> hook.on_delete(old_data)
+        :insert -> :ok = hook.on_insert(data)
+        :update -> :ok = hook.on_update(old_data, data)
+        :delete -> :ok = hook.on_delete(old_data)
       end
     else
       log_warning(op, table)

elixir/apps/domain/lib/domain/events/topics.ex

@@ -1,47 +0,0 @@
-defmodule Domain.Events.Topics do
-  @moduledoc """
-    A simple module to house all of the topics and broadcasts so we can see
-    them and verify them in one place.
-  """
-  alias Domain.PubSub
-
-  defmodule Account do
-    def subscribe(account_id) do
-      account_id
-      |> topic()
-      |> PubSub.subscribe()
-    end
-
-    defp topic(account_id) do
-      "accounts:" <> account_id
-    end
-  end
-
-  defmodule Presence do
-    defmodule Account do
-      defmodule Clients do
-        def subscribe(account_id) do
-          account_id
-          |> topic()
-          |> PubSub.subscribe()
-        end
-
-        defp topic(account_id) do
-          "presences:account_clients:" <> account_id
-        end
-      end
-
-      defmodule Gateways do
-        def subscribe(account_id) do
-          account_id
-          |> topic()
-          |> PubSub.subscribe()
-        end
-
-        defp topic(account_id) do
-          "presences:account_gateways:" <> account_id
-        end
-      end
-    end
-  end
-end

elixir/apps/domain/lib/domain/flows.ex

@@ -1,45 +1,47 @@
 defmodule Domain.Flows do
   alias Domain.Repo
-  alias Domain.{Auth, Actors, Clients, Gateways, Resources, Policies, Tokens}
+  alias Domain.{Auth, Actors, Clients, Gateways, Resources, Policies}
   alias Domain.Flows.{Authorizer, Flow}
   require Ecto.Query
   require Logger
 
-  def authorize_flow(
+  # TODO: Optimization
+  # Connection setup latency - doesn't need to block setting up flow. Authorizing the flow
+  # is now handled in memory and this only logs it, so these can be done in parallel.
+  def create_flow(
         %Clients.Client{
           id: client_id,
           account_id: account_id,
           actor_id: actor_id
-        } = client,
+        },
         %Gateways.Gateway{
           id: gateway_id,
           last_seen_remote_ip: gateway_remote_ip,
           account_id: account_id
         },
         resource_id,
+        %Policies.Policy{} = policy,
         %Auth.Subject{
           account: %{id: account_id},
           actor: %{id: actor_id},
-          expires_at: expires_at,
           token_id: token_id,
           context: %Auth.Context{
             remote_ip: client_remote_ip,
             user_agent: client_user_agent
           }
-        } = subject,
-        opts \\ []
+        } = subject
       ) do
     with :ok <- Auth.ensure_has_permissions(subject, Authorizer.create_flows_permission()),
-         {:ok, resource} <-
-           Resources.fetch_and_authorize_resource_by_id(resource_id, subject, opts),
-         {:ok, policy, conformation_expires_at} <- fetch_conforming_policy(resource, client) do
+         {:ok, membership} <-
+           Actors.fetch_membership_by_actor_id_and_group_id(actor_id, policy.actor_group_id) do
       flow =
         Flow.Changeset.create(%{
           token_id: token_id,
           policy_id: policy.id,
           client_id: client_id,
           gateway_id: gateway_id,
-          resource_id: resource.id,
+          resource_id: resource_id,
+          actor_group_membership_id: membership.id,
           account_id: account_id,
           client_remote_ip: client_remote_ip,
           client_user_agent: client_user_agent,
@@ -47,28 +49,7 @@ defmodule Domain.Flows do
         })
         |> Repo.insert!()
 
-      expires_at = conformation_expires_at || expires_at
-
-      {:ok, resource, flow, expires_at}
-    end
-  end
-
-  defp fetch_conforming_policy(%Resources.Resource{} = resource, client) do
-    Enum.reduce_while(resource.authorized_by_policies, {:error, []}, fn policy, {:error, acc} ->
-      case Policies.ensure_client_conforms_policy_conditions(client, policy) do
-        {:ok, expires_at} ->
-          {:halt, {:ok, policy, expires_at}}
-
-        {:error, {:forbidden, violated_properties: violated_properties}} ->
-          {:cont, {:error, violated_properties ++ acc}}
-      end
-    end)
-    |> case do
-      {:error, violated_properties} ->
-        {:error, {:forbidden, violated_properties: violated_properties}}
-
-      {:ok, policy, expires_at} ->
-        {:ok, policy, expires_at}
+      {:ok, flow}
     end
   end
 
@@ -85,6 +66,14 @@ defmodule Domain.Flows do
     end
   end
 
+  def all_gateway_flows_for_cache!(%Gateways.Gateway{} = gateway) do
+    Flow.Query.all()
+    |> Flow.Query.by_account_id(gateway.account_id)
+    |> Flow.Query.by_gateway_id(gateway.id)
+    |> Flow.Query.for_cache()
+    |> Repo.all()
+  end
+
   def list_flows_for(assoc, subject, opts \\ [])
 
   def list_flows_for(%Policies.Policy{} = policy, %Auth.Subject{} = subject, opts) do
@@ -125,121 +114,62 @@ defmodule Domain.Flows do
     end
   end
 
-  # TODO: WAL
-  # Remove all of the indexes used for these after flow expiration is moved to state
-  # broadcasts
-
-  def expire_flows_for(%Auth.Identity{} = identity) do
+  def delete_flows_for(%Domain.Accounts.Account{} = account) do
     Flow.Query.all()
-    |> Flow.Query.by_identity_id(identity.id)
-    |> expire_flows()
+    |> Flow.Query.by_account_id(account.id)
+    |> Repo.delete_all()
   end
 
-  def expire_flows_for(%Clients.Client{} = client) do
+  def delete_flows_for(%Domain.Actors.Membership{} = membership) do
     Flow.Query.all()
+    |> Flow.Query.by_account_id(membership.account_id)
+    |> Flow.Query.by_actor_group_membership_id(membership.id)
+    |> Repo.delete_all()
+  end
+
+  def delete_flows_for(%Domain.Clients.Client{} = client) do
+    Flow.Query.all()
+    |> Flow.Query.by_account_id(client.account_id)
     |> Flow.Query.by_client_id(client.id)
-    |> expire_flows()
+    |> Repo.delete_all()
   end
 
-  def expire_flows_for(%Actors.Group{} = actor_group) do
+  def delete_flows_for(%Domain.Gateways.Gateway{} = gateway) do
     Flow.Query.all()
-    |> Flow.Query.by_policy_actor_group_id(actor_group.id)
-    |> expire_flows()
+    |> Flow.Query.by_account_id(gateway.account_id)
+    |> Flow.Query.by_gateway_id(gateway.id)
+    |> Repo.delete_all()
   end
 
-  def expire_flows_for(%Tokens.Token{} = token, %Auth.Subject{} = subject) do
+  def delete_flows_for(%Domain.Policies.Policy{} = policy) do
     Flow.Query.all()
-    |> Flow.Query.by_token_id(token.id)
-    |> expire_flows(subject)
+    |> Flow.Query.by_account_id(policy.account_id)
+    |> Flow.Query.by_policy_id(policy.id)
+    |> Repo.delete_all()
   end
 
-  def expire_flows_for(%Actors.Actor{} = actor, %Auth.Subject{} = subject) do
-    Flow.Query.all()
-    |> Flow.Query.by_actor_id(actor.id)
-    |> expire_flows(subject)
-  end
-
-  def expire_flows_for(%Auth.Identity{} = identity, %Auth.Subject{} = subject) do
-    Flow.Query.all()
-    |> Flow.Query.by_identity_id(identity.id)
-    |> expire_flows(subject)
-  end
-
-  def expire_flows_for(%Resources.Resource{} = resource, %Auth.Subject{} = subject) do
+  def delete_flows_for(%Domain.Resources.Resource{} = resource) do
     Flow.Query.all()
+    |> Flow.Query.by_account_id(resource.account_id)
     |> Flow.Query.by_resource_id(resource.id)
-    |> expire_flows(subject)
+    |> Repo.delete_all()
   end
 
-  def expire_flows_for(%Actors.Group{} = actor_group, %Auth.Subject{} = subject) do
+  def delete_flows_for(%Domain.Tokens.Token{} = token) do
     Flow.Query.all()
-    |> Flow.Query.by_policy_actor_group_id(actor_group.id)
-    |> expire_flows(subject)
+    |> Flow.Query.by_account_id(token.account_id)
+    |> Flow.Query.by_token_id(token.id)
+    |> Repo.delete_all()
   end
 
-  def expire_flows_for(%Auth.Provider{} = provider, %Auth.Subject{} = subject) do
+  def delete_stale_flows_on_connect(%Clients.Client{} = client, resources)
+      when is_list(resources) do
+    authorized_resource_ids = Enum.map(resources, & &1.id)
+
     Flow.Query.all()
-    |> Flow.Query.by_identity_provider_id(provider.id)
-    |> expire_flows(subject)
-  end
-
-  def expire_flows_for(account_id, actor_id, group_id) do
-    Flow.Query.all()
-    |> Flow.Query.by_account_id(account_id)
-    |> Flow.Query.by_actor_id(actor_id)
-    |> Flow.Query.by_policy_actor_group_id(group_id)
-    |> expire_flows()
-  end
-
-  def expire_flows_for_resource_id(account_id, resource_id) do
-    Flow.Query.all()
-    |> Flow.Query.by_account_id(account_id)
-    |> Flow.Query.by_resource_id(resource_id)
-    |> expire_flows()
-  end
-
-  def expire_flows_for_policy_id(account_id, policy_id) do
-    Flow.Query.all()
-    |> Flow.Query.by_account_id(account_id)
-    |> Flow.Query.by_policy_id(policy_id)
-    |> expire_flows()
-  end
-
-  defp expire_flows(queryable, subject) do
-    with :ok <- Auth.ensure_has_permissions(subject, Authorizer.create_flows_permission()) do
-      queryable
-      |> Authorizer.for_subject(Flow, subject)
-      |> expire_flows()
-    end
-  end
-
-  defp expire_flows(queryable) do
-    {:ok, :ok} =
-      Repo.transaction(fn ->
-        queryable
-        |> Repo.stream()
-        |> Stream.chunk_every(100)
-        |> Enum.each(fn chunk ->
-          Enum.each(chunk, &broadcast_flow_expiration/1)
-        end)
-      end)
-
-    :ok
-  end
-
-  defp broadcast_flow_expiration(flow) do
-    case Domain.PubSub.Flow.broadcast(
-           flow.id,
-           {:expire_flow, flow.id, flow.client_id, flow.resource_id}
-         ) do
-      :ok ->
-        :ok
-
-      {:error, reason} ->
-        Logger.error("Failed to broadcast flow expiration",
-          reason: inspect(reason),
-          flow_id: flow.id
-        )
-    end
+    |> Flow.Query.by_account_id(client.account_id)
+    |> Flow.Query.by_client_id(client.id)
+    |> Flow.Query.by_not_in_resource_ids(authorized_resource_ids)
+    |> Repo.delete_all()
   end
 end

elixir/apps/domain/lib/domain/flows/flow.ex

@@ -7,6 +7,7 @@ defmodule Domain.Flows.Flow do
     belongs_to :gateway, Domain.Gateways.Gateway
     belongs_to :resource, Domain.Resources.Resource
     belongs_to :token, Domain.Tokens.Token
+    belongs_to :actor_group_membership, Domain.Actors.Membership
 
     belongs_to :account, Domain.Accounts.Account
 

elixir/apps/domain/lib/domain/flows/flow/changeset.ex

@@ -2,7 +2,7 @@ defmodule Domain.Flows.Flow.Changeset do
   use Domain, :changeset
   alias Domain.Flows.Flow
 
-  @fields ~w[token_id policy_id client_id gateway_id resource_id
+  @fields ~w[token_id policy_id client_id gateway_id resource_id actor_group_membership_id
              account_id
              client_remote_ip client_user_agent
              gateway_remote_ip]a
@@ -16,6 +16,7 @@ defmodule Domain.Flows.Flow.Changeset do
     |> assoc_constraint(:client)
     |> assoc_constraint(:gateway)
     |> assoc_constraint(:resource)
+    |> assoc_constraint(:actor_group_membership)
     |> assoc_constraint(:account)
   end
 end

elixir/apps/domain/lib/domain/flows/flow/query.ex

@@ -21,12 +21,28 @@ defmodule Domain.Flows.Flow.Query do
     where(queryable, [flows: flows], flows.policy_id == ^policy_id)
   end
 
+  # Return the latest {client_id, resource_id} pairs over the last 14 days
+  def for_cache(queryable) do
+    cutoff = DateTime.utc_now() |> DateTime.add(-14, :day)
+
+    where(queryable, [flows: flows], flows.inserted_at > ^cutoff)
+    |> group_by([flows: flows], [flows.client_id, flows.resource_id])
+    |> select(
+      [flows: flows],
+      {{flows.client_id, flows.resource_id}, max(flows.inserted_at)}
+    )
+  end
+
   def by_policy_actor_group_id(queryable, actor_group_id) do
     queryable
     |> with_joined_policy()
     |> where([policy: policy], policy.actor_group_id == ^actor_group_id)
   end
 
+  def by_actor_group_membership_id(queryable, membership_id) do
+    where(queryable, [flows: flows], flows.actor_group_membership_id == ^membership_id)
+  end
+
   def by_identity_id(queryable, identity_id) do
     queryable
     |> with_joined_client()
@@ -43,6 +59,10 @@ defmodule Domain.Flows.Flow.Query do
     where(queryable, [flows: flows], flows.resource_id == ^resource_id)
   end
 
+  def by_not_in_resource_ids(queryable, resource_ids) do
+    where(queryable, [flows: flows], flows.resource_id not in ^resource_ids)
+  end
+
   def by_client_id(queryable, client_id) do
     where(queryable, [flows: flows], flows.client_id == ^client_id)
   end

elixir/apps/domain/lib/domain/gateways/presence.ex

@@ -9,8 +9,7 @@ defmodule Domain.Gateways.Presence do
   def connect(%Gateway{} = gateway) do
     with {:ok, _} <- __MODULE__.Group.track(gateway.group_id, gateway.id),
          {:ok, _} <- __MODULE__.Account.track(gateway.account_id, gateway.id) do
-      :ok = PubSub.Gateway.subscribe(gateway.id)
-      :ok = PubSub.Account.Gateways.subscribe(gateway.account_id)
+      :ok
     end
   end
 

elixir/apps/domain/lib/domain/policies.ex

@@ -58,6 +58,22 @@ defmodule Domain.Policies do
     end
   end
 
+  def all_policies_for_actor!(%Actors.Actor{} = actor) do
+    Policy.Query.not_disabled()
+    |> Policy.Query.by_account_id(actor.account_id)
+    |> Policy.Query.by_actor_id(actor.id)
+    |> Policy.Query.with_preloaded_resource_gateway_groups()
+    |> Repo.all()
+  end
+
+  def all_policies_for_actor_group_id!(account_id, actor_group_id) do
+    Policy.Query.not_disabled()
+    |> Policy.Query.by_account_id(account_id)
+    |> Policy.Query.by_actor_group_id(actor_group_id)
+    |> Policy.Query.with_preloaded_resource_gateway_groups()
+    |> Repo.all()
+  end
+
   def new_policy(attrs, %Auth.Subject{} = subject) do
     Policy.Changeset.create(attrs, subject)
   end
@@ -161,18 +177,8 @@ defmodule Domain.Policies do
     {:ok, policies}
   end
 
-  def pre_filter_non_conforming_resources(resources, %Clients.Client{} = client) do
-    resources
-    |> Enum.flat_map(fn resource ->
-      case client_conforms_any_on_connect?(client, resource.authorized_by_policies) do
-        true -> [resource]
-        false -> []
-      end
-    end)
-  end
-
-  def client_conforms_any_on_connect?(%Clients.Client{} = client, policies) do
-    Enum.any?(policies, fn policy ->
+  def filter_by_conforming_policies_for_client(policies, %Clients.Client{} = client) do
+    Enum.filter(policies, fn policy ->
       policy.conditions
       |> Enum.filter(&Condition.Evaluator.evaluable_on_connect?/1)
       |> Condition.Evaluator.ensure_conforms(client)

elixir/apps/domain/lib/domain/policies/policy/query.ex

@@ -108,6 +108,11 @@ defmodule Domain.Policies.Policy.Query do
     end)
   end
 
+  def with_preloaded_resource_gateway_groups(queryable) do
+    queryable
+    |> preload(resource: :gateway_groups)
+  end
+
   # Pagination
 
   @impl Domain.Repo.Query

elixir/apps/domain/lib/domain/pubsub.ex

@@ -42,7 +42,6 @@ defmodule Domain.PubSub do
     Phoenix.PubSub.unsubscribe(__MODULE__, topic)
   end
 
-  # TODO: These are quite repetitive. We could simplify this with a `__using__` macro.
   defmodule Account do
     def subscribe(account_id) do
       account_id
@@ -59,315 +58,5 @@ defmodule Domain.PubSub do
     defp topic(account_id) do
       Atom.to_string(__MODULE__) <> ":" <> account_id
     end
-
-    defmodule Clients do
-      def subscribe(account_id) do
-        account_id
-        |> topic()
-        |> Domain.PubSub.subscribe()
-      end
-
-      def broadcast(account_id, payload) do
-        account_id
-        |> topic()
-        |> Domain.PubSub.broadcast(payload)
-      end
-
-      def disconnect(account_id) do
-        account_id
-        |> topic()
-        |> Domain.PubSub.broadcast("disconnect")
-      end
-
-      defp topic(account_id) do
-        Atom.to_string(__MODULE__) <> ":" <> account_id
-      end
-    end
-
-    defmodule Gateways do
-      def subscribe(account_id) do
-        account_id
-        |> topic()
-        |> Domain.PubSub.subscribe()
-      end
-
-      def disconnect(account_id) do
-        account_id
-        |> topic()
-        |> Domain.PubSub.broadcast("disconnect")
-      end
-
-      defp topic(account_id) do
-        Atom.to_string(__MODULE__) <> ":" <> account_id
-      end
-    end
-
-    defmodule Policies do
-      def subscribe(account_id) do
-        account_id
-        |> topic()
-        |> Domain.PubSub.subscribe()
-      end
-
-      def broadcast(account_id, payload) do
-        account_id
-        |> topic()
-        |> Domain.PubSub.broadcast(payload)
-      end
-
-      defp topic(account_id) do
-        Atom.to_string(__MODULE__) <> ":" <> account_id
-      end
-    end
-
-    defmodule Resources do
-      def subscribe(account_id) do
-        account_id
-        |> topic()
-        |> Domain.PubSub.subscribe()
-      end
-
-      def broadcast(account_id, payload) do
-        account_id
-        |> topic()
-        |> Domain.PubSub.broadcast(payload)
-      end
-
-      defp topic(account_id) do
-        Atom.to_string(__MODULE__) <> ":" <> account_id
-      end
-    end
-  end
-
-  defmodule Actor do
-    def subscribe(actor_id) do
-      actor_id
-      |> topic()
-      |> Domain.PubSub.subscribe()
-    end
-
-    defp topic(actor_id) do
-      Atom.to_string(__MODULE__) <> ":" <> actor_id
-    end
-
-    defmodule Memberships do
-      def subscribe(actor_id) do
-        actor_id
-        |> topic()
-        |> Domain.PubSub.subscribe()
-      end
-
-      def broadcast(actor_id, payload) do
-        actor_id
-        |> topic()
-        |> Domain.PubSub.broadcast(payload)
-      end
-
-      def broadcast_access(action, actor_id, group_id) do
-        Domain.Policies.Policy.Query.not_deleted()
-        |> Domain.Policies.Policy.Query.by_actor_group_id(group_id)
-        |> Domain.Repo.all()
-        |> Enum.each(fn policy ->
-          payload = {:"#{action}_access", policy.id, policy.actor_group_id, policy.resource_id}
-          :ok = Actor.Policies.broadcast(actor_id, payload)
-        end)
-      end
-
-      defp topic(actor_id) do
-        Atom.to_string(__MODULE__) <> ":" <> actor_id
-      end
-    end
-
-    defmodule Policies do
-      def subscribe(actor_id) do
-        actor_id
-        |> topic()
-        |> Domain.PubSub.subscribe()
-      end
-
-      def broadcast(actor_id, payload) do
-        actor_id
-        |> topic()
-        |> Domain.PubSub.broadcast(payload)
-      end
-
-      defp topic(actor_id) do
-        Atom.to_string(__MODULE__) <> ":" <> actor_id
-      end
-    end
-  end
-
-  defmodule ActorGroup do
-    defmodule Policies do
-      def subscribe(actor_group_id) do
-        actor_group_id
-        |> topic()
-        |> Domain.PubSub.subscribe()
-      end
-
-      def unsubscribe(actor_group_id) do
-        actor_group_id
-        |> topic()
-        |> Domain.PubSub.unsubscribe()
-      end
-
-      def broadcast(actor_group_id, payload) do
-        actor_group_id
-        |> topic()
-        |> Domain.PubSub.broadcast(payload)
-      end
-
-      defp topic(actor_group_id) do
-        Atom.to_string(__MODULE__) <> ":" <> actor_group_id
-      end
-    end
-  end
-
-  defmodule Client do
-    def subscribe(client_id) do
-      client_id
-      |> topic()
-      |> Domain.PubSub.subscribe()
-    end
-
-    def broadcast(client_id, payload) do
-      client_id
-      |> topic()
-      |> Domain.PubSub.broadcast(payload)
-    end
-
-    def disconnect(client_id) do
-      client_id
-      |> topic()
-      |> Domain.PubSub.broadcast("disconnect")
-    end
-
-    defp topic(client_id) do
-      Atom.to_string(__MODULE__) <> ":" <> client_id
-    end
-  end
-
-  defmodule Flow do
-    def subscribe(flow_id) do
-      flow_id
-      |> topic()
-      |> Domain.PubSub.subscribe()
-    end
-
-    def unsubscribe(flow_id) do
-      flow_id
-      |> topic()
-      |> Domain.PubSub.unsubscribe()
-    end
-
-    def broadcast(flow_id, payload) do
-      flow_id
-      |> topic()
-      |> Domain.PubSub.broadcast(payload)
-    end
-
-    defp topic(flow_id) do
-      Atom.to_string(__MODULE__) <> ":" <> flow_id
-    end
-  end
-
-  defmodule GatewayGroup do
-    def subscribe(gateway_group_id) do
-      gateway_group_id
-      |> topic()
-      |> Domain.PubSub.subscribe()
-    end
-
-    def unsubscribe(gateway_group_id) do
-      gateway_group_id
-      |> topic()
-      |> Domain.PubSub.unsubscribe()
-    end
-
-    defp topic(gateway_group_id) do
-      Atom.to_string(__MODULE__) <> ":" <> gateway_group_id
-    end
-  end
-
-  defmodule Gateway do
-    def subscribe(gateway_id) do
-      gateway_id
-      |> topic()
-      |> Domain.PubSub.subscribe()
-    end
-
-    def broadcast(gateway_id, payload) do
-      gateway_id
-      |> topic()
-      |> Domain.PubSub.broadcast(payload)
-    end
-
-    def disconnect(gateway_id) do
-      gateway_id
-      |> topic()
-      |> Domain.PubSub.broadcast("disconnect")
-    end
-
-    defp topic(gateway_id) do
-      Atom.to_string(__MODULE__) <> ":" <> gateway_id
-    end
-  end
-
-  defmodule Policy do
-    def subscribe(policy_id) do
-      policy_id
-      |> topic()
-      |> Domain.PubSub.subscribe()
-    end
-
-    def broadcast(policy_id, payload) do
-      policy_id
-      |> topic()
-      |> Domain.PubSub.broadcast(payload)
-    end
-
-    defp topic(policy_id) do
-      Atom.to_string(__MODULE__) <> ":" <> policy_id
-    end
-  end
-
-  defmodule Resource do
-    def subscribe(resource_id) do
-      resource_id
-      |> topic()
-      |> Domain.PubSub.subscribe()
-    end
-
-    def unsubscribe(resource_id) do
-      resource_id
-      |> topic()
-      |> Domain.PubSub.unsubscribe()
-    end
-
-    def broadcast(resource_id, payload) do
-      resource_id
-      |> topic()
-      |> Domain.PubSub.broadcast(payload)
-    end
-
-    defp topic(resource_id) do
-      Atom.to_string(__MODULE__) <> ":" <> resource_id
-    end
-  end
-
-  defmodule Token do
-    def disconnect(token_id) do
-      token_id
-      |> topic()
-      |> Domain.PubSub.broadcast(%Phoenix.Socket.Broadcast{
-        topic: topic(token_id),
-        event: "disconnect"
-      })
-    end
-
-    defp topic(token_id) do
-      # This topic is managed by Phoenix
-      Domain.Tokens.socket_id(token_id)
-    end
   end
 end

elixir/apps/domain/lib/domain/relays.ex

@@ -344,6 +344,8 @@ defmodule Domain.Relays do
     |> Enum.map(&Enum.random(elem(&1, 1)))
   end
 
+  # TODO: WAL
+  # Refactor to use new conventions
   def connect_relay(%Relay{} = relay, secret) do
     with {:ok, _} <-
            Presence.track(self(), group_presence_topic(relay.group_id), relay.id, %{}),

elixir/apps/domain/lib/domain/replication/connection.ex

@@ -579,34 +579,13 @@ defmodule Domain.Replication.Connection do
         {:delete, old, nil}
       end
 
-      defp decode_value({value, %{type: type} = column})
-           when type in ["json", "jsonb"] and is_binary(value) do
-        case JSON.decode(value) do
-          {:ok, decoded} ->
-            {column.name, decoded}
-
-          {:error, reason} ->
-            Logger.warning("Could not decode JSONB value, using as-is",
-              reason: reason,
-              value: value,
-              column: column.name
-            )
-
-            {column.name, value}
-        end
-      end
-
-      defp decode_value({value, column}) do
-        {column.name, value}
-      end
-
       defp zip(nil, _), do: nil
 
       defp zip(tuple_data, columns) do
         tuple_data
         |> Tuple.to_list()
         |> Enum.zip(columns)
-        |> Map.new(&decode_value/1)
+        |> Map.new(&Decoder.decode_json/1)
       end
     end
   end

elixir/apps/domain/lib/domain/replication/decoder.ex

@@ -150,6 +150,73 @@ defmodule Domain.Replication.Decoder do
 
   alias Domain.Replication.OidDatabase
 
+  @doc """
+  Helper for decoding JSON data inside messages.
+  """
+
+  # Postgrex uses `_jsonb` to mean `jsonb[]`. These array types are returned as string literals from
+  # Postgrex and need to be split, and then double-decoded.
+  def decode_json({value, %{type: type} = column})
+      when type in ["_json", "_jsonb"] and is_binary(value) do
+    decoded_list = parse_postgres_jsonb_array(value)
+    {column.name, decoded_list}
+  end
+
+  def decode_json({value, %{type: type} = column})
+      when type in ["json", "jsonb"] and is_binary(value) do
+    case JSON.decode(value) do
+      {:ok, decoded} ->
+        {column.name, decoded}
+
+      {:error, reason} ->
+        Logger.warning("Failed to decode JSON, using as-is",
+          json: value,
+          reason: reason
+        )
+
+        {column.name, value}
+    end
+  end
+
+  def decode_json({value, column}) do
+    {column.name, value}
+  end
+
+  defp parse_postgres_jsonb_array("{}"), do: []
+
+  defp parse_postgres_jsonb_array("{" <> content) do
+    content
+    |> String.trim_trailing("}")
+    |> split_json_array_elements()
+    |> Enum.map(&double_decode_json/1)
+  end
+
+  defp parse_postgres_jsonb_array(_), do: []
+
+  # Split JSON elements in PostgreSQL array using regex
+  defp split_json_array_elements(content) do
+    ~r/,(?=(?:[^"]*"[^"]*")*[^"]*$)(?![^{]*})/
+    |> Regex.split(content)
+    |> Enum.map(&String.trim/1)
+    |> Enum.reject(&(&1 == ""))
+  end
+
+  # PostgreSQL double-encodes JSON in arrays, so we need to decode twice
+  defp double_decode_json(json_str) do
+    with {:ok, first} <- Jason.decode(json_str),
+         {:ok, second} <- Jason.decode(first) do
+      second
+    else
+      {:error, reason} ->
+        Logger.warning("Failed to decode JSON, using as-is",
+          json: json_str,
+          reason: reason
+        )
+
+        json_str
+    end
+  end
+
   @doc """
   Parses logical replication messages from Postgres
 

elixir/apps/domain/lib/domain/resources.ex

@@ -80,21 +80,6 @@ defmodule Domain.Resources do
     end
   end
 
-  def fetch_and_authorize_resource_by_id(id, %Auth.Subject{} = subject, opts \\ []) do
-    with :ok <-
-           Auth.ensure_has_permissions(subject, Authorizer.view_available_resources_permission()),
-         true <- Repo.valid_uuid?(id) do
-      Resource.Query.not_deleted()
-      |> Resource.Query.by_id(id)
-      |> Resource.Query.by_account_id(subject.account.id)
-      |> Resource.Query.by_authorized_actor_id(subject.actor.id)
-      |> Repo.fetch(Resource.Query, opts)
-    else
-      false -> {:error, :not_found}
-      other -> other
-    end
-  end
-
   def fetch_resource_by_id!(id) do
     if Repo.valid_uuid?(id) do
       Resource.Query.not_deleted()

elixir/apps/domain/lib/domain/tokens.ex

@@ -183,10 +183,6 @@ defmodule Domain.Tokens do
        ]}
 
     with :ok <- Auth.ensure_has_permissions(subject, required_permissions) do
-      # TODO: WAL
-      # Broadcast flow side effects directly
-      :ok = Domain.Flows.expire_flows_for(token, subject)
-
       Token.Query.not_deleted()
       |> Token.Query.by_id(token.id)
       |> Authorizer.for_subject(subject)
@@ -203,23 +199,9 @@ defmodule Domain.Tokens do
     |> Token.Query.by_id(subject.token_id)
     |> Authorizer.for_subject(subject)
     |> delete_tokens()
-    |> case do
-      {:ok, [token]} ->
-        # TODO: WAL
-        # Broadcast flow side effects directly
-        :ok = Domain.Flows.expire_flows_for(token, subject)
-        {:ok, token}
-
-      {:ok, []} ->
-        {:ok, []}
-    end
   end
 
   def delete_tokens_for(%Auth.Identity{} = identity) do
-    # TODO: WAL
-    # Broadcast flow side effects directly
-    :ok = Domain.Flows.expire_flows_for(identity)
-
     Token.Query.not_deleted()
     |> Token.Query.by_identity_id(identity.id)
     |> delete_tokens()
@@ -227,10 +209,6 @@ defmodule Domain.Tokens do
 
   def delete_tokens_for(%Actors.Actor{} = actor, %Auth.Subject{} = subject) do
     with :ok <- Auth.ensure_has_permissions(subject, Authorizer.manage_tokens_permission()) do
-      # TODO: WAL
-      # Broadcast flow side effects directly
-      :ok = Domain.Flows.expire_flows_for(actor, subject)
-
       Token.Query.not_deleted()
       |> Token.Query.by_actor_id(actor.id)
       |> Authorizer.for_subject(subject)
@@ -240,10 +218,6 @@ defmodule Domain.Tokens do
 
   def delete_tokens_for(%Auth.Identity{} = identity, %Auth.Subject{} = subject) do
     with :ok <- Auth.ensure_has_permissions(subject, Authorizer.manage_tokens_permission()) do
-      # TODO: WAL
-      # Broadcast flow side effects directly
-      :ok = Domain.Flows.expire_flows_for(identity, subject)
-
       Token.Query.not_deleted()
       |> Token.Query.by_identity_id(identity.id)
       |> Authorizer.for_subject(subject)
@@ -253,10 +227,6 @@ defmodule Domain.Tokens do
 
   def delete_tokens_for(%Auth.Provider{} = provider, %Auth.Subject{} = subject) do
     with :ok <- Auth.ensure_has_permissions(subject, Authorizer.manage_tokens_permission()) do
-      # TODO: WAL
-      # Broadcast flow side effects directly
-      :ok = Domain.Flows.expire_flows_for(provider, subject)
-
       Token.Query.not_deleted()
       |> Token.Query.by_provider_id(provider.id)
       |> Authorizer.for_subject(subject)

elixir/apps/domain/priv/repo/manual_migrations/20250704201940_index_flows_on_token_id.exs

@@ -0,0 +1,17 @@
+defmodule Domain.Repo.Migrations.IndexFlowsOnTokenId do
+  use Ecto.Migration
+
+  @disable_ddl_transaction true
+
+  def up do
+    execute("""
+    CREATE INDEX CONCURRENTLY IF NOT EXISTS flows_account_id_token_id_index ON flows USING BTREE (account_id, token_id, inserted_at DESC, id DESC);
+    """)
+  end
+
+  def down do
+    execute("""
+    DROP INDEX CONCURRENTLY IF EXISTS flows_account_id_token_id_index;
+    """)
+  end
+end

elixir/apps/domain/priv/repo/migrations/20250712010323_add_id_to_actor_group_memberships.exs

@@ -0,0 +1,70 @@
+defmodule Domain.Repo.Migrations.AddIdToActorGroupMemberships do
+  use Ecto.Migration
+
+  @moduledoc """
+  This migration will lock the `actor_group_memberships` table, so it's
+  best to run this when a brief period of downtime is acceptable.
+  """
+
+  def up do
+    # Step 1: Add the new column with a default value
+    execute("""
+    ALTER TABLE actor_group_memberships
+    ADD COLUMN IF NOT EXISTS id UUID DEFAULT uuid_generate_v4()
+    """)
+
+    # Step 2: Backfill the new column for existing rows
+    execute("""
+    UPDATE actor_group_memberships SET id = uuid_generate_v4() WHERE id IS NULL
+    """)
+
+    # Step 3: Enforce the NOT NULL constraint
+    execute("""
+    ALTER TABLE actor_group_memberships
+    ALTER COLUMN id SET NOT NULL
+    """)
+
+    # Step 4: Drop the old composite primary key
+    execute("""
+    ALTER TABLE actor_group_memberships
+    DROP CONSTRAINT IF EXISTS actor_group_memberships_pkey
+    """)
+
+    # Step 5: Add the new primary key on the id column
+    execute("""
+    ALTER TABLE actor_group_memberships
+    ADD PRIMARY KEY (id)
+    """)
+
+    # Step 6: Recreate the actor_id, group_id index with unique constraint
+    execute("""
+    CREATE UNIQUE INDEX IF NOT EXISTS actor_group_memberships_actor_id_group_id_index
+    ON actor_group_memberships (actor_id, group_id)
+    """)
+  end
+
+  def down do
+    # Step 1: Drop the unique index on actor_id and group_id
+    execute("""
+    DROP INDEX IF EXISTS actor_group_memberships_actor_id_group_id_index
+    """)
+
+    # Step 2: Drop the new single-column primary key
+    execute("""
+    ALTER TABLE actor_group_memberships
+    DROP CONSTRAINT IF EXISTS actor_group_memberships_pkey
+    """)
+
+    # Step 3: Restore the original composite primary key
+    execute("""
+    ALTER TABLE actor_group_memberships
+    ADD CONSTRAINT IF NOT EXISTS actor_group_memberships_pkey PRIMARY KEY (actor_id, group_id)
+    """)
+
+    # Step 4: Drop the id column
+    execute("""
+    ALTER TABLE actor_group_memberships
+    DROP COLUMN IF EXISTS id
+    """)
+  end
+end

elixir/apps/domain/priv/repo/migrations/20250712155618_backfill_flows_with_actor_group_membership_id.exs

@@ -0,0 +1,85 @@
+defmodule Domain.Repo.Migrations.BackfillFlowsWithActorGroupMembershipId do
+  use Ecto.Migration
+
+  @moduledoc """
+  This migration will lock the `flows` table, so it's best to run this when a brief period of
+  downtime is acceptable.
+  """
+
+  def up do
+    # Step 1: Truncate flows table to remove entries older than 14 days
+    execute("""
+    DELETE FROM flows
+    WHERE inserted_at < NOW() - INTERVAL '14 days'
+    """)
+
+    # Step 2: Add the new foreign key column if it doesn't already exist
+    execute("""
+    ALTER TABLE flows
+    ADD COLUMN IF NOT EXISTS actor_group_membership_id UUID
+    """)
+
+    # Step 3: Backfill the new column by finding the correct membership ID
+    execute("""
+    UPDATE flows AS f
+    SET actor_group_membership_id = agm.id
+    FROM
+      clients AS c,
+      policies AS p,
+      actor_group_memberships AS agm
+    WHERE
+      f.client_id = c.id
+      AND f.policy_id = p.id
+      AND c.actor_id = agm.actor_id
+      AND p.actor_group_id = agm.group_id
+    """)
+
+    # Step 4: Delete flow records where a membership couldn't be found
+    execute("""
+    DELETE FROM flows
+    WHERE actor_group_membership_id IS NULL
+    """)
+
+    # Step 5: Now that all rows are populated, make the column NOT NULL
+    execute("""
+    ALTER TABLE flows
+    ALTER COLUMN actor_group_membership_id SET NOT NULL
+    """)
+
+    # Step 6: Add an index on the new foreign key for performance
+    execute("""
+    CREATE INDEX IF NOT EXISTS flows_actor_group_membership_id_index
+    ON flows USING BTREE (account_id, actor_group_membership_id, inserted_at DESC, id DESC)
+    """)
+
+    # Step 7: Add the foreign key constraint
+    execute("""
+    ALTER TABLE flows
+    ADD CONSTRAINT flows_actor_group_membership_id_fkey
+    FOREIGN KEY (actor_group_membership_id)
+    REFERENCES actor_group_memberships(id)
+    ON DELETE CASCADE
+    """)
+  end
+
+  def down do
+    # Step 1: Drop the foreign key constraint
+    execute("""
+    ALTER TABLE flows
+    DROP CONSTRAINT IF EXISTS flows_actor_group_membership_id_fkey
+    """)
+
+    # Step 2: Drop the index
+    execute("""
+    DROP INDEX IF EXISTS flows_actor_group_membership_id_index
+    """)
+
+    # Step 3: Drop the column
+    execute("""
+    ALTER TABLE flows
+    DROP COLUMN IF EXISTS actor_group_membership_id
+    """)
+
+    # Note: The data deleted in the 'up' migration is not restored.
+  end
+end

elixir/apps/domain/priv/repo/seeds.exs

@@ -1048,7 +1048,7 @@ defmodule Domain.Repo.Seeds do
     IO.puts("  #{search_domain_resource.address} - DNS - gateways: #{gateway_name}")
     IO.puts("")
 
-    {:ok, _} =
+    {:ok, policy} =
       Policies.create_policy(
         %{
           name: "All Access To Google",
@@ -1175,11 +1175,12 @@ defmodule Domain.Repo.Seeds do
 
     IO.puts("")
 
-    {:ok, _resource, _flow, _expires_at} =
-      Flows.authorize_flow(
+    {:ok, _flow} =
+      Flows.create_flow(
         user_iphone,
         gateway1,
         cidr_resource.id,
+        policy,
         unprivileged_subject
       )
   end

elixir/apps/domain/test/domain/actors_test.exs

@@ -4,7 +4,6 @@ defmodule Domain.ActorsTest do
   alias Domain.Auth
   alias Domain.Clients
   alias Domain.Actors
-  alias Domain.Events
 
   describe "fetch_groups_count_grouped_by_provider_id/1" do
     test "returns empty map when there are no groups" do
@@ -1169,28 +1168,9 @@ defmodule Domain.ActorsTest do
       provider: provider,
       group1: group1,
       group2: group2,
-      actor1: actor1,
       identity1: identity1,
       identity2: identity2
     } do
-      policy = Fixtures.Policies.create_policy(account: account, actor_group: group1)
-
-      client =
-        Fixtures.Clients.create_client(
-          account: account,
-          actor: actor1,
-          identity: identity1
-        )
-
-      flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          actor_group: group1,
-          client: client,
-          resource_id: policy.resource_id,
-          policy: policy
-        )
-
       Fixtures.Actors.create_membership(
         account: account,
         group: group1,
@@ -1233,27 +1213,6 @@ defmodule Domain.ActorsTest do
 
       assert Repo.aggregate(Actors.Membership, :count) == 0
       assert Repo.aggregate(Actors.Membership.Query.all(), :count) == 0
-
-      # TODO: WAL
-      # These tests will be made redundant soon
-      Events.Hooks.ActorGroupMemberships.on_delete(%{
-        "account_id" => account.id,
-        "actor_id" => identity1.actor_id,
-        "group_id" => group1.id
-      })
-
-      Events.Hooks.ActorGroupMemberships.on_delete(%{
-        "account_id" => account.id,
-        "actor_id" => identity2.actor_id,
-        "group_id" => group2.id
-      })
-
-      :ok = Domain.PubSub.Flow.subscribe(flow.id)
-
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
     end
 
     test "deletes memberships of removed groups", %{
@@ -3036,30 +2995,6 @@ defmodule Domain.ActorsTest do
       assert token.deleted_at
     end
 
-    test "expires actor flows" do
-      account = Fixtures.Accounts.create_account()
-      actor = Fixtures.Actors.create_actor(account: account, type: :account_admin_user)
-      identity = Fixtures.Auth.create_identity(account: account, actor: actor)
-      subject = Fixtures.Auth.create_subject(identity: identity)
-      client = Fixtures.Clients.create_client(account: account, identity: identity)
-
-      flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          subject: subject,
-          client: client
-        )
-
-      :ok = Domain.PubSub.Flow.subscribe(flow.id)
-
-      assert {:ok, _actor} = disable_actor(actor, subject)
-
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-    end
-
     test "returns error when trying to disable the last admin actor" do
       account = Fixtures.Accounts.create_account()
       actor = Fixtures.Actors.create_actor(account: account, type: :account_admin_user)
@@ -3284,31 +3219,6 @@ defmodule Domain.ActorsTest do
       assert Repo.aggregate(Actors.Membership, :count) == 0
     end
 
-    test "expires actor flows", %{
-      account: account,
-      actor: actor,
-      identity: identity,
-      subject: subject
-    } do
-      client = Fixtures.Clients.create_client(account: account, identity: identity)
-
-      flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          subject: subject,
-          client: client
-        )
-
-      :ok = Domain.PubSub.Flow.subscribe(flow.id)
-
-      assert {:ok, _actor} = delete_actor(actor, subject)
-
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-    end
-
     test "returns error when trying to delete the last admin actor", %{
       actor: actor,
       subject: subject

elixir/apps/domain/test/domain/auth/adapters/google_workspace/jobs/sync_directory_test.exs

@@ -1,6 +1,6 @@
 defmodule Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectoryTest do
   use Domain.DataCase, async: true
-  alias Domain.{Auth, Actors, Events, PubSub}
+  alias Domain.{Auth, Actors}
   alias Domain.Mocks.GoogleWorkspaceDirectory
   import Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectory
 
@@ -575,20 +575,6 @@ defmodule Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectoryTest do
           identity: deleted_identity
         )
 
-      deleted_identity_client =
-        Fixtures.Clients.create_client(
-          account: account,
-          actor: other_actor,
-          identity: deleted_identity
-        )
-
-      deleted_identity_flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          client: deleted_identity_client,
-          token_id: deleted_identity_token.id
-        )
-
       group =
         Fixtures.Actors.create_group(
           account: account,
@@ -610,34 +596,12 @@ defmodule Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectoryTest do
           provider_identifier: "OU:OU_ID1"
         )
 
-      policy = Fixtures.Policies.create_policy(account: account, actor_group: group)
-
-      deleted_policy =
-        Fixtures.Policies.create_policy(account: account, actor_group: deleted_group)
-
-      deleted_group_flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          actor_group: deleted_group,
-          resource_id: deleted_policy.resource_id,
-          policy: deleted_policy
-        )
-
       Fixtures.Actors.create_membership(account: account, actor: actor)
       Fixtures.Actors.create_membership(account: account, actor: actor, group: group)
       deleted_membership = Fixtures.Actors.create_membership(account: account, group: group)
       Fixtures.Actors.create_membership(account: account, actor: actor, group: deleted_group)
       Fixtures.Actors.create_membership(account: account, actor: actor, group: org_unit)
 
-      :ok = PubSub.Flow.subscribe(deleted_group_flow.id)
-      :ok = PubSub.Flow.subscribe(deleted_identity_flow.id)
-      :ok = PubSub.Actor.Memberships.subscribe(actor.id)
-      :ok = PubSub.Actor.Memberships.subscribe(other_actor.id)
-      :ok = PubSub.Actor.Memberships.subscribe(deleted_membership.actor_id)
-      :ok = PubSub.Actor.Policies.subscribe(actor.id)
-      :ok = PubSub.Actor.Policies.subscribe(other_actor.id)
-      :ok = PubSub.ActorGroup.Policies.subscribe(deleted_group.id)
-
       GoogleWorkspaceDirectory.override_endpoint_url("http://localhost:#{bypass.port}/")
 
       GoogleWorkspaceDirectory.mock_groups_list_endpoint(
@@ -706,18 +670,6 @@ defmodule Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectoryTest do
       # Created membership for a member of existing group
       assert Repo.get_by(Domain.Actors.Membership, actor_id: other_actor.id, group_id: group.id)
 
-      # Broadcasts allow_access for it
-      policy_id = policy.id
-      group_id = group.id
-      resource_id = policy.resource_id
-
-      Events.Hooks.ActorGroupMemberships.on_insert(%{
-        "actor_id" => actor.id,
-        "group_id" => group.id
-      })
-
-      assert_receive {:allow_access, ^policy_id, ^group_id, ^resource_id}
-
       # Deletes membership that is not found on IdP end
       refute Repo.get_by(Domain.Actors.Membership,
                actor_id: deleted_membership.actor_id,
@@ -727,47 +679,6 @@ defmodule Domain.Auth.Adapters.GoogleWorkspace.Jobs.SyncDirectoryTest do
       # Signs out users which identity has been deleted
       deleted_identity_token = Repo.reload(deleted_identity_token)
       assert deleted_identity_token.deleted_at
-
-      # Deleted group deletes all policies and broadcasts reject access events for them
-      policy_id = deleted_policy.id
-      group_id = deleted_group.id
-      resource_id = deleted_policy.resource_id
-
-      # Simulate WAL events
-      Events.Hooks.ActorGroupMemberships.on_delete(%{
-        "account_id" => deleted_identity.account_id,
-        "actor_id" => deleted_identity.actor_id,
-        "group_id" => deleted_group.id
-      })
-
-      Events.Hooks.Policies.on_delete(%{
-        "id" => policy_id,
-        "actor_group_id" => group_id,
-        "resource_id" => resource_id,
-        "account_id" => deleted_identity.account_id
-      })
-
-      assert_receive {:reject_access, ^policy_id, ^group_id, ^resource_id}
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      # Deleted policies expire all flows authorized by them
-      flow_id = deleted_group_flow.id
-      client_id = deleted_group_flow.client_id
-      resource_id = deleted_group_flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-
-      # Expires flows for signed out user
-      flow_id = deleted_identity_flow.id
-      client_id = deleted_identity_flow.client_id
-      resource_id = deleted_identity_flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-
-      # Should not do anything else
-      refute_received {:allow_access, _policy_id, _group_id, _resource_id}
-      refute_received {:reject_access, _policy_id, _group_id, _resource_id}
     end
 
     test "resurrects deleted identities that reappear on the next sync", %{

elixir/apps/domain/test/domain/auth/adapters/jumpcloud/jobs/sync_directory_test.exs

@@ -1,6 +1,6 @@
 defmodule Domain.Auth.Adapters.JumpCloud.Jobs.SyncDirectoryTest do
   use Domain.DataCase, async: true
-  alias Domain.{Auth, Actors, Events, PubSub}
+  alias Domain.{Auth, Actors}
   alias Domain.Mocks.WorkOSDirectory
   import Domain.Auth.Adapters.JumpCloud.Jobs.SyncDirectory
 
@@ -363,20 +363,6 @@ defmodule Domain.Auth.Adapters.JumpCloud.Jobs.SyncDirectoryTest do
           identity: deleted_identity
         )
 
-      deleted_identity_client =
-        Fixtures.Clients.create_client(
-          account: account,
-          actor: other_actor,
-          identity: deleted_identity
-        )
-
-      deleted_identity_flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          client: deleted_identity_client,
-          token_id: deleted_identity_token.id
-        )
-
       group =
         Fixtures.Actors.create_group(
           account: account,
@@ -398,33 +384,10 @@ defmodule Domain.Auth.Adapters.JumpCloud.Jobs.SyncDirectoryTest do
           provider_identifier: "G:DELETED_GROUP_ID!"
         )
 
-      policy = Fixtures.Policies.create_policy(account: account, actor_group: group)
-
-      deleted_policy =
-        Fixtures.Policies.create_policy(account: account, actor_group: deleted_group)
-
-      deleted_group_flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          actor_group: deleted_group,
-          resource_id: deleted_policy.resource_id,
-          policy: deleted_policy
-        )
-
       Fixtures.Actors.create_membership(account: account, actor: actor)
       Fixtures.Actors.create_membership(account: account, actor: actor, group: group)
-      deleted_membership = Fixtures.Actors.create_membership(account: account, group: group)
       Fixtures.Actors.create_membership(account: account, actor: actor, group: deleted_group)
 
-      :ok = PubSub.Flow.subscribe(deleted_identity_flow.id)
-      :ok = PubSub.Flow.subscribe(deleted_group_flow.id)
-      :ok = PubSub.Actor.Memberships.subscribe(actor.id)
-      :ok = PubSub.Actor.Memberships.subscribe(other_actor.id)
-      :ok = PubSub.Actor.Memberships.subscribe(deleted_membership.actor_id)
-      :ok = PubSub.Actor.Policies.subscribe(actor.id)
-      :ok = PubSub.Actor.Policies.subscribe(other_actor.id)
-      :ok = PubSub.ActorGroup.Policies.subscribe(deleted_group.id)
-
       WorkOSDirectory.override_base_url("http://localhost:#{bypass.port}")
       WorkOSDirectory.mock_list_directories_endpoint(bypass)
       WorkOSDirectory.mock_list_users_endpoint(bypass, users)
@@ -473,65 +436,12 @@ defmodule Domain.Auth.Adapters.JumpCloud.Jobs.SyncDirectoryTest do
                group_id: group.id
              )
 
-      # Broadcasts allow_access for it
-      policy_id = policy.id
-      group_id = group.id
-      resource_id = policy.resource_id
-
-      Events.Hooks.ActorGroupMemberships.on_insert(%{
-        "actor_id" => actor.id,
-        "group_id" => group.id
-      })
-
-      assert_receive {:allow_access, ^policy_id, ^group_id, ^resource_id}
-
       # Deletes membership that is not found on IdP end
       refute Repo.get_by(Domain.Actors.Membership, group_id: deleted_group.id)
 
       # Signs out users which identity has been deleted
       deleted_identity_token = Repo.reload(deleted_identity_token)
       assert deleted_identity_token.deleted_at
-
-      # Deleted group deletes all policies and broadcasts reject access events for them
-      policy_id = deleted_policy.id
-      group_id = deleted_group.id
-      resource_id = deleted_policy.resource_id
-
-      # Simulate the WAL events
-      Events.Hooks.ActorGroupMemberships.on_delete(%{
-        "account_id" => deleted_group.account_id,
-        "actor_id" => actor.id,
-        "group_id" => deleted_group.id
-      })
-
-      Events.Hooks.Policies.on_delete(%{
-        "id" => policy_id,
-        "account_id" => deleted_policy.account_id,
-        "actor_group_id" => group_id,
-        "resource_id" => resource_id
-      })
-
-      assert_receive {:reject_access, ^policy_id, ^group_id, ^resource_id}
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      # Deleted policies expire all flows authorized by them
-      flow_id = deleted_group_flow.id
-      client_id = deleted_group_flow.client_id
-      resource_id = deleted_group_flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-
-      # Expires flows for signed out user
-      flow_id = deleted_identity_flow.id
-      client_id = deleted_identity_flow.client_id
-      resource_id = deleted_identity_flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-
-      # Should not do anything else
-      refute_received {:allow_access, _policy_id, _group_id, _resource_id}
-      refute_received {:reject_access, _policy_id, _group_id, _resource_id}
     end
 
     test "resurrects deleted identities that reappear on the next sync", %{

elixir/apps/domain/test/domain/auth/adapters/microsoft_entra/jobs/sync_directory_test.exs

@@ -1,6 +1,6 @@
 defmodule Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectoryTest do
   use Domain.DataCase, async: true
-  alias Domain.{Auth, Actors, Events, PubSub}
+  alias Domain.{Auth, Actors}
   alias Domain.Mocks.MicrosoftEntraDirectory
   import Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectory
 
@@ -411,20 +411,6 @@ defmodule Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectoryTest do
           identity: deleted_identity
         )
 
-      deleted_identity_client =
-        Fixtures.Clients.create_client(
-          account: account,
-          actor: other_actor,
-          identity: deleted_identity
-        )
-
-      deleted_identity_flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          client: deleted_identity_client,
-          token_id: deleted_identity_token.id
-        )
-
       group =
         Fixtures.Actors.create_group(
           account: account,
@@ -446,33 +432,11 @@ defmodule Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectoryTest do
           provider_identifier: "G:DELETED_GROUP_ID!"
         )
 
-      policy = Fixtures.Policies.create_policy(account: account, actor_group: group)
-
-      deleted_policy =
-        Fixtures.Policies.create_policy(account: account, actor_group: deleted_group)
-
-      deleted_group_flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          actor_group: deleted_group,
-          resource_id: deleted_policy.resource_id,
-          policy: deleted_policy
-        )
-
       Fixtures.Actors.create_membership(account: account, actor: actor)
       Fixtures.Actors.create_membership(account: account, actor: actor, group: group)
       deleted_membership = Fixtures.Actors.create_membership(account: account, group: group)
       Fixtures.Actors.create_membership(account: account, actor: actor, group: deleted_group)
 
-      :ok = PubSub.Flow.subscribe(deleted_identity_flow.id)
-      :ok = PubSub.Flow.subscribe(deleted_group_flow.id)
-      :ok = PubSub.Actor.Memberships.subscribe(actor.id)
-      :ok = PubSub.Actor.Memberships.subscribe(other_actor.id)
-      :ok = PubSub.Actor.Memberships.subscribe(deleted_membership.actor_id)
-      :ok = PubSub.Actor.Policies.subscribe(actor.id)
-      :ok = PubSub.Actor.Policies.subscribe(other_actor.id)
-      :ok = PubSub.ActorGroup.Policies.subscribe(deleted_group.id)
-
       MicrosoftEntraDirectory.mock_groups_list_endpoint(
         bypass,
         200,
@@ -536,18 +500,6 @@ defmodule Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectoryTest do
       # Created membership for a member of existing group
       assert Repo.get_by(Domain.Actors.Membership, actor_id: other_actor.id, group_id: group.id)
 
-      # Broadcasts allow_access for it
-      policy_id = policy.id
-      group_id = group.id
-      resource_id = policy.resource_id
-
-      Events.Hooks.ActorGroupMemberships.on_insert(%{
-        "actor_id" => actor.id,
-        "group_id" => group.id
-      })
-
-      assert_receive {:allow_access, ^policy_id, ^group_id, ^resource_id}
-
       # Deletes membership that is not found on IdP end
       refute Repo.get_by(
                Domain.Actors.Membership,
@@ -558,47 +510,6 @@ defmodule Domain.Auth.Adapters.MicrosoftEntra.Jobs.SyncDirectoryTest do
       # Signs out users which identity has been deleted
       deleted_identity_token = Repo.reload(deleted_identity_token)
       assert deleted_identity_token.deleted_at
-
-      # Deleted group deletes all policies and broadcasts reject access events for them
-      policy_id = deleted_policy.id
-      group_id = deleted_group.id
-      resource_id = deleted_policy.resource_id
-
-      # Simulate WAL events
-      Events.Hooks.ActorGroupMemberships.on_delete(%{
-        "account_id" => deleted_identity.account_id,
-        "actor_id" => deleted_identity.actor_id,
-        "group_id" => deleted_group.id
-      })
-
-      Events.Hooks.Policies.on_delete(%{
-        "id" => policy_id,
-        "account_id" => deleted_policy.account_id,
-        "actor_group_id" => group_id,
-        "resource_id" => resource_id
-      })
-
-      assert_receive {:reject_access, ^policy_id, ^group_id, ^resource_id}
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      # Deleted policies expire all flows authorized by them
-      flow_id = deleted_group_flow.id
-      client_id = deleted_group_flow.client_id
-      resource_id = deleted_group_flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-
-      # Expires flows for signed out user
-      flow_id = deleted_identity_flow.id
-      client_id = deleted_identity_flow.client_id
-      resource_id = deleted_identity_flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-
-      # Should not do anything else
-      refute_received {:allow_access, _policy_id, _group_id, _resource_id}
-      refute_received {:reject_access, _policy_id, _group_id, _resource_id}
     end
 
     test "stops the sync retries on 401 error on the provider", %{

elixir/apps/domain/test/domain/auth/adapters/okta/jobs/sync_directory_test.exs

@@ -1,6 +1,6 @@
 defmodule Domain.Auth.Adapters.Okta.Jobs.SyncDirectoryTest do
   use Domain.DataCase, async: true
-  alias Domain.{Auth, Actors, Events, PubSub}
+  alias Domain.{Auth, Actors}
   alias Domain.Mocks.OktaDirectory
   import Domain.Auth.Adapters.Okta.Jobs.SyncDirectory
 
@@ -655,20 +655,6 @@ defmodule Domain.Auth.Adapters.Okta.Jobs.SyncDirectoryTest do
           identity: deleted_identity
         )
 
-      deleted_identity_client =
-        Fixtures.Clients.create_client(
-          account: account,
-          actor: other_actor,
-          identity: deleted_identity
-        )
-
-      deleted_identity_flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          client: deleted_identity_client,
-          token_id: deleted_identity_token.id
-        )
-
       group =
         Fixtures.Actors.create_group(
           account: account,
@@ -690,33 +676,11 @@ defmodule Domain.Auth.Adapters.Okta.Jobs.SyncDirectoryTest do
           provider_identifier: "G:DELETED_GROUP_ID!"
         )
 
-      policy = Fixtures.Policies.create_policy(account: account, actor_group: group)
-
-      deleted_policy =
-        Fixtures.Policies.create_policy(account: account, actor_group: deleted_group)
-
-      deleted_group_flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          actor_group: deleted_group,
-          resource_id: deleted_policy.resource_id,
-          policy: deleted_policy
-        )
-
       Fixtures.Actors.create_membership(account: account, actor: actor)
       Fixtures.Actors.create_membership(account: account, actor: actor, group: group)
       deleted_membership = Fixtures.Actors.create_membership(account: account, group: group)
       Fixtures.Actors.create_membership(account: account, actor: actor, group: deleted_group)
 
-      :ok = PubSub.Flow.subscribe(deleted_identity_flow.id)
-      :ok = PubSub.Flow.subscribe(deleted_group_flow.id)
-      :ok = PubSub.Actor.Memberships.subscribe(actor.id)
-      :ok = PubSub.Actor.Memberships.subscribe(other_actor.id)
-      :ok = PubSub.Actor.Memberships.subscribe(deleted_membership.actor_id)
-      :ok = PubSub.Actor.Policies.subscribe(actor.id)
-      :ok = PubSub.Actor.Policies.subscribe(other_actor.id)
-      :ok = PubSub.ActorGroup.Policies.subscribe(deleted_group.id)
-
       OktaDirectory.mock_groups_list_endpoint(bypass, 200, Jason.encode!(groups))
       OktaDirectory.mock_users_list_endpoint(bypass, 200, Jason.encode!(users))
 
@@ -771,18 +735,6 @@ defmodule Domain.Auth.Adapters.Okta.Jobs.SyncDirectoryTest do
       # Creates membership for a member of existing group
       assert Repo.get_by(Domain.Actors.Membership, actor_id: other_actor.id, group_id: group.id)
 
-      # Broadcasts allow_access for it
-      policy_id = policy.id
-      group_id = group.id
-      resource_id = policy.resource_id
-
-      Events.Hooks.ActorGroupMemberships.on_insert(%{
-        "actor_id" => other_actor.id,
-        "group_id" => group.id
-      })
-
-      assert_receive {:allow_access, ^policy_id, ^group_id, ^resource_id}
-
       # Deletes membership that is not found on IdP end
       refute Repo.get_by(Domain.Actors.Membership,
                actor_id: deleted_membership.actor_id,
@@ -792,48 +744,6 @@ defmodule Domain.Auth.Adapters.Okta.Jobs.SyncDirectoryTest do
       # Signs out users which identity has been deleted
       deleted_identity_token = Repo.reload(deleted_identity_token)
       assert deleted_identity_token.deleted_at
-
-      # Deleted group deletes all policies and broadcasts reject access events for them
-      policy_id = deleted_policy.id
-      group_id = deleted_group.id
-      resource_id = deleted_policy.resource_id
-      account_id = deleted_policy.account_id
-
-      # Simulate WAL events
-      Events.Hooks.ActorGroupMemberships.on_delete(%{
-        "account_id" => deleted_membership.account_id,
-        "actor_id" => actor.id,
-        "group_id" => deleted_group.id
-      })
-
-      Events.Hooks.Policies.on_delete(%{
-        "id" => policy_id,
-        "actor_group_id" => group_id,
-        "resource_id" => resource_id,
-        "account_id" => account_id
-      })
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      assert_receive {:reject_access, ^policy_id, ^group_id, ^resource_id}
-
-      # Deleted policies expire all flows authorized by them
-      flow_id = deleted_group_flow.id
-      client_id = deleted_group_flow.client_id
-      resource_id = deleted_group_flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-
-      # Expires flows for signed out user
-      flow_id = deleted_identity_flow.id
-      client_id = deleted_identity_flow.client_id
-      resource_id = deleted_identity_flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-
-      # Should not do anything else
-      refute_received {:allow_access, _policy_id, _group_id, _resource_id}
-      refute_received {:reject_access, _policy_id, _group_id, _resource_id}
     end
 
     test "resurrects deleted identities that reappear on the next sync", %{

elixir/apps/domain/test/domain/auth_test.exs

@@ -1054,31 +1054,6 @@ defmodule Domain.AuthTest do
       assert token.deleted_at
     end
 
-    test "expires provider flows", %{
-      account: account,
-      provider: provider,
-      identity: identity,
-      subject: subject
-    } do
-      client = Fixtures.Clients.create_client(account: account, identity: identity)
-
-      flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          subject: subject,
-          client: client
-        )
-
-      :ok = Domain.PubSub.Flow.subscribe(flow.id)
-
-      flow_id = flow.id
-      client_id = client.id
-      resource_id = flow.resource_id
-
-      assert {:ok, _provider} = disable_provider(provider, subject)
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-    end
-
     test "returns error when trying to disable the last provider", %{
       subject: subject,
       provider: provider
@@ -1279,31 +1254,6 @@ defmodule Domain.AuthTest do
       assert actor_group.deleted_at
     end
 
-    test "expires provider flows", %{
-      account: account,
-      provider: provider,
-      identity: identity,
-      subject: subject
-    } do
-      client = Fixtures.Clients.create_client(account: account, identity: identity)
-
-      flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          subject: subject,
-          client: client
-        )
-
-      :ok = Domain.PubSub.Flow.subscribe(flow.id)
-
-      flow_id = flow.id
-      client_id = client.id
-      resource_id = flow.resource_id
-
-      assert {:ok, _provider} = delete_provider(provider, subject)
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-    end
-
     test "returns error when trying to delete the last provider", %{
       subject: subject,
       provider: provider
@@ -1878,22 +1828,6 @@ defmodule Domain.AuthTest do
           identity: deleted_identity
         )
 
-      deleted_identity_client =
-        Fixtures.Clients.create_client(
-          account: account,
-          actor: deleted_identity_actor,
-          identity: deleted_identity
-        )
-
-      deleted_identity_flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          client: deleted_identity_client,
-          token_id: deleted_identity_token.id
-        )
-
-      :ok = Domain.PubSub.Flow.subscribe(deleted_identity_flow.id)
-
       for n <- 1..4 do
         Fixtures.Auth.create_identity(
           account: account,
@@ -1958,12 +1892,6 @@ defmodule Domain.AuthTest do
       # Signs out users which identity has been deleted
       deleted_identity_token = Repo.reload(deleted_identity_token)
       assert deleted_identity_token.deleted_at
-
-      # Expires flows for signed out user
-      flow_id = deleted_identity_flow.id
-      client_id = deleted_identity_flow.client_id
-      resource_id = deleted_identity_flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
     end
 
     test "circuit breaker prevents mass deletions of identities", %{
@@ -2789,33 +2717,6 @@ defmodule Domain.AuthTest do
       assert token.deleted_at
     end
 
-    test "expires all flows created using deleted tokens", %{
-      account: account,
-      actor: actor,
-      identity: identity,
-      subject: subject
-    } do
-      client = Fixtures.Clients.create_client(account: account, identity: identity)
-
-      flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          identity: identity,
-          actor: actor,
-          subject: subject,
-          client: client
-        )
-
-      :ok = Domain.PubSub.Flow.subscribe(flow.id)
-
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-
-      assert delete_identities_for(actor, subject) == :ok
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-    end
-
     test "does not remove identities that belong to another actor", %{
       account: account,
       provider: provider,

elixir/apps/domain/test/domain/clients_test.exs

@@ -1,7 +1,7 @@
 defmodule Domain.ClientsTest do
   use Domain.DataCase, async: true
   import Domain.Clients
-  alias Domain.{Clients, PubSub}
+  alias Domain.Clients
 
   setup do
     account = Fixtures.Accounts.create_account()
@@ -119,8 +119,6 @@ defmodule Domain.ClientsTest do
 
       {:ok, _} = Clients.Presence.Account.track(client.account_id, client.id)
       {:ok, _} = Clients.Presence.Actor.track(client.actor_id, client.id)
-      :ok = PubSub.Client.subscribe(client.id)
-      :ok = PubSub.Account.Clients.subscribe(client.account_id)
 
       assert {:ok, client} = fetch_client_by_id(client.id, subject, preload: [:online?])
       assert client.online? == true
@@ -228,8 +226,7 @@ defmodule Domain.ClientsTest do
 
       {:ok, _} = Clients.Presence.Account.track(client.account_id, client.id)
       {:ok, _} = Clients.Presence.Actor.track(client.actor_id, client.id)
-      :ok = PubSub.Client.subscribe(client.id)
-      :ok = PubSub.Account.Clients.subscribe(client.account_id)
+
       assert client = fetch_client_by_id!(client.id, preload: [:online?])
       assert client.online? == true
     end
@@ -290,8 +287,7 @@ defmodule Domain.ClientsTest do
 
       {:ok, _} = Clients.Presence.Account.track(client.account_id, client.id)
       {:ok, _} = Clients.Presence.Actor.track(client.actor_id, client.id)
-      :ok = PubSub.Client.subscribe(client.id)
-      :ok = PubSub.Account.Clients.subscribe(client.account_id)
+
       assert {:ok, [client], _metadata} = list_clients(subject, preload: [:online?])
       assert client.online? == true
     end
@@ -1077,32 +1073,6 @@ defmodule Domain.ClientsTest do
       assert is_nil(client.verified_by_subject)
     end
 
-    test "expires flows for the unverified client", %{
-      account: account,
-      admin_actor: actor,
-      admin_subject: subject
-    } do
-      client = Fixtures.Clients.create_client(actor: actor)
-
-      flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          actor: actor,
-          client: client,
-          subject: subject
-        )
-
-      :ok = Domain.PubSub.Flow.subscribe(flow.id)
-
-      assert {:ok, client} = verify_client(client, subject)
-      assert {:ok, _client} = remove_client_verification(client, subject)
-
-      flow_id = flow.id
-      client_id = client.id
-      resource_id = flow.resource_id
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-    end
-
     test "returns error when subject has no permission to verify clients", %{
       admin_actor: actor,
       admin_subject: subject

elixir/apps/domain/test/domain/events/hooks/accounts_test.exs

@@ -1,92 +1,84 @@
 defmodule Domain.Events.Hooks.AccountsTest do
   use Domain.DataCase, async: true
+  alias Domain.Accounts
   import Domain.Events.Hooks.Accounts
 
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
   describe "insert/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_insert(data)
+    test "returns :ok" do
+      assert :ok == on_insert(%{})
     end
   end
 
   describe "update/2" do
-    test "disconnects gateways if slug changes" do
-      account = Fixtures.Accounts.create_account()
-      gateway = Fixtures.Gateways.create_gateway(account: account)
-      :ok = Domain.Gateways.Presence.connect(gateway)
+    test "sends delete when account is disabled" do
+      account_id = "00000000-0000-0000-0000-000000000001"
 
-      old_data = %{"slug" => "old"}
-      data = %{"slug" => "new", "id" => account.id}
-
-      assert :ok == on_update(old_data, data)
-
-      assert_receive "disconnect"
-    end
-
-    test "sends :config_changed if config changes" do
-      account = Fixtures.Accounts.create_account()
-      gateway = Fixtures.Gateways.create_gateway(account: account)
-
-      :ok = Domain.PubSub.Account.subscribe(account.id)
-      :ok = Domain.Gateways.Presence.connect(gateway)
+      :ok = Domain.PubSub.Account.subscribe(account_id)
 
       old_data = %{
-        "id" => account.id,
-        "config" => %{"search_domain" => "old_value", "clients_upstream_dns" => []}
+        "id" => account_id,
+        "disabled_at" => nil
       }
 
       data = %{
-        "id" => account.id,
-        "config" => %{
-          "search_domain" => "new_value",
-          "clients_upstream_dns" => [%{"protocol" => "ip_port", "address" => "8.8.8.8"}]
-        }
+        "id" => account_id,
+        "disabled_at" => "2023-10-01T00:00:00Z"
       }
 
       assert :ok == on_update(old_data, data)
-      assert_receive :config_changed
-      refute_receive "disconnect"
+      assert_receive {:deleted, %Accounts.Account{} = account}
+
+      assert account.id == account_id
     end
 
-    test "does not send :config_changed if config does not change" do
-      account = Fixtures.Accounts.create_account()
-
-      :ok = Domain.PubSub.Account.subscribe(account.id)
+    test "sends delete when soft-deleted" do
+      account_id = "00000000-0000-0000-0000-000000000002"
+      :ok = Domain.PubSub.Account.subscribe(account_id)
 
       old_data = %{
-        "id" => account.id,
-        "config" => %{"search_domain" => "old_value", "clients_upstream_dns" => []}
+        "id" => account_id,
+        "deleted_at" => nil
       }
 
       data = %{
-        "id" => account.id,
-        "config" => %{"search_domain" => "old_value", "clients_upstream_dns" => []}
+        "id" => account_id,
+        "deleted_at" => "2023-10-01T00:00:00Z"
       }
 
       assert :ok == on_update(old_data, data)
-      refute_receive :config_changed
-    end
+      assert_receive {:deleted, %Accounts.Account{} = account}
 
-    test "sends disconnect to clients if account is disabled" do
-      account_id = Fixtures.Accounts.create_account().id
-
-      old_data = %{"id" => account_id, "disabled_at" => nil}
-      data = %{"id" => account_id, "disabled_at" => DateTime.utc_now()}
-
-      :ok = Domain.PubSub.Account.Clients.subscribe(account_id)
-
-      assert :ok == on_update(old_data, data)
-
-      assert_receive "disconnect"
+      assert account.id == account_id
     end
   end
 
   describe "delete/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_delete(data)
+    test "delete broadcasts deleted account" do
+      account_id = "00000000-0000-0000-0000-000000000003"
+      :ok = Domain.PubSub.Account.subscribe(account_id)
+
+      old_data = %{
+        "id" => account_id,
+        "deleted_at" => "2023-10-01T00:00:00Z"
+      }
+
+      assert :ok == on_delete(old_data)
+      assert_receive {:deleted, %Accounts.Account{} = account}
+      assert account.id == account_id
+      assert account.deleted_at == ~U[2023-10-01 00:00:00.000000Z]
+    end
+
+    test "deletes associated flows when account is deleted" do
+      account = Fixtures.Accounts.create_account()
+      flow = Fixtures.Flows.create_flow(account: account)
+
+      old_data = %{
+        "id" => account.id,
+        "deleted_at" => "2023-10-01T00:00:00Z"
+      }
+
+      assert :ok == on_delete(old_data)
+      assert Repo.get_by(Domain.Flows.Flow, id: flow.id) == nil
     end
   end
 end

elixir/apps/domain/test/domain/events/hooks/actor_group_memberships_test.exs

@@ -1,37 +1,34 @@
 defmodule Domain.Events.Hooks.ActorGroupMembershipsTest do
   use API.ChannelCase, async: true
   import Domain.Events.Hooks.ActorGroupMemberships
+  alias Domain.Actors
   alias Domain.PubSub
 
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
   describe "insert/1" do
-    test "returns :ok" do
-      actor_id = "#{Ecto.UUID.generate()}"
-      group_id = "#{Ecto.UUID.generate()}"
+    test "broadcasts membership" do
+      account_id = "00000000-0000-0000-0000-000000000001"
+      actor_id = "00000000-0000-0000-0000-000000000002"
+      group_id = "00000000-0000-0000-0000-000000000003"
+
+      :ok = PubSub.Account.subscribe(account_id)
 
       data = %{
+        "account_id" => account_id,
         "actor_id" => actor_id,
         "group_id" => group_id
       }
 
-      :ok = PubSub.Actor.Memberships.subscribe(actor_id)
-
       assert :ok == on_insert(data)
-
-      # TODO: WAL
-      # Remove this when direct broadcast is implement
-      Process.sleep(100)
-
-      assert_receive {:create_membership, ^actor_id, ^group_id}
+      assert_receive {:created, %Actors.Membership{} = membership}
+      assert membership.account_id == account_id
+      assert membership.actor_id == actor_id
+      assert membership.group_id == group_id
     end
   end
 
   describe "update/2" do
-    test "returns :ok", %{old_data: old_data, data: data} do
-      assert :ok == on_update(old_data, data)
+    test "returns :ok" do
+      assert :ok == on_update(%{}, %{})
     end
   end
 
@@ -40,80 +37,53 @@ defmodule Domain.Events.Hooks.ActorGroupMembershipsTest do
       account = Fixtures.Accounts.create_account()
       actor_group = Fixtures.Actors.create_group(account: account)
       actor = Fixtures.Actors.create_actor(account: account, type: :account_admin_user)
-      Fixtures.Actors.create_membership(account: account, group: actor_group, actor: actor)
-      identity = Fixtures.Auth.create_identity(account: account, actor: actor)
-      subject = Fixtures.Auth.create_subject(identity: identity)
-      client = Fixtures.Clients.create_client(subject: subject)
 
-      resource = Fixtures.Resources.create_resource(account: account)
-
-      policy =
-        Fixtures.Policies.create_policy(
-          account: account,
-          resource: resource,
-          actor_group: actor_group
-        )
-
-      {:ok, _reply, socket} =
-        API.Client.Socket
-        |> socket("client:#{client.id}", %{
-          opentelemetry_ctx: OpenTelemetry.Ctx.new(),
-          opentelemetry_span_ctx: OpenTelemetry.Tracer.start_span("test"),
-          client: client,
-          subject: subject,
-          turn_salt: "test_salt"
-        })
-        |> subscribe_and_join(API.Client.Channel, "client")
+      membership =
+        Fixtures.Actors.create_membership(account: account, group: actor_group, actor: actor)
 
       %{
         account: account,
         actor_group: actor_group,
         actor: actor,
-        identity: identity,
-        subject: subject,
-        client: client,
-        resource: resource,
-        policy: policy,
-        socket: socket
+        membership: membership
       }
     end
 
-    test "returns :ok" do
-      actor_id = "#{Ecto.UUID.generate()}"
-      group_id = "#{Ecto.UUID.generate()}"
+    test "broadcasts deleted membership" do
+      account_id = "00000000-0000-0000-0000-000000000001"
+      :ok = PubSub.Account.subscribe(account_id)
 
-      data = %{
-        "account_id" => "#{Ecto.UUID.generate()}",
-        "actor_id" => actor_id,
-        "group_id" => group_id
+      old_data = %{
+        "id" => "00000000-0000-0000-0000-000000000000",
+        "account_id" => "00000000-0000-0000-0000-000000000001",
+        "actor_id" => "00000000-0000-0000-0000-000000000002",
+        "group_id" => "00000000-0000-0000-0000-000000000003"
       }
 
-      :ok = PubSub.Actor.Memberships.subscribe(actor_id)
+      assert :ok == on_delete(old_data)
 
-      assert :ok == on_delete(data)
-      assert_receive {:delete_membership, ^actor_id, ^group_id}
+      assert_receive {:deleted, %Actors.Membership{} = membership}
+      assert membership.id == "00000000-0000-0000-0000-000000000000"
+      assert membership.account_id == "00000000-0000-0000-0000-000000000001"
+      assert membership.actor_id == "00000000-0000-0000-0000-000000000002"
+      assert membership.group_id == "00000000-0000-0000-0000-000000000003"
     end
 
-    test "client channel pushes \"resource_deleted\" when affected membership is deleted", %{
-      actor: actor,
-      subject: subject,
-      actor_group: actor_group
-    } do
-      assert_push "init", %{}
-      # TODO: WAL
-      # This is needed because the :reject_access received in the client channel re-fetches allowed resources for this client.
-      # Remove this when that's cleaned up.
-      {:ok, _actor} = Domain.Actors.update_actor(actor, %{memberships: []}, subject)
+    test "deletes flows for membership", %{account: account, membership: membership} do
+      flow = Fixtures.Flows.create_flow(account: account, actor_group_membership: membership)
+      unrelated_flow = Fixtures.Flows.create_flow(account: account)
 
-      assert :ok =
-               on_delete(%{
-                 "account_id" => actor.account_id,
-                 "actor_id" => actor.id,
-                 "group_id" => actor_group.id
-               })
+      old_data = %{
+        "id" => membership.id,
+        "account_id" => membership.account_id,
+        "actor_id" => membership.actor_id,
+        "group_id" => membership.group_id
+      }
 
-      assert_push "resource_deleted", _payload
-      refute_push "resource_created_or_updated", _payload
+      assert ^flow = Repo.get_by(Domain.Flows.Flow, actor_group_membership_id: membership.id)
+      assert :ok == on_delete(old_data)
+      assert nil == Repo.get_by(Domain.Flows.Flow, actor_group_membership_id: membership.id)
+      assert ^unrelated_flow = Repo.get_by(Domain.Flows.Flow, id: unrelated_flow.id)
     end
   end
 end

elixir/apps/domain/test/domain/events/hooks/actor_groups_test.exs

@@ -1,26 +0,0 @@
-defmodule Domain.Events.Hooks.ActorGroupsTest do
-  use ExUnit.Case, async: true
-  import Domain.Events.Hooks.ActorGroups
-
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
-  describe "insert/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_insert(data)
-    end
-  end
-
-  describe "update/2" do
-    test "returns :ok", %{old_data: old_data, data: data} do
-      assert :ok == on_update(old_data, data)
-    end
-  end
-
-  describe "delete/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_delete(data)
-    end
-  end
-end

elixir/apps/domain/test/domain/events/hooks/actors_test.exs

@@ -1,26 +0,0 @@
-defmodule Domain.Events.Hooks.ActorsTest do
-  use ExUnit.Case, async: true
-  import Domain.Events.Hooks.Actors
-
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
-  describe "insert/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_insert(data)
-    end
-  end
-
-  describe "update/2" do
-    test "returns :ok", %{old_data: old_data, data: data} do
-      assert :ok == on_update(old_data, data)
-    end
-  end
-
-  describe "delete/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_delete(data)
-    end
-  end
-end

elixir/apps/domain/test/domain/events/hooks/auth_identities_test.exs

@@ -1,26 +0,0 @@
-defmodule Domain.Events.Hooks.AuthIdentitiesTest do
-  use ExUnit.Case, async: true
-  import Domain.Events.Hooks.AuthIdentities
-
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
-  describe "insert/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_insert(data)
-    end
-  end
-
-  describe "update/2" do
-    test "returns :ok", %{old_data: old_data, data: data} do
-      assert :ok == on_update(old_data, data)
-    end
-  end
-
-  describe "delete/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_delete(data)
-    end
-  end
-end

elixir/apps/domain/test/domain/events/hooks/auth_providers_test.exs

@@ -1,26 +0,0 @@
-defmodule Domain.Events.Hooks.AuthProvidersTest do
-  use ExUnit.Case, async: true
-  import Domain.Events.Hooks.AuthProviders
-
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
-  describe "insert/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_insert(data)
-    end
-  end
-
-  describe "update/2" do
-    test "returns :ok", %{old_data: old_data, data: data} do
-      assert :ok == on_update(old_data, data)
-    end
-  end
-
-  describe "delete/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_delete(data)
-    end
-  end
-end

elixir/apps/domain/test/domain/events/hooks/clients_test.exs

@@ -3,81 +3,90 @@ defmodule Domain.Events.Hooks.ClientsTest do
   import Domain.Events.Hooks.Clients
   alias Domain.{Clients, PubSub}
 
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
   describe "insert/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_insert(data)
+    test "returns :ok" do
+      assert :ok == on_insert(%{})
     end
   end
 
   describe "update/2" do
-    test "soft-delete broadcasts disconnect" do
+    test "soft-delete broadcasts deleted client" do
       client = Fixtures.Clients.create_client()
-      :ok = Clients.Presence.connect(client)
+      :ok = PubSub.Account.subscribe(client.account_id)
 
-      old_data = %{"id" => client.id, "deleted_at" => nil}
-      data = %{"id" => client.id, "deleted_at" => DateTime.utc_now()}
+      old_data = %{"id" => client.id, "deleted_at" => nil, "account_id" => client.account_id}
+
+      data = %{
+        "id" => client.id,
+        "deleted_at" => DateTime.utc_now(),
+        "account_id" => client.account_id
+      }
 
       assert :ok == on_update(old_data, data)
-
-      assert_receive "disconnect"
-      refute_receive :updated
+      assert_receive {:deleted, %Clients.Client{} = deleted_client}
+      assert deleted_client.id == client.id
     end
 
-    test "update broadcasts :update" do
-      client = Fixtures.Clients.create_client()
-      :ok = Clients.Presence.connect(client)
+    test "update broadcasts updated client" do
+      account = Fixtures.Accounts.create_account()
+      client = Fixtures.Clients.create_client(account: account)
+      :ok = PubSub.Account.subscribe(client.account_id)
 
-      old_data = %{"id" => client.id, "name" => "Old Client"}
-      data = %{"id" => client.id, "name" => "Updated Client"}
+      old_data = %{"id" => client.id, "name" => "Old Name", "account_id" => client.account_id}
+      data = %{"id" => client.id, "name" => "New Name", "account_id" => client.account_id}
 
       assert :ok == on_update(old_data, data)
 
-      assert_receive {:updated, %Clients.Client{} = updated_client}
-      assert updated_client.id == client.id
-      refute_receive "disconnect"
+      assert_receive {:updated, %Clients.Client{} = old_client, %Clients.Client{} = new_client}
+      assert old_client.name == "Old Name"
+      assert new_client.name == "New Name"
+      assert new_client.id == client.id
+    end
+
+    test "update unverifies client and deletes associated flows" do
+      account = Fixtures.Accounts.create_account()
+      client = Fixtures.Clients.create_client(account: account, verified_at: DateTime.utc_now())
+      :ok = PubSub.Account.subscribe(client.account_id)
+
+      old_data = %{
+        "id" => client.id,
+        "verified_at" => "2023-10-01T00:00:00Z",
+        "account_id" => client.account_id
+      }
+
+      data = %{"id" => client.id, "verified_at" => nil, "account_id" => client.account_id}
+
+      assert flow = Fixtures.Flows.create_flow(client: client, account: account)
+      assert :ok == on_update(old_data, data)
+      assert_receive {:updated, %Clients.Client{}, %Clients.Client{} = new_client}
+      assert is_nil(new_client.verified_at)
+      assert new_client.id == client.id
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
     end
   end
 
   describe "delete/1" do
-    test "broadcasts disconnect" do
-      client = Fixtures.Clients.create_client()
-      :ok = Clients.Presence.connect(client)
+    test "broadcasts deleted client" do
+      account = Fixtures.Accounts.create_account()
+      client = Fixtures.Clients.create_client(account: account)
+      :ok = PubSub.Account.subscribe(client.account_id)
 
-      old_data = %{"id" => client.id}
+      old_data = %{"id" => client.id, "account_id" => client.account_id}
 
       assert :ok == on_delete(old_data)
-
-      assert_receive "disconnect"
-      refute_receive :updated
+      assert_receive {:deleted, %Clients.Client{} = deleted_client}
+      assert deleted_client.id == client.id
     end
-  end
 
-  describe "connect/1" do
-    test "tracks client presence and subscribes to topics" do
-      client = Fixtures.Clients.create_client()
-      assert :ok == Clients.Presence.connect(client)
+    test "deletes associated flows" do
+      account = Fixtures.Accounts.create_account()
+      client = Fixtures.Clients.create_client(account: account)
 
-      assert Clients.Presence.Account.get(client.account_id, client.id)
-      assert Clients.Presence.Actor.get(client.actor_id, client.id)
+      old_data = %{"id" => client.id, "account_id" => client.account_id}
 
-      PubSub.Account.Clients.broadcast(client.account_id, :test_event)
-
-      assert_receive :test_event
-    end
-  end
-
-  describe "broadcast/2" do
-    test "broadcasts payload to client topic" do
-      client = Fixtures.Clients.create_client()
-      :ok = Clients.Presence.connect(client)
-
-      assert :ok == PubSub.Client.broadcast(client.id, :updated)
-
-      assert_receive :updated
+      assert flow = Fixtures.Flows.create_flow(client: client, account: account)
+      assert :ok == on_delete(old_data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
     end
   end
 end

elixir/apps/domain/test/domain/events/hooks/flows_test.exs

@@ -0,0 +1,49 @@
+defmodule Domain.Events.Hooks.FlowsTest do
+  use Domain.DataCase, async: true
+  import Domain.Events.Hooks.Flows
+  alias Domain.Flows
+
+  describe "insert/1" do
+    test "returns :ok" do
+      assert :ok == on_insert(%{})
+    end
+  end
+
+  describe "update/2" do
+    test "returns :ok" do
+      assert :ok == on_update(%{}, %{})
+    end
+  end
+
+  describe "delete/1" do
+    test "delete broadcasts deleted flow" do
+      :ok = Domain.PubSub.Account.subscribe("00000000-0000-0000-0000-000000000000")
+
+      old_data = %{
+        "id" => "00000000-0000-0000-0000-000000000001",
+        "account_id" => "00000000-0000-0000-0000-000000000000",
+        "client_id" => "00000000-0000-0000-0000-000000000002",
+        "gateway_id" => "00000000-0000-0000-0000-000000000003",
+        "resource_id" => "00000000-0000-0000-0000-000000000004",
+        "token_id" => "00000000-0000-0000-0000-000000000005",
+        "actor_group_membership_id" => "00000000-0000-0000-0000-000000000006",
+        "policy_id" => "00000000-0000-0000-0000-000000000007",
+        "inserted_at" => "2023-01-01T00:00:00Z"
+      }
+
+      assert :ok == on_delete(old_data)
+
+      assert_receive {:deleted, %Flows.Flow{} = flow}
+
+      assert flow.id == "00000000-0000-0000-0000-000000000001"
+      assert flow.account_id == "00000000-0000-0000-0000-000000000000"
+      assert flow.client_id == "00000000-0000-0000-0000-000000000002"
+      assert flow.gateway_id == "00000000-0000-0000-0000-000000000003"
+      assert flow.resource_id == "00000000-0000-0000-0000-000000000004"
+      assert flow.token_id == "00000000-0000-0000-0000-000000000005"
+      assert flow.actor_group_membership_id == "00000000-0000-0000-0000-000000000006"
+      assert flow.policy_id == "00000000-0000-0000-0000-000000000007"
+      assert flow.inserted_at == ~U[2023-01-01 00:00:00.000000Z]
+    end
+  end
+end

elixir/apps/domain/test/domain/events/hooks/gateway_groups_test.exs

@@ -1,26 +1,50 @@
 defmodule Domain.Events.Hooks.GatewayGroupsTest do
   use ExUnit.Case, async: true
   import Domain.Events.Hooks.GatewayGroups
-
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
+  alias Domain.Gateways
 
   describe "insert/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_insert(data)
+    test "returns :ok" do
+      assert :ok == on_insert(%{})
     end
   end
 
   describe "update/2" do
-    test "returns :ok", %{old_data: old_data, data: data} do
+    test "returns :ok for soft-deleted gateway group" do
+      # Deleting a gateway group will delete the associated gateways which
+      # handles all side effects we need to handle, including removing any
+      # resources from the client's resource list.
+      assert :ok = on_delete(%{})
+    end
+
+    test "broadcasts updated gateway group" do
+      account_id = "00000000-0000-0000-0000-000000000000"
+
+      :ok = Domain.PubSub.Account.subscribe(account_id)
+
+      old_data = %{
+        "id" => "00000000-0000-0000-0000-000000000001",
+        "account_id" => account_id,
+        "name" => "Old Gateway Group",
+        "deleted_at" => nil
+      }
+
+      data = Map.put(old_data, "name", "Updated Gateway Group")
+
       assert :ok == on_update(old_data, data)
+
+      assert_receive {:updated, %Gateways.Group{} = old_group, %Gateways.Group{} = new_group}
+      assert old_group.id == old_data["id"]
+      assert new_group.name == data["name"]
     end
   end
 
   describe "delete/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_delete(data)
+    test "returns :ok" do
+      # Deleting a gateway group will delete the associated gateways which
+      # handles all side effects we need to handle, including removing any
+      # resources from the client's resource list.
+      assert :ok = on_delete(%{})
     end
   end
 end

elixir/apps/domain/test/domain/events/hooks/gateways_test.exs

@@ -2,52 +2,74 @@ defmodule Domain.Events.Hooks.GatewaysTest do
   use Domain.DataCase, async: true
   import Domain.Events.Hooks.Gateways
 
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
   describe "insert/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_insert(data)
+    test "returns :ok" do
+      assert :ok == on_insert(%{})
     end
   end
 
   describe "update/2" do
-    test "soft-delete broadcasts disconnect" do
-      gateway = Fixtures.Gateways.create_gateway()
+    test "soft-delete broadcasts deleted gateway" do
+      account = Fixtures.Accounts.create_account()
+      gateway = Fixtures.Gateways.create_gateway(account: account)
 
-      old_data = %{"id" => gateway.id, "deleted_at" => nil}
-      data = %{"id" => gateway.id, "deleted_at" => "2023-10-01T00:00:00Z"}
+      :ok = Domain.PubSub.Account.subscribe(account.id)
 
-      :ok = Domain.Gateways.Presence.connect(gateway)
-      :ok = on_update(old_data, data)
+      old_data = %{"id" => gateway.id, "deleted_at" => nil, "account_id" => account.id}
+      data = Map.put(old_data, "deleted_at", "2023-01-01T00:00:00Z")
 
-      assert_receive "disconnect"
+      assert :ok = on_update(old_data, data)
+
+      assert_receive {:deleted, %Domain.Gateways.Gateway{} = deleted_gateway}
+      assert deleted_gateway.id == gateway.id
     end
 
-    test "regular update does not broadcast disconnect" do
-      gateway = Fixtures.Gateways.create_gateway()
+    test "soft-delete deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      gateway = Fixtures.Gateways.create_gateway(account: account)
 
-      old_data = %{"id" => gateway.id}
-      data = %{"id" => gateway.id, "name" => "New Gateway Name"}
+      old_data = %{"id" => gateway.id, "deleted_at" => nil, "account_id" => account.id}
+      data = Map.put(old_data, "deleted_at", "2023-01-01T00:00:00Z")
 
-      :ok = Domain.Gateways.Presence.connect(gateway)
-      :ok = on_update(old_data, data)
+      assert flow = Fixtures.Flows.create_flow(gateway: gateway, account: account)
+      assert :ok = on_update(old_data, data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
+    end
 
-      refute_receive "disconnect"
+    test "update returns :ok" do
+      assert :ok = on_update(%{}, %{})
     end
   end
 
   describe "delete/1" do
-    test "delete broadcasts disconnect" do
-      gateway = Fixtures.Gateways.create_gateway()
+    test "delete broadcasts deleted gateway" do
+      account = Fixtures.Accounts.create_account()
+      gateway = Fixtures.Gateways.create_gateway(account: account)
 
-      old_data = %{"id" => gateway.id}
+      :ok = Domain.PubSub.Account.subscribe(account.id)
 
-      :ok = Domain.Gateways.Presence.connect(gateway)
-      :ok = on_delete(old_data)
+      old_data = %{
+        "id" => gateway.id,
+        "account_id" => account.id,
+        "name" => "Test Gateway",
+        "deleted_at" => nil
+      }
 
-      assert_receive "disconnect"
+      assert :ok = on_delete(old_data)
+
+      assert_receive {:deleted, %Domain.Gateways.Gateway{} = deleted_gateway}
+      assert deleted_gateway.id == gateway.id
+    end
+
+    test "deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      gateway = Fixtures.Gateways.create_gateway(account: account)
+
+      old_data = %{"id" => gateway.id, "account_id" => account.id, "deleted_at" => nil}
+
+      assert flow = Fixtures.Flows.create_flow(gateway: gateway, account: account)
+      assert :ok = on_delete(old_data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
     end
   end
 end

elixir/apps/domain/test/domain/events/hooks/policies_test.exs

@@ -1,277 +1,280 @@
 defmodule Domain.Events.Hooks.PoliciesTest do
   use Domain.DataCase, async: true
   import Domain.Events.Hooks.Policies
-  alias Domain.PubSub
+  alias Domain.{Policies, PubSub}
 
   describe "insert/1" do
-    test "broadcasts :create_policy and :allow_access" do
-      policy_id = "policy-123"
-      account_id = "account-456"
-      actor_group_id = "group-456"
-      resource_id = "resource-789"
+    test "broadcasts created policy" do
+      account = Fixtures.Accounts.create_account()
+      policy = Fixtures.Policies.create_policy(account: account)
+      :ok = PubSub.Account.subscribe(account.id)
 
       data = %{
-        "id" => policy_id,
-        "account_id" => account_id,
-        "actor_group_id" => actor_group_id,
-        "resource_id" => resource_id
+        "id" => policy.id,
+        "account_id" => account.id,
+        "actor_group_id" => policy.actor_group_id,
+        "resource_id" => policy.resource_id,
+        "disabled_at" => nil,
+        "deleted_at" => nil
       }
 
-      :ok = PubSub.Policy.subscribe(policy_id)
-      :ok = PubSub.Account.Policies.subscribe(account_id)
-      :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id)
-
       assert :ok == on_insert(data)
-      assert_receive {:create_policy, ^policy_id}
-      assert_receive {:create_policy, ^policy_id}
-      assert_receive {:allow_access, ^policy_id, ^actor_group_id, ^resource_id}
+      assert_receive {:created, %Policies.Policy{} = policy}
+
+      assert policy.id == data["id"]
+      assert policy.account_id == data["account_id"]
+      assert policy.actor_group_id == data["actor_group_id"]
+      assert policy.resource_id == data["resource_id"]
     end
   end
 
   describe "update/2" do
-    test "enable: broadcasts :enable_policy and :allow_access" do
-      policy_id = "policy-123"
-      account_id = "account-456"
-      actor_group_id = "group-456"
-      resource_id = "resource-789"
+    test "disable policy broadcasts deleted policy and deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      resource = Fixtures.Resources.create_resource(account: account)
+      policy = Fixtures.Policies.create_policy(account: account, resource: resource)
+      :ok = PubSub.Account.subscribe(account.id)
 
       old_data = %{
-        "id" => policy_id,
-        "account_id" => account_id,
-        "actor_group_id" => actor_group_id,
-        "resource_id" => resource_id,
-        "disabled_at" => "2023-10-01T00:00:00Z"
-      }
-
-      data = Map.put(old_data, "disabled_at", nil)
-
-      :ok = Domain.PubSub.Policy.subscribe(policy_id)
-      :ok = Domain.PubSub.Account.Policies.subscribe(account_id)
-      :ok = Domain.PubSub.ActorGroup.Policies.subscribe(actor_group_id)
-
-      assert :ok == on_update(old_data, data)
-      assert_receive {:enable_policy, ^policy_id}
-      assert_receive {:enable_policy, ^policy_id}
-      assert_receive {:allow_access, ^policy_id, ^actor_group_id, ^resource_id}
-    end
-
-    test "disable: broadcasts :disable_policy and :reject_access" do
-      flow = Fixtures.Flows.create_flow()
-      flow_id = flow.id
-      client_id = flow.client_id
-      policy_id = flow.policy_id
-      account_id = flow.account_id
-      actor_group_id = "group-456"
-      resource_id = flow.resource_id
-
-      old_data = %{
-        "id" => policy_id,
-        "account_id" => account_id,
-        "actor_group_id" => actor_group_id,
-        "resource_id" => resource_id,
-        "disabled_at" => nil
+        "id" => policy.id,
+        "account_id" => account.id,
+        "actor_group_id" => policy.actor_group_id,
+        "resource_id" => policy.resource_id,
+        "disabled_at" => nil,
+        "deleted_at" => nil
       }
 
       data = Map.put(old_data, "disabled_at", "2023-10-01T00:00:00Z")
 
-      :ok = PubSub.Flow.subscribe(flow_id)
-      :ok = PubSub.Policy.subscribe(policy_id)
-      :ok = PubSub.Account.Policies.subscribe(account_id)
-      :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id)
+      # Create a flow that should be deleted
+      flow = Fixtures.Flows.create_flow(policy: policy, resource: resource, account: account)
 
       assert :ok == on_update(old_data, data)
-      assert_receive {:disable_policy, ^policy_id}
-      assert_receive {:disable_policy, ^policy_id}
-      assert_receive {:reject_access, ^policy_id, ^actor_group_id, ^resource_id}
+      assert_receive {:deleted, %Policies.Policy{} = broadcasted_policy}
 
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
+      assert broadcasted_policy.id == data["id"]
+      assert broadcasted_policy.account_id == data["account_id"]
 
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
+      # Verify flow was deleted
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
     end
 
-    test "soft-delete: broadcasts :delete_policy and :reject_access" do
-      flow = Fixtures.Flows.create_flow()
-      flow_id = flow.id
-      client_id = flow.client_id
-      policy_id = flow.policy_id
-      account_id = flow.account_id
-      actor_group_id = "group-456"
-      resource_id = flow.resource_id
+    test "enable policy broadcasts created policy" do
+      account = Fixtures.Accounts.create_account()
+      policy = Fixtures.Policies.create_policy(account: account)
+      :ok = PubSub.Account.subscribe(account.id)
 
       old_data = %{
-        "id" => policy_id,
-        "account_id" => account_id,
-        "actor_group_id" => actor_group_id,
-        "resource_id" => resource_id,
+        "id" => policy.id,
+        "account_id" => account.id,
+        "actor_group_id" => policy.actor_group_id,
+        "resource_id" => policy.resource_id,
+        "disabled_at" => "2023-09-01T00:00:00Z",
+        "deleted_at" => nil
+      }
+
+      data = Map.put(old_data, "disabled_at", nil)
+
+      assert :ok == on_update(old_data, data)
+      assert_receive {:created, %Policies.Policy{} = policy}
+
+      assert policy.id == data["id"]
+      assert policy.account_id == data["account_id"]
+      assert policy.actor_group_id == data["actor_group_id"]
+      assert policy.resource_id == data["resource_id"]
+    end
+
+    test "soft-delete broadcasts deleted policy" do
+      account = Fixtures.Accounts.create_account()
+      policy = Fixtures.Policies.create_policy(account: account)
+      :ok = PubSub.Account.subscribe(account.id)
+
+      old_data = %{
+        "id" => policy.id,
+        "account_id" => account.id,
+        "actor_group_id" => policy.actor_group_id,
+        "resource_id" => policy.resource_id,
+        "disabled_at" => nil,
         "deleted_at" => nil
       }
 
       data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z")
 
-      :ok = PubSub.Flow.subscribe(flow_id)
-      :ok = PubSub.Policy.subscribe(policy_id)
-      :ok = PubSub.Account.Policies.subscribe(account_id)
-      :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id)
-
       assert :ok == on_update(old_data, data)
-      assert_receive {:delete_policy, ^policy_id}
-      assert_receive {:delete_policy, ^policy_id}
-      assert_receive {:reject_access, ^policy_id, ^actor_group_id, ^resource_id}
+      assert_receive {:deleted, %Policies.Policy{} = policy}
 
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
+      assert policy.id == old_data["id"]
+      assert policy.account_id == old_data["account_id"]
+      assert policy.actor_group_id == old_data["actor_group_id"]
+      assert policy.resource_id == old_data["resource_id"]
     end
 
-    test "breaking update: broadcasts :delete_policy, :reject_access, :create_policy, :allow_access" do
-      flow = Fixtures.Flows.create_flow()
-      flow_id = flow.id
-      client_id = flow.client_id
-      policy_id = flow.policy_id
-      account_id = flow.account_id
-      actor_group_id = "group-456"
-      resource_id = flow.resource_id
+    test "soft-delete deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      resource = Fixtures.Resources.create_resource(account: account)
+
+      policy =
+        Fixtures.Policies.create_policy(
+          account: account,
+          resource: resource
+        )
 
       old_data = %{
-        "id" => policy_id,
-        "account_id" => account_id,
-        "actor_group_id" => actor_group_id,
-        "resource_id" => resource_id,
-        "conditions" => []
+        "id" => policy.id,
+        "account_id" => account.id,
+        "actor_group_id" => policy.actor_group_id,
+        "resource_id" => resource.id,
+        "deleted_at" => nil
       }
 
-      data = Map.put(old_data, "resource_id", "new-resource-123")
+      data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z")
 
-      :ok = PubSub.Flow.subscribe(flow_id)
-      :ok = PubSub.Policy.subscribe(policy_id)
-      :ok = PubSub.Account.Policies.subscribe(account_id)
-      :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id)
+      assert flow =
+               Fixtures.Flows.create_flow(
+                 policy: policy,
+                 resource: resource,
+                 account: account
+               )
 
-      assert :ok == on_update(old_data, data)
-
-      # TODO: WAL
-      # Remove this when side effects are directly broadcasted
-      Process.sleep(100)
-
-      assert_receive {:delete_policy, ^policy_id}
-      assert_receive {:delete_policy, ^policy_id}
-      assert_receive {:reject_access, ^policy_id, ^actor_group_id, ^resource_id}
-
-      assert_receive {:create_policy, ^policy_id}
-      assert_receive {:create_policy, ^policy_id}
-      assert_receive {:allow_access, ^policy_id, ^actor_group_id, "new-resource-123"}
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
+      assert :ok = on_update(old_data, data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
     end
 
-    test "breaking update: disabled policy has no side-effects" do
-      flow = Fixtures.Flows.create_flow()
-      flow_id = flow.id
-      client_id = flow.client_id
-      policy_id = flow.policy_id
-      account_id = flow.account_id
-      actor_group_id = "group-456"
-      resource_id = flow.resource_id
+    test "non-breaking update broadcasts updated policy" do
+      account = Fixtures.Accounts.create_account()
+      policy = Fixtures.Policies.create_policy(account: account)
+      :ok = PubSub.Account.subscribe(account.id)
 
       old_data = %{
-        "id" => policy_id,
-        "account_id" => account_id,
-        "actor_group_id" => actor_group_id,
-        "resource_id" => resource_id,
-        "disabled_at" => "2023-10-01T00:00:00Z"
+        "id" => policy.id,
+        "description" => "Old description",
+        "account_id" => account.id,
+        "actor_group_id" => policy.actor_group_id,
+        "resource_id" => policy.resource_id,
+        "disabled_at" => nil,
+        "deleted_at" => nil
       }
 
-      data = Map.put(old_data, "resource_id", "new-resource-123")
-
-      :ok = PubSub.Policy.subscribe(policy_id)
-      :ok = PubSub.Account.Policies.subscribe(account_id)
-      :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id)
+      data = Map.put(old_data, "description", "Updated description")
 
       assert :ok == on_update(old_data, data)
-
-      refute_receive {:delete_policy, ^policy_id}
-      refute_receive {:reject_access, ^policy_id, ^actor_group_id, ^resource_id}
-      refute_receive {:create_policy, ^policy_id}
-      refute_receive {:allow_access, ^policy_id, ^actor_group_id, "new-resource-123"}
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      refute_receive {:expire_flow, ^flow_id, ^client_id, "new-resource-123"}
+      assert_receive {:updated, %Policies.Policy{} = old_policy, %Policies.Policy{} = new_policy}
+      assert old_policy.id == old_data["id"]
+      assert new_policy.description == data["description"]
+      assert new_policy.account_id == old_data["account_id"]
+      assert new_policy.actor_group_id == old_data["actor_group_id"]
+      assert new_policy.resource_id == old_data["resource_id"]
     end
 
-    test "non-breaking-update: broadcasts :update_policy" do
-      policy_id = "policy-123"
-      account_id = "account-456"
-      actor_group_id = "group-456"
-      resource_id = "resource-789"
+    test "breaking update deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      policy = Fixtures.Policies.create_policy(account: account)
 
       old_data = %{
-        "description" => "Old Policy",
-        "id" => policy_id,
-        "account_id" => account_id,
-        "actor_group_id" => actor_group_id,
-        "resource_id" => resource_id,
-        "disabled_at" => "2023-10-01T00:00:00Z"
+        "id" => policy.id,
+        "account_id" => account.id,
+        "actor_group_id" => policy.actor_group_id,
+        "resource_id" => policy.resource_id,
+        "deleted_at" => nil
       }
 
-      data = Map.put(old_data, "resource_id", "new-resource-123")
+      data = Map.put(old_data, "resource_id", "00000000-0000-0000-0000-000000000001")
 
-      :ok = PubSub.Policy.subscribe(policy_id)
-      :ok = PubSub.Account.Policies.subscribe(account_id)
-      :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id)
+      assert flow =
+               Fixtures.Flows.create_flow(
+                 policy: policy,
+                 account: account
+               )
 
-      assert :ok == on_update(old_data, data)
+      assert :ok = on_update(old_data, data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
+    end
 
-      assert_receive {:update_policy, ^policy_id}
-      assert_receive {:update_policy, ^policy_id}
+    test "breaking update on actor_group_id deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      policy = Fixtures.Policies.create_policy(account: account)
+
+      old_data = %{
+        "id" => policy.id,
+        "account_id" => account.id,
+        "actor_group_id" => policy.actor_group_id,
+        "resource_id" => policy.resource_id,
+        "deleted_at" => nil
+      }
+
+      data = Map.put(old_data, "actor_group_id", "00000000-0000-0000-0000-000000000001")
+
+      assert flow = Fixtures.Flows.create_flow(policy: policy, account: account)
+
+      assert :ok = on_update(old_data, data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
+    end
+
+    test "breaking update on conditions deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      policy = Fixtures.Policies.create_policy(account: account)
+
+      old_data = %{
+        "id" => policy.id,
+        "account_id" => account.id,
+        "actor_group_id" => policy.actor_group_id,
+        "resource_id" => policy.resource_id,
+        "conditions" => [
+          %{"property" => "remote_ip", "operator" => "is_in", "values" => ["10.0.0.1"]}
+        ],
+        "deleted_at" => nil
+      }
+
+      data =
+        Map.put(old_data, "conditions", [
+          %{"property" => "remote_ip", "operator" => "is_in", "values" => ["10.0.0.2"]}
+        ])
+
+      assert flow = Fixtures.Flows.create_flow(policy: policy, account: account)
+
+      assert :ok = on_update(old_data, data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
     end
   end
 
   describe "delete/1" do
-    test "broadcasts :delete_policy and :reject_access" do
-      flow = Fixtures.Flows.create_flow()
-      flow_id = flow.id
-      client_id = flow.client_id
-      policy_id = flow.policy_id
-      account_id = flow.account_id
-      actor_group_id = "group-456"
-      resource_id = flow.resource_id
+    test "broadcasts deleted policy" do
+      account = Fixtures.Accounts.create_account()
+      policy = Fixtures.Policies.create_policy(account: account)
+      :ok = PubSub.Account.subscribe(account.id)
 
       old_data = %{
-        "id" => policy_id,
-        "account_id" => account_id,
-        "actor_group_id" => actor_group_id,
-        "resource_id" => resource_id,
+        "id" => policy.id,
+        "account_id" => account.id,
+        "actor_group_id" => policy.actor_group_id,
+        "resource_id" => policy.resource_id
+      }
+
+      assert :ok == on_delete(old_data)
+      assert_receive {:deleted, %Policies.Policy{} = policy}
+
+      assert policy.id == old_data["id"]
+      assert policy.account_id == old_data["account_id"]
+      assert policy.actor_group_id == old_data["actor_group_id"]
+      assert policy.resource_id == old_data["resource_id"]
+    end
+
+    test "deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      policy = Fixtures.Policies.create_policy(account: account)
+
+      old_data = %{
+        "id" => policy.id,
+        "account_id" => account.id,
+        "actor_group_id" => policy.actor_group_id,
+        "resource_id" => policy.resource_id,
         "deleted_at" => nil
       }
 
-      data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z")
+      assert flow = Fixtures.Flows.create_flow(policy: policy, account: account)
 
-      :ok = PubSub.Flow.subscribe(flow_id)
-      :ok = PubSub.Policy.subscribe(policy_id)
-      :ok = PubSub.Account.Policies.subscribe(account_id)
-      :ok = PubSub.ActorGroup.Policies.subscribe(actor_group_id)
-
-      assert :ok == on_update(old_data, data)
-      assert_receive {:delete_policy, ^policy_id}
-      assert_receive {:delete_policy, ^policy_id}
-      assert_receive {:reject_access, ^policy_id, ^actor_group_id, ^resource_id}
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
+      assert :ok = on_delete(old_data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
     end
   end
 end

elixir/apps/domain/test/domain/events/hooks/relay_groups_test.exs

@@ -1,26 +0,0 @@
-defmodule Domain.Events.Hooks.RelayGroupsTest do
-  use ExUnit.Case, async: true
-  import Domain.Events.Hooks.RelayGroups
-
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
-  describe "insert/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_insert(data)
-    end
-  end
-
-  describe "update/2" do
-    test "returns :ok", %{old_data: old_data, data: data} do
-      assert :ok == on_update(old_data, data)
-    end
-  end
-
-  describe "delete/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_delete(data)
-    end
-  end
-end

elixir/apps/domain/test/domain/events/hooks/relays_test.exs

@@ -1,26 +0,0 @@
-defmodule Domain.Events.Hooks.RelaysTest do
-  use ExUnit.Case, async: true
-  import Domain.Events.Hooks.Relays
-
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
-  describe "insert/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_insert(data)
-    end
-  end
-
-  describe "update/2" do
-    test "returns :ok", %{old_data: old_data, data: data} do
-      assert :ok == on_update(old_data, data)
-    end
-  end
-
-  describe "delete/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_delete(data)
-    end
-  end
-end

elixir/apps/domain/test/domain/events/hooks/resource_connections_test.exs

@@ -2,35 +2,57 @@ defmodule Domain.Events.Hooks.ResourceConnectionsTest do
   use Domain.DataCase, async: true
   import Domain.Events.Hooks.ResourceConnections
 
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
   describe "insert/1" do
-    test "returns :ok", %{data: data} do
+    test "broadcasts created resource connection" do
+      account = Fixtures.Accounts.create_account()
+      resource = Fixtures.Resources.create_resource(account: account)
+      gateway_group = Fixtures.Gateways.create_group(account: account)
+
+      :ok = Domain.PubSub.Account.subscribe(account.id)
+
+      data = %{
+        "account_id" => account.id,
+        "resource_id" => resource.id,
+        "gateway_group_id" => gateway_group.id,
+        "deleted_at" => nil
+      }
+
       assert :ok == on_insert(data)
+
+      assert_receive {:created, %Domain.Resources.Connection{} = connection}
+      assert connection.account_id == data["account_id"]
+      assert connection.resource_id == data["resource_id"]
+      assert connection.gateway_group_id == data["gateway_group_id"]
     end
   end
 
   describe "update/2" do
-    test "returns :ok", %{old_data: old_data, data: data} do
-      assert :ok == on_update(old_data, data)
+    test "returns :ok" do
+      assert :ok = on_update(%{}, %{})
     end
   end
 
   describe "delete/1" do
-    test "returns :ok" do
-      flow = Fixtures.Flows.create_flow()
-      :ok = Domain.PubSub.Flow.subscribe(flow.id)
+    test "broadcasts deleted connection" do
+      account = Fixtures.Accounts.create_account()
+      resource = Fixtures.Resources.create_resource(account: account)
+      gateway_group = Fixtures.Gateways.create_group(account: account)
 
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
+      :ok = Domain.PubSub.Account.subscribe(account.id)
 
-      assert :ok ==
-               on_delete(%{"account_id" => flow.account_id, "resource_id" => flow.resource_id})
+      old_data = %{
+        "account_id" => account.id,
+        "resource_id" => resource.id,
+        "gateway_group_id" => gateway_group.id,
+        "deleted_at" => nil
+      }
 
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
+      assert :ok == on_delete(old_data)
+
+      assert_receive {:deleted, %Domain.Resources.Connection{} = deleted_connection}
+      assert deleted_connection.account_id == old_data["account_id"]
+      assert deleted_connection.resource_id == old_data["resource_id"]
+      assert deleted_connection.gateway_group_id == old_data["gateway_group_id"]
     end
   end
 end

elixir/apps/domain/test/domain/events/hooks/resources_test.exs

@@ -4,207 +4,202 @@ defmodule Domain.Events.Hooks.ResourcesTest do
   alias Domain.PubSub
 
   describe "insert/1" do
-    test "broadcasts :create_resource to subscribed" do
-      resource_id = "test_resource"
-      account_id = "test_account"
-      :ok = PubSub.Resource.subscribe(resource_id)
-      :ok = PubSub.Account.Resources.subscribe(account_id)
+    test "broadcasts created resource" do
+      account = Fixtures.Accounts.create_account()
+      filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}]
+      resource = Fixtures.Resources.create_resource(account: account, filters: filters)
 
-      data = %{"id" => resource_id, "account_id" => account_id}
+      :ok = PubSub.Account.subscribe(account.id)
+
+      data = %{
+        "id" => resource.id,
+        "account_id" => account.id,
+        "address_description" => resource.address_description,
+        "type" => resource.type,
+        "address" => resource.address,
+        "filters" => filters,
+        "ip_stack" => resource.ip_stack,
+        "deleted_at" => nil
+      }
 
       assert :ok == on_insert(data)
-
-      # we expect two - once for the resource subscription, and once for the account
-      assert_receive {:create_resource, ^resource_id}
-      assert_receive {:create_resource, ^resource_id}
-
-      :ok = PubSub.Resource.unsubscribe(resource_id)
-
-      assert :ok = on_insert(data)
-      assert_receive {:create_resource, ^resource_id}
-      refute_receive {:create_resource, ^resource_id}
+      assert_receive {:created, %Domain.Resources.Resource{} = created_resource}
+      assert created_resource.id == resource.id
+      assert created_resource.account_id == resource.account_id
+      assert created_resource.type == resource.type
+      assert created_resource.address == resource.address
+      assert created_resource.filters == resource.filters
+      assert created_resource.ip_stack == resource.ip_stack
+      assert created_resource.address_description == resource.address_description
     end
   end
 
   describe "update/2" do
-    setup do
-      flow = Fixtures.Flows.create_flow()
+    test "soft-delete broadcasts deleted resource" do
+      account = Fixtures.Accounts.create_account()
+      filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}]
+      resource = Fixtures.Resources.create_resource(account: account, filters: filters)
+
+      :ok = PubSub.Account.subscribe(account.id)
 
       old_data = %{
-        "type" => "dns",
-        "address" => "1.2.3.4",
-        "filters" => [],
-        "ip_stack" => "dual",
-        "id" => flow.resource_id,
-        "account_id" => flow.account_id
+        "id" => resource.id,
+        "account_id" => account.id,
+        "address_description" => resource.address_description,
+        "type" => resource.type,
+        "address" => resource.address,
+        "filters" => filters,
+        "ip_stack" => resource.ip_stack,
+        "deleted_at" => nil
       }
 
-      %{flow: flow, old_data: old_data}
-    end
-
-    test "broadcasts :delete_resource to subscribed for soft-deletions" do
-      resource_id = "test_resource"
-      account_id = "test_account"
-      :ok = PubSub.Resource.subscribe(resource_id)
-      :ok = PubSub.Account.Resources.subscribe(account_id)
-
-      old_data = %{"id" => resource_id, "account_id" => account_id, "deleted_at" => nil}
-
-      data = %{
-        "id" => resource_id,
-        "account_id" => account_id,
-        "deleted_at" => DateTime.utc_now()
-      }
+      data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z")
 
       assert :ok == on_update(old_data, data)
-      assert_receive {:delete_resource, ^resource_id}
-      assert_receive {:delete_resource, ^resource_id}
 
-      :ok = PubSub.Resource.unsubscribe(resource_id)
+      assert_receive {:deleted, %Domain.Resources.Resource{} = deleted_resource}
 
-      assert :ok = on_update(old_data, data)
-      assert_receive {:delete_resource, ^resource_id}
-      refute_receive {:delete_resource, ^resource_id}
+      assert deleted_resource.id == resource.id
+      assert deleted_resource.account_id == resource.account_id
+      assert deleted_resource.type == resource.type
+      assert deleted_resource.address == resource.address
+      assert deleted_resource.filters == resource.filters
+      assert deleted_resource.ip_stack == resource.ip_stack
+      assert deleted_resource.address_description == resource.address_description
     end
 
-    test "expires flows when resource type changes", %{flow: flow, old_data: old_data} do
-      :ok = PubSub.Flow.subscribe(flow.id)
-      :ok = PubSub.Resource.subscribe(flow.resource_id)
-      :ok = PubSub.Account.Resources.subscribe(flow.account_id)
+    test "soft-delete deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}]
+      resource = Fixtures.Resources.create_resource(account: account, filters: filters)
+
+      old_data = %{
+        "id" => resource.id,
+        "account_id" => account.id,
+        "address_description" => resource.address_description,
+        "type" => resource.type,
+        "address" => resource.address,
+        "filters" => filters,
+        "ip_stack" => resource.ip_stack,
+        "deleted_at" => nil
+      }
+
+      data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z")
+
+      assert flow = Fixtures.Flows.create_flow(resource: resource, account: account)
+      assert :ok = on_update(old_data, data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
+    end
+
+    test "regular update broadcasts updated resource" do
+      account = Fixtures.Accounts.create_account()
+      filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}]
+      resource = Fixtures.Resources.create_resource(account: account, filters: filters)
+
+      :ok = PubSub.Account.subscribe(account.id)
+
+      old_data = %{
+        "id" => resource.id,
+        "account_id" => account.id,
+        "address_description" => resource.address_description,
+        "type" => resource.type,
+        "address" => resource.address,
+        "filters" => filters,
+        "ip_stack" => resource.ip_stack,
+        "deleted_at" => nil
+      }
+
+      data = Map.put(old_data, "address", "new-address.example.com")
+
+      assert :ok == on_update(old_data, data)
+
+      assert_receive {:updated, %Domain.Resources.Resource{},
+                      %Domain.Resources.Resource{} = updated_resource}
+
+      assert updated_resource.id == resource.id
+      assert updated_resource.account_id == resource.account_id
+      assert updated_resource.type == resource.type
+      assert updated_resource.address == "new-address.example.com"
+      assert updated_resource.filters == resource.filters
+      assert updated_resource.ip_stack == resource.ip_stack
+      assert updated_resource.address_description == resource.address_description
+    end
+
+    test "breaking update deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}]
+      resource = Fixtures.Resources.create_resource(account: account, filters: filters)
+
+      old_data = %{
+        "id" => resource.id,
+        "account_id" => account.id,
+        "address_description" => resource.address_description,
+        "type" => "dns",
+        "address" => resource.address,
+        "filters" => filters,
+        "ip_stack" => resource.ip_stack,
+        "deleted_at" => nil
+      }
 
       data = Map.put(old_data, "type", "cidr")
 
-      assert :ok == on_update(old_data, data)
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-      assert_receive {:delete_resource, ^resource_id}
-      assert_receive {:delete_resource, ^resource_id}
-      assert_receive {:create_resource, ^resource_id}
-      assert_receive {:create_resource, ^resource_id}
-    end
-
-    test "expires flows when resource address changes", %{flow: flow, old_data: old_data} do
-      :ok = PubSub.Flow.subscribe(flow.id)
-      :ok = PubSub.Resource.subscribe(flow.resource_id)
-      :ok = PubSub.Account.Resources.subscribe(flow.account_id)
-
-      data = Map.put(old_data, "address", "4.3.2.1")
-
-      assert :ok == on_update(old_data, data)
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-      assert_receive {:delete_resource, ^resource_id}
-      assert_receive {:delete_resource, ^resource_id}
-      assert_receive {:create_resource, ^resource_id}
-      assert_receive {:create_resource, ^resource_id}
-    end
-
-    test "expires flows when resource filters change", %{flow: flow, old_data: old_data} do
-      :ok = PubSub.Flow.subscribe(flow.id)
-      :ok = PubSub.Resource.subscribe(flow.resource_id)
-      :ok = PubSub.Account.Resources.subscribe(flow.account_id)
-
-      data = Map.put(old_data, "filters", ["new_filter"])
-
-      assert :ok == on_update(old_data, data)
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-      assert_receive {:delete_resource, ^resource_id}
-      assert_receive {:delete_resource, ^resource_id}
-      assert_receive {:create_resource, ^resource_id}
-      assert_receive {:create_resource, ^resource_id}
-    end
-
-    test "expires flows when resource ip_stack changes", %{flow: flow, old_data: old_data} do
-      :ok = PubSub.Flow.subscribe(flow.id)
-      :ok = PubSub.Resource.subscribe(flow.resource_id)
-      :ok = PubSub.Account.Resources.subscribe(flow.account_id)
-
-      data = Map.put(old_data, "ip_stack", "ipv4_only")
-
-      assert :ok == on_update(old_data, data)
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-      assert_receive {:delete_resource, ^resource_id}
-      assert_receive {:delete_resource, ^resource_id}
-      assert_receive {:create_resource, ^resource_id}
-      assert_receive {:create_resource, ^resource_id}
-    end
-
-    test "broadcasts update for non-addressability change", %{flow: flow, old_data: old_data} do
-      :ok = PubSub.Resource.subscribe(flow.resource_id)
-      :ok = PubSub.Account.Resources.subscribe(flow.account_id)
-
-      data = Map.put(old_data, "name", "New Name")
-
-      assert :ok == on_update(old_data, data)
-
-      # TODO: WAL
-      # Remove this after direct broadcast
-      Process.sleep(100)
-
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-
-      refute_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-      assert_receive {:update_resource, ^resource_id}
-      assert_receive {:update_resource, ^resource_id}
-      refute_receive {:delete_resource, ^resource_id}
-      refute_receive {:create_resource, ^resource_id}
+      assert flow = Fixtures.Flows.create_flow(resource: resource, account: account)
+      assert :ok = on_update(old_data, data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
     end
   end
 
   describe "delete/1" do
-    test "broadcasts :delete_resource to subscribed" do
-      resource_id = "test_resource"
-      account_id = "test_account"
-      :ok = PubSub.Resource.subscribe(resource_id)
-      :ok = PubSub.Account.Resources.subscribe(account_id)
+    test "broadcasts deleted resource" do
+      account = Fixtures.Accounts.create_account()
+      filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}]
+      resource = Fixtures.Resources.create_resource(account: account, filters: filters)
 
-      old_data = %{"id" => resource_id, "account_id" => account_id}
+      :ok = PubSub.Account.subscribe(account.id)
+
+      old_data = %{
+        "id" => resource.id,
+        "account_id" => account.id,
+        "address_description" => resource.address_description,
+        "type" => resource.type,
+        "address" => resource.address,
+        "filters" => filters,
+        "ip_stack" => resource.ip_stack,
+        "deleted_at" => nil
+      }
 
       assert :ok == on_delete(old_data)
-      assert_receive {:delete_resource, ^resource_id}
-      assert_receive {:delete_resource, ^resource_id}
 
-      :ok = PubSub.Resource.unsubscribe(resource_id)
+      assert_receive {:deleted, %Domain.Resources.Resource{} = deleted_resource}
 
+      assert deleted_resource.id == resource.id
+      assert deleted_resource.account_id == resource.account_id
+      assert deleted_resource.type == resource.type
+      assert deleted_resource.address == resource.address
+      assert deleted_resource.filters == resource.filters
+      assert deleted_resource.ip_stack == resource.ip_stack
+      assert deleted_resource.address_description == resource.address_description
+    end
+
+    test "deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      filters = [%{"protocol" => "tcp", "ports" => ["80", "443"]}]
+      resource = Fixtures.Resources.create_resource(account: account, filters: filters)
+
+      old_data = %{
+        "id" => resource.id,
+        "account_id" => account.id,
+        "address_description" => resource.address_description,
+        "type" => resource.type,
+        "address" => resource.address,
+        "filters" => filters,
+        "ip_stack" => resource.ip_stack,
+        "deleted_at" => nil
+      }
+
+      assert flow = Fixtures.Flows.create_flow(resource: resource, account: account)
       assert :ok = on_delete(old_data)
-      assert_receive {:delete_resource, ^resource_id}
-      refute_receive {:delete_resource, ^resource_id}
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
     end
   end
 end

elixir/apps/domain/test/domain/events/hooks/tokens_test.exs

@@ -1,62 +1,98 @@
 defmodule Domain.Events.Hooks.TokensTest do
-  use ExUnit.Case, async: true
+  use Domain.DataCase, async: true
   import Domain.Events.Hooks.Tokens
 
-  setup do
-    %{old_data: %{}, data: %{}}
-  end
-
   describe "insert/1" do
-    test "returns :ok", %{data: data} do
-      assert :ok == on_insert(data)
+    test "returns :ok" do
+      assert :ok == on_insert(%{})
     end
   end
 
   describe "update/2" do
-    test "does not broadcast for email token updates" do
-      token_id = "token-id-123"
-      topic = "sessions:#{token_id}"
-      old_data = %{"id" => token_id, "type" => "email"}
-
-      :ok = Domain.PubSub.subscribe("sessions:#{token_id}")
-
-      assert :ok = on_update(old_data, %{})
-
-      refute_receive %Phoenix.Socket.Broadcast{
-        topic: ^topic,
-        event: "disconnect"
-      }
+    test "returns :ok for email token updates" do
+      assert :ok = on_update(%{"type" => "email"}, %{"type" => "email"})
     end
 
-    test "broadcasts disconnect for soft-deletions" do
-      token_id = "token-id-123"
-      topic = "sessions:#{token_id}"
-      old_data = %{"id" => token_id, "deleted_at" => nil}
-      data = %{"id" => token_id, "deleted_at" => DateTime.utc_now()}
-      :ok = Domain.PubSub.subscribe("sessions:#{token_id}")
+    test "soft-delete broadcasts deleted token" do
+      account = Fixtures.Accounts.create_account()
+      token = Fixtures.Tokens.create_token(account: account)
+      :ok = Domain.PubSub.Account.subscribe(account.id)
 
-      assert :ok = on_update(old_data, data)
-
-      assert_receive %Phoenix.Socket.Broadcast{
-        topic: ^topic,
-        event: "disconnect"
+      old_data = %{
+        "id" => token.id,
+        "account_id" => account.id,
+        "type" => token.type,
+        "deleted_at" => nil
       }
+
+      data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z")
+
+      assert :ok == on_update(old_data, data)
+      assert_receive {:deleted, %Domain.Tokens.Token{} = deleted_token}
+      assert deleted_token.id == old_data["id"]
+      assert deleted_token.account_id == old_data["account_id"]
+      assert deleted_token.type == old_data["type"]
+    end
+
+    test "soft-delete deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      token = Fixtures.Tokens.create_token(account: account)
+
+      old_data = %{
+        "id" => token.id,
+        "account_id" => account.id,
+        "type" => token.type,
+        "deleted_at" => nil
+      }
+
+      data = Map.put(old_data, "deleted_at", "2023-10-01T00:00:00Z")
+
+      assert flow = Fixtures.Flows.create_flow(account: account, token: token)
+      assert flow.token_id == token.id
+      assert :ok = on_update(old_data, data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
+    end
+
+    test "regular update returns :ok" do
+      assert :ok = on_update(%{}, %{})
     end
   end
 
   describe "delete/1" do
-    test "broadcasts disconnect for deletions" do
-      token_id = "token-id-123"
-      topic = "sessions:#{token_id}"
-      old_data = %{"id" => token_id}
-      :ok = Domain.PubSub.subscribe("sessions:#{token_id}")
+    test "broadcasts deleted token" do
+      account = Fixtures.Accounts.create_account()
+      token = Fixtures.Tokens.create_token(account: account)
+      :ok = Domain.PubSub.Account.subscribe(account.id)
 
-      assert :ok = on_delete(old_data)
-
-      assert_receive %Phoenix.Socket.Broadcast{
-        topic: ^topic,
-        event: "disconnect"
+      old_data = %{
+        "id" => token.id,
+        "account_id" => account.id,
+        "type" => token.type,
+        "deleted_at" => nil
       }
+
+      assert :ok == on_delete(old_data)
+
+      assert_receive {:deleted, %Domain.Tokens.Token{} = deleted_token}
+      assert deleted_token.id == old_data["id"]
+      assert deleted_token.account_id == old_data["account_id"]
+      assert deleted_token.type == old_data["type"]
+    end
+
+    test "deletes flows" do
+      account = Fixtures.Accounts.create_account()
+      token = Fixtures.Tokens.create_token(account: account)
+
+      old_data = %{
+        "id" => token.id,
+        "account_id" => account.id,
+        "type" => token.type,
+        "deleted_at" => nil
+      }
+
+      assert flow = Fixtures.Flows.create_flow(account: account, token: token)
+      assert :ok = on_delete(old_data)
+      refute Repo.get_by(Domain.Flows.Flow, id: flow.id)
     end
   end
 end

elixir/apps/domain/test/domain/events/replication_connection_test.exs

@@ -1,5 +1,5 @@
 defmodule Domain.Events.ReplicationConnectionTest do
-  use ExUnit.Case, async: true
+  use Domain.DataCase, async: true
 
   import ExUnit.CaptureLog
   alias Domain.Events.ReplicationConnection
@@ -15,7 +15,12 @@ defmodule Domain.Events.ReplicationConnectionTest do
   describe "on_write/6 for inserts" do
     test "logs warning for unknown table" do
       table = "unknown_table"
-      data = %{"id" => Ecto.UUID.generate(), "name" => "test"}
+
+      data = %{
+        "account_id" => Ecto.UUID.generate(),
+        "id" => Ecto.UUID.generate(),
+        "name" => "test"
+      }
 
       log_output =
         capture_log(fn ->
@@ -29,7 +34,11 @@ defmodule Domain.Events.ReplicationConnectionTest do
 
     test "handles known tables without errors", %{tables: tables} do
       for table <- tables do
-        data = %{"id" => Ecto.UUID.generate(), "table" => table}
+        data = %{
+          "account_id" => Ecto.UUID.generate(),
+          "id" => Ecto.UUID.generate(),
+          "table" => table
+        }
 
         # The actual hook call might fail if the hook modules aren't available,
         # but we can test that our routing logic works
@@ -51,6 +60,7 @@ defmodule Domain.Events.ReplicationConnectionTest do
           capture_log(fn ->
             try do
               ReplicationConnection.on_write(%{}, 0, :insert, table, nil, %{
+                "account_id" => Ecto.UUID.generate(),
                 "id" => Ecto.UUID.generate()
               })
             rescue
@@ -68,8 +78,14 @@ defmodule Domain.Events.ReplicationConnectionTest do
   describe "on_write/6 for updates" do
     test "logs warning for unknown table" do
       table = "unknown_table"
-      old_data = %{"id" => Ecto.UUID.generate(), "name" => "old"}
-      data = %{"id" => old_data["id"], "name" => "new"}
+
+      old_data = %{
+        "account_id" => Ecto.UUID.generate(),
+        "id" => Ecto.UUID.generate(),
+        "name" => "old"
+      }
+
+      data = %{"account_id" => Ecto.UUID.generate(), "id" => old_data["id"], "name" => "new"}
 
       log_output =
         capture_log(fn ->
@@ -82,8 +98,13 @@ defmodule Domain.Events.ReplicationConnectionTest do
     end
 
     test "handles known tables", %{tables: tables} do
-      old_data = %{"id" => Ecto.UUID.generate(), "name" => "old name"}
-      data = %{"id" => old_data["id"], "name" => "new name"}
+      old_data = %{
+        "account_id" => Ecto.UUID.generate(),
+        "id" => Ecto.UUID.generate(),
+        "name" => "old name"
+      }
+
+      data = %{"account_id" => Ecto.UUID.generate(), "id" => old_data["id"], "name" => "new name"}
 
       for table <- tables do
         try do
@@ -101,7 +122,12 @@ defmodule Domain.Events.ReplicationConnectionTest do
   describe "on_write/6 for deletes" do
     test "logs warning for unknown table" do
       table = "unknown_table"
-      old_data = %{"id" => Ecto.UUID.generate(), "name" => "deleted"}
+
+      old_data = %{
+        "account_id" => Ecto.UUID.generate(),
+        "id" => Ecto.UUID.generate(),
+        "name" => "deleted"
+      }
 
       log_output =
         capture_log(fn ->
@@ -114,7 +140,11 @@ defmodule Domain.Events.ReplicationConnectionTest do
     end
 
     test "handles known tables", %{tables: tables} do
-      old_data = %{"id" => Ecto.UUID.generate(), "name" => "deleted item"}
+      old_data = %{
+        "account_id" => Ecto.UUID.generate(),
+        "id" => Ecto.UUID.generate(),
+        "name" => "deleted item"
+      }
 
       for table <- tables do
         try do
@@ -139,7 +169,11 @@ defmodule Domain.Events.ReplicationConnectionTest do
 
       # Insert
       try do
-        result = ReplicationConnection.on_write(state, 1, :insert, table, nil, %{"id" => "123"})
+        result =
+          ReplicationConnection.on_write(state, 1, :insert, table, nil, %{
+            "id" => "00000000-0000-0000-0000-000000000001"
+          })
+
         assert result == state
       rescue
         FunctionClauseError -> :ok
@@ -148,10 +182,17 @@ defmodule Domain.Events.ReplicationConnectionTest do
       # Update
       try do
         result =
-          ReplicationConnection.on_write(state, 2, :update, table, %{"id" => "123"}, %{
-            "id" => "123",
-            "updated" => true
-          })
+          ReplicationConnection.on_write(
+            state,
+            2,
+            :update,
+            table,
+            %{"id" => "00000000-0000-0000-0000-000000000001"},
+            %{
+              "id" => "00000000-0000-0000-0000-000000000001",
+              "updated" => true
+            }
+          )
 
         assert result == state
       rescue
@@ -160,7 +201,16 @@ defmodule Domain.Events.ReplicationConnectionTest do
 
       # Delete
       try do
-        result = ReplicationConnection.on_write(state, 3, :delete, table, %{"id" => "123"}, nil)
+        result =
+          ReplicationConnection.on_write(
+            state,
+            3,
+            :delete,
+            table,
+            %{"id" => "00000000-0000-0000-0000-000000000001"},
+            nil
+          )
+
         assert result == state
       rescue
         FunctionClauseError -> :ok
@@ -223,11 +273,8 @@ defmodule Domain.Events.ReplicationConnectionTest do
       tables_to_hooks = %{
         "accounts" => Domain.Events.Hooks.Accounts,
         "actor_group_memberships" => Domain.Events.Hooks.ActorGroupMemberships,
-        "actor_groups" => Domain.Events.Hooks.ActorGroups,
-        "actors" => Domain.Events.Hooks.Actors,
-        "auth_identities" => Domain.Events.Hooks.AuthIdentities,
-        "auth_providers" => Domain.Events.Hooks.AuthProviders,
         "clients" => Domain.Events.Hooks.Clients,
+        "flows" => Domain.Events.Hooks.Flows,
         "gateway_groups" => Domain.Events.Hooks.GatewayGroups,
         "gateways" => Domain.Events.Hooks.Gateways,
         "policies" => Domain.Events.Hooks.Policies,
@@ -241,11 +288,8 @@ defmodule Domain.Events.ReplicationConnectionTest do
                [
                  "accounts",
                  "actor_group_memberships",
-                 "actor_groups",
-                 "actors",
-                 "auth_identities",
-                 "auth_providers",
                  "clients",
+                 "flows",
                  "gateway_groups",
                  "gateways",
                  "policies",

elixir/apps/domain/test/domain/notifications/jobs/outdated_gateways_test.exs

@@ -1,7 +1,7 @@
 defmodule Domain.Notifications.Jobs.OutdatedGatewaysTest do
   use Domain.DataCase, async: true
   import Domain.Notifications.Jobs.OutdatedGateways
-  alias Domain.{ComponentVersions, Gateways, PubSub}
+  alias Domain.{ComponentVersions, Gateways}
 
   describe "execute/1" do
     setup do
@@ -44,7 +44,6 @@ defmodule Domain.Notifications.Jobs.OutdatedGatewaysTest do
       :ok = Gateways.Presence.Group.subscribe(gateway_group.id)
       {:ok, _} = Gateways.Presence.Group.track(gateway.group_id, gateway.id)
       {:ok, _} = Gateways.Presence.Account.track(gateway.account_id, gateway.id)
-      :ok = PubSub.Gateway.subscribe(gateway.id)
       assert_receive %Phoenix.Socket.Broadcast{topic: "presences:group_gateways:" <> _}
 
       assert execute(%{}) == :ok
@@ -70,7 +69,7 @@ defmodule Domain.Notifications.Jobs.OutdatedGatewaysTest do
       :ok = Gateways.Presence.Group.subscribe(gateway_group.id)
       {:ok, _} = Gateways.Presence.Group.track(gateway.group_id, gateway.id)
       {:ok, _} = Gateways.Presence.Account.track(gateway.account_id, gateway.id)
-      :ok = PubSub.Gateway.subscribe(gateway.id)
+
       assert_receive %Phoenix.Socket.Broadcast{topic: "presences:group_gateways:" <> _}
 
       assert execute(%{}) == :ok

elixir/apps/domain/test/domain/resources_test.exs

@@ -3,7 +3,6 @@ defmodule Domain.ResourcesTest do
   import Domain.Resources
   alias Domain.Resources
   alias Domain.Actors
-  alias Domain.Events
 
   setup do
     account = Fixtures.Accounts.create_account()
@@ -222,222 +221,6 @@ defmodule Domain.ResourcesTest do
     end
   end
 
-  describe "fetch_and_authorize_resource_by_id/3" do
-    test "returns error when resource does not exist", %{subject: subject} do
-      assert fetch_and_authorize_resource_by_id(Ecto.UUID.generate(), subject) ==
-               {:error, :not_found}
-    end
-
-    test "returns error when UUID is invalid", %{subject: subject} do
-      assert fetch_and_authorize_resource_by_id("foo", subject) == {:error, :not_found}
-    end
-
-    test "returns authorized resource for account admin", %{
-      account: account,
-      actor: actor,
-      subject: subject
-    } do
-      resource = Fixtures.Resources.create_resource(account: account)
-      actor_group = Fixtures.Actors.create_group(account: account)
-      Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group)
-
-      assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found}
-
-      policy =
-        Fixtures.Policies.create_policy(
-          account: account,
-          actor_group: actor_group,
-          resource: resource
-        )
-
-      assert {:ok, fetched_resource} = fetch_and_authorize_resource_by_id(resource.id, subject)
-      assert fetched_resource.id == resource.id
-      assert Enum.map(fetched_resource.authorized_by_policies, & &1.id) == [policy.id]
-    end
-
-    test "returns authorized resource for account user", %{
-      account: account
-    } do
-      actor_group = Fixtures.Actors.create_group(account: account)
-      actor = Fixtures.Actors.create_actor(type: :account_user, account: account)
-      Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group)
-
-      identity = Fixtures.Auth.create_identity(account: account, actor: actor)
-      subject = Fixtures.Auth.create_subject(identity: identity)
-
-      resource = Fixtures.Resources.create_resource(account: account)
-
-      assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found}
-
-      policy =
-        Fixtures.Policies.create_policy(
-          account: account,
-          actor_group: actor_group,
-          resource: resource
-        )
-
-      assert {:ok, fetched_resource} = fetch_and_authorize_resource_by_id(resource.id, subject)
-      assert fetched_resource.id == resource.id
-      assert Enum.map(fetched_resource.authorized_by_policies, & &1.id) == [policy.id]
-    end
-
-    test "returns authorized resource using one of multiple policies for account user", %{
-      account: account
-    } do
-      actor = Fixtures.Actors.create_actor(type: :account_user, account: account)
-      identity = Fixtures.Auth.create_identity(account: account, actor: actor)
-      subject = Fixtures.Auth.create_subject(identity: identity)
-      resource = Fixtures.Resources.create_resource(account: account)
-
-      actor_group1 = Fixtures.Actors.create_group(account: account)
-      Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group1)
-
-      policy1 =
-        Fixtures.Policies.create_policy(
-          account: account,
-          actor_group: actor_group1,
-          resource: resource
-        )
-
-      actor_group2 = Fixtures.Actors.create_group(account: account)
-      Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group2)
-
-      policy2 =
-        Fixtures.Policies.create_policy(
-          account: account,
-          actor_group: actor_group2,
-          resource: resource
-        )
-
-      assert {:ok, fetched_resource} = fetch_and_authorize_resource_by_id(resource.id, subject)
-      assert fetched_resource.id == resource.id
-
-      authorized_by_policy_ids = Enum.map(fetched_resource.authorized_by_policies, & &1.id)
-      policy_ids = [policy1.id, policy2.id]
-      assert Enum.sort(authorized_by_policy_ids) == Enum.sort(policy_ids)
-    end
-
-    test "does not return deleted resources", %{account: account, actor: actor, subject: subject} do
-      {:ok, resource} =
-        Fixtures.Resources.create_resource(account: account)
-        |> delete_resource(subject)
-
-      actor_group = Fixtures.Actors.create_group(account: account)
-      Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group)
-
-      Fixtures.Policies.create_policy(
-        account: account,
-        actor_group: actor_group,
-        resource: resource
-      )
-
-      assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found}
-    end
-
-    test "does not authorize using deleted policies", %{
-      account: account,
-      actor: actor,
-      subject: subject
-    } do
-      resource = Fixtures.Resources.create_resource(account: account)
-      actor_group = Fixtures.Actors.create_group(account: account)
-      Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group)
-
-      Fixtures.Policies.create_policy(
-        account: account,
-        actor_group: actor_group,
-        resource: resource
-      )
-      |> Fixtures.Policies.delete_policy()
-
-      assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found}
-    end
-
-    test "does not authorize using deleted group membership", %{
-      account: account,
-      subject: subject
-    } do
-      resource = Fixtures.Resources.create_resource(account: account)
-      actor_group = Fixtures.Actors.create_group(account: account)
-
-      # memberships are not soft deleted
-      # Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group)
-
-      Fixtures.Policies.create_policy(
-        account: account,
-        actor_group: actor_group,
-        resource: resource
-      )
-
-      assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found}
-    end
-
-    test "does not authorize using disabled policies", %{
-      account: account,
-      actor: actor,
-      subject: subject
-    } do
-      resource = Fixtures.Resources.create_resource(account: account)
-      actor_group = Fixtures.Actors.create_group(account: account)
-      Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group)
-
-      policy =
-        Fixtures.Policies.create_policy(
-          account: account,
-          actor_group: actor_group,
-          resource: resource
-        )
-
-      {:ok, _policy} = Domain.Policies.disable_policy(policy, subject)
-
-      assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found}
-    end
-
-    test "does not return resources in other accounts", %{subject: subject} do
-      resource = Fixtures.Resources.create_resource()
-      assert fetch_and_authorize_resource_by_id(resource.id, subject) == {:error, :not_found}
-    end
-
-    test "returns error when subject has no permission to view resources", %{subject: subject} do
-      subject = Fixtures.Auth.remove_permissions(subject)
-
-      assert fetch_and_authorize_resource_by_id(Ecto.UUID.generate(), subject) ==
-               {:error,
-                {:unauthorized,
-                 reason: :missing_permissions,
-                 missing_permissions: [Resources.Authorizer.view_available_resources_permission()]}}
-    end
-
-    test "associations are preloaded when opts given", %{
-      account: account,
-      actor: actor,
-      subject: subject
-    } do
-      actor_group = Fixtures.Actors.create_group(account: account)
-      Fixtures.Actors.create_membership(account: account, actor: actor, group: actor_group)
-
-      gateway_group = Fixtures.Gateways.create_group(account: account)
-
-      resource =
-        Fixtures.Resources.create_resource(
-          account: account,
-          connections: [%{gateway_group_id: gateway_group.id}]
-        )
-
-      Fixtures.Policies.create_policy(
-        account: account,
-        actor_group: actor_group,
-        resource: resource
-      )
-
-      assert {:ok, resource} =
-               fetch_and_authorize_resource_by_id(resource.id, subject, preload: :connections)
-
-      assert Ecto.assoc_loaded?(resource.connections)
-      assert length(resource.connections) == 1
-    end
-  end
-
   describe "all_authorized_resources/1" do
     test "returns empty list when there are no resources", %{subject: subject} do
       assert {:ok, []} = all_authorized_resources(subject)
@@ -1525,23 +1308,6 @@ defmodule Domain.ResourcesTest do
       assert resource.address_description == attrs["address_description"]
     end
 
-    test "does not expire flows when connections are not updated", %{
-      account: account,
-      resource: resource,
-      subject: subject
-    } do
-      flow = Fixtures.Flows.create_flow(account: account, resource: resource, subject: subject)
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-
-      :ok = Domain.PubSub.Flow.subscribe(flow_id)
-
-      attrs = %{"name" => "foo"}
-      assert {:ok, _resource} = update_resource(resource, attrs, subject)
-      refute_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-    end
-
     test "allows to update connections", %{account: account, resource: resource, subject: subject} do
       group = Fixtures.Gateways.create_group(account: account, subject: subject)
       gateway1 = Fixtures.Gateways.create_gateway(account: account, group: group)
@@ -1553,13 +1319,6 @@ defmodule Domain.ResourcesTest do
 
       gateway2 = Fixtures.Gateways.create_gateway(account: account)
 
-      flow = Fixtures.Flows.create_flow(account: account, resource: resource, subject: subject)
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-
-      :ok = Domain.PubSub.Flow.subscribe(flow_id)
-
       attrs = %{
         "connections" => [
           %{gateway_group_id: gateway1.group_id},
@@ -1575,15 +1334,6 @@ defmodule Domain.ResourcesTest do
       assert {:ok, resource} = update_resource(resource, attrs, subject)
       gateway_group_ids = Enum.map(resource.connections, & &1.gateway_group_id)
       assert gateway_group_ids == [gateway2.group_id]
-
-      # TODO: WAL
-      # Remove this when directly broadcasting flow removals
-      Events.Hooks.ResourceConnections.on_delete(%{
-        "account_id" => resource.account_id,
-        "resource_id" => resource.id
-      })
-
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
     end
 
     test "does not allow to remove all connections", %{resource: resource, subject: subject} do

elixir/apps/domain/test/domain/schema_helpers_test.exs

@@ -7,21 +7,15 @@ defmodule Domain.SchemaHelpersTest do
 
   defmodule NestedEmbedSchema do
     use Ecto.Schema
-    import Ecto.Changeset
 
     @primary_key false
     embedded_schema do
       field :nested_field, :string
     end
-
-    def changeset(struct, params) do
-      cast(struct, params, [:nested_field])
-    end
   end
 
   defmodule EmbeddedSchema do
     use Ecto.Schema
-    import Ecto.Changeset
 
     @primary_key false
     embedded_schema do
@@ -29,26 +23,34 @@ defmodule Domain.SchemaHelpersTest do
       field :sub_field2, :string
       embeds_one :nested_item, NestedEmbedSchema
     end
-
-    def changeset(struct, params) do
-      struct
-      |> cast(params, [:sub_field1, :sub_field2])
-      |> cast_embed(:nested_item)
-    end
   end
 
   defmodule ListItemSchema do
     use Ecto.Schema
-    import Ecto.Changeset
 
     @primary_key false
     embedded_schema do
       field :list_field1, :string
       field :list_field2, :string, default: "default_value"
     end
+  end
 
-    def changeset(struct, params) do
-      cast(struct, params, [:list_field1, :list_field2])
+  defmodule DateTimeSchema do
+    use Ecto.Schema
+
+    @primary_key false
+    embedded_schema do
+      field :datetime_field, :utc_datetime_usec
+    end
+  end
+
+  defmodule NestedEnumSchema do
+    use Ecto.Schema
+
+    @primary_key false
+    embedded_schema do
+      field :enum_field, Ecto.Enum, values: ~w[option1 option2 option3]a
+      field :values, {:array, :string}
     end
   end
 
@@ -183,5 +185,144 @@ defmodule Domain.SchemaHelpersTest do
       assert %RootSchema{field1: "Empty Embedded", embedded_item: item} = result
       assert %EmbeddedSchema{sub_field1: nil, sub_field2: nil} = item
     end
+
+    test "correctly casts ISO 8601 datetime string" do
+      params = %{
+        "datetime_field" => "2023-12-25T10:30:45Z"
+      }
+
+      result = SchemaHelpers.struct_from_params(DateTimeSchema, params)
+
+      assert %DateTimeSchema{datetime_field: datetime} = result
+      assert %DateTime{} = datetime
+      assert datetime.year == 2023
+      assert datetime.month == 12
+      assert datetime.day == 25
+      assert datetime.hour == 10
+      assert datetime.minute == 30
+      assert datetime.second == 45
+      assert datetime.time_zone == "Etc/UTC"
+    end
+
+    test "correctly casts ISO 8601 datetime string with microseconds" do
+      params = %{
+        "datetime_field" => "2023-12-25T10:30:45.123456Z"
+      }
+
+      result = SchemaHelpers.struct_from_params(DateTimeSchema, params)
+
+      assert %DateTimeSchema{datetime_field: datetime} = result
+      assert %DateTime{} = datetime
+      assert datetime.microsecond == {123_456, 6}
+    end
+
+    test "correctly casts ISO 8601 datetime string with timezone offset" do
+      params = %{
+        "datetime_field" => "2023-12-25T10:30:45+02:00"
+      }
+
+      result = SchemaHelpers.struct_from_params(DateTimeSchema, params)
+
+      assert %DateTimeSchema{datetime_field: datetime} = result
+      assert %DateTime{} = datetime
+      # Should be converted to UTC
+      assert datetime.time_zone == "Etc/UTC"
+      # Should be adjusted for timezone (10:30 +02:00 = 08:30 UTC)
+      assert datetime.hour == 8
+      assert datetime.minute == 30
+    end
+
+    test "correctly casts ISO 8601 datetime string with negative timezone offset" do
+      params = %{
+        "datetime_field" => "2023-12-25T10:30:45-05:00"
+      }
+
+      result = SchemaHelpers.struct_from_params(DateTimeSchema, params)
+
+      assert %DateTimeSchema{datetime_field: datetime} = result
+      assert %DateTime{} = datetime
+      # Should be converted to UTC
+      assert datetime.time_zone == "Etc/UTC"
+      # Should be adjusted for timezone (10:30 -05:00 = 15:30 UTC)
+      assert datetime.hour == 15
+      assert datetime.minute == 30
+    end
+
+    test "handles nil datetime field" do
+      params = %{
+        "datetime_field" => nil
+      }
+
+      result = SchemaHelpers.struct_from_params(DateTimeSchema, params)
+
+      assert %DateTimeSchema{datetime_field: nil} = result
+    end
+
+    test "handles missing datetime field" do
+      params = %{}
+
+      result = SchemaHelpers.struct_from_params(DateTimeSchema, params)
+
+      assert %DateTimeSchema{datetime_field: nil} = result
+    end
+
+    test "handles DateTime struct input" do
+      datetime = DateTime.utc_now()
+
+      params = %{
+        "datetime_field" => datetime
+      }
+
+      result = SchemaHelpers.struct_from_params(DateTimeSchema, params)
+
+      assert %DateTimeSchema{datetime_field: ^datetime} = result
+    end
+
+    test "handles invalid datetime string gracefully" do
+      params = %{
+        "datetime_field" => "invalid-datetime"
+      }
+
+      result = SchemaHelpers.struct_from_params(DateTimeSchema, params)
+
+      # Ecto casting should handle invalid datetime strings by setting to nil
+      # or keeping the original value depending on changeset validation
+      assert %DateTimeSchema{} = result
+    end
+
+    test "correctly casts datetime string without 'Z' suffix" do
+      params = %{
+        "datetime_field" => "2023-12-25T10:30:45"
+      }
+
+      result = SchemaHelpers.struct_from_params(DateTimeSchema, params)
+
+      assert %DateTimeSchema{datetime_field: datetime} = result
+      # Should still parse as UTC when no timezone is specified
+      assert %DateTime{} = datetime
+      assert datetime.year == 2023
+      assert datetime.month == 12
+      assert datetime.day == 25
+    end
+
+    test "correctly casts NaiveDateTime to DateTime" do
+      naive_datetime = ~N[2023-12-25 10:30:45]
+
+      params = %{
+        "datetime_field" => naive_datetime
+      }
+
+      result = SchemaHelpers.struct_from_params(DateTimeSchema, params)
+
+      assert %DateTimeSchema{datetime_field: datetime} = result
+      assert %DateTime{} = datetime
+      assert datetime.year == 2023
+      assert datetime.month == 12
+      assert datetime.day == 25
+      assert datetime.hour == 10
+      assert datetime.minute == 30
+      assert datetime.second == 45
+      assert datetime.time_zone == "Etc/UTC"
+    end
   end
 end

elixir/apps/domain/test/domain/tokens_test.exs

@@ -498,7 +498,7 @@ defmodule Domain.TokensTest do
     } do
       other_token = Fixtures.Tokens.create_token(account: account, identity: identity)
 
-      assert {:ok, deleted_token} = delete_token_for(subject)
+      assert {:ok, [deleted_token]} = delete_token_for(subject)
       assert deleted_token.id == subject.token_id
 
       assert Repo.get(Tokens.Token, subject.token_id).deleted_at
@@ -506,29 +506,6 @@ defmodule Domain.TokensTest do
       refute Repo.get(Tokens.Token, other_token.id).deleted_at
     end
 
-    test "expires flows for given subject", %{
-      account: account,
-      identity: identity,
-      subject: subject
-    } do
-      flow =
-        Fixtures.Flows.create_flow(
-          account: account,
-          identity: identity,
-          subject: subject
-        )
-
-      :ok = Domain.PubSub.Flow.subscribe(flow.id)
-
-      assert {:ok, _token} = delete_token_for(subject)
-
-      flow_id = flow.id
-      client_id = flow.client_id
-      resource_id = flow.resource_id
-
-      assert_receive {:expire_flow, ^flow_id, ^client_id, ^resource_id}
-    end
-
     test "does not delete tokens for other actors", %{account: account, subject: subject} do
       token = Fixtures.Tokens.create_token(account: account)
 

elixir/apps/domain/test/support/fixtures/flows.ex

@@ -52,11 +52,15 @@ defmodule Domain.Fixtures.Flows do
         |> Fixtures.Resources.create_resource()
       end)
 
-    {actor_group_id, attrs} =
-      pop_assoc_fixture_id(attrs, :actor_group, fn assoc_attrs ->
+    {membership, attrs} =
+      pop_assoc_fixture(attrs, :actor_group_membership, fn assoc_attrs ->
         assoc_attrs
-        |> Enum.into(%{account: account, subject: subject})
-        |> Fixtures.Actors.create_group()
+        |> Enum.into(%{
+          account: account,
+          subject: subject,
+          actor_id: client.actor_id
+        })
+        |> Fixtures.Actors.create_membership()
       end)
 
     {policy_id, attrs} =
@@ -64,14 +68,17 @@ defmodule Domain.Fixtures.Flows do
         assoc_attrs
         |> Enum.into(%{
           account: account,
-          actor_group_id: actor_group_id,
+          actor_group_id: membership.group_id,
           resource_id: resource_id,
           subject: subject
         })
         |> Fixtures.Policies.create_policy()
       end)
 
-    {token_id, _attrs} = Map.pop(attrs, :token_id, subject.token_id)
+    {token_id, _attrs} =
+      pop_assoc_fixture_id(attrs, :token, fn _assoc_attrs ->
+        %{id: subject.token_id}
+      end)
 
     Flows.Flow.Changeset.create(%{
       token_id: token_id,
@@ -79,6 +86,7 @@ defmodule Domain.Fixtures.Flows do
       client_id: client.id,
       gateway_id: gateway.id,
       resource_id: resource_id,
+      actor_group_membership_id: membership.id,
       account_id: account.id,
       client_remote_ip: client.last_seen_remote_ip,
       client_user_agent: client.last_seen_user_agent,

elixir/apps/web/lib/web.ex

@@ -67,12 +67,6 @@ defmodule Web do
           {:noreply, socket}
         end
       end
-
-      # ignore "disconnect" message that is broadcasted for some pages
-      # because of subscription for relay/gateway group events
-      def handle_info("disconnect", socket) do
-        {:noreply, socket}
-      end
     end
   end
 

elixir/apps/web/lib/web/live/actors/show.ex

@@ -630,6 +630,8 @@ defmodule Web.Actors.Show do
     {:noreply, reload_live_table!(socket, "clients")}
   end
 
+  def handle_info(_message, socket), do: {:noreply, socket}
+
   def handle_event(event, params, socket) when event in ["paginate", "order_by", "filter"],
     do: handle_live_table_event(event, params, socket)
 

elixir/apps/web/lib/web/live/policies/index.ex

@@ -4,7 +4,7 @@ defmodule Web.Policies.Index do
 
   def mount(_params, _session, socket) do
     if connected?(socket) do
-      :ok = PubSub.Account.Policies.subscribe(socket.assigns.account.id)
+      :ok = PubSub.Account.subscribe(socket.assigns.account.id)
     end
 
     socket =
@@ -123,7 +123,15 @@ defmodule Web.Policies.Index do
       when event in ["paginate", "order_by", "filter", "reload"],
       do: handle_live_table_event(event, params, socket)
 
-  def handle_info({_action, _policy_id}, socket) do
+  def handle_info({_action, %Policies.Policy{}, %Policies.Policy{}}, socket) do
     {:noreply, assign(socket, stale: true)}
   end
+
+  def handle_info({_action, %Policies.Policy{}}, socket) do
+    {:noreply, assign(socket, stale: true)}
+  end
+
+  def handle_info(_, socket) do
+    {:noreply, socket}
+  end
 end

elixir/apps/web/lib/web/live/policies/show.ex

@@ -16,7 +16,7 @@ defmodule Web.Policies.Show do
       providers = Auth.all_active_providers_for_account!(socket.assigns.account)
 
       if connected?(socket) do
-        :ok = PubSub.Policy.subscribe(policy.id)
+        :ok = PubSub.Account.subscribe(policy.account_id)
       end
 
       socket =
@@ -302,7 +302,12 @@ defmodule Web.Policies.Show do
     """
   end
 
-  def handle_info({_action, _policy_id}, socket) do
+  # TODO: Do we really want to update the view in place?
+  def handle_info(
+        {_action, _old_policy, %Policies.Policy{id: policy_id}},
+        %{assigns: %{policy: %{id: id}}} = socket
+      )
+      when policy_id == id do
     {:ok, policy} =
       Policies.fetch_policy_by_id_or_persistent_id(
         socket.assigns.policy.id,
@@ -318,6 +323,10 @@ defmodule Web.Policies.Show do
     {:noreply, assign(socket, policy: policy)}
   end
 
+  def handle_info(_, socket) do
+    {:noreply, socket}
+  end
+
   def handle_event(event, params, socket) when event in ["paginate", "order_by", "filter"],
     do: handle_live_table_event(event, params, socket)
 

elixir/apps/web/lib/web/live/resources/index.ex

@@ -5,7 +5,7 @@ defmodule Web.Resources.Index do
 
   def mount(_params, _session, socket) do
     if connected?(socket) do
-      :ok = PubSub.Account.Resources.subscribe(socket.assigns.account.id)
+      :ok = PubSub.Account.subscribe(socket.assigns.account.id)
     end
 
     socket =
@@ -170,7 +170,15 @@ defmodule Web.Resources.Index do
       when event in ["paginate", "order_by", "filter", "reload"],
       do: handle_live_table_event(event, params, socket)
 
-  def handle_info({_action, _resource_id}, socket) do
+  def handle_info({_action, %Resources.Resource{}, %Resources.Resource{}}, socket) do
     {:noreply, assign(socket, stale: true)}
   end
+
+  def handle_info({_action, %Resources.Resource{}}, socket) do
+    {:noreply, assign(socket, stale: true)}
+  end
+
+  def handle_info(_, socket) do
+    {:noreply, socket}
+  end
 end

elixir/apps/web/lib/web/live/resources/show.ex

@@ -9,7 +9,7 @@ defmodule Web.Resources.Show do
          {:ok, actor_groups_peek} <-
            Resources.peek_resource_actor_groups([resource], 3, socket.assigns.subject) do
       if connected?(socket) do
-        :ok = PubSub.Resource.subscribe(resource.id)
+        :ok = PubSub.Account.subscribe(resource.account_id)
       end
 
       socket =
@@ -399,7 +399,12 @@ defmodule Web.Resources.Show do
     """
   end
 
-  def handle_info({_action, _resource_id}, socket) do
+  # TODO: Do we really want to update the view in place?
+  def handle_info(
+        {_action, _old_resource, %Resources.Resource{id: resource_id}},
+        %{assigns: %{resource: %{id: id}}} = socket
+      )
+      when resource_id == id do
     {:ok, resource} =
       Resources.fetch_resource_by_id(socket.assigns.resource.id, socket.assigns.subject,
         preload: [
@@ -413,6 +418,10 @@ defmodule Web.Resources.Show do
     {:noreply, assign(socket, resource: resource)}
   end
 
+  def handle_info(_, socket) do
+    {:noreply, socket}
+  end
+
   def handle_event(event, params, socket) when event in ["paginate", "order_by", "filter"],
     do: handle_live_table_event(event, params, socket)
 

elixir/apps/web/test/web/acceptance/auth_test.exs

@@ -92,7 +92,17 @@ defmodule Web.Acceptance.AuthTest do
 
     for token <- tokens do
       assert %DateTime{} = token.deleted_at
-      Domain.Events.Hooks.Tokens.on_delete(%{"id" => token.id})
+
+      Domain.Events.Hooks.Tokens.on_delete(%{
+        "remaining_attempts" => token.remaining_attempts,
+        "actor_id" => token.actor_id,
+        "name" => token.name,
+        "type" => "#{token.type}",
+        "account_id" => token.account_id,
+        "id" => token.id,
+        "identity_id" => token.identity_id,
+        "expires_at" => "#{token.expires_at}"
+      })
     end
 
     wait_for(

elixir/apps/web/test/web/live/actors/show_test.exs

@@ -184,7 +184,7 @@ defmodule Web.Live.Actors.ShowTest do
              "#{flow.gateway.group.name}-#{flow.gateway.name} #{flow.gateway.last_seen_remote_ip}"
   end
 
-  test "renders flows even for deleted policy assocs", %{
+  test "does not render flows for deleted policy assocs", %{
     conn: conn
   } do
     account = Fixtures.Accounts.create_account()
@@ -207,21 +207,11 @@ defmodule Web.Live.Actors.ShowTest do
       |> authorize_conn(identity)
       |> live(~p"/#{account}/actors/#{actor}")
 
-    [row] =
-      lv
-      |> element("#flows")
-      |> render()
-      |> table_to_map()
-
-    assert row["authorized"]
-    assert row["policy"] =~ flow.policy.actor_group.name
-    assert row["policy"] =~ flow.policy.resource.name
-
-    assert row["client"] ==
-             "#{flow.client.name} #{client.last_seen_remote_ip}"
-
-    assert row["gateway"] ==
-             "#{flow.gateway.group.name}-#{flow.gateway.name} #{flow.gateway.last_seen_remote_ip}"
+    assert [] ==
+             lv
+             |> element("#flows")
+             |> render()
+             |> table_to_map()
   end
 
   test "renders groups table", %{

elixir/apps/web/test/web/live/clients/show_test.exs

@@ -248,7 +248,7 @@ defmodule Web.Live.Clients.ShowTest do
              "#{flow.gateway.group.name}-#{flow.gateway.name} #{flow.gateway.last_seen_remote_ip}"
   end
 
-  test "renders flows even for deleted policy assocs", %{
+  test "does not render flows for deleted policy assocs", %{
     account: account,
     identity: identity,
     client: client,
@@ -269,19 +269,11 @@ defmodule Web.Live.Clients.ShowTest do
       |> authorize_conn(identity)
       |> live(~p"/#{account}/clients/#{client}")
 
-    [row] =
-      lv
-      |> element("#flows")
-      |> render()
-      |> table_to_map()
-
-    assert row["authorized"]
-    assert row["remote ip"] == to_string(client.last_seen_remote_ip)
-    assert row["policy"] =~ flow.policy.actor_group.name
-    assert row["policy"] =~ flow.policy.resource.name
-
-    assert row["gateway"] ==
-             "#{flow.gateway.group.name}-#{flow.gateway.name} #{flow.gateway.last_seen_remote_ip}"
+    assert [] ==
+             lv
+             |> element("#flows")
+             |> render()
+             |> table_to_map()
   end
 
   test "allows editing clients", %{

elixir/apps/web/test/web/live/resources/edit_test.exs

@@ -1,6 +1,6 @@
 defmodule Web.Live.Resources.EditTest do
   use Web.ConnCase, async: true
-  alias Domain.{Events, PubSub}
+  alias Domain.{Events, PubSub, Resources}
 
   setup do
     account = Fixtures.Accounts.create_account()
@@ -229,7 +229,7 @@ defmodule Web.Live.Resources.EditTest do
       }
     }
 
-    :ok = PubSub.Account.Resources.subscribe(account.id)
+    :ok = PubSub.Account.subscribe(account.id)
 
     {:ok, lv, _html} =
       conn
@@ -240,7 +240,7 @@ defmodule Web.Live.Resources.EditTest do
       "account_id" => resource.account_id,
       "type" => resource.type,
       "address" => resource.address,
-      "filters" => resource.filters,
+      "filters" => [%{"protocol" => "icmp", "ports" => ["80"]}],
       "ip_stack" => resource.ip_stack
     }
 
@@ -253,9 +253,9 @@ defmodule Web.Live.Resources.EditTest do
       |> render_submit()
       |> follow_redirect(conn, ~p"/#{account}/resources")
 
-    assert_receive {:delete_resource, delete_msg_resource_id}
-    assert_receive {:create_resource, create_msg_resource_id}
-    assert delete_msg_resource_id == create_msg_resource_id
+    assert_receive {:updated, %Resources.Resource{}, %Resources.Resource{} = updated_resource}
+
+    assert updated_resource.id == resource.id
 
     assert updated_resource = Repo.get_by(Domain.Resources.Resource, id: resource.id)
     assert updated_resource.name == attrs.name

elixir/config/config.exs

@@ -99,13 +99,10 @@ config :domain, Domain.Events.ReplicationConnection,
   table_subscriptions: ~w[
     accounts
     actor_group_memberships
-    actor_groups
-    actors
-    auth_identities
-    auth_providers
     clients
-    gateway_groups
+    flows
     gateways
+    gateway_groups
     policies
     resource_connections
     resources