feat(connlib): introduce FZ p2p control protocol (#6939)

At present, `connlib` utilises the portal as a signalling layer for any
kind of control message that needs to be exchanged between clients and
gateways. For anything regard to connectivity, this is crucial: Before
we have a direct connection to the gateway, we don't really have a
choice other than using the portal as a "relay" to e.g. exchange address
candidates for ICE.

However, once a direct connection has been established, exchanging
information directly with the gateway is faster and removes the portal
as a potential point of failure for the data plane.

For DNS resources, `connlib` intercepts all DNS requests on the client
and assigns its own IPs within the CG-NAT range to all domains that are
configured as resources. Thus, all packets targeting DNS resources will
have one of these IPs set as their destination. The gateway needs to
learn about all the IPs that have been assigned to a certain domain by
the client and perform NAT. We call this concept "DNS resource NAT".

Currently, the domain + the assigned IPs are sent together with the
`allow_access` or `request_connection` message via the portal. The new
control protocol defined in #6732 purposely excludes this information
and only authorises traffic to the entire resource which could also be a
wildcard-DNS resource.

To exchange the assigned IPs for a certain domain with the gateway, we
introduce our own p2p control protocol built on top of IP. All control
protocol messages are sent through the tunnel and thus encrypted at all
times. They are differentiated from regular application traffic as
follows:

- IP src is set to the unspecified IPv6 address (`::`)
- IP dst is set to the unspecified IPv6 address (`::`)
- IP protocol is set to reserved (`0xFF`)

The combination of all three should never appear as regular traffic.

To ensure forwards-compatibility, the control protocol utilises a fixed
8-byte header where the first byte denotes the message kind. In this
current design, there is no concept of a request or response in the
wire-format. Each message is unidirectional and the fact that the two
messages we define in here appear in tandem is purely by convention. We
use the IPv6 payload length to determine the total length of the packet.
The payloads are JSON-encoded. Message types are free to chose whichever
encoding they'd like.

This protocol is sent through the WireGuard tunnel, meaning we are
effectively limited by our device MTU of 1280, otherwise we'd have to
implement fragmentation. For the messages of setting up the DNS resource
NAT, we are below this limit:

- UUIDs are 16 bytes
- Domain names are at most 255 bytes
- IPv6 addresses are 16 bytes * 4
- IPv4 addressers are 4 bytes * 4

Including the JSON serialisation overhead, this results in a total
maximum payload size of 402 bytes, which is well below our MTU.

Finally, another thing to consider here is that IP is unreliable,
meaning each use of this protocol needs to make sure that:

- It is resilient against message re-ordering
- It is resilient against packet loss

The details of how this is ensured for setting up the DNS resource NAT
is left to #6732.

rust/connlib/tunnel/Cargo.toml

@@ -28,6 +28,7 @@ rand = "0.8.5"
 rangemap = "1.5.1"
 secrecy = { workspace = true, features = ["serde"] }
 serde = { version = "1.0", default-features = false, features = ["derive", "std"] }
+serde_json = "1.0"
 snownet = { workspace = true }
 socket-factory = { workspace = true }
 socket2 = { workspace = true }
@@ -43,7 +44,6 @@ firezone-relay = { workspace = true, features = ["proptest"] }
 ip-packet = { workspace = true, features = ["proptest"] }
 proptest-state-machine = "0.3"
 rand = "0.8"
-serde_json = "1.0"
 test-case = "3.3.1"
 test-strategy = "0.3.1"
 tracing-subscriber = { version = "0.3", features = ["env-filter"] }

rust/connlib/tunnel/src/device_channel.rs

@@ -60,6 +60,10 @@ impl Device {
             }
         }
 
+        if packet.is_fz_p2p_control() {
+            tracing::warn!("Packet matches heuristics of FZ-internal p2p control protocol");
+        }
+
         tracing::trace!(target: "wire::dev::recv", dst = %packet.destination(), src = %packet.source(), bytes = %packet.packet().len());
 
         Poll::Ready(Ok(packet))
@@ -74,6 +78,11 @@ impl Device {
 
         tracing::trace!(target: "wire::dev::send", dst = %packet.destination(), src = %packet.source(), bytes = %packet.packet().len());
 
+        debug_assert!(
+            !packet.is_fz_p2p_control(),
+            "FZ p2p control protocol packets should never leave `connlib`"
+        );
+
         match packet {
             IpPacket::Ipv4(msg) => self.tun()?.write4(msg.packet()),
             IpPacket::Ipv6(msg) => self.tun()?.write6(msg.packet()),

rust/connlib/tunnel/src/lib.rs

@@ -30,6 +30,7 @@ mod dns;
 mod gateway;
 mod io;
 pub mod messages;
+mod p2p_control;
 mod peer;
 mod peer_store;
 #[cfg(all(test, feature = "proptest"))]

rust/connlib/tunnel/src/p2p_control.rs

@@ -0,0 +1,149 @@
+//! Firezone's P2P control protocol between clients and gateways.
+
+#[cfg_attr(not(test), expect(dead_code, reason = "Will be used soon."))]
+pub mod setup_dns_resource_nat {
+    use anyhow::{Context, Result};
+    use connlib_model::{DomainName, ResourceId};
+    use ip_packet::{FzP2pControlSlice, IpPacket};
+    use std::net::IpAddr;
+
+    pub const REQ_CODE: u8 = 0;
+    pub const RES_CODE: u8 = 1;
+
+    pub fn request(resource: ResourceId, domain: DomainName, proxy_ips: Vec<IpAddr>) -> IpPacket {
+        debug_assert_eq!(proxy_ips.len(), 8);
+
+        let payload = serde_json::to_vec(&Request {
+            resource,
+            domain,
+            proxy_ips,
+        })
+        .unwrap();
+
+        ip_packet::make::fz_p2p_control([REQ_CODE, 0, 0, 0, 0, 0, 0, 0], &payload)
+            .expect("with only 8 proxy IPs, payload should be less than max packet size")
+    }
+
+    pub fn response(resource: ResourceId, domain: DomainName, code: u16) -> IpPacket {
+        let payload = serde_json::to_vec(&Response {
+            code,
+            resource,
+            domain,
+        })
+        .unwrap();
+
+        ip_packet::make::fz_p2p_control([RES_CODE, 0, 0, 0, 0, 0, 0, 0], &payload)
+            .expect("payload is less than max packet size")
+    }
+
+    pub fn decode_request(packet: FzP2pControlSlice) -> Result<Request> {
+        anyhow::ensure!(
+            packet.message_type() == REQ_CODE,
+            "Control protocol packet is not a setup-dns-resource-nat request"
+        );
+
+        serde_json::from_slice::<Request>(packet.payload())
+            .context("Failed to deserialize `setup_dns_resource_nat::Request`")
+    }
+
+    pub fn decode_response(packet: FzP2pControlSlice) -> Result<Response> {
+        anyhow::ensure!(
+            packet.message_type() == RES_CODE,
+            "Control protocol packet is not a setup-dns-resource-nat request"
+        );
+
+        serde_json::from_slice::<Response>(packet.payload())
+            .context("Failed to deserialize `setup_dns_resource_nat::Response`")
+    }
+
+    #[derive(serde::Serialize, serde::Deserialize)]
+    pub struct Request {
+        pub resource: ResourceId,
+        pub domain: DomainName,
+        pub proxy_ips: Vec<IpAddr>,
+    }
+
+    #[derive(serde::Serialize, serde::Deserialize)]
+    pub struct Response {
+        pub resource: ResourceId,
+        pub domain: DomainName,
+        pub code: u16, // Loosely follows the semantics of HTTP.
+    }
+
+    #[cfg(test)]
+    mod tests {
+        use domain::base::Name;
+
+        use super::*;
+        use std::net::{Ipv4Addr, Ipv6Addr};
+
+        #[test]
+        fn max_payload_length_request() {
+            let request = Request {
+                resource: ResourceId::from_u128(100),
+                domain: longest_domain_possible(),
+                proxy_ips: eight_proxy_ips(),
+            };
+
+            let serialized = serde_json::to_vec(&request).unwrap();
+
+            assert_eq!(serialized.len(), 402);
+            assert!(serialized.len() <= ip_packet::PACKET_SIZE);
+        }
+
+        #[test]
+        fn request_serde_roundtrip() {
+            let packet = request(
+                ResourceId::from_u128(101),
+                domain("example.com"),
+                eight_proxy_ips(),
+            );
+
+            let slice = packet.as_fz_p2p_control().unwrap();
+            let request = decode_request(slice).unwrap();
+
+            assert_eq!(request.resource, ResourceId::from_u128(101));
+            assert_eq!(request.domain, domain("example.com"));
+            assert_eq!(request.proxy_ips, eight_proxy_ips())
+        }
+
+        #[test]
+        fn response_serde_roundtrip() {
+            let packet = response(ResourceId::from_u128(101), domain("example.com"), 200);
+
+            let slice = packet.as_fz_p2p_control().unwrap();
+            let request = decode_response(slice).unwrap();
+
+            assert_eq!(request.resource, ResourceId::from_u128(101));
+            assert_eq!(request.domain, domain("example.com"));
+            assert_eq!(request.code, 200)
+        }
+
+        fn domain(d: &str) -> DomainName {
+            d.parse().unwrap()
+        }
+
+        fn longest_domain_possible() -> DomainName {
+            let label = "a".repeat(49);
+            let domain =
+                DomainName::vec_from_str(&format!("{label}.{label}.{label}.{label}.{label}.com"))
+                    .unwrap();
+            assert_eq!(domain.len(), Name::MAX_LEN);
+
+            domain
+        }
+
+        fn eight_proxy_ips() -> Vec<IpAddr> {
+            vec![
+                IpAddr::V4(Ipv4Addr::LOCALHOST),
+                IpAddr::V4(Ipv4Addr::LOCALHOST),
+                IpAddr::V4(Ipv4Addr::LOCALHOST),
+                IpAddr::V4(Ipv4Addr::LOCALHOST),
+                IpAddr::V6(Ipv6Addr::LOCALHOST),
+                IpAddr::V6(Ipv6Addr::LOCALHOST),
+                IpAddr::V6(Ipv6Addr::LOCALHOST),
+                IpAddr::V6(Ipv6Addr::LOCALHOST),
+            ]
+        }
+    }
+}

rust/ip-packet/src/fz_p2p_control.rs

@@ -0,0 +1,14 @@
+use std::net::Ipv6Addr;
+
+use etherparse::IpNumber;
+
+/// The src and dst of a FZ p2p control protocol packet.
+///
+/// No actual IP packet can be sent to the unspecified IPv6 addr.
+/// This allows us to unambiguously identify our control protocol packets among the others.
+pub const ADDR: Ipv6Addr = Ipv6Addr::UNSPECIFIED;
+
+/// The IP protocol of FZ p2p control protocol packets.
+///
+/// `0xFF` is reserved and should thus never appear as real-world traffic.
+pub const IP_NUMBER: IpNumber = IpNumber(0xFF);

rust/ip-packet/src/fz_p2p_control_slice.rs

@@ -0,0 +1,32 @@
+use etherparse::LenSource;
+
+pub struct FzP2pControlSlice<'a> {
+    slice: &'a [u8],
+}
+
+impl<'a> FzP2pControlSlice<'a> {
+    /// Creates a new [`FzP2pControlSlice`].
+    pub fn from_slice(slice: &'a [u8]) -> Result<Self, etherparse::err::LenError> {
+        if slice.len() < 8 {
+            return Err(etherparse::err::LenError {
+                required_len: 8,
+                len: slice.len(),
+                len_source: LenSource::Slice,
+                layer: etherparse::err::Layer::Ipv6Header,
+                layer_start_offset: 0,
+            });
+        }
+
+        Ok(Self { slice })
+    }
+
+    pub fn message_type(&self) -> u8 {
+        self.slice[0]
+    }
+
+    pub fn payload(&self) -> &[u8] {
+        let (_, payload) = self.slice.split_at(8);
+
+        payload
+    }
+}

rust/ip-packet/src/lib.rs

@@ -1,5 +1,7 @@
 pub mod make;
 
+mod fz_p2p_control;
+mod fz_p2p_control_slice;
 mod icmpv4_header_slice_mut;
 mod icmpv6_header_slice_mut;
 mod ipv4_header_slice_mut;
@@ -13,6 +15,7 @@ mod tcp_header_slice_mut;
 mod udp_header_slice_mut;
 
 pub use etherparse::*;
+pub use fz_p2p_control_slice::FzP2pControlSlice;
 
 #[cfg(all(test, feature = "proptest"))]
 mod proptests;
@@ -610,6 +613,14 @@ impl IpPacket {
         Icmpv6EchoHeaderSliceMut::from_slice(self.payload_mut()).ok()
     }
 
+    pub fn as_fz_p2p_control(&self) -> Option<FzP2pControlSlice> {
+        if !self.is_fz_p2p_control() {
+            return None;
+        }
+
+        FzP2pControlSlice::from_slice(self.payload()).ok()
+    }
+
     fn icmpv4_echo_header(&self) -> Option<IcmpEchoHeader> {
         let p = self.as_icmpv4()?;
 
@@ -745,6 +756,13 @@ impl IpPacket {
         self.next_header() == IpNumber::ICMP
     }
 
+    /// Whether the packet is a Firezone p2p control protocol packet.
+    pub fn is_fz_p2p_control(&self) -> bool {
+        self.next_header() == fz_p2p_control::IP_NUMBER
+            && self.source() == fz_p2p_control::ADDR
+            && self.destination() == fz_p2p_control::ADDR
+    }
+
     pub fn is_icmpv6(&self) -> bool {
         self.next_header() == IpNumber::IPV6_ICMP
     }

rust/ip-packet/src/make.rs

@@ -1,6 +1,7 @@
 //! Factory module for making all kinds of packets.
 
-use crate::IpPacket;
+use crate::{IpPacket, IpPacketBuf};
+use anyhow::{Context, Result};
 use domain::{
     base::{
         iana::{Class, Opcode, Rcode},
@@ -26,6 +27,36 @@ macro_rules! build {
     }};
 }
 
+pub fn fz_p2p_control(header: [u8; 8], control_payload: &[u8]) -> Result<IpPacket> {
+    let ip_payload_size = header.len() + control_payload.len();
+
+    anyhow::ensure!(ip_payload_size <= crate::PACKET_SIZE);
+
+    let builder = etherparse::PacketBuilder::ipv6(
+        crate::fz_p2p_control::ADDR.octets(),
+        crate::fz_p2p_control::ADDR.octets(),
+        0,
+    );
+    let packet_size = builder.size(ip_payload_size);
+
+    let mut packet_buf = IpPacketBuf::new();
+
+    let mut payload_buf = vec![0u8; 8 + control_payload.len()];
+    payload_buf[..8].copy_from_slice(&header);
+    payload_buf[8..].copy_from_slice(control_payload);
+
+    builder
+        .write(
+            &mut std::io::Cursor::new(packet_buf.buf()),
+            crate::fz_p2p_control::IP_NUMBER,
+            &payload_buf,
+        )
+        .expect("Buffer should be big enough");
+    let ip_packet = IpPacket::new(packet_buf, packet_size).context("Unable to create IP packet")?;
+
+    Ok(ip_packet)
+}
+
 pub fn icmp_request_packet(
     src: IpAddr,
     dst: impl Into<IpAddr>,