End-to-end Tests
This directory contains our suite of end-to-end tests for the following clients, running on baremetal hardware:
- Windows
- Android
- ChromeOS
- macOS
- iOS
- iPadOS
We use Appium as our test orchestrator because it supports all of the above platforms with a consistent architecture and test language.
Running
You shouldn't run these tests yourself. Instead, these tests are intended to be run from our bare metal testbed running in the Firezone HQ. See the e2e.yml workflow for how these tests are triggered and run.
Architecture
Our baremetal testbed consists of the following hardware:
| Hostname | Purpose | OS / Model | CPU | RAM | Disk | Network |
|---|---|---|---|---|---|---|
linux.firezone.test |
Orchestration server + Android / ChromeOS test runner | Ubuntu 22.04 | Ryzen 5950x | 128 GB DDR4 3200 ECC | 500 GB NVMe SSD | 10 GbE internet uplink + 1 GbE testnet |
macos.firezone.test |
macOS, iPadOS, iOS test driver | macOS 14 | Apple M1 | 8 GB | 256 GB | 1 GbE testnet + WiFi 6 |
windows.firezone.test |
Windows test driver | Windows 11 Home | Intel i3-1115G4 | 8 GB | 256 GB | 1 GbE testnet + WiFi 6 |
ios.firezone.test |
iOS test client | iOS 17 | A14 Bionic (iPhone 12) | 4 GB | 128 GB | WiFi 6 |
ipados.firezone.test |
iPadOS test client | iPadOS 17 | A12 (iPad 8th gen) | 3 GB | 128 GB | 802.11ax WiFi |
android.firezone.test |
Android test client | Android 14 (Pixel 6) | Tensor SoC | 8 GB | 128 GB | WiFi 6 |
gateway.firezone.test |
Remote access (using a Firezone gateway!) | Raspbian 12 (Raspberry Pi 2 Model B) | Cortex A7 4-core armv7l | 4 GB | 32 GB | 1 GbE testnet |
Topology
Since we can't run the test driver directly on mobile platforms, we run it on
the host instead and connect over USB to the client. iOS/iPadOS clients are
tested from the macos.firezone.test host, and the Android/ChromeOS clients are
tested from the linux.firezone.test host.
There's also a Raspberry Pi in the testnet running Firezone, used for remote access.
Network diagram
Addressing
| Host | Interface | IPv4 Address | IPv6 Address |
|---|---|---|---|
linux.firezone.test |
egress | 192.168.1.249 |
2600:1700:3ecb:2410::/64 (DHCP address) |
linux.firezone.test |
testnet ethernet | 10.0.0.1 |
fc01:2345:6789:abcd::/64 (DHCP address) |
macos.firezone.test |
testnet ethernet | 10.0.0.2 |
fc01:2345:6789:abcd::/64 (DHCP address) |
windows.firezone.test |
testnet ethernet | 10.0.0.3 |
fc01:2345:6789:abcd::/64 (DHCP address) |
linux.firezone.test |
testnet WiFi | 10.0.1.1 |
fc01:2345:6789:abce::/64 (DHCP address) |
macos.firezone.test |
testnet WiFi | 10.0.1.2 |
fc01:2345:6789:abce::/64 (DHCP address) |
windows.firezone.test |
testnet WiFi | 10.0.1.3 |
fc01:2345:6789:abce::/64 (DHCP address) |
ios.firezone.test |
testnet WiFi | 10.0.1.4 |
fc01:2345:6789:abce::/64 (DHCP address) |
ipados.firezone.test |
testnet WiFi | 10.0.1.5 |
fc01:2345:6789:abce::/64 (DHCP address) |
android.firezone.test |
testnet WiFi | 10.0.1.6 |
fc01:2345:6789:abce::/64 (DHCP address) |
gateway.firezone.test |
testnet ethernet | 10.0.0.254 |
fc01:2345:6789:abcd::/64 (DHCP address) |