Files
firezone/scripts
Thomas Eizinger a8aafc9e14 ci: use bencher.dev for continuous benchmarking (#5915)
Currently, we have a homegrown benchmark suite that reports results of
the iperf runs within CI by comparing a run on `main` with the current
branch.

These comments are noisy because they happen on every PR, regardless of
the performance results. As a result, they tend to be skimmed over by
devs and not actually considered. To properly track performance, we need
to record benchmark results over time and use statistics to detect
regressions.

https://bencher.dev does exactly that. it supports various benchmark
harnesses to automatically collect benchmarks. For our case, we simply
use the generic JSON adapter to extract the relevant metrics from the
iperf results and report them to the bencher backend.

With these metrics in place, bencher can plot the results over time, and
alert us in the case of regressions using thresholds based on
statistical tests.

Resolves: #5818.

---------

Signed-off-by: Thomas Eizinger <thomas@eizinger.io>
Co-authored-by: Reactor Scram <ReactorScram@users.noreply.github.com>
2024-07-24 01:22:17 +00:00
..

Firezone shell scripts

This directory contains various shell scripts used for development, testing, and deployment of the Firezone product.

Developer Setup

We lint shell scripts in CI. To get your PR to pass, you'll want to ensure your local development environment is set up to lint shell scripts:

  1. Install shfmt:
  2. Install shellcheck:
    • brew install shellcheck on macOS
    • sudo apt-get install shellcheck on Ubuntu

Then just lint and format your shell scripts before you commit:

shfmt -i 4 **/*.sh
shellcheck --severity=warning **/*.sh

You can achieve this more easily by using pre-commit. See CONTRIBUTING.

Editor setup

Scripting tips

  • Use #!/usr/bin/env bash along with set -euox pipefail in general for dev and test scripts.
  • In Docker images and other minimal envs, stick to #!/bin/sh and simply set -eu.