This PR fixes a bunch of small things to allow a new flow to test
clients pinging a resource within docker compose.
Masquerade/Forwarding is enabled directly in the container for now, this
might change in the future.
Also added a README to be able to run this locally.
---------
Signed-off-by: Gabi <gabrielalejandro7@gmail.com>
Co-authored-by: Jamil <jamilbk@users.noreply.github.com>
With this PR the full control-plane message flow is working.
Meaning that if you do:
```
docker compose up -d
docker compose exec -it client "ping 172.20.0.2" # will fix this IP later
```
Messages start flowing to gateway. The gateway still not correctly
forwards the messages to the resource since masquerading is still not
working, although I suspect there might be an additional problem. Will
fix this in my next PR along with a README on how to test this whole
flow.
This PR also fixes how we sent the stamp secret to the gateway from the
relay, but I still see some warnings in the webrtc that I'm sure that
are due to a mismatch between how webrtc-rs and the relay handle
messages (The most important being `bind() failed: unexpected response
type`), I will take a look at that and a way to test that the flow works
when:
1. hole-punching is available
2. through relay when it's not
Since the flow right now works without hole-punching or relay since the
gateway is in the same network in the docker compose.
This PR fixes `docker compose up` but it doesn't have the test client ->
resource flow working but it prevent anything from erroring at startup.
This fixes:
* tokens (use the correct token for the client user agent we are using)
* randomize `name_suffix` at start up for connlib (we will eventually
allow options to set it manually)
* remove port ranges for relay (see firezone/product#613)
**Update CONTRIBUTING.md**
Why:
* The CONTRIBUTING.md doc seems to have fallen slightly out of date with
how Firezone now works. This commit updates the doc to provide a
quick start guide for getting all of the various Firezone components
up and running as quick as possible. The doc then links to the more
specific `Elixir` and `Rust` README.md files in the respective
directories to help developers who would like to contribute.
**Update docker-compose vault health check**
Why:
* The current Vault health check listed in the docker-compose file does
not seem to be working when using `localhost` in the `wget` command.
Updating the URL to use `127.0.0.1` seems to have fixed it.
---------
Signed-off-by: bmanifold <bmanifold@users.noreply.github.com>
Co-authored-by: Jamil <jamilbk@users.noreply.github.com>
There are problems building the docker images in macos using musl due to
ring's problems therefore we started using slim-debian with glibc for
development.
This brindgs connlib from its own separated repo to firezone's monorepo.
On top of bringing connlib we also add and unify the Dockerfile for all
rust binaries and add a docker-compose that can run a headless client, a
relay and a gateway which eventually will test the whole flow between a
client and a resource. For this to work we also incorporated some elixir
scripts to generate portal tokens for those components.
TODO:
- [x] Cluster formation for all API and web nodes
- [x] Injest Docker logs to Stackdriver
- [x] Fix assets building for prod
To finish later:
- [ ] Structured logging:
https://issuetracker.google.com/issues/285950891
- [ ] Better networking policy (eg. use public postmark ranges and deny
all unwanted egress)
- [ ] OpenTelemetry collector for Google Stackdriver
- [ ] LoggerJSON.Plug integration
---------
Signed-off-by: Andrew Dryga <andrew@dryga.com>
Co-authored-by: Jamil <jamilbk@users.noreply.github.com>
- [x] All configs should support ENV variable overrides over DB values
- [ ] ~Adding a new field to DB value should automatically write ENV
config to DB on app boot (so that we don't need migrations)~
- [x] Validate configs and report human-readable errors when something
is wrong, telling where it's invalid (eg. env key X) and what's wrong
with it
- [x] Reuse Changeset validations (we still have a DB schema and UI
form, and want to make sure it's valid)
- [x] Auto-generate docs
- [x] Merge `Config` and `Configurations` into one `Config` context
- [x] Lock out UI fields for configurations when there is an ENV
override
- [x] Lock out corresponding REST API configuration field if overridden
via ENV var
- [x] Log a warning when deprecated legacy var is used
- [x] Document precedence: ENV -> Legacy ENV -> File -> DB
- [x] Change type to `inet[]` for `configurations.{default_client_dns,
default_client_allowed_ips}`, `devices.{dns, allowed_ips}`,
- [x] Drop `EctoNetwork` dep
- [x] `s/phoenix_port/phoenix_http_port` because it doesn't configure
HTTPS server
- [x] Do not load DB configs when config can be resolved from other
sources
Maybe:
- [ ] ~Auto-generate Ecto types to automatically cast/dump values
to/from DB~
- [ ] Allow JSON file config source
- [x] DB-related configs will not be validated?
Closes#1162Closes#1313Closes#1374Closes#1432
Regression was introduced in #1350: the path order did not allow auth
callback to be ever called.
Additionally, acceptance tests for SAML are added and we added a
blacklist of SAML config ids to prevent further route collisions.
Fixes#1362
This codepath was being skipped in the `test` env, which is no longer
necessary. This caused a runtime error that failed to show up in tests
because the codepath was being skipped.
Refs #1341
1. `auto_create_users` default value is removed. We want to avoid
situations when admins integrate OIDC/SAML providers and don't expect
anyone that has access to it to automatically gain access to VPN, which
is especially critical for providers like Google Workspace, where all
employees typically have access.
2. OpenID library was completely rewritten and a new version is
integrated. It will allow async tests and better scales for the cloud
version of the panel.
3. `Mox` was removed, we don't test modules by overriding them to
prevent breaking changes that tests can't capture.
4. Deps are reordered and unused ones are removed.
5. Browser/e2e tests are added to ensure we won't break UI features in
the future, allowing for front-end refactoring.
6. Users context was overhauled for better code clarity.
* Remove _build folders for umbrella apps
For umbrella apps everything goes into /_build directory so there no need to ignore directories that should never be created
* Change mix aliases to be more aligned with what OS community would expect
1. We want ecto.create and ecto.migrate to be run on each tests, this will simplify setup steps (no need to run migrations manually)
2. ecto.remigrate is not needed because now you can just run ecto.drop and on tests migrations would be executed anyways.
* Rename docker-compose step name in CONTRIBUTING.md
The step was renamed here: dd67baf629 (diff-67a4805fdcc6145d7b3ada2a6099a9b2e91c9d0fd108c22f95d2f01d219793d1R10)
* Remove .devcontainer
This an is opinionated change. Right now devcontainer doesn't work but should be easy to fix (with renaming step name), but at the same time it forces developers that use VS code to have unified development environment (including plugins for the editor itself).
I feel like it's not a good path to go for OS and for small team - everyone should be allowed to use setup they like. Especially for people like me that tend to recompile ls-elixir for Elixir plugin from master branch.
Plus it's yet another thing to maintain while nobody on the team is using it, which means it will be always causing issues.
* Make fz_http mix.exs aliases aligned with umbrella app ones
* Redirect stderr to stdout in a command called from dev.exs
Otherwise I'm getting this on my MacOS (that has a `route` implementation that doesn't show interfaces) when `mix phx.server` is executed:
```
usage: route [-dnqtv] command [[modifiers] args]
```
* Fix race condition due to static device field values
Both public_key and name are unique and we should not use static values for field covered by unique index, otherwise deadlocks and slow tests are expected.
* Remove unwanted transaction block
The changeset code doesn't have any code that accesses the database and individual Ecto.SQL commands are already wrapped in transactions by default, so there is no need to start it manually and hold for longer than expected (while irrelevant Elixir code is running).
* Use netstat to identify egress interface on MacOS
* Rename uninstall.sh to omnibus-uninstall.sh
* Fix uninstall path in omnibus_build.yml
* Test docker prod build in CI
* Need uses
* Set build-args
* Split builds
* Fix build-args format
* wtf
* hmmm
* Jeebus
* build version based on drafted release
* Build multi-platform images
* Only build for supported platforms
* Use newer OTP fingers crossed
* Use OTP 24 for arm64
* Finalize test build of Docker image
* Update comment
* Bump to OTP 25.1
* Use proper ver
* D'oh proper sha
* Use OTP 24 and install python for node build
* Use new Docker base
* Use python3
* Use newly-built base images with other platform support
* Don't build for ppc64le and s390x
* Build only for amd64/arm64
* Don't rebuild on publish; simply copy
* See where the image is being pushed
* Remove echo
* Add docker updates
* Match platforms from base image
* Use docker-compose over docker compose
* Use our own base
* we need python3
* use consistent service name
* trim trailing slash from external_url
* Build for latest tag in staging
* rename dockerfile
* use renamed dockerfile
* add Dockerfile.prod
* digest
* add publish image workflow
* add helper scripts
* start example docker compose
* fix workflow
* stage
* chmod
* stage
* nft
* must be root
* create path
* list address
* try alpine again
* remove nobody
* fix shared lib
* set wireguard endpoint
* config interface and route before starting server
* fix env var name
* move env var defaults into dockerfile
* persist private key
* migrate on start
* add create-or-reset-admin
* link env values
* clean up
* fix permission
* rename service
* add deploy
* persist pg data
* build version
* set version on build
* add gen_secrets
* fix typo
* move version to the end
* move up a little
* use map_join
* remove gen_secrets
* add gen-env
* cat
* gen more
* add telemetry
* remove cast role in create
* remove cast in update
this will fail
* Revert "remove cast role in create"
This reverts commit 217c62170f1f09987c6adbf7a5b3f467dd84034c.
* remove role in generic cast
* separate role and sign in token out from generic update
* handle empty value in put_password_hash
* separate last signed in at
* remove usage of removed changeset function
* fix clear sign in token
* improve tests
* fix tests
* split update user
* require password change
* fix test
* remove unused env var
* checkpoint
* Docker dev environment final root cause analysis
* Update CONTRIBUTING.md
* Update apps/fz_common/lib/fz_net.ex
Co-authored-by: Po Chen <chenpaul914@gmail.com>
Co-authored-by: Po Chen <chenpaul914@gmail.com>
