Since the user-mode-linux backend is flaky in trixie remove the runtime deps
from the docker container so that it cannot be used.
Signed-off-by: Christopher Obbard <christopher.obbard@linaro.org>
With new qemu-user-static from trixie we no longer need to register
the binfmts manually.
This reverts commit 306d2ea194.
Signed-off-by: Christopher Obbard <christopher.obbard@linaro.org>
Since Debian trixie has now been released; upgrade the debos container
to the new release.
Signed-off-by: Christopher Obbard <christopher.obbard@linaro.org>
To allow building in environments with TLS MITM proxies add a way to
expose the host cert store to the *build* container. This is needed for
for the copilot coding agent as that seems to run in such an environment
when the recommended firewall is enabled.
The runner images install a kernel, which pulls in initramfs-tools to
create an initramfs. For debos we don't need to have an initramfs, we
just need the kernel image. Generating this on the native
architecture (amd64) is reasonably fast. However in emulation, when
building for arm64, this is *very* slow.
In current github actions this change seems to save about 20 minutes per
docker build. For PR triggered runs this (which don't upload the
containers) this should save 20 minutes end-to-end (one build on the
critical path), while for runs that do it should save about 40 minutes
end to end (three builds, of which 2 on the critical path).
Convert Dockerfile so multiple target architecturebuilds are possible. Also
modify the CI pipeline, in order to built a single multi-platform container.
Note that at the current time-being docker cannot export multi-platform
containers without an intermediate registry. Thus only the amd64 image
is exported for the test stage. Nevertheless the arm64 image is built in
the pipeline and also published later to the registry.
Co-authored-by: Thomas Böhler <t.boehler@kunbus.com>
Signed-off-by: Nicolai Buchwitz <n.buchwitz@kunbus.com>
zstd compresses as well as gzip or xz (depending on compression ratio)
but has superiour performance; Add it to the docker image so it can be
used in recipes more easily.
Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
Currently rebuilding the arch keyring on Debian bookworm is broken as
the build depends on newer sq options. There is no reason to re-build
the keyring though, debian also uses a pre-build version to package it
for example.
Switch to use a script to download the latest release from arch gitlab
Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
I got a `permission denied` error when trying to use the KVM fakemachine backend on a host. The Docker documentation indicates that "the docker container process runs with the supplementary groups looked up for the specified user," but I suspect this means the groups _inside_ the container are looked up. I added the owning group of `/dev/kvm` (which happened to be `kvm`) to the container and all is well.
The arch linux tests require a newer version of archlinux-keyring than
what is in Debian stable; so revert back to pulling the latest version.
This partially reverts 36cf33366b.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
Update parent image to bookworm.
This allows us to remove all the backports and also to build Ubuntu
images again as it requires dpkg zstd support.
Signed-off-by: Klaus Goger <klaus.goger@theobroma-systems.com>
Nothing special here - a simple test.yaml akin to the existing ones.
The pacman.conf file is effectively vanilla config that comes with the
pacman package, while the mirrorlist file is a list of UK https mirrors
that I've been using for years.
Note that there is no canonical mirror for Arch. The top-level/tier 1
Arch server should _not_ be used as per the official recommendation.
Instead we use the mirrors set in the official arch docker tooling.
v2:
- Also install archlinux-keyring and makepkg
- Use arch + backend=qemu only testing matrix
v3:
- Use the same mirrors as the Arch docker images tooling
- Build and install pacman locally, until we get an official package
v4:
- Remove the local pacman build - it's available in backports
v5:
- Explicitly pull arch-install-scripts from backports, normal one lack
pacstrap
v6:
- Manually pull the latest keyring - Debian one is outdated
Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
The docker-compose test strategy was mainly done for integration with
dockerhub, but as dockerhub no longer builds our images it's not that
relevant.
What's more interesting is to use the docker container we've build to
run a selection of debos recipes, whose successfull build indicate
success. This both makes it easier to test locally as well (just run the
debos recipe) and makes the test jobs more specific.
On top of the existing test this also adds a "debian" test which does some basic
debian smoketesting (debootstrap and apt) as well as a basic
"partitioning" test. The partitioning test unforutunately doesn't work
in the nofakemachine run as udev isn't available in the container, so
that only runs on UML and Qemu based backends.
Current the kvm backend isn't tested because the standard github action
runners don't support kvm. But qemu, though being lots slower, covers
some part of it.
Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
This is a continuation of work done by @eds-collabora in !275
This replaces the old, simpler pipeline with a three phase process:
- First, build the image and cache it using docker buildx.
- Second, run all the tests in parallel, restoring the image from the cache.
- Thirdly, if the tests pass:
- if this is a push to the main branch, push to DockerHub.
- push to GitHub Container registry (PS: will push to a user's own fork).
This uses Buildkit caching aggressively, and will make use of the entire 5GiB allocation of cache space that GitHub provides over time.
It requires the following additional repository secrets:
- DOCKERHUB_USERNAME: the username to login as on DockerHub (e.g. go-debos)
- DOCKERHUB_PASSWORD: an access token for the DockerHub repository.
Closes: #275
Based on original work by: Ed Smith <ed.smith@collabora.com>
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
Since we are now building using go modules, let's remove
the hacks for various dependencies and build the docker
container using the go modules.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
Running the tests on the host is a good first step; to test Debos
properly we should run inside a Fakemachine. Since GitHub actions
do not support creating nested virtual machines, use the
user-mode-linux backend in Fakemachine to create a user process
to run the tests inside of.
Since Docker autobuild does not support UML the docker-compose
file purposely does not have a suffix of `.test.yml` so that the
test will not be picked up to run on Docker autobuild.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
Currently we assume all of the tests are run without fakemachine;
since we are looking to run the tests with fakemachine as well,
let's allow arguments to be passed to the test script which are
then passed to debos.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
Currently only the unit-tests under the actions directory are ran. This
patch runs all of the available unit tests in the project.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
This patch adds support for XFS, including the ability
to specify the UUID through the fsuuid property of the
image-partition action.
Signed-off-by: Nguyen Thi Huong <huong4.nguyenthi@toshiba.co.jp>
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
Debian's qemu-user-static package no longer registers binfmts on postinst
when running inside a virtualmachine; dockerhub builds are now built inside
a vm so the binfmts are not generated inside the docker container.
Fixes: 91af617bea ("docker: Install qemu-user-static 6.0 to fix segfault")
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
There are issues with qemu-user-static 5.2 crashing when attempting
to allocate guest memory when compiled as a proper statically-linked
binary.
From testing, qemu 6.0 fixes the bug but it's not yet clear which
patch fixes the bug. So until the correct patch is backported to
bullseye, let's install qemu from experimental to pickup the bugfix.
See: #245
See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=988174
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
Since we have the support for the new user-mode-linux backend in the
docker container, Debos defaults to attempting to use that.
Unfortunately that isn't possible to run inside the Docker Hub where the
test suite runs, so let's explicitly disable the fakemachine library
when running the Docker Hub test suite.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
The fakemachine uml backend uses user-mode-linux and libslirp-helper
packages available from bullseye, let's include those inside the container.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
The equivs package is used to create dummy Debian packages which can
be useful in development to satisfy dependencies without installing the
real package. Note that this package is not the recommended way of
dealing with broken dependencies: a bug report should be filed instead.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
Testify is currently not built unless the manual tests are ran at which
point testify compiliation fails due to requiring a newer version of
golang than in buster. Unfortunately the main branch is no longer
compatible, so install from source to get the unit tests running again.
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
Debos currently sets the partition type to filesystem type and
does not allow the partition type to be set from the recipe.
In some situations setting the partition type is required, so add the
property `PartType` to the Partition and set the partition type to the value
of that property. If the `PartType` property is unset, the original partition
type is retained thus not breaking backwards compatibility.
For msdos, the partition type should be hexadecimal and 2-characters long.
For gpt, the partition GUID should be in GUID format and 36-characters long.
Some examples and further reading is included in the documentation.
Resolves: #98
Signed-off-by: Christopher Obbard <chris.obbard@collabora.com>
Add the u-boot-tools package to be able to use binaries such as
mkimage in debos' recipes.
Signed-off-by: Mylène Josserand <mylene.josserand@collabora.com>
Move simple test to simple dir.
Add test with recipe included from sub-directory.
Add test with recipe included from directory outside of main recipe path.
Add script to run tests in sequence.
Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
The usage of the container as an executable does not seems to be
straightforward.
Add a warning in docker/README.md and add container installation reference
in main README.md.
Signed-off-by: Frédéric Danis <frederic.danis@collabora.com>
