198 Commits

Author SHA1 Message Date
Sjoerd Simons
4f6c82a5a0 Switch yaml parsing to goccy/go-yaml
go-yaml/yaml has been deprecated by its maintainer; Move to goccy/go-yaml, which is
an actively maintained from a scratch yaml implementation in pure go. It
should able to parse all yaml file go-yaml could, and more. As an extra
benefit its errors are a lot better.

Fixes: #592
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2025-11-01 22:04:02 +01:00
copilot-swe-agent[bot]
6d347a63b1 Make origin property optional for raw action, default to recipe directory
Co-authored-by: sjoerdsimons <22603932+sjoerdsimons@users.noreply.github.com>
2025-11-01 16:04:11 +01:00
Sjoerd Simons
b59af4ed28 actions/mmdebstrap_action.go: persist cleaned keyring paths
Let verify (indirectly) save the cleaned keyring paths back to the
action arguments so those are used for the Run as well

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2025-10-26 10:47:46 +01:00
copilot-swe-agent[bot]
f566e04888 chore: enable more linters and fix all critical issues
Enabled additional linters from fakemachine configuration:
- errorlint: Error wrapping with %w
- misspell: Spelling checks
- revive: Code quality checks
- whitespace: Formatting checks

Fixed all issues including:
- Error handling: Added proper error checks for all function returns
- Error wrapping: Changed %v to %w for proper error wrapping
- Type assertions: Used errors.As instead of direct type assertions
- Unused parameters: Renamed to underscore where appropriate
- Variable naming: Fixed ALL_CAPS constants and underscored names
- Whitespace: Removed unnecessary leading/trailing newlines
- Code flow: Removed unnecessary else blocks

Renamed types (breaking internal API changes):
- DebosState → State
- DebosContext → Context
- DownloadHttpUrl → DownloadHTTPURL

Fixed struct field naming with proper YAML tags:
- Url → URL (with yaml:"url" tag)
- TlsClientCertPath → TLSClientCertPath (kept yaml:"tls-client-cert-path")
- TlsClientKeyPath → TLSClientKeyPath (kept yaml:"tls-client-key-path")
- validateUrl → validateURL method

Co-authored-by: sjoerdsimons <22603932+sjoerdsimons@users.noreply.github.com>
2025-10-01 21:21:22 +02:00
Sjoerd Simons
226b78e20d Merge pull request #585 from lool/sha256sum
feat: Optional sha256sum for download action
2025-09-11 19:04:58 +00:00
Loïc Minier
9e3fd35d3f ci: Enable govet linter in golangci.yml
Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 13:44:23 +00:00
Loïc Minier
f3b4ced5ee fix: struct literal uses unkeyed fields
Reported by go vet

Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 13:22:57 +00:00
Loïc Minier
2a40c09840 fix: struct field tag not compatible with reflect.StructTag.Get
Reported by go vet

Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 13:07:26 +00:00
Loïc Minier
eba3cb0369 feat: sha256sum for download action
Unconditionally log SHA256 sum of downloads and add an optional
sha256sum property. Setting this property will ensure verification of
the downloaded file and delete it on mismatch.

Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 12:41:15 +00:00
Loïc Minier
fe573bbeac fix: Omit redundant type from variable declaration (QF1011)
Reported by staticcheck

Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 09:24:07 +02:00
Loïc Minier
ba97245068 fix: Omit redundant nil check on slices, maps, and channels (S1009)
Reported by staticcheck

Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 09:24:07 +02:00
Loïc Minier
4b751ec129 fix: Break statement with no effect (SA4011)
Reported by staticcheck

Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 09:24:07 +02:00
Loïc Minier
e0fe2786cf fix: Deferring Close before checking for a possible error (SA5001)
Reported by staticcheck

Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 09:24:07 +02:00
Loïc Minier
38ffb9dd28 fix: Using a deprecated function, variable, constant or field (SA1019)
Reported by staticcheck

Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 09:24:07 +02:00
Loïc Minier
16ab444c53 fix: Omit comparison with boolean constant (S1002)
Reported by staticcheck

Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 09:24:07 +02:00
Loïc Minier
be75ca645e fix: Unnecessary use of fmt.Sprintf (S1039)
Reported by staticcheck

Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 09:24:07 +02:00
Loïc Minier
ae9d5574cc fix: Incorrectly formatted error string (ST1005)
Reported by staticcheck

Signed-off-by: Loïc Minier <loic.minier@oss.qualcomm.com>
2025-09-05 09:24:07 +02:00
Christopher Obbard
158ed7c09a chore: run go fmt
Signed-off-by: Christopher Obbard <christopher.obbard@linaro.org>
2025-08-11 14:33:38 +01:00
Christopher Obbard
76735871cd Merge pull request #560 from daissi/wip/daissi/more-compress-options
Add more compress options
2025-08-08 08:04:08 +00:00
Dylan Aïssi
b950155e20 actions: make run action fail if a command fails
Passing -e to sh makes the action to fail immediately instead of
continuing to the next commands.

Fixes: #290

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>
2025-08-06 11:38:40 +02:00
rektide de la faye
d7135c3787 add lzop/zlma/lzip/zstd unpack compress options.
Signed-off-by: rektide de la faye <rektide@voodoowarez.com>
Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>
2025-08-06 10:02:02 +02:00
rektide de la faye
a494e5e218 add auto/zstd/lzop/lzma/lzip compression options to pack action.
Signed-off-by: rektide de la faye <rektide@voodoowarez.com>
Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>
2025-08-06 10:02:02 +02:00
Dylan Aïssi
63e05fe1ea partitioning: partattrs: Convert bit numbers to bit names to workaround a sfdisk limitation
sfdisk supports only bit names (and not bit numbers) for bits 0 to 2.
See https://github.com/util-linux/util-linux/issues/3353

Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>
2025-06-19 12:31:22 +02:00
Nícolas F. R. A. Prado
d0b9c0435a partitioning: Add partattrs property to partition
Add a 'partattrs' property to partitions. It allows the GPT partition
attribute bits to be set for a partition. Notably this is needed to set
bits 48 and 56, which are part of the GUID specific range, to allow a
ChromeOS Kernel partition to be booted on Chromebooks.

Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com>
2025-06-19 12:31:22 +02:00
Julien Massot
e470bd1be4 actions: pack/unpack: Add support for zstd compression
Debian Bookworm includes a version of tar with native support
for Zstandard compression.
Add mapping to enable zstd in pack and unpack operations.

Signed-off-by: Julien Massot <julien.massot@collabora.com>
2025-04-28 15:59:03 +02:00
Christopher Obbard
c0e1e5adfd apt: clarify default value for parameters
Signed-off-by: Christopher Obbard <christopher.obbard@linaro.org>
2025-03-19 15:47:14 +00:00
Sjoerd Simons
ba4449b7b4 Merge pull request #536 from julien-massot/main
Add support for alternate sector size
2025-03-04 10:38:37 +00:00
Sjoerd Simons
e3e2e4982b Merge pull request #543 from sjoerdsimons/shellescape-mod-path
Adjust Shellescape mod path
2025-03-04 06:54:08 +00:00
Julien Massot
840b1df5c2 Add support for alternate sectorsize
Newer storage, such as UFS or NVMe disk, may not support the
default 512 sector size.

For these devices, the partition table and the partition offset
need to be aligned with sector size boundaries. The GPT partition
table is 34 sectors in length, so we can't have compatible disk images
between 512 and 4096 sector-size devices.

This patch introduces a major breakage with the {{ sector 64 }}
template.
At template processing, we can't guess the appropriate sector size
for the image since this value can be defined as a YAML header.
Instead of multiplying the sector number by the default 512 bytes
boundary, append a 's' suffix. This defers the work of computing
the value to the different Actions.
The 's' suffix is chosen to be compatible with parted (ImagePartition)
and support has been added to raw action.

Signed-off-by: Julien Massot <julien.massot@collabora.com>
2025-03-04 07:16:15 +01:00
Sjoerd Simons
3e0f842ca5 Adjust shellescape go module path
The shellescape module renamed itself to al.essio.dev/pkg/shellescape.
Adjust imports to match
2025-03-04 07:14:49 +01:00
Nobuhiro Iwamatsu
dc4ce466a5 actions: mmdebstrap: Remove single quotation from append strings of dpkgopt and aptopt
Remove single quotation (') from the append strings of dpkgopt and aptopt.
The mmdebstrap process fails because the format contains single quotation.

Debos yaml file:
```
architecture: amd64

actions:
  - action: mmdebstrap
    suite: sid
    variant: minbase
    merged-usr: true
    dpkg-opts:
      - path-exclude=/usr/share/man/*
```

Error message:
```
[...]
2025/03/03 06:59:12 Mmdebstrap | dpkg: error: configuration error: /etc/dpkg/dpkg.cfg.d/99mmdebstrap:1: unknown option ''
[...]
```

From debug message of mmdebstrap:
```
[...]
D: 1 3180 content of /etc/dpkg/dpkg.cfg.d/99mmdebstrap:
'path-exclude=/usr/share/man/*' <--here
apt 2.9.30 (amd64)
Supported modules:
*Ver: Standard .deb
[...]
```

Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
2025-03-03 13:18:51 +01:00
Nicolai Buchwitz
9ed961408c actions: mmdebstrap: Add aptopt option
Allow to set arbitrary apt options via yaml keyword `apt-opts`. One
particular use case in to create minimal embedded image where things
like recommendations are not needed. These can be omitted during
setup with the apt options `APT::Install-Recommends` and
`APT::Install-Suggests` to "false. All other apt options do also work.

As stated in the mmdebstrap man page [1] these options are persisted in
/etc/apt/apt.conf.d/99mmdebstrap, so further calls to apt will respect
these options too.

[1] https://manpages.debian.org/testing/mmdebstrap/mmdebstrap.1.en.html#aptopt

Signed-off-by: Nicolai Buchwitz <nb+github@tipi-net.de>
2025-02-23 23:32:28 +01:00
Nicolai Buchwitz
75978c4478 actions: mmdebstrap: Add dpkgopt option
Allow to set arbitrary dpkg options via yaml keyword `dpkg-opts`. One
particular use case in to create minimal embedded image where things
like docs and man pages are not needed. These can be omitted during
setup with the dpkg option `path-exclude`. All other dpkg options do
also work.

As stated in the mmdebstrap man page [1] these options are persisted in
/etc/dpkg/dpkg.cfg.d/99mmdebstrap, so further calls to dpkg will respect
these options too.

[1] https://manpages.debian.org/testing/mmdebstrap/mmdebstrap.1.en.html#dpkgopt

Signed-off-by: Nicolai Buchwitz <nb+github@tipi-net.de>
2025-02-23 23:32:28 +01:00
Nobuhiro Iwamatsu
6a56ddbf6d action: mmdebstrap: Add include option
Add include option for mmdebstrap.
The include option of mmdebstrap specifies the packages to be installed during
bootstrap image.

Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
2025-02-23 23:32:28 +01:00
Nicolai Buchwitz
b58fb75f5e actions: Add mmdebstrap action
Add mmdebstrap [1] as a faster and more flexible alternative to debootstrap.

[1] https://gitlab.mister-muffin.de/josch/mmdebstrap

Signed-off-by: Nicolai Buchwitz <nb+github@tipi-net.de>
2025-02-23 23:32:17 +01:00
Vignesh Raman
2b9ada8318 actions/image-partition: truncate filesystem label to maximum supported length
Added optional fslabel property to partition to allow truncation of filesystem
label and allow user to modify filesystem label without modifying the name.
Filesystem label defaults to the name property of the partition.

The filesystem label can be up to 11 characters long for {v}fat{12|16|32},
16 characters long for ext2/3/4, 255 characters long for btrfs, 512 characters
long for hfs/hfsplus and 12 characters long for xfs.

Fixes: #251

Suggested-by: Christopher Obbard <chris.obbard@collabora.com>
Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Signed-off-by: Dylan Aïssi <dylan.aissi@collabora.com>
2025-01-28 16:19:24 +01:00
Christopher Obbard
260c01fd64 actions: filesystem-deploy: Return error rather than fatal log
We should be consistent and return an error to bubble up rather
than logging the error.

Signed-off-by: Christopher Obbard <christopher.obbard@linaro.org>
2024-12-17 16:04:08 +00:00
Christopher Obbard
8dc9cbce60 actions: filesystem-deploy: Defer Close() call on file handles
Make use of defer when closing file handles.

Signed-off-by: Christopher Obbard <christopher.obbard@linaro.org>
2024-12-17 16:03:02 +00:00
Christopher Obbard
0159549e44 actions: filesystem-deploy: Be consistent with paths in log
The paths used aren't consistent in the log, e.g.:

    2024/12/17 15:56:10 ==== Deploy onto image ====
    2024/12/17 15:56:21 Setting up fstab
    2024/12/17 15:56:21 Setting up /etc/kernel/cmdline

For all log and error messages, use the full path to fstab and cmdline.

Signed-off-by: Christopher Obbard <christopher.obbard@linaro.org>
2024-12-17 16:01:16 +00:00
Jakob Unterwurzacher
db6b23c20a ImagePartitionAction: return specific error on failure
go-losetup v2 now returns a meaningful error. Add it to the returned
error message.
2024-12-02 11:36:22 +01:00
Jakob Unterwurzacher
0bc5983e8a ImagePartitionAction: retry losetup.Attach()
losetup.Attach() can fail due to concurrent attaches in other processes
as seen in https://github.com/go-debos/debos/issues/522 .

The problem is a race condition between finding a free loop device
and attaching the image.

Now that we have go-losetup v2, which does report the error, we can do
what util-linux does
( 4c4b248c68/sys-utils/losetup.c (L662) )
and retry on failure.

I only sleep for 200 ms as opposed to 1 second as in
78aad24dc0/actions/image_partition_action.go (L668)
because the race condition should immediately resolve without waiting
at all.

I still sleep for 200 ms as this is what util-linux does to
prevent spinning ( 3ff6fb802d ).

Fixes: https://github.com/go-debos/debos/issues/522
2024-12-02 11:06:12 +01:00
Jakob Unterwurzacher
958e68dbe6 deps: bump go-losetup to latest v2 release
We were stuck in 2017 (v1.0.0-20170407175016-fc9adea44124).

Related: https://github.com/go-debos/debos/issues/522
2024-11-19 16:56:35 +01:00
Christopher Obbard
227cf70102 Merge pull request #501 from azayet01/fix-fstab-mount
actions: fix fstab generated in case of FAT{12|16|32}.
2024-05-21 10:15:24 +00:00
Aymen Zayet
de367146a4 actions: fix fstab generated in case of FAT{12|16|32}.
Recent changes introduced a tuned formatting of the partitions created
with FAT{12|16|32}. The issue is that fstab is generated using the fs
property which won't match the system expectations.

If the fs property is "fat", "fat12", "fat16", "fat32" or "msdos", then
fstab must use "vfat".

Signed-off-by: Aymen Zayet <aymen.zayet@flex.ai>
2024-05-21 11:59:56 +02:00
Christopher Obbard
d00c9f0de5 Merge pull request #499 from azayet01/fix-mount-fat-parts
actions: Fix regression introduced with FAT{12|16|32} support.
2024-05-20 20:13:50 +00:00
Christopher Obbard
09629ab9cc Merge pull request #496 from fooishbar/no-silent-yaml-parse-fail
Don't ignore YAML parsing errors
2024-05-20 20:04:21 +00:00
Aymen Zayet
84c4e58a1d actions: Fix regression introduced with FAT{12|16|32} support.
The regression was introduced with the following commit :
  commit 94fedb2c0c
  Author: Christopher Obbard <chris.obbard@collabora.com>
  Date:   Tue Mar 19 17:43:42 2024 +0000

  actions: image-partition: enable creation of FAT{12|16|32} partitions

This one added more options when it comes to creating FAT partitions.
So when partition fs is defined as "fat", "fat12", "fat16", "fat32",
"msdos" or "vfat", then mkfs.vfat is used to create the partition,
and different options were used depending on the FAT type.

The main issue is that  mounting a FAT partition should  use "vfat"
 as fs type when using syscall.Mount().

So, in order to fix this issue, "vfat" is simply used to mount "fat",
"fat12", "fat16", "fat32" or "msdos" partitions.

Signed-off-by: Aymen Zayet <aymen.zayet@flex.ai>
2024-05-19 14:33:11 +02:00
Daniel Stone
ecbea7a290 download: Actually validate empty pathnames
From the path module doc:
    If the path is empty, Base returns ".". If the path consists
    entirely of slashes, Base returns "/".

Actually validate that we provide a filename when one is required,
rather than failing with an error that /scratch exists but is not a
file.

Signed-off-by: Daniel Stone <daniels@collabora.com>
2024-05-13 15:48:12 +01:00
Daniel Stone
27da56fa82 recipe: Add test for type mismatch during recipe parse
Check that debos more obviously errors out when you do what I tried to
do.

Signed-off-by: Daniel Stone <daniels@collabora.com>
2024-05-01 13:26:04 +01:00
Daniel Stone
7c80d18ed3 actions: Don't ignore YAML parsing errors
When we're unpacking individual actions, pass YAML parsing errors up to
the caller instead of silently ignoring them.

Signed-off-by: Daniel Stone <daniels@collabora.com>
2024-05-01 13:20:20 +01:00