Skip to content

Use gzip tar layers for bundle OCI images - #490

Merged
rahultee merged 2 commits into
mainfrom
gzip-oci-layer
Oct 15, 2026
Merged

rahultee merged 2 commits into
mainfrom
gzip-oci-layer

Conversation

@rahultee

@rahultee rahultee commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Why

Compressing files inside EROFS prevents those files from using DAX. Use gzip for OCI transport compression instead, so an image unpacker can create an uncompressed EROFS filesystem.

What changed

  • Build one application/vnd.oci.image.layer.v1.tar+gzip layer instead of a native compressed EROFS layer.
  • Keep the complete store closure, module metadata, file modes, symlinks and hardlinks. Include extended attributes in the tar stream.
  • Set stable tar entry order, timestamps, numeric ownership and extended headers. Use gzip level 9 without a timestamp or source filename.
  • Record the compressed blob digest and size in the manifest. Record the uncompressed tar digest in rootfs.diff_ids.
  • Update the fixture tests and documentation. Existing disk and per-module image builders do not change.

Test plan

  • Passed locally: scripts/ci_check.sh (Nix formatting, deprecated shell-helper check, all module derivation evaluation, dev-shell smoke test, bundle evaluation and nix build .#bundle-oci-check).
  • The level-9 gzip fixture verified compressed OCI descriptors, uncompressed diffID, exact repeatability, complete closure and metadata, contents, modes, numeric ownership, symlinks, hardlinks, long paths and digest-preserving Skopeo copy.
  • Streamed the actual gzip-built fixture through mkfs.erofs --tar=f --aufs --quiet -Enoinline_data. Integrity and extraction checks passed; closure contents, modes, symlinks, metadata and hardlinks were preserved.
  • Mounted the converted fixture over virtio-pmem in an isolated VM with a DAX-enabled Linux kernel. Both small and large fixture files reported STATX_ATTR_DAX. Compressed negative controls did not, even with the same DAX mount option.
  • The xattr round-trip skips only ENOTSUP on filesystems without xattrs. The level-9 fixture also passed archive-header and extraction checks on an xattr-capable local filesystem.
  • GitHub Actions checks and security scanners passed for the current commit; policy review is pending.
  • Not tested: full bundle producer build or full-image VM startup.

Rollout

This PR does not publish or deploy an image. A separate publisher compatibility update is required before publishing this output: the validator must accept gzip-tar as well as existing native EROFS images while keeping provenance checks.

Before using a new full bundle, check available unpacked disk space and smoke-test target VM startup. The small fixture test does not validate a complete bundle or every consumer configuration. The tar-to-EROFS unpack path must leave filesystem compression disabled.

The expanded filesystem uses more disk space than compressed EROFS. Revert this PR and rebuild to restore native compressed EROFS output. Existing image digests are unchanged.

  • This is fully backward and forward compatible

~ written by ⠕ Replit

@rahultee
rahultee marked this pull request as ready for review October 14, 2026 23:55
@rahultee
rahultee requested a review from a team as a code owner October 14, 2026 23:55
@rahultee
rahultee requested review from vlinkz and removed request for a team October 14, 2026 23:55
@rahultee
rahultee merged commit 7e9ace9 into main Oct 15, 2026
8 checks passed
@rahultee
rahultee deleted the gzip-oci-layer branch October 15, 2026 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants