Skip to content

Antalya 26.8: Iceberg: reuse the Puffin object metadata across deletion-vector reads - #2665

Merged
zvonand merged 1 commit into
antalya-26.8from
feature/antalya-26.8/pr-2419
Oct 25, 2026
Merged

zvonand merged 1 commit into
antalya-26.8from
feature/antalya-26.8/pr-2419

Conversation

@zvonand

@zvonand zvonand commented Oct 24, 2026

Copy link
Copy Markdown
Member

Changelog category (leave one):

  • Performance Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Reduce number of Puffin file HEAD requests (#2419 by @UnamedRus).

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Performance tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All with Aarch64
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

Cherry-picked from #2419.


Documentation entry for user-facing changes

...

Iceberg: reuse the Puffin object metadata across deletion-vector reads
@zvonand zvonand added releasy Created/managed by RelEasy antalya-26.8 Session label (releasy session config) forwardport This is a frontport of code that existed in previous Antalya versions labels Oct 24, 2026
@github-actions

Copy link
Copy Markdown

Workflow [PR], commit [8546561]

@zvonand zvonand mentioned this pull request Oct 24, 2026
37 tasks done
@zvonand

zvonand commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

RelEasy analyze-fails — UNRELATED

run completed at 2026-10-25T09:47:16Z

  • Head SHA: 85465612b3 (feature/antalya-26.8/pr-2419)
  • Failed CI checks: 2
  • Tests considered: 4 across 2 CI shard(s)
  • Baseline (cdbcf6d5f8, 2026-10-23T21:17:55Z, before this PR): 4 of 4 failure(s) were already red there
  • Outcomes: 0 done · 0 partial · 2 unrelated · 0 unresolved
  • Second opinion: no shard was in doubt (nothing committed, and the evidence backs each verdict).
  • Commits added by AI: 0 (—)
  • Anthropic cost: —
  • Checks covered by another shard (same failures, investigated once):
    • Regression release swarms: 3 of 4 failure(s) duplicate Regression aarch64 swarms; the remaining 1 form this shard.

Per-shard outcomes

⏭️ UNRELATED — Regression aarch64 swarms

3 failed test(s) considered — cost $0.2736
full report

AI narration

This PR didn't cause any of the 3 failures in this shard. All three were already failing on antalya-26.8 at baseline commit cdbcf6d5f8 (2026-10-23), which doesn't contain this PR's change. I didn't change any code, run a build, or reproduce anything.

I couldn't do the spot-check against the reports: fetching fails.log.txt from S3 needed approval and was denied. The baseline verdict therefore rests on the annotations that came with the task, plus a check of the diff. The PR being tested (#2419, puffin-dv-metadata-threading) threads Iceberg puffin deletion-vector metadata. That has nothing to do with swarm cluster discovery or how nodes behave when a CPU is overloaded.

  • [unrelated] /swarms/feature/cluster discovery/check cluster discovery with wrong cluster name — already failing at baseline cdbcf6d5f8 (Regression release swarms).
  • [unrelated] /swarms/feature/node failure/cpu overload — already failing at baseline cdbcf6d5f8; also fails in the Regression release swarms shard.
  • [unrelated] /swarms/feature/node failure/cpu overload all swarm nodes — already failing at baseline cdbcf6d5f8; also fails in the Regression release swarms shard.

git status --porcelain is not empty. It lists three untracked .releasy-discover-deps-*/ directories, which were already there when I started. I didn't create them and left them in place.

UNRELATED
This PR didn't cause any of the 3 failures in this shard. All three were already failing on antalya-26.8 at baseline commit cdbcf6d5f8 (2026-10-23), which doesn't contain this PR's change. I didn't change any code, run a build, or reproduce anything.

I couldn't do the spot-check against the reports: fetching fails.log.txt from S3 needed approval and was denied. The baseline verdict therefore rests on the annotations that came with the task, plus a check of the diff. The PR being tested (#2419, puffin-dv-metadata-threading) threads Iceberg puffin deletion-vector metadata. That has nothing to do with swarm cluster discovery or how nodes behave when a CPU is overloaded.

  • [unrelated] /swarms/feature/cluster discovery/check cluster discovery with wrong cluster name — already failing at baseline cdbcf6d5f8 (Regression release swarms).
  • [unrelated] /swarms/feature/node failure/cpu overload — already failing at baseline cdbcf6d5f8; also fails in the Regression release swarms shard.
  • [unrelated] /swarms/feature/node failure/cpu overload all swarm nodes — already failing at baseline cdbcf6d5f8; also fails in the Regression release swarms shard.

git status --porcelain is not empty. It lists three untracked .releasy-discover-deps-*/ directories, which were already there when I started. I didn't create them and left them in place.

UNRELATED

⏭️ UNRELATED — Regression release swarms

1 failed test(s) considered — cost $0.2289
full report

AI narration

The only failure in this shard was already failing before this PR, so I changed no code and committed nothing.

Triage:

  • Baseline: the failing test was already red on antalya-26.8 at commit cdbcf6d5f8 (2026-10-23T21:17:55Z), in the same Regression release swarms shard, before this PR's diff existed. That rules this PR out as the cause.
  • Spot-check not done: I tried to download this run's fails.log.txt to compare it against the baseline, but the download needed approval and didn't run. I also didn't open the baseline report, so I'm relying on the baseline verdict I was given.
  • Diff: the PR is a large rebase-style diff (CI workflows, contrib bumps, Poco SSL and more). I didn't find anything in it specific to swarm node-restart handling, but I only looked at the file list.
  • Not done: I didn't build or run the regression suite, because the baseline already settles it.

git status --porcelain isn't empty, but nothing in it is mine. It lists three untracked .releasy-discover-deps-*/ directories (5b7950, 7b21af, ff3f2d) that were already there when the session started. I left them alone.

Failing tests:

  • /swarms/feature/node failure/check restart swarm node — [unrelated]: already failing at baseline commit cdbcf6d5f8 on antalya-26.8, without this PR's changes. The CI excerpt only says AssertionError.

UNRELATED
The only failure in this shard was already failing before this PR, so I changed no code and committed nothing.

Triage:

  • Baseline: the failing test was already red on antalya-26.8 at commit cdbcf6d5f8 (2026-10-23T21:17:55Z), in the same Regression release swarms shard, before this PR's diff existed. That rules this PR out as the cause.
  • Spot-check not done: I tried to download this run's fails.log.txt to compare it against the baseline, but the download needed approval and didn't run. I also didn't open the baseline report, so I'm relying on the baseline verdict I was given.
  • Diff: the PR is a large rebase-style diff (CI workflows, contrib bumps, Poco SSL and more). I didn't find anything in it specific to swarm node-restart handling, but I only looked at the file list.
  • Not done: I didn't build or run the regression suite, because the baseline already settles it.

git status --porcelain isn't empty, but nothing in it is mine. It lists three untracked .releasy-discover-deps-*/ directories (5b7950, 7b21af, ff3f2d) that were already there when the session started. I left them alone.

Failing tests:

  • /swarms/feature/node failure/check restart swarm node — [unrelated]: already failing at baseline commit cdbcf6d5f8 on antalya-26.8, without this PR's changes. The CI excerpt only says AssertionError.

UNRELATED


🤖 Posted automatically by releasy analyze-fails. Re-run the command to refresh.

@zvonand

zvonand commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

@blau-ai

@blau-ai

blau-ai commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator

CI triage for #2665

Verdict: 1 failing check family, 0 PR-caused. The only red is the Regression swarms suite (aarch64 + release). Every build, Fast test, stateless/stateful, integration, stress, AST fuzzer, SQLLogic, compatibility and install-packages check is green. The swarm failures are pre-existing / flaky on antalya-26.8 and unrelated to this change.


What's failing

Regression aarch64 swarms / Regression release swarms (+ the matching RegressionTests*/Swarms/swarms GH jobs). Failing leaf scenarios:

Scenario arm x86
cluster discovery/check cluster discovery with wrong cluster name ✗ ✗
node failure/cpu overload ✗ ✗
node failure/cpu overload all swarm nodes ✗ ✗
node failure/check restart swarm node — ✗

All are AssertionError on query exit codes inside swarm cluster-management scenarios (cluster discovery, node restart under load, CPU overload) from Altinity/clickhouse-regression — e.g. assert result.exitcode in expected_exitcodes in swarms/tests/node_failure.py:108.

Why this is not caused by the PR

  • Scope mismatch. This PR changes exactly one file — src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergDeletionVector.cpp — threading an optional ObjectMetadata into the Puffin read path to avoid redundant S3 HEADs on deletion-vector loads. There is no path by which that touches swarm cluster discovery, node-failure handling, or CPU-overload behavior. The Iceberg-related swarm scenarios (iceberg fallback…, iceberg engine table fallback) are green/skipped.
  • Same failures on unrelated PRs. The identical scenarios (wrong cluster name, cpu overload, cpu overload all swarm nodes) fail on:
  • Arch-dependent flakiness. 3 fail on aarch64 vs 4 on x86, differing by one scenario (check restart swarm node) — a genuine regression would fail deterministically, not drift by arch/run. These are timing/load-sensitive cluster tests.

Suggested next step

  • No PR-side fix is needed or possible here — the swarm suite is red branch-wide independent of PR content.
  • Safe to re-run the swarm jobs if you want a cleaner board, but don't expect them to reliably go green until the underlying flakiness on antalya-26.8 is addressed (tracked at the suite/infra level, not in this PR). It should not block merge.

PR health check

A small, well-scoped optimization: the Puffin ObjectMetadata is now seeded into RelativePathWithMetadata so createReadBuffer skips its per-buffer getObjectMetadata (S3 HEAD) when the caller already has it — paying one HEAD per deletion-vector load instead of one per buffer, with std::nullopt passed on the paths that don't have the metadata yet (behavior-preserving). All functional CI is green; the change carries no new tests, which is fine for a pure request-count optimization with no observable semantic change.

🤖 Automated triage by @blau-ai. CI reports read from the praktika S3 bucket; swarm comparisons against #2661/#2666.

@zvonand
zvonand merged commit f34df52 into antalya-26.8 Oct 25, 2026
323 of 327 checks passed
@zvonand zvonand added the verified Approved for release label Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

antalya-26.8 Session label (releasy session config) forwardport This is a frontport of code that existed in previous Antalya versions releasy Created/managed by RelEasy verified Approved for release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants