Skip to content

Antalya 26.8: Backport of #117106 support dynamic type in parquet reader - #2666

Merged
zvonand merged 2 commits into
antalya-26.8from
backports/antalya-26.8/117106
Oct 26, 2026
Merged

zvonand merged 2 commits into
antalya-26.8from
backports/antalya-26.8/117106

Conversation

@subkanthi

@subkanthi subkanthi commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

This backport is a pre-requisite to support variant datatype.(ClickHouse#117106 by @scanhex12)

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

upport reading the Parquet VARIANT type (unshredded, as written for example by Spark 4.0) into the Dynamic type, with objects read as JSON. A variant column can also be read directly as JSON if all its values are objects. Shredded variants are not supported yet. (ClickHouse#117106 by @scanhex12)

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Unit tests
  • Performance tests
  • Aarch64 tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • CAS (content-addressed storage; Antalya only)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • OAuth (5m)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

@subkanthi
subkanthi marked this pull request as ready for review October 24, 2026 20:05
@github-actions

Copy link
Copy Markdown

Workflow [PR], commit [c0b8b0c]

@subkanthi subkanthi added antalya antalya-26.8 Session label (releasy session config) backport Backport port-antalya PRs to be ported to all new Antalya releases labels Oct 24, 2026
@subkanthi

Copy link
Copy Markdown
Collaborator Author

@blau-ai

@blau-ai

blau-ai commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator

CI triage — #2666 (backport of ClickHouse#117106, dynamic type in Parquet reader)

Verdict: 3 failing checks, 0 PR-caused. All failures are in unrelated subsystems (Iceberg+Spark export, OAuth regression, swarm clusters) — none touch Parquet read code. Every Parquet-relevant check is green, including the new stateless tests this PR adds.

Why I'm confident it's not this PR

This PR only changes Parquet format reading (src/Processors/Formats/Impl/Parquet/*, ParquetV3BlockInputFormat.cpp, FormatFactory*, FormatSettings.h, SettingsChangesHistory.cpp) and adds 0492x/0522x/0529x stateless Parquet-variant tests. The green checks that do exercise that code:

  • ✅ Stateless tests (amd_debug, parallel) — where the new 04928…05307_parquet_variant_* tests ran
  • ✅ Stateless tests (amd_asan_ubsan, distributed plan, parallel), Fast test, Quick functional tests
  • ✅ All Builds, Unit tests (asan_ubsan), AST fuzzer (targeted)

1. Integration tests (amd_asan_ubsan, db disk, old analyzer, 5/8) — not PR-caused (test-fixture/access-control)

21/21 failures are all in test_storage_iceberg_with_spark/test_export_partition_iceberg.py, all with the identical root cause while creating the IcebergS3 table in the fixture:

Code: 497. DB::Exception: default: Not enough privileges. To execute this query, it's necessary
to have the grant SHOW NAMED COLLECTIONS SECRETS ON s3. (ACCESS_DENIED)
(query: CREATE TABLE ... ENGINE=IcebergS3(s3, ...) SETTINGS iceberg_format_version = 2;)

The default user in this test lacks SHOW NAMED COLLECTIONS SECRETS. This is Iceberg export test-setup, with no code path to Parquet reading, and nothing in this diff touches access control, named collections, or Iceberg. Pre-existing on the branch.
→ Next step: safe to ignore for this PR. A fix belongs in the test fixture (grant SHOW NAMED COLLECTIONS SECRETS ON s3 to default, or run the DDL as an admin user) on the branch, not here.

2. Regression release oauth — not PR-caused (infra / Altinity regression suite)

/oauth/access control fails in missing_default_profile_does_not_grant_unrestricted_access (expected HTTP 500, got otherwise). This is the external-OAuth-provider regression suite (token-based auth), unrelated to Parquet.
→ Next step: safe to re-run; not actionable from this PR.

3. Regression release swarms — not PR-caused (infra / connectivity)

Cluster-discovery and node-failure scenarios fail with:

Code: 279. Cannot connect to any replica for query execution. (ALL_CONNECTION_TRIES_FAILED)
(query: SELECT hostName(), count() FROM s3Cluster('swarm', 'http://minio:9000/...**.parquet', ...))

Swarm-cluster connectivity / node-overload scenarios — infra flakiness, unrelated to the Parquet reader.
→ Next step: safe to re-run.


Health check

The functional backport looks healthy: the whole point of the PR — Dynamic/Variant type support in the Parquet reader — is exercised by the new stateless tests, which pass alongside builds and unit tests. The three red checks are all in orthogonal subsystems (Iceberg-export access grants, OAuth, swarm clusters) that fail the same way regardless of this change. I'd treat this as green for merge purposes; re-running the two regression suites may clear them, and the Iceberg-export failures are a pre-existing fixture-grant issue tracked at the branch level, not something to fix in this backport.

🤖 Triage by @blau-ai — evidence from the praktika result_pr.json and the oauth/swarms regression reports for c0b8b0c.

@zvonand
zvonand merged commit 08370ba into antalya-26.8 Oct 26, 2026
275 of 282 checks passed
@zvonand zvonand added the verified Approved for release label Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

antalya antalya-26.8 Session label (releasy session config) backport Backport port-antalya PRs to be ported to all new Antalya releases verified Approved for release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants