Repository navigation
HDDS-16661. Validate unsigned S3 checksum trailer completion and framing - #11586
Merged
Merged
Conversation
This was referenced Oct 17, 2026
rich7420
marked this pull request as ready for review
October 17, 2026 06:38
adoroszlai
marked this pull request as draft
October 18, 2026 11:40
Contributor
|
Converted to draft only to "stack" CI runs: resolving conflicts will first trigger CI only in fork, then after successful run it can be marked "ready for review". |
Contributor
Author
|
@adoroszlai got it! thanks |
smengcl
marked this pull request as ready for review
October 18, 2026 20:29
# Conflicts: # hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/endpoint/EndpointBase.java # hadoop-ozone/s3gateway/src/test/java/org/apache/hadoop/ozone/s3/endpoint/TestObjectPut.java
Contributor
Author
|
Flaky test |
Contributor
|
Thanks for the patch. I would like to defer merging this after #11735, which will (hopefully) allow skipping most integration tests for S3-only changes. |
Contributor
Author
|
@adoroszlai no problem, thanks |
Contributor
|
@chungen0126 would you like to take a look? |
Contributor
|
Hi @adoroszlai , sorry for the delay. I'll review this as soon as possible. |
chungen0126
approved these changes
Oct 26, 2026
Contributor
There was a problem hiding this comment.
Thanks @rich7420 for working on this. Overall LGTM. There are a few points that can be tracked as follow-ups:
- HDDS-17476: Return correct S3 errors for malformed aws-chunked request bodies
- HDDS-17477: Support SHA512 and XXHASH trailing checksums in S3 Gateway
- HDDS-17478: Refactor SignedChunksInputStream and UnsignedChunksInputStream. Extract shared aws-chunked parsing from SignedChunksInputStream and UnsignedChunksInputStream
I suggest we merge this PR first. I've created these Jiras to track them.
Contributor
|
Merged. Thanks @rich7420 for working on this, @adoroszlai @chihsuan for the review. |
Contributor
Author
|
@adoroszlai @chungen0126 @chihsuan @smengcl thanks for the review! |
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Oct 27, 2026
* master: (114 commits) HDDS-17411. Show one initialization notice on the Namespace Usage page (apache#11737) HDDS-17375. Create new submodule ozone-integration-test-tools (apache#11754) HDDS-17452. Introduce ClientProtocol delegation for S3 Gateway (apache#11757) HDDS-16661. Validate unsigned S3 checksum trailer completion and framing (apache#11586) HDDS-17450. Speed up TestDirectoryDeletingService (apache#11765) HDDS-17308. ListObjectsV2 Incorrectly Filters CommonPrefixes When StartAfter Is Before the Matching Prefixes (apache#11715). Clamp Echo sleep time and response payload size. (apache#44) HDDS-17433. Add a freon command for positioned reads on shared input streams (apache#11741) HDDS-17239. Initialize usage counters on bucket create (apache#11742) HDDS-17357. Initialize Netty in background when Ozone client is created (apache#11726) HDDS-17426. Disable ozone.tracing.client.application-aware in acceptance tests (apache#11758) HDDS-16737. Share OM follower read logic between the Hadoop RPC and gRPC clients (apache#11418) HDDS-17446. Copy datanode container set in NodeStateMap.updateNode only once (apache#11746) HDDS-17454. Bump asm to 9.11.1 (apache#11750) HDDS-17453. Bump jackson to 2.21.8 (apache#11749) HDDS-17376. Scale repeated-defrag footprint tolerance with the number of OM tables (apache#11708) HDDS-17380. Fix deadlock when BackgroundContainerDataScanner shutdown during volume failure (apache#11729) HDDS-17251. List open multipart uploads in ListOpenFiles (apache#11739) HDDS-17361. S3 Gateway can set another user as the owner of a new key (apache#11707) HDDS-17238. Initialize refCount and usedNamespace on volume create (apache#11720) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Consume and validate unsigned checksum trailers before committing PutObject or UploadPart. Cover the FSO directory path as well. Payload checksum comparison is in #11589.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16661
How was this patch tested?