Repository navigation
HDDS-17055. Reject malformed x-amz-content-sha256 before writing the body - #11562
andyhuangdev wants to merge 6 commits into
Conversation
andyhuangdev
changed the title
| } | ||
|
|
||
| if (!hasUnsignedPayload(xAmzContentSha256Header) && !hasMultiChunksPayload(xAmzContentSha256Header) | ||
| && !xAmzContentSha256Header.matches("[0-9a-fA-F]{64}")) { |
There was a problem hiding this comment.
The new format check deliberately accepts uppercase hex, and the added test covers it, but the upload paths still compare the header with the lowercase digest using String.equals (for example S3ObjectWriteGuard.checkContentSha256).
Thus a header containing the uppercase form of the correct SHA-256 passes this validation and is then rejected as XAmzContentSHA256Mismatch, so the accepted format is not actually usable.
Normalize the expected value (or compare case-insensitively) before returning/validating it?
There was a problem hiding this comment.
Agreed. I’ll normalize validated hex digests to lowercase in validateSignatureHeader, preserving the unsigned and streaming constants, and add upload tests for uppercase and mixed-case digests across regular, datastream, and multipart uploads.
Normalize hexadecimal payload digests to lowercase while preserving unsigned and streaming values. Cover lowercase, uppercase, and mixed-case headers in validation and upload tests. Tests: 641 tests passed across TestS3Utils, TestObjectPut, and TestPartUpload; project checkstyle and git diff --check passed.
Resolve multipart upload test fixtures and preserve case-insensitive SHA-256 validation.
Adapt SHA-256 validation to the upstream payload classifier and update malformed-header test expectations. Tests: TestS3Utils, TestObjectPut, TestPartUpload (673 passed); checkstyle.
|
Thanks for the patch. I would like to defer merging this after #11735, which will (hopefully) allow skipping most integration tests for S3-only changes. |
What changes were proposed in this pull request?
S3 Gateway currently reads and writes the request body before rejecting a malformed x-amz-content-sha256 value with XAmzContentSHA256Mismatch. Such values cannot match a valid SHA-256 digest, so processing the body wastes I/O and network bandwidth.
This patch adds format validation to the shared header validation method. Hash values that are not exactly 64 hexadecimal characters are rejected with InvalidArgument before the body is read or a write stream is opened. Existing unsigned and streaming payload handling remains unchanged. Well-formed hashes that do not match the body continue to return XAmzContentSHA256Mismatch.
Existing test fixtures using malformed placeholder hashes are updated to use well-formed values so those tests continue exercising their intended behavior.
What is the link to the Apache JIRA
HDDS-17055
How was this patch tested?