Skip to content

HDDS-10362. Add S3 Gateway metric for pending operations - #10990

Open
rich7420 wants to merge 8 commits into
apache:masterfrom
rich7420:HDDS-10362
Open

rich7420 wants to merge 8 commits into
apache:masterfrom
rich7420:HDDS-10362

Conversation

@rich7420

@rich7420 rich7420 commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Add a gateway-wide pendingOperations gauge to track S3 operations being processed. Requests are counted when the post-matching filter runs and released exactly once when Jersey finishes processing the response, including streaming and error responses. Requests rejected before the filter runs are excluded.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-10362

How was this patch tested?

  • Clean verify for S3 Gateway and its dependencies: 106 related tests passed, including 16 Jersey lifecycle cases and dependency analysis.
  • Checkstyle passed locally.
  • Fork CI is running for the latest commit.

Copilot AI lite review requested due to automatic review settings August 11, 2026 12:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@chihsuan chihsuan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch! @rich7420 Nice addition overall.

I noticed two edge cases. Please see inline comments. Thanks!

responseContext.setEntityStream(new WrappedOutputStream(out) {
@Override
public void close() throws IOException {
super.close();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If super.close() throws, the decrement is skipped, and the gauge remains permanently inflated. Should the decrement run in a finally block?

requestContext.removeProperty(COUNTED_METRICS);
final S3GatewayMetrics metrics = (S3GatewayMetrics) counted;

if (isStreamingGetObject(requestContext)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some GetObject responses, such as 304 Not Modified, have no response body. Since there is no body to stream, Jersey does not close this wrapped stream, and the pending count is never decremented.

Would it make sense to decrement immediately when the response has no entity?

@Russole
Russole self-requested a review September 14, 2026 15:54
@github-actions github-actions Bot added the s3 S3 Gateway label Sep 23, 2026
@rich7420
rich7420 marked this pull request as draft October 17, 2026 12:09
@rich7420
rich7420 marked this pull request as ready for review October 17, 2026 12:30
@adoroszlai

Copy link
Copy Markdown
Contributor

Thanks for the patch. I would like to defer merging this after #11735, which will (hopefully) allow skipping most integration tests for S3-only changes.

@rich7420

Copy link
Copy Markdown
Contributor Author

@adoroszlai sure, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

s3 S3 Gateway

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants