Skip to content

Adopt shared request-compression configuration in the Bedrock Runtime client - #111

Open
gouveags wants to merge 2 commits into
aws:developfrom
gouveags:feat/request-compression-adoption
Open

gouveags wants to merge 2 commits into
aws:developfrom
gouveags:feat/request-compression-adoption

Conversation

@gouveags

@gouveags gouveags commented Oct 19, 2026 •

Copy link
Copy Markdown

Related issue: #110.

Request compression would let AWS-hosted agents upload less data as conversations and tool results grow. This PR adopts the companion Smithy Python implementation in the Bedrock Runtime client. This PR is ready for review; merging depends on an upstream runtime release and a subsequent minimum-version update. Local validation uses the unpublished sibling Smithy checkout.

The client was regenerated using the normal generator workflow. Current Smithy source has already moved to compact schemas and APIOperation(static_schema=...), while the client at the original base uses the older schema= API. Adopting the current runtime therefore requires regenerating models.py and _private/schemas.py. That prerequisite migration is isolated in commit 73a81b9 and accounts for the large generated diff; it needs review separately from the compression feature.

Compression configuration is inherited from the shared AsyncAwsConfig, so regeneration does not add duplicate service configuration fields. This contribution adds client-level coverage and usage guidance for the shared behavior.

The shared configuration offers automatic compression or explicit gzip without a size threshold. The operation must advertise support, and unsupported explicit choices fail before sending. Current Bedrock operations remain unsupported: the client tests verify inherited configuration, unchanged default serialization, and explicit gzip failing before transport. Successful compression is covered by the companion Smithy runtime and protocol tests.

We use AWS-hosted models for long agent conversations, where uploading accumulated context can delay the first token. Anthropic's announcement and Claude Code changelog, together with OpenAI's Responses zstd documentation, show this is a useful option for LLM applications. Bedrock and Mantle supporting these capabilities would make AWS-hosted inference more attractive for this workload. No end-to-end latency improvement has been measured for this patch.

The companion Smithy implementation supports gzip under the current contract; zstd needs follow-up agreement. Bedrock Runtime also needs service acceptance and upstream metadata. This PR leaves service compression metadata unchanged. Mantle's provider-compatible APIs require separate API/client coordination; they are not implemented by this generated Bedrock Runtime client.

I'd appreciate feedback on the generated prerequisite and dependency workflow in this diff, and help coordinating service-side support.

Related contributions: botocore: PR; smithy-python: PR.

Verification

Reviewed implementation commit: 1cd64afc87a1d06988428260c9e9bd50b4e2f687.

  • Before implementation, the six client regression cases produced five failures and one pass against published dependencies.
  • Before regeneration, the same tests could not collect against current local Smithy source because generated APIOperation(schema=...) is incompatible with its static_schema= API.
  • After normal regeneration with the companion Smithy generator/runtime: all ten client cases pass. They cover inherited defaults, explicit gzip, environment settings and precedence, unchanged default serialization across four inference operations, and rejecting unsupported explicit gzip.
  • Ruff lint and formatting passed for generated source and the new tests; strict Pyright reports zero errors; git diff --check passed.

Generation used make publish-codegen, then make gen bedrock-runtime, with SMITHY_PYTHON set to the absolute sibling checkout path, Java 17, and the CI-pinned Pandoc 3.8.2 on PATH. A second generation with that Pandoc version reproduced both generated files byte for byte. Generated source files were not manually edited. Existing dependency constraints were preserved pending a real upstream release.

Tests ran on Python 3.12 using the local Smithy workspace environment. They are hermetic and send no AWS requests. Hosted CI against released dependencies, the supported-Python matrix, and live inference have not been validated for this dependent PR.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant