Skip to content

feat(client): pass through sampling-mask logprobs - #170

Open
faresobeid wants to merge 1 commit into
perf/packed-completion-logprobsfrom
feat/top-logprobs-v2
Open

faresobeid wants to merge 1 commit into
perf/packed-completion-logprobsfrom
feat/top-logprobs-v2

Conversation

@faresobeid

@faresobeid faresobeid commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

On top of #174's packed completion logprobs, passes prime-rl's sampler logprobs at the sampling-mask ids through renderers.client.generate().

  • generate() returns sampling_mask_logprobs = choice.get("sampling_mask_logprobs") next to sampling_mask. prime-rl's /inference/v1/generate sends it packed: base64 float32 {data, shape, dtype}, parallel to the packed sampling_mask ids. Stock vLLM does not send it, and then the key is None.
  • Pure pass-through: no request-side change, no parsing or validation (same as sampling_mask), no payload change when absent. verifiers decodes it (feat(v1): optional sampler logprobs on SamplingMask verifiers#2756).

Consumer

prime-rl score centering, PrimeIntellect-ai/prime-rl#3794 (stacked on prime-rl#3900), pins this head. Without that consumer the field is unused.

Stack

Base: #174 (perf/packed-completion-logprobs). Order: #174 -> PrimeIntellect-ai/verifiers#2778 -> prime-rl#3900; then this PR + PrimeIntellect-ai/verifiers#2756 -> prime-rl#3794. Rebased on current main (via #174).

Validation

  • uv run pytest tests/test_client.py, ruff check renderers tests, ruff format --check renderers tests.

Note

Pass through sampling_mask_logprobs in client generate result

The async generation client in renderers/client.py now reads the optional sampling_mask_logprobs field from the vLLM generation choice. The server may send packed float32 log probabilities parallel to the kept sampling-mask IDs when score centering is on. Behavioral Change: generate returns None for this value when the response omits the field.

Macroscope summarized ecfeef7.


Note

Low Risk
Optional response field pass-through with no parsing or request changes; behavior unchanged when the engine omits the key.

Overview
generate() now surfaces optional sampling_mask_logprobs from the /inference/v1/generate choice, alongside the existing sampling_mask. When prime-rl score centering is enabled, the server can send packed float32 logprobs parallel to the kept mask token ids; stock vLLM omits the field and callers see None.

This is a pass-through only: no new request fields, no decoding or validation (same pattern as sampling_mask). The inline comment documents when the server sends the payload.

Reviewed by Cursor Bugbot for commit ecfeef7. Bugbot is set up for automated code reviews on this repo. Configure here.

@faresobeid faresobeid changed the title feat(client): opt-in per-token top-k logprobs feat(client): pass through sampling-mask logprobs Oct 3, 2026
@faresobeid
faresobeid marked this pull request as ready for review October 3, 2026 11:52
macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Oct 3, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Approvability

Verdict: Approved at 1cd3446

Macroscope's review found this PR approvable — This is a narrowly scoped, backwards-compatible pass-through of optional sampling-mask logprobs from the existing server response. It changes no request behavior, generation logic, or validation and adds no security, schema, or deployment risk.

Notes:

  • Macroscope's correctness review did not run, so approvability was decided on eligibility alone.

No code changes detected at ecfeef7. Prior analysis still applies.

You can add or adjust custom eligibility rules. Learn more.

@faresobeid
faresobeid force-pushed the feat/top-logprobs-v2 branch 3 times, most recently from 01c9cce to 1cd3446 Compare October 6, 2026 17:19
@faresobeid
faresobeid changed the base branch from main to perf/packed-completion-logprobs October 6, 2026 17:19
@faresobeid
faresobeid force-pushed the perf/packed-completion-logprobs branch from e134d56 to e161362 Compare October 8, 2026 14:35
@faresobeid
faresobeid force-pushed the feat/top-logprobs-v2 branch from 1cd3446 to ea01237 Compare October 8, 2026 14:35
@faresobeid
faresobeid force-pushed the perf/packed-completion-logprobs branch from e161362 to 19c558f Compare October 9, 2026 23:48
@faresobeid
faresobeid force-pushed the feat/top-logprobs-v2 branch from ea01237 to ecfeef7 Compare October 9, 2026 23:48

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant