Repository navigation
feat(client): pass through sampling-mask logprobs - #170
Open
faresobeid wants to merge 1 commit into
Open
faresobeid wants to merge 1 commit into
faresobeid wants to merge 1 commit into
Conversation
faresobeid
marked this pull request as ready for review
October 3, 2026 11:52
ApprovabilityVerdict: Approved at Macroscope's review found this PR approvable — This is a narrowly scoped, backwards-compatible pass-through of optional sampling-mask logprobs from the existing server response. It changes no request behavior, generation logic, or validation and adds no security, schema, or deployment risk. Notes:
No code changes detected at You can add or adjust custom eligibility rules. Learn more. |
faresobeid
force-pushed
the
feat/top-logprobs-v2
branch
3 times, most recently
from
October 6, 2026 17:19
01c9cce to
1cd3446
Compare
faresobeid
changed the base branch from
main
to
perf/packed-completion-logprobs
October 6, 2026 17:19
faresobeid
force-pushed
the
perf/packed-completion-logprobs
branch
from
October 8, 2026 14:35
e134d56 to
e161362
Compare
faresobeid
force-pushed
the
feat/top-logprobs-v2
branch
from
October 8, 2026 14:35
1cd3446 to
ea01237
Compare
faresobeid
force-pushed
the
perf/packed-completion-logprobs
branch
from
October 9, 2026 23:48
e161362 to
19c558f
Compare
faresobeid
force-pushed
the
feat/top-logprobs-v2
branch
from
October 9, 2026 23:48
ea01237 to
ecfeef7
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On top of #174's packed completion logprobs, passes prime-rl's sampler logprobs at the sampling-mask ids through
renderers.client.generate().generate()returnssampling_mask_logprobs = choice.get("sampling_mask_logprobs")next tosampling_mask. prime-rl's/inference/v1/generatesends it packed: base64 float32{data, shape, dtype}, parallel to the packedsampling_maskids. Stock vLLM does not send it, and then the key isNone.sampling_mask), no payload change when absent. verifiers decodes it (feat(v1): optional sampler logprobs on SamplingMask verifiers#2756).Consumer
prime-rl score centering, PrimeIntellect-ai/prime-rl#3794 (stacked on prime-rl#3900), pins this head. Without that consumer the field is unused.
Stack
Base: #174 (
perf/packed-completion-logprobs). Order: #174 -> PrimeIntellect-ai/verifiers#2778 -> prime-rl#3900; then this PR + PrimeIntellect-ai/verifiers#2756 -> prime-rl#3794. Rebased on current main (via #174).Validation
uv run pytest tests/test_client.py,ruff check renderers tests,ruff format --check renderers tests.Note
Pass through
sampling_mask_logprobsin client generate resultThe async generation client in renderers/client.py now reads the optional
sampling_mask_logprobsfield from the vLLM generation choice. The server may send packed float32 log probabilities parallel to the kept sampling-mask IDs when score centering is on. Behavioral Change:generatereturnsNonefor this value when the response omits the field.Macroscope summarized ecfeef7.
Note
Low Risk
Optional response field pass-through with no parsing or request changes; behavior unchanged when the engine omits the key.
Overview
generate()now surfaces optionalsampling_mask_logprobsfrom the/inference/v1/generatechoice, alongside the existingsampling_mask. When prime-rl score centering is enabled, the server can send packed float32 logprobs parallel to the kept mask token ids; stock vLLM omits the field and callers seeNone.This is a pass-through only: no new request fields, no decoding or validation (same pattern as
sampling_mask). The inline comment documents when the server sends the payload.Reviewed by Cursor Bugbot for commit ecfeef7. Bugbot is set up for automated code reviews on this repo. Configure here.