[svonava]

Problem

Qwen3-Reranker's model card builds each pair as one string, <Instruct>: {instruction}\n<Query>: {query}\n<Document>: {doc}, tokenizes it once, and wraps it in the chat template's prefix and suffix tokens. The adapter tokenized the text up to <Document>: (with its trailing space) and the document separately. Qwen2's pre-tokenizer then emits a lone space token and the document's first word without its space (Ġ, License), where the card has one ĠLicense token. The model scores token ids it never sees in the reference prompt.

On a 3,840-row workload (48 contract questions, 8 instructions, 10 candidates each), 3,800 rows on main differ from the card's ids. The 40 that match are documents that start with a digit, where the space stays a token of its own either way.

Fix

End the prefix at <Document>: and tokenize the document as " " + document. The pre-tokenizer always splits after that colon, so prefix + tail equal the card's single tokenization, and prefix reuse still shares one prefix per (instruction, query). The separator is not billed as document text where it stays a token of its own. Where it merges into the first word (ĠLicense), the merged token is one document token, the same count as before; a document whose first character joins the space differently ((g becomes (, g) can bill one token more or less than before.

The mxbai input format is unchanged.

Evidence

One H100 on Modal, the server and its client in one container (the reranker serving study's recipe: released sie-server v0.10.1 default image, pinned by digest, with main and this branch overlaid), plain HTTP, one request in flight. Reference: the Transformers code from the Qwen3-Reranker-4B model card at revision 22e683669bc0f0bd69640a1354a6d0aebcfeede5, float32. Noise floor: the largest change of any candidate-pair gap between float32 and bfloat16 in the card's own code on the same rows, 0.47 log-odds.

	main	this branch
Rows identical to the card's token ids	40 / 3,840	3,840 / 3,840
Candidate-pair rank flips against the card, reference margin above noise	23	0
Of those, between the two documents that decide the task	3	0
Median / max abs log-odds difference from the card	0.132 / 2.34	0.041 / 0.33
fast profile against default	bit-equal, 384 / 384 requests	383 / 384 bit-equal, max 1.4e-4 log-odds, same rankings

Feeding main's token rows to the card's own code gives zero flips above noise against main's scores, so the numerics are fine and the whole difference is the tokenization.

Root-cause record, harness and per-request scores: superlinked/sie-web#859, apps/site/public/reference/legal-supersession/root-cause/ (ls_app.py, ref_card.py, check_rows.py, analyze.py, summary.json).

Compatibility

Scores for input_format: qwen3 models (Qwen3-Reranker 0.6B and 4B, and decision or routing jobs served through this adapter) move toward the reference, by a median 0.13 log-odds on the workload above. Recordings made before this change will not reproduce bit-for-bit. Billed content tokens are unchanged for documents that start with a letter or digit. No API or wire change.

Validation

• pytest tests/adapters/test_qwen2_flash_cross_encoder.py tests/adapters/test_qwen2_flash_cross_encoder_prefix.py tests/adapters/test_qwen2_flash_cross_encoder_fused.py on CPU: 49 passed, 6 skipped (CUDA).
• The same three files on the H100, CUDA tests included: 55 passed.
• Every test file that touches this adapter (test_factory_integration.py, test_runtime_options.py, tests/config/test_native_zerank_catalog.py, test_all_models.py and the three above): 171 passed, 7 skipped.
• ruff check and ruff format --check on the changed files: clean. ty check on them reports the same 6 diagnostics as main (flash_attn imports and two existing ones), none new.
• New test test_qwen3_rows_are_the_model_cards_token_ids builds an offline word-level tokenizer over Qwen2's own pre-tokenizer and checks that _build_input_ids and the batched score_pairs path both produce the card's single tokenization, for documents starting with a letter, a digit, a bracket, a quote, a newline and a space. On main it fails for 4 of the 6.

Summary by CodeRabbit

• Bug Fixes
  • Improved Qwen3 document processing to match the model card’s tokenization behavior.
  • Ensured individual and batched scoring produce consistent tokenization and results.
  • Improved token counting so document counts exclude formatting separator tokens.
  • Preserved accurate document handling when inputs are truncated.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Team
• Run ID: a12c4b25-11d0-4adf-a903-af72c9bd2483

📥 Commits

Reviewing files that changed from the base of the PR and between 23204d8 and 9ca0c60.

📒 Files selected for processing: 3

• packages/sie_server/src/sie_server/adapters/qwen2_flash_cross_encoder/adapter.py
• packages/sie_server/tests/adapters/test_qwen2_flash_cross_encoder.py
• packages/sie_server/tests/adapters/test_qwen2_flash_cross_encoder_prefix.py

Included review availability: This review used your included allowance. 7 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 10 reviews per hour.

📝 Walkthrough

Qwen3 document construction now preserves the model-card tokenization by adding a pre-tokenized separator before document text. Document token counts exclude the separator when it remains distinct after tokenization.

Changes: Qwen3 document separator tokenization and counts

Layer / File(s)	Summary
Separator tokenization and count handling packages/sie_server/src/sie_server/adapters/qwen2_flash_cross_encoder/adapter.py, packages/sie_server/tests/adapters/test_qwen2_flash_cross_encoder.py, packages/sie_server/tests/adapters/test_qwen2_flash_cross_encoder_prefix.py	The adapter adds a pre-tokenized separator to batched and individual Qwen3 document tokenization. It excludes distinct leading separator tokens from document counts. Tests cover model-card tokenization, batched scoring counts, document filtering, and CUDA graph tail sizing.

Suggested reviewers: krisztian-gajdar

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 9ca0c

No actionable merge-blocking risk is established for the Qwen3 tokenization change; merge after normal checks.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 35.71% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: updating Qwen3 reranker tokenization to match the model card.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}