[svonava]

Continuous cheap arrivals can repeatedly bypass an older expensive item in the Python worker's cost-sorted batcher. This adds the explicit bounded_batch_progress profile option, disabled by default. When enabled, a bypassed oldest item is reserved for the next batchable turn; an item with k older batchable items receives a dispatch opportunity within 2 * (k + 1) nonempty batchable turns. Isolated-item turns may interleave; this is not a wall-clock deadline.

The ordinary BatchFormer remains unchanged, and all 378 shipped resolved profiles remain disabled. The worker selects the batcher class once, including for lazy LoRA batchers, so disabled workers have no progress bookkeeping. Profile inheritance, strict validation, config-service schema and model-loader plumbing are covered. Preformed/sidecar batches bypass this policy.

Enabled scheduling changes native batch composition and can change FP16 scores, labels and throughput. The earlier #798 GLiClass qualification FAILED its original 0.002 numerical bound: maximum entropy-confidence difference 0.0232588 and 26 changed choices, which were repetitions of one exact-tie decision. That failure is retained. This PR ships only the opt-in scheduler capability; it establishes no numerical equivalence or new GPU performance/model-quality qualification. Validate the intended model's output and throughput contract before enabling a profile. #390 remains open for the default-profile acceptance question; #798 is unchanged.

Validation at 945898acf1b4a63c73e5b30d3522123c4d9494df:

• 123 affected CPU controls pass after integration with main 17605365; scoped lint, formatting and type checks pass, with fix(server): wake idle workers on admitted work #807 admission/wakeup/stop behavior preserved.
• All 21 GitHub checks passed, including CI / Required; independent source/configuration review confirms the original policy delta, unchanged legacy batcher and disabled shipped profiles.

Related to #390.

Summary by CodeRabbit

• New Features
  • Added an opt-in profile setting that helps older batchable requests get dispatched within a bounded number of batchable turns, even when newer, lower-cost requests arrive.
  • The setting defaults to off and is not enabled in shipped profiles. Existing request and cost limits remain in effect.
• Documentation
  • Documented how the setting is inherited and how enabling it may affect batch composition, throughput, scores, or selected labels.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Team
• Run ID: 90d101b4-df6d-4ae6-8ace-d2c49bc460d1

📥 Commits

Reviewing files that changed from the base of the PR and between 118ba91 and 30d38e5.

📒 Files selected for processing: 10

• packages/sie_config/src/sie_config/model_config.schema.json
• packages/sie_server/README.md
• packages/sie_server/src/sie_server/config/model.py
• packages/sie_server/src/sie_server/core/loader.py
• packages/sie_server/src/sie_server/core/model_loader.py
• packages/sie_server/src/sie_server/core/progress_batcher.py
• packages/sie_server/src/sie_server/core/worker/model_worker.py
• packages/sie_server/src/sie_server/core/worker/types.py
• packages/sie_server/tests/config/test_batch_progress.py
• packages/sie_server/tests/core/test_progress_batcher.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 5 remain after this review.

📝 Walkthrough

Adds an opt-in bounded_batch_progress profile setting and a progress-aware batcher for Python workers. The setting inherits through profile extensions and defaults to disabled. When enabled, the batcher reserves bypassed requests for later batchable turns while retaining existing request, cost, and item limits.

Changes: Bounded batch progress

Layer / File(s)	Summary
Profile setting and resolution packages/sie_config/src/sie_config/model_config.schema.json, packages/sie_server/src/sie_server/config/model.py, packages/sie_server/src/sie_server/core/loader.py, packages/sie_server/tests/config/test_batch_progress.py	Adds the nullable setting, parent inheritance, serialization behavior, and schema validation. Generated variants preserve enabled values. Tests cover validation, inheritance, serialization, and configuration loading.
Progress-aware batch formation packages/sie_server/src/sie_server/core/progress_batcher.py, packages/sie_server/tests/core/test_progress_batcher.py, packages/sie_server/README.md	Adds ProgressBatchFormer, which tracks request arrival order and reserves a bypassed batchable request for a later turn. Tests cover progress bounds, reservation behavior, limits, cleanup, and compatibility. The README documents the setting and scheduling behavior.
Worker batcher selection packages/sie_server/src/sie_server/core/worker/types.py, packages/sie_server/src/sie_server/core/model_loader.py, packages/sie_server/src/sie_server/core/worker/model_worker.py, packages/sie_server/tests/config/test_batch_progress.py, packages/sie_server/tests/core/test_progress_batcher.py	Passes the resolved setting into WorkerConfig. The worker selects ProgressBatchFormer for base-model and LoRA batchers when enabled, and retains BatchFormer when disabled.

Sequence Diagram(s)

sequenceDiagram
  participant ProfileResolution
  participant ModelLoader
  participant WorkerConfig
  participant ModelWorker
  participant ProgressBatchFormer
  ProfileResolution->>ModelLoader: Provide resolved bounded_batch_progress
  ModelLoader->>WorkerConfig: Set bounded_batch_progress
  WorkerConfig->>ModelWorker: Configure worker
  ModelWorker->>ProgressBatchFormer: Construct batcher when enabled

Loading

Suggested reviewers: krisztian-gajdar

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 30d38

The change adds an opt-in batching option that is off by default and leaves existing behavior unchanged unless a profile enables it. No concrete merge-blocking risk was found. The PR remains a draft, and the author has already said the compatibility decision is pending.

Pre-merge checks: 3 · 2

Failed checks: 2 warnings

Check name	Status	Explanation	Resolution
Linked Issues check		Direct issue [#390] asks for investigation of separate-label-group request splitting and scheduling, timeout or cancellation visibility, and reproduction under mixed extract load. This PR adds an opt-…	Add automated coverage for the reported separate-label-group and mixed-extract workload. Add coverage or implementation for the required per-request timeout and cancellation visibility. If this PR intentionally addresses only scheduler prog…
Docstring Coverage		Docstring coverage is 21.62% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 37 functions across 8 files. (2 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 3 passed

Check name	Status	Explanation
Out of Scope Changes check		The new ProgressBatchFormer, worker selection, profile inheritance, schema and serialization changes, loader plumbing, documentation, and tests support the opt-in scheduling objective described for [#…
Title check		The title clearly and concisely identifies the main change: adding opt-in bounded batch progress to the server.
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.

Full details: Out of Scope Changes check

Explanation

The new ProgressBatchFormer, worker selection, profile inheritance, schema and serialization changes, loader plumbing, documentation, and tests support the opt-in scheduling objective described for [#390]. The changes preserve the default batcher and do not enable the option in shipped profiles. No unrelated change is established by the reviewed diff.

Full details: Linked Issues check

Explanation

Direct issue [#390] asks for investigation of separate-label-group request splitting and scheduling, timeout or cancellation visibility, and reproduction under mixed extract load. This PR adds an opt-in bounded-progress scheduler and unit tests for cost-sorted batch progress. The reviewed changes do not add a GLiClass separate-group split or mixed-load reproduction, and they do not add or test per-request timeout or cancellation visibility. The PR description also states that it does not close [#390].

Resolution

Add automated coverage for the reported separate-label-group and mixed-extract workload. Add coverage or implementation for the required per-request timeout and cancellation visibility. If this PR intentionally addresses only scheduler progress, remove [#390] as a direct linked target or split the remaining investigation into a separate change.

Full details: Docstring Coverage

Explanation

Docstring coverage is 21.62% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 37 functions across 8 files. (2 skipped: 2 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

• Fix all pre-merge checks with AI

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[svonava]

@coderabbitai full review

Please review exact head 30d38e5f20b5794b55f38470618b1416f048fb69, including disabled-path preservation, profile inheritance and serialization, config-service validation, and opt-in progress/cleanup. This remains a draft with the prior numerical failure and compatibility decision explicitly retained.

[coderabbitai]

✅ Action performed

Full review finished.