Skip to content

Speed up offer lookups and consume them off the event loop - #4374

Merged
r4victor merged 4 commits into
masterfrom
issue_4373_offers_disk_step
Oct 15, 2026
Merged

r4victor merged 4 commits into
masterfrom
issue_4373_offers_disk_step

Conversation

@r4victor

@r4victor r4victor commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #4373. ComputeWithAllOffersCached.get_offers() deep-copied every cached offer to set its disk before filtering, and the lazy offers iterator was consumed by sorted() on the event loop, so a burst of offer lookups blocked the server.

  • Fit the offer disk after filtering by requirements, via the new get_offer_disk_size_range() hook instead of a disk offer modifier, and copy only the offer, instance and resources instead of deep-copying: a lookup over GCP's 12.9k cached offers (44 regions) drops from 0.36–0.42 s to 0.02–0.03 s for GPU or default-disk requirements and to 0.09 s for broad requirements with an explicit disk range, with offers identical to master for the AWS, GCP, Azure, OCI, Nebius, RunPod and Verda catalogs.
  • Compute the fitted disk once per configurable disk size range in each lookup and share it between offers.
  • Consume the lazy offers chain in the executor instead of on the event loop in get_offers_by_requirements(): with 20 concurrent GCP lookups, the longest event-loop stall drops from 7–10 s, i.e. the whole burst, to 0.36 s for GPU lookups limited to 100 offers, 0.40 s for default-disk lookups and 2.1 s for broad explicit-disk lookups.

Measured on an M1 Pro. Next step, outside this PR: consume offers in a separate single-worker pool. Concurrent lookups add no throughput under the GIL and take it away from the event loop; running them one at a time in the same 20-lookup bursts cut the longest stall to 0.18 s, 0.21 s and 0.49 s with the same total time.

@r4victor
r4victor merged commit 0c8f753 into master Oct 15, 2026
26 checks passed
@r4victor
r4victor deleted the issue_4373_offers_disk_step branch October 15, 2026 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant