Skip to content

fix(files): run the provider listing off the loop in acleanup_provider_files - #8642

Merged
Vidit-Ostwal merged 1 commit into
crewAIInc:mainfrom
Wilson0511:fix/acleanup-listing-blocks-loop
Oct 27, 2026
Merged

Vidit-Ostwal merged 1 commit into
crewAIInc:mainfrom
Wilson0511:fix/acleanup-listing-blocks-loop

Conversation

@Wilson0511

@Wilson0511 Wilson0511 commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

acleanup_provider_files(..., delete_all_from_provider=True) called uploader.list_files() — the sync method that hits the provider's HTTP API — directly on the event loop. Deletes were already off-loop (adelete under a semaphore + gather), but the listing step froze every other coroutine for the duration of the provider listing. The listing now runs via asyncio.to_thread, a one-line fix with no uploader API change (alist_files does not exist on the uploaders).

Closes #8641

Change

  • lib/crewai-files/src/crewai_files/cache/cleanup.py: files = await asyncio.to_thread(uploader.list_files) in the delete_all_from_provider branch of acleanup_provider_files (the sync cleanup_provider_files twin is unchanged).
  • lib/crewai-files/tests/test_acleanup_listing_nonblocking.py: regression test with a stub uploader whose list_files sleeps 0.5 s; a ticker coroutine must keep ticking while the listing runs. Fails on main (ticker frozen), passes with the fix.

Proof

  • Red on main: AssertionError: event loop was blocked during listing: 3 ticks (expected ≥ 6).
  • Green with the fix: 1 passed; full lib/crewai-files suite: 180 passed, 4 skipped.

Risk

Low: behavior is identical except the listing executes in a worker thread; result ordering and error handling (except Exception path) are unchanged.

🤖 Generated with ZCode (AI agent) on behalf of Wilson0511

Co-Authored-By: ZCode [email protected]


Note

Low Risk
Same cleanup semantics and error handling; only moves synchronous listing to a worker thread so the event loop stays responsive.

Overview
Fixes event-loop blocking when acleanup_provider_files runs with delete_all_from_provider=True. Provider file listing used synchronous list_files() (HTTP) on the asyncio loop while deletes were already async; other coroutines stalled for the whole listing call.

Listing now runs with await asyncio.to_thread(uploader.list_files) so the sync uploader API stays unchanged. The sync cleanup_provider_files path is untouched.

Adds a regression test with a stub uploader whose list_files sleeps 0.5s and a ticker task that must keep firing during cleanup—fails if listing blocks the loop.

Reviewed by Cursor Bugbot for commit 34ecc95. Bugbot is set up for automated code reviews on this repo. Configure here.

@strix-security

strix-security Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 2 commits after the last Strix review (34ecc95). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Moves the synchronous provider file listing in `acleanup_provider_files(delete_all_from_provider=True)` off the event loop via `asyncio.to_thread`, and adds a test that fails if the listing blocks the loop.

Review summary

The diff wraps the synchronous provider listing in acleanup_provider_files with asyncio.to_thread, plus a regression test asserting the event loop keeps ticking during the listing. No security issues found: the listing is read-only in every uploader, uploaders are instantiated per call so the worker thread has exclusive access, error handling is unchanged, and no privilege, credential, or input-handling path is touched.

Review 1 on this PR · 34ecc95 · requested Oct 27 04:38 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings

…r_files

acleanup_provider_files(delete_all_from_provider=True) called the
synchronous uploader.list_files() — a provider HTTP listing (Anthropic/
OpenAI files list, Bedrock paginated list_objects_v2, Gemini files.list)
— directly on the running event loop. Deletes were already off-loop via
adelete under a semaphore, but the listing froze every other coroutine
for the duration of the provider round-trips.

Change: files = await asyncio.to_thread(uploader.list_files) in the
delete_all_from_provider branch; the sync cleanup_provider_files twin is
unchanged.

Why: an async API must not block the loop for seconds-scale network I/O.

Proof: tests/test_acleanup_listing_nonblocking.py — a stub uploader
whose list_files sleeps 0.5s freezes a 0.05s ticker coroutine on main
(~3 ticks) and lets it keep ticking with the fix (~12); full
lib/crewai-files suite 180 passed, 4 skipped.

Risk: low — identical behavior except the listing runs in a worker
thread; error handling and result ordering unchanged.

🤖 Generated with ZCode (AI agent) on behalf of Wilson0511

Co-Authored-By: ZCode <[email protected]>
@Vidit-Ostwal
Vidit-Ostwal force-pushed the fix/acleanup-listing-blocks-loop branch from 34ecc95 to cd6a9ae Compare October 27, 2026 07:03

@Vidit-Ostwal Vidit-Ostwal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Vidit-Ostwal Vidit-Ostwal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the event-loop-blocking cleanup path and its regression coverage. Moving the synchronous provider listing to asyncio.to_thread is the minimal correct fix, and the ticker-based test verifies that other coroutines keep running.

@Vidit-Ostwal
Vidit-Ostwal enabled auto-merge (squash) October 27, 2026 07:07
@Vidit-Ostwal
Vidit-Ostwal merged commit c5f996a into crewAIInc:main Oct 27, 2026
55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

acleanup_provider_files blocks the event loop on the provider listing

2 participants