Skip to content

fix(media-uploads): copy media upload files when cloning an activity; freeze modern-format upload paths - #643

Open
caseylocker wants to merge 13 commits into
mainfrom
fix/clone-media-upload-files
Open

caseylocker wants to merge 13 commits into
mainfrom
fix/clone-media-upload-files

Conversation

@caseylocker

@caseylocker caseylocker commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

ref: https://app.clickup.com/t/9014802374/86bchyjfg

Problem

  • SummitService::cloneEvent cloned PresentationMediaUpload records but not their files. The clone's path is built from the clone's presentation ID (always legacy format), so every cloned upload pointed at a path with no file. Seen in prod on OCP Global 2026 (activity 10346 cloned from 10177).
  • A modern-format upload's path is re-derived from the current title slug in getPath(), so renaming an activity (or changing the summit slug) moved the path away from the stored file. Legacy-format paths use only IDs and were unaffected.

Fix

  • Rename: new nullable PresentationMediaUpload.FrozenPathSegment. getPath() uses it for modern-format rows; output is byte-identical for every row where it is null. Version20261014120000 adds the column (Builder), Version20261014120001 backfills the LegacyPathFormat = 0 rows with the segment getPath() produces today (same empty-title rule as generateSlug()).
  • Clone: after cloneEvent commits, it queues one CopyClonedMediaUploadFilesJob per cloned upload (JobDispatcher::withDbFallback). The job is a thin delegate to SummitService::copyClonedMediaUploadFiles (and removeUncopiedClonedMediaUpload from failed()). The logic lives on SummitService rather than PresentationService because it shares a storage lock with processPendingMediaUploads, which lives there. The copy streams each enabled storage leg from the original's path to the clone's (readStream -> saveFromStream, overwriting every attempt), skips a leg the original does not have, and retries (5 tries, timeout 1200 s, below retry_after).
  • The job never writes to or deletes a clone upload staff have re-uploaded since (filename changed, or any PendingMediaUpload row for it), and retries while a re-upload of the original is still in flight.
  • A per-storage-location Redis lock (ILockManagerService, SummitService::mediaUploadStorageLockName = media_upload_storage.{presentation_id}.{sha1(filename)}.lock) is shared by the job and processPendingMediaUploads, so a staff upload landing mid-copy is written after the copy, never overwritten by it. The processor skips a locked row untouched and picks it up on the next minute's run.
  • If every attempt fails (including a final-attempt timeout, which Laravel handles in-process), the cleanup removes the clone's upload record before marking its files deleted, so the clone never points at a missing file. Staff then re-upload on the clone.

Tests

  • New: CopyClonedMediaUploadFilesJobTest (21), PresentationMediaUploadPathTest (7), FrozenPathSegmentBackfillMigrationTest (4).
  • Updated: SummitServiceTest (clone queues the copy and the copy lands; dispatch failure; processor skips a locked upload), ProcessPendingMediaUploadsTest (lock dependency; locked row untouched).
  • push.yml: adds the MigrationUnitTests shard and the new root test file to the PresentationMediaUploads shard.
  • Unchanged by this branch and failing identically on main locally: the OAuth2PresentationApiTest + OAuth2SummitEventsApiTest pair (17 failures, 1 error), SponsorUserPermissionTrackingTest (2), and 3 in tests/Unit/Entities.

Deploy notes

  • Hold the merge until after OCP Global (Oct 16). This changes the pending upload processor.
  • Squash-merge (one intermediate commit has a red unit test).
  • Two migrations run on the image build. Afterwards SELECT COUNT(*) FROM PresentationMediaUpload WHERE LegacyPathFormat = 0 AND FrozenPathSegment IS NULL should be 0.
  • Smoke test on a non-show summit: clone an activity with a deck and confirm the file lands on both storage legs (Spaces and Dropbox). CI covers the Local driver only.
  • Expect LockManagerService ... MAX RETRIES ERROR lines while a clone copy holds a lock the processor wants. They are lock misses, not failures.

Known limits

  • A modern-format row whose title changed before the backfill is frozen at the current slug; those rows (2020, summits 8 and 12) were already broken.
  • Between the clone request and the copy finishing, the clone's upload briefly has no file.
  • If a copy times out after the clone's upload was deleted mid-copy, its lock lives out its 1300 s lifetime, delaying a same-name re-add on that clone by up to about 22 minutes.

@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 4c695417-78b6-4401-a5e2-abcf43525a91
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

📘 OpenAPI / Swagger preview

➡️ https://OpenStackweb.github.io/summit-api/openapi/pr-643/

This page is automatically updated on each push to this PR.

@github-actions

Copy link
Copy Markdown

📘 OpenAPI / Swagger preview

➡️ https://OpenStackweb.github.io/summit-api/openapi/pr-643/

This page is automatically updated on each push to this PR.

@caseylocker
caseylocker marked this pull request as ready for review October 15, 2026 00:47
@caseylocker
caseylocker requested review from smarcet and a balanced review from Copilot October 15, 2026 00:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved concurrency gaps can copy incomplete source files or overwrite and delete staff replacements.

2 open findings
What changed in this PR

Fixes activity cloning to copy media files and preserves modern upload paths across title or summit slug changes.

Changes:

  • Adds queued file copying, shared storage locks, and failure cleanup.
  • Adds and backfills frozen path segments.
  • Expands regression tests and CI coverage.
File Description
tests/​Unit/​Services/​ProcessPendingMediaUploadsTest.php Tests processor lock handling.
tests/​Unit/​Migrations/​FrozenPathSegmentBackfillMigrationTest.php Tests backfill slug rules.
tests/​Unit/​Entities/​PresentationMediaUploadPathTest.php Tests stable upload paths.
tests/​SummitServiceTest.php Tests clone dispatch and processor locking.
tests/​CopyClonedMediaUploadFilesJobTest.php Tests copying, replacement guards, and cleanup.
database/​migrations/​model/​Version20261014120001.php Backfills frozen paths.
database/​migrations/​model/​Version20261014120000.php Adds the frozen-path column.
app/​Services/​Model/​ISummitService.php Defines copy and cleanup contracts.
app/​Services/​Model/​Imp/​SummitService.php Implements copying, locking, and cleanup.
app/​Repositories/​Summit/​DoctrinePendingMediaUploadRepository.php Adds pending-upload existence queries.
app/​Models/​Foundation/​Summit/​Repositories/​IPendingMediaUploadRepository.php Declares pending-upload queries.
app/​Models/​Foundation/​Summit/​PendingMediaUpload.php Centralizes in-flight statuses.
app/​Models/​Foundation/​Summit/​Events/​Presentations/​Presentation.php Extracts reusable slug generation.
app/​Models/​Foundation/​Summit/​Events/​Presentations/​Materials/​PresentationMediaUpload.php Uses frozen modern paths.
app/​Jobs/​CopyClonedMediaUploadFilesJob.php Adds retryable copy job.
.github/​workflows/​push.yml Includes new tests in CI.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

return;
}
// UnacquiredLockException propagates: the processor is writing this location, so retry later
$token = $this->lock_service->acquireLock($lock_name, ISummitService::MEDIA_UPLOAD_STORAGE_LOCK_LIFETIME);
private function resolveClonedMediaUploadLegs(int $source_media_upload_id, int $target_media_upload_id, string $filename): array
{
return $this->tx_service->transaction(function () use ($source_media_upload_id, $target_media_upload_id, $filename) {
$target = $this->presentation_media_upload_repository->getById($target_media_upload_id);

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants