Repository navigation
Cache a screen-sized copy of TIFF/HEIC images served by /images/ - #423
Merged
Merged
Conversation
/images/ re-encoded every TIFF/HEIC to a full-resolution PNG on every request, with no validators, so each view and slideshow preload paid for the decode and a 30-70 MB encode again. It now serves a JPEG (WebP when the image has transparency) capped at 4096px on the long edge, cached in the album's thumbnail directory and served with a content ETag/304. - Named with the tile digest, so the reindex sweep and the per-image delete reclaim display copies with no second keep-set. - Per-file lock plus write-then-rename: concurrent requests convert once and never read a partial file. Abandoned temporaries are swept. - Rebuilt when the source is newer; a failed cache write still serves. - 16-bit scans are reduced to 8-bit after _png_safe's rescale. On a 6000x4000 LZW TIFF: 2.55 s / 53 MB before, 0.67 s / 3.9 MB cold, ~1 ms warm. Downloads keep using ?original=1. Fixes #414 Co-Authored-By: Claude Opus 5.5 <[email protected]>
Adversarial review found the cached copy could show the wrong picture: same-named files under a symlinked root (or a mapped Windows drive) shared one copy, as did albums whose indexes share a directory, and a source replaced with an older mtime, or edited mid-conversion, kept its old copy. The copy's name now carries the album and a digest of the source's mtime+size taken before decoding; the relative path is taken from the resolved file, as the index stores it. Also: a copy written after its source was deleted is removed, a greyscale profile is not carried onto RGBA pixels, and an unwritable cache warns once. Co-Authored-By: Claude Opus 5.5 <[email protected]>
From a second adversarial review: - prune/discard matched any <32 hex>.jpg/.webp/.png, so a hash-named user photo beside a custom index path could be deleted. Both now match only the exact names the routes write. - A source edited between the up-front stat and the decode was cached under the old stamp; restoring the original then showed the edit. Re-stat after encoding and serve without caching if it moved. - A temporary swept mid-write marked the directory unwritable for good, silencing real failures; one undeletable sibling skipped the vanished-source check. Each step now handles its own errors. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #414.
Summary
/images/used to re-encode every TIFF/HEIC as a full-resolution PNG on every request, with no validators. It now serves a JPEG capped at 4096 px on the long edge, or a WebP when the image has real transparency. The copy is cached in the album's thumbnail directory and served with a content ETag (304 on revalidation). Downloads are unchanged because they use?original=1.Measured on a 6000×4000 LZW TIFF (97 MB):
How it works
<tile digest>_display_<album tag>_<source stamp>.<jpg|webp>.keep_hashes_forproduces. That lets the post-index sweep and the per-image delete (discard) reclaim display copies with no second keep-set.cp -p, a backup restore) is therefore reconverted. The stat is repeated after encoding, and if the source changed in between, the result is served but not cached._png_safe's 16-bit, 32-bit, float and CMYK/ICC handling still applies. 16-bit is then reduced to 8-bit for JPEG. An alpha channel that is fully opaque is dropped, and an ICC profile is dropped whenever the mode changes.prune/discardnow delete only the exact filename shapes the routes write. A hash-named user photo in athumbnails/folder next to a custom index path is left alone.Review
Two rounds of adversarial fresh-context review. Every confirmed finding has a regression test that fails on the earlier code.
Known and deliberately out of scope, because each predates this change:
thumbnails/are indexed as photos. The defaultphotomap_index/is excluded from scanning.thumbnails/folder.Test plan
pytest tests: 1308 passedruff check photomap tests🤖 Generated with Claude Code