Repository navigation
feat: replace the Kodak demo photographs with three from CID22 - #57
Merged
Merged
Conversation
The demo set's photographs were three Kodak images, one of them a single-woman close-up portrait, the kind of test image that has drawn criticism. They are replaced with three references from the CID22 validation set (CC BY-SA 4.0), the dataset ssimulacra2 was tuned and validated against: two people at a table (5055743), a church with grass (2936831) and red blossoms over a sky gradient (5458393). fetch.sh downloads the 17 MB validation zip once, checks its SHA-256 and extracts the three, each checked too. The lesson tables in compare/README.md are re-measured for the new photographs in the image version each table names, and the extra measurements behind their text (HEIC decoded with sips, JPEG at matched subsampling, encode times as the median of five runs) are repeated on all seven images. The conclusions change where the photographs do: these photos pack more color detail than the Kodak film scans, so 4:2:0 costs more on them. Plain WebP reaches ssimulacra2 90 on none of them, every 4:2:0 encoder fails on the blossoms photograph, and at 4:4:4 mozjpeg is larger than libjpeg-turbo, so mozjpeg's loss on photographs comes from its 4:4:4 encode, not from its subsampling default.
Codec comparison ✅ no size or quality regression📥 Download the codec report (unzip and open Size check |
Rebuilt from the CI amd64 results.csv of this pull request (42 image and codec pairs): the three Kodak entries are replaced by the three CID22 photographs, and the other four images are unchanged.
Cites Jake Archibald's JPEG XL vs AVIF for web use-cases where it bears on the method: the RD example shows the winner changing with the target (AVIF ahead from 70 to 85, JPEG XL from 90), as his photographs do; the matched-effort section notes that a ranking at the defaults is not one at maximum effort; the scope section says ssimulacra2 comes from the JPEG XL authors and names ColorVideoVDP as an independent cross-check; and the SVT-AV1 and JPEG lessons cite his "4:2:0 shouldn't be your default".
docs: note how target, effort and metric shape the comparison
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The demo set's photographs were
kodim04,kodim19andkodim23from the Kodak suite.kodim04is a close-up portrait of a single woman, the kind of test image that has drawn criticism in the imaging community. The set now uses photographs from CID22 instead, with the one photo of people showing two people.What
compare/demos/fetch.sh: downloads the CID22 validation set without distorted images (17 MB zip, SHA-256 checked) only when a photo is missing, and extractscid22-friends.png(5055743),cid22-church.png(2936831) andcid22-blossoms.png(5458393), each SHA-256 checked. Tested: first fetch, re-extraction of a damaged file, and a wrong checksum (fails and removes the file).compare/demos/README.md: sources, the CC BY-SA 4.0 license, the paper citation, and the note that the references are downscaled to 512×512 (which hides compression in their sources). Reports that embed them carry the same license.compare/demos/baseline.json: regenerated from this PR's CI amd64results.csv(second commit).compare/README.md: the RD example (nowcid22-friends.png, image 0.4.0) and the opt-in variant lessons, with the photo rows re-measured in the image version each table names (0.2.0, 0.3.0 for SVT-AV1). The texture and graphics rows are unchanged (deterministic encoders; re-runs reproduce them). The extra measurements behind the text are repeated on all seven images: HEIC decoded withsips(macOS 27.0.1), JPEG with-sample 2x2and-sample 1x1forced on both encoders, and encode times as the median of five runs.What the new photographs change in the lessons
These are modern photos downscaled to 512 px, with more color detail per pixel than the Kodak film scans, so 4:2:0 costs more on them:
sipsdecodes 4:2:0 1.5–8.4 points lower on six of seven images.Verification
node --test: 96 pass.