diff --git a/compare/README.md b/compare/README.md index 5765fad..9cdfaa6 100644 --- a/compare/README.md +++ b/compare/README.md @@ -63,16 +63,16 @@ For each image it prints the bpp each codec needs at every target, and its **BD- +---------+---------+---------+---------+---------+---------+--------------------+ | Codec | bpp @70 | bpp @80 | bpp @85 | bpp @90 | bpp @95 | BD-rate vs mozjpeg | +---------+---------+---------+---------+---------+---------+--------------------+ -| JPEG XL | 0.745 | 1.208 | 1.640 | 2.452 | 3.296 * | -27.3% | -| HEIC | 0.731 * | 1.209 * | 1.859 * | 2.881 | 5.957 * | -17.1% | -| AVIF | 0.767 | 1.316 | 1.805 | 3.052 | 8.343 * | -16.2% | -| jpegli | 0.911 | 1.381 | 1.830 | 2.658 | 3.516 * | -15.5% | -| mozjpeg | 0.901 | 1.451 | 2.335 | 4.097 | 8.048 * | baseline | -| WebP | 1.010 * | 1.543 | 2.052 | 3.535 | 3.597 * | +0.7% | +| AVIF | 0.416 | 0.654 * | 0.942 | 1.657 | 7.378 | -33.8% | +| HEIC | 0.424 * | 0.649 * | 1.043 | 2.420 | 4.716 * | -32.9% | +| JPEG XL | 0.566 | 0.862 | 1.114 | 1.611 | 2.417 * | -26.6% | +| WebP | 0.588 | 0.876 | 1.320 | 2.662 * | 2.662 * | -13.6% | +| jpegli | 0.733 | 1.103 | 1.389 | 1.966 | 2.794 * | -8.2% | +| mozjpeg | 0.667 | 1.150 * | 1.536 | 2.794 | 5.897 | baseline | +---------+---------+---------+---------+---------+---------+--------------------+ ``` -That is `kodim04.png` in the Docker image. A `*` marks a target the codec could not reach (its range is exhausted), as in the single-target table; the curve uses the quality the codec actually reached there, not the target. +That is `cid22-friends.png` in the Docker image (0.4.0). AVIF needs the fewest bits from 70 to 85, and JPEG XL is smaller from 90 up, so a single-target run at the default 90 and one at 70 name different winners. Jake Archibald's [JPEG XL vs AVIF for web use-cases](https://jakearchibald.com/2026/jpeg-xl-vs-avif/) finds the same pattern across his photographs: AVIF ahead at ssimulacra2 70, with JPEG XL closing the gap at very high quality. A `*` marks a target the codec could not reach (its range is exhausted), as in the single-target table; the curve uses the quality the codec actually reached there, not the target. BD-rate (the Bjøntegaard delta rate) is the average difference in bits at equal quality across the range both curves cover: `-30%` means the codec needs, on average, 30% fewer bits than the baseline for the same ssimulacra2 score. Rate is interpolated piecewise-linearly in the log domain between the measured points, rather than with the classic cubic fit, so it stays well-behaved with few targets and never overshoots. A codec whose curve does not overlap the baseline's (for example, one pinned at its range bound at every target) shows `n/a`. @@ -192,6 +192,8 @@ deliberate, comparable operating point: compare-codecs reference.png --target 90 --effort 9 --time-runs 3 ``` +Effort moves sizes as well as times, and not equally across codecs, so a ranking at the defaults is not a ranking at maximum effort. [JPEG XL vs AVIF for web use-cases](https://jakearchibald.com/2026/jpeg-xl-vs-avif/) compares at each encoder's maximum effort (`avifenc -s 0`) and finds that "libjxl's higher efforts are much slower, use a lot more memory, and barely improve the result"; read results at the defaults as a comparison of the presets most tools ship with. + ## Opt-in variants The default set compares six formats, one encoder configuration each. A variant is one of them with a single setting changed, so a comparison can isolate what that setting is worth on your images. Variants reuse the base codec's quality knob, decoder and effort mapping, and run only when named in `--codecs` (or with `--codecs all`), so the default comparison and the regression baseline are unaffected. The weekly comparison runs them all. @@ -208,17 +210,17 @@ compare-codecs images/ --target 90 --codecs webp,webp-sharp | Image | WebP | WebP sharp YUV | | --- | --- | --- | -| `kodim04.png` (photo) | 169.7 KB, 89.8 | **150.6 KB**, 89.8 (−11%) | -| `kodim19.png` (photo) | 169.8 KB, 90.0 | **165.0 KB**, 89.9 (−3%) | +| `cid22-friends.png` (photo, people) | 85.2 KB, 89.4 * | **76.5 KB**, 89.9 | +| `cid22-church.png` (photo, detail) | 92.0 KB, 88.9 * | **83.4 KB**, 89.6 | +| `cid22-blossoms.png` (photo, color) | 115.4 KB, 76.9 * | 119.4 KB, **85.9** * | | `forest-leaves.png` (texture) | 735.8 KB, 89.5 | **657.9 KB**, 90.3 (−11%) | -| `kodim23.png` (photo) | 137.1 KB, 89.0 * | 143.9 KB, **89.8** | | `corporate-memphis.png` (illustration) | 45.7 KB, 85.9 * | 45.8 KB, **89.5** | | `report-ui.png` (UI, text) | 114.8 KB, 77.4 * | 123.4 KB, **89.4** * | | `pipeline-diagram.png` (diagram) | 61.6 KB, 73.5 * | 70.9 KB, **86.2** * | -Where both reach the target, sharp YUV is 3–11% smaller at the same quality. Where plain WebP cannot reach it, even at `-q 100`, sharp YUV raises the ceiling, from 77 to 89 on the UI screenshot; the extra bytes buy quality that was unreachable before, so those rows are not a size comparison. Encoding takes about 1.2–2× as long. +Plain WebP reaches the target only on the texture, where sharp YUV is 11% smaller at the same quality. On every other image plain WebP stops short even at `-q 99`–`100`, photographs included, and sharp YUV raises the ceiling: on the people and church photographs it reaches the target with 9–10% fewer bytes than plain WebP's best attempt, and on the UI screenshot it lifts the score from 77 to 89. Where it still falls short, the extra bytes buy quality that was unreachable before, so those rows are not a size comparison. Encoding takes about 1.2–2× as long. -**Takeaway.** Sharp YUV is close to a free improvement for lossy WebP, and most valuable on graphics, text and saturated edges. It does not remove the 4:2:0 limit: for UI and diagrams, a format that keeps full-resolution color (AVIF, which `avifenc` encodes as 4:4:4 by default, or JPEG XL) still does better. +**Takeaway.** Sharp YUV is close to a free improvement for lossy WebP, and valuable wherever color changes sharply: graphics and text, but also detailed photographs, where it is what lets WebP reach high quality at all. It does not remove the 4:2:0 limit: on the blossoms photograph, the UI and the diagram, a format that keeps full-resolution color (AVIF, which `avifenc` encodes as 4:4:4 by default, or JPEG XL) still does better. **Using it.** `cwebp -sharp_yuv`; in [sharp](https://sharp.pixelplumbing.com/api-output/#webp), `webp({ smartSubsample: true })` ("use high quality chroma subsampling", off by default); for 4:2:0 AVIF, `avifenc --sharpyuv`. @@ -232,19 +234,19 @@ Where both reach the target, sharp YUV is 3–11% smaller at the same quality. W | Image | HEIC (4:2:0) | HEIC 4:4:4 | | --- | --- | --- | -| `kodim04.png` (photo) | 138.3 KB, 89.8 | 137.7 KB, 90.2 (−0.5%) | -| `kodim19.png` (photo) | 151.7 KB, 89.9 | **139.7 KB**, 89.7 (−8%) | -| `kodim23.png` (photo) | 153.9 KB, 90.2 | **111.6 KB**, 90.2 (−28%) | +| `cid22-friends.png` (photo, people) | 77.4 KB, 90.0 | **47.9 KB**, 90.1 (−38%) | +| `cid22-church.png` (photo, detail) | **69.3 KB**, 90.1 | 72.7 KB, 90.4 (+5%) | +| `cid22-blossoms.png` (photo, color) | 190.3 KB, 82.3 * | **124.1 KB**, 90.0 | | `forest-leaves.png` (texture) | 696.5 KB, 90.5 | 683.8 KB, 90.2 (−2%) | | `corporate-memphis.png` (illustration) | 67.1 KB, 88.0 * | **25.0 KB**, 89.7 | | `report-ui.png` (UI, text) | 194.3 KB, 83.2 * | **82.1 KB**, 90.2 | | `pipeline-diagram.png` (diagram) | 94.8 KB, 73.4 * | **84.2 KB**, 89.7 | -4:4:4 reaches the target on all seven images, against four for 4:2:0, and is never larger: on the UI screenshot it is 58% smaller *and* reaches a quality 4:2:0 cannot. At this quality level, halving the color resolution does not save bytes; it costs them, because the search has to raise the quality setting to win back the lost color (`-q 94–97` for 4:2:0 against `-q 57–75` for 4:4:4 on the three images 4:2:0 cannot reach). Encoding takes up to about 1.4× as long. +4:4:4 reaches the target on all seven images, against three for 4:2:0. It is 38% smaller on the people photograph, and on the UI screenshot it is 58% smaller *and* reaches a quality 4:2:0 cannot. It is larger only on the church photograph (+5%), where halving the color resolution loses little and 4:2:0 keeps its byte advantage. Elsewhere, at this quality level, halving the color resolution does not save bytes; it costs them, because the search has to raise the quality setting to win back the lost color (`-q 94–97` for 4:2:0 against `-q 57–75` for 4:4:4 on the four images 4:2:0 cannot reach). Encoding takes up to about 1.4× as long. -**The decoder matters too.** The harness decodes with libheif. Decoding the same files with Apple's ImageIO (`sips`, macOS 27.0.1) gives the same score for 4:4:4 (within ±0.5), but 0.9 to 6.6 points *lower* for 4:2:0 on six of the seven images (report UI 83.2 → 77.8, illustration 88.0 → 81.5): Apple upsamples the color differently from libheif. A 4:2:0 HEIC therefore looks worse on an Apple device than the harness measures, and 4:4:4 avoids the gap. +**The decoder matters too.** The harness decodes with libheif. Decoding the same files with Apple's ImageIO (`sips`, macOS 27.0.1) gives the same score for 4:4:4 (within ±0.5), but 1.5 to 8.4 points *lower* for 4:2:0 on six of the seven images (blossoms photograph 82.3 → 73.9, illustration 88.0 → 81.5, report UI 83.2 → 77.8): Apple upsamples the color differently from libheif. A 4:2:0 HEIC therefore looks worse on an Apple device than the harness measures, and 4:4:4 avoids the gap. -**Takeaway.** For HEIC at high quality, 4:4:4 is smaller and better, and Apple's decoder reads it. Check the decoders you target before relying on it, since 4:4:4 needs HEVC RExt and many hardware decoders only implement Main and Main 10. +**Takeaway.** For HEIC at high quality, 4:4:4 is almost always smaller and better, and Apple's decoder reads it. Check the decoders you target before relying on it, since 4:4:4 needs HEVC RExt and many hardware decoders only implement Main and Main 10. **Using it.** `heif-enc -p chroma=444`; in libheif's API, the x265 encoder parameter `chroma` (`heif-enc -P` lists it, `{ 420, 422, 444 }`, default 420). @@ -254,19 +256,19 @@ Where both reach the target, sharp YUV is 3–11% smaller at the same quality. W **What it changes.** AVIF is the AV1 format, and AV1 has several independent encoders. `avifenc` auto-selects one from those libavif was built with, which in the image is **aom** (libaom, the Alliance for Open Media reference encoder); `-c rav1e` selects **[rav1e](https://github.com/xiph/rav1e)** instead, a Rust encoder from Xiph that describes itself as "the fastest and safest AV1 encoder" and is the engine behind pure-Rust tools such as [cavif](https://github.com/kornelski/cavif-rs). The bitstream is AV1 either way, decoded by the same decoders; only the encoder's decisions differ. The variant keeps AVIF's quantizer knob and speed mapping, and needs an `avifenc` built with rav1e (`avifenc --version` lists `rav1e [enc]`; Homebrew's is built without it). -**What it is worth on the demo images** (target ssimulacra2 90, image 0.2.0, both at avifenc's default speed 6): +**What it is worth on the demo images** (target ssimulacra2 90, image 0.2.0, both at avifenc's default speed 6; encode time is the median of five runs): | Image | AVIF (aom) | AVIF rav1e | Encode time | | --- | --- | --- | --- | -| `kodim04.png` (photo) | 146.5 KB, 90.1 | **142.2 KB**, 90.0 (−3%) | 110 ms → 1.07 s | -| `kodim19.png` (photo) | 145.1 KB, 89.8 | **142.2 KB**, 89.8 (−2%) | 110 ms → 1.00 s | -| `kodim23.png` (photo) | **108.9 KB**, 89.7 | 119.7 KB, 90.3 (+10%) | 80 ms → 1.05 s | -| `forest-leaves.png` (texture) | 805.2 KB, 90.4 | **771.2 KB**, 90.2 (−4%) | 290 ms → 3.95 s | -| `corporate-memphis.png` (illustration) | **18.7 KB**, 89.9 | 26.6 KB, 90.5 (+43%) | 40 ms → 950 ms | -| `report-ui.png` (UI, text) | **62.6 KB**, 89.9 | 105.5 KB, 90.2 (+69%) | 640 ms → 1.45 s | -| `pipeline-diagram.png` (diagram) | **61.3 KB**, 90.1 | 88.5 KB, 90.0 (+44%) | 50 ms → 1.19 s | +| `cid22-friends.png` (photo, people) | 53.0 KB, 90.0 | **51.8 KB**, 90.2 (−2%) | 130 ms → 1.00 s | +| `cid22-church.png` (photo, detail) | 66.5 KB, 89.7 | **64.6 KB**, 89.7 (−3%) | 130 ms → 980 ms | +| `cid22-blossoms.png` (photo, color) | **125.0 KB**, 89.7 | 130.4 KB, 90.0 (+4%) | 190 ms → 1.60 s | +| `forest-leaves.png` (texture) | 805.2 KB, 90.4 | **771.2 KB**, 90.2 (−4%) | 560 ms → 7.55 s | +| `corporate-memphis.png` (illustration) | **18.7 KB**, 89.9 | 26.6 KB, 90.5 (+43%) | 70 ms → 1.84 s | +| `report-ui.png` (UI, text) | **62.6 KB**, 89.9 | 105.5 KB, 90.2 (+69%) | 1.26 s → 2.84 s | +| `pipeline-diagram.png` (diagram) | **61.3 KB**, 90.1 | 88.5 KB, 90.0 (+44%) | 110 ms → 2.35 s | -On photographs and the texture rav1e is within a few percent of aom, slightly smaller on three of the four. On flat graphics and text aom is far ahead, 43–69% smaller. rav1e is also 2–24× slower here, and its CPU time equals its wall time, so it ran on one core, while aom spread the same work over several. +On photographs and the texture rav1e is within a few percent of aom, slightly smaller on three of the four. On flat graphics and text aom is far ahead, 43–69% smaller. rav1e is also 2–26× slower here, and its CPU time equals its wall time, so it ran on one core, while aom spread the same work over several. **Why the gap on graphics?** AV1 has tools aimed at screen content, such as palette mode and intra block copy. libaom counts the 16×16 blocks with few colors and turns them on when enough of the image qualifies ([`encoder.c`](https://aomedia.googlesource.com/aom/+/refs/heads/main/av1/encoder/encoder.c), `allow_screen_content_tools`), while using palette mode is still an [open rav1e issue](https://github.com/xiph/rav1e/issues/1066). They explain only part of it, though: turning them off in aom (`-a enable-palette=0 -a enable-intrabc=0`) grows the UI screenshot by 18%, still far below rav1e, and leaves the illustration and the diagram byte-identical. The rest comes from the encoders' overall decisions rather than any single tool, which is why measuring beats reading feature lists. @@ -284,21 +286,21 @@ On photographs and the texture rav1e is within a few percent of aom, slightly sm | Image | AVIF (aom, 4:4:4) | aom 4:2:0 | AVIF SVT-AV1 (4:2:0) | | --- | --- | --- | --- | -| `kodim04.png` (photo) | **146.5 KB**, 90.1 | 156.7 KB, 89.9 | 154.9 KB, 89.6 | -| `kodim19.png` (photo) | **145.1 KB**, 89.8 | 153.6 KB, 90.1 | 148.3 KB, 89.4 * | -| `kodim23.png` (photo) | **108.9 KB**, 89.7 | 148.5 KB, 89.9 | 168.5 KB, 90.3 | +| `cid22-friends.png` (photo, people) | **53.0 KB**, 90.0 | 62.9 KB, 89.5 | 72.2 KB, 89.7 | +| `cid22-church.png` (photo, detail) | **66.5 KB**, 89.7 | 87.1 KB, 90.3 | 71.9 KB, 89.7 | +| `cid22-blossoms.png` (photo, color) | **125.0 KB**, 89.7 | 202.9 KB, 78.6 * | 166.7 KB, 78.4 * | | `forest-leaves.png` (texture) | 805.2 KB, 90.4 | **779.3 KB**, 90.4 | 839.0 KB, 90.5 | | `corporate-memphis.png` (illustration) | **18.7 KB**, 89.9 | 56.3 KB, 87.8 * | 43.6 KB, 87.7 * | | `report-ui.png` (UI, text) | **62.6 KB**, 89.9 | 156.4 KB, 78.8 * | 104.2 KB, 78.7 * | | `pipeline-diagram.png` (diagram) | **61.3 KB**, 90.1 | 57.3 KB, 75.4 * | failed (odd height) | -At matched 4:2:0, SVT-AV1 and aom are close on the photographs (−3% to +13%), and on the UI screenshot and the illustration SVT-AV1 reaches the same quality ceiling with fewer bytes (−33% and −22%). What holds it back is the chroma format, not the encoder: forcing aom to 4:2:0 costs it as much (kodim23 +36%, and the illustration, UI and diagram no longer reach the target). Encode times at the default speed are similar to aom's here. +At matched 4:2:0, SVT-AV1 and aom trade places on the photographs (+15% on the people photograph, −17% on the church), and where 4:2:0 cannot reach the target, on the blossoms photograph, the UI screenshot and the illustration, SVT-AV1 reaches the same quality ceiling with fewer bytes (−18%, −33% and −22%). What holds it back is the chroma format, not the encoder: forcing aom to 4:2:0 costs it as much (+19% and +31% on the people and church photographs, and the blossoms photograph, the illustration, the UI and the diagram no longer reach the target). Encode times at the default speed are similar to aom's here. -**Takeaway.** SVT-AV1 is built for video, where 4:2:0 is the norm; for still images AVIF's 4:4:4 default is worth more than the choice of encoder. Use SVT-AV1 for AVIF stills only where 4:2:0 is acceptable (photographs), and expect it to reject odd-sized images. +**Takeaway.** SVT-AV1 is built for video, where 4:2:0 is the norm; for still images AVIF's 4:4:4 default is worth more than the choice of encoder. Use SVT-AV1 for AVIF stills only where 4:2:0 is acceptable, which is a property of the image rather than of photographs as a category: the blossoms photograph, saturated red over a smooth sky, cannot reach the target in 4:2:0 with either encoder. Expect it to reject odd-sized images too. **Using it.** `avifenc -c svt -y 420`; with odd dimensions, crop or pad to even first. -**References.** [SVT-AV1](https://gitlab.com/AOMediaCodec/SVT-AV1) and its [`Parameters.md`](https://gitlab.com/AOMediaCodec/SVT-AV1/-/blob/master/Docs/Parameters.md) (`--color-format`: "only yuv420 is supported at this time"); [`avifenc` manual](https://github.com/AOMediaCodec/libavif/blob/main/doc/avifenc.1.md) (`-y`, and `-c` per build). +**References.** [JPEG XL vs AVIF for web use-cases](https://jakearchibald.com/2026/jpeg-xl-vs-avif/) (Jake Archibald, 2026: "AVIF's 4:2:0 mode (half colour resolution) shouldn't be your default", since it rarely reduces size and makes higher qualities harder to reach); [SVT-AV1](https://gitlab.com/AOMediaCodec/SVT-AV1) and its [`Parameters.md`](https://gitlab.com/AOMediaCodec/SVT-AV1/-/blob/master/Docs/Parameters.md) (`--color-format`: "only yuv420 is supported at this time"); [`avifenc` manual](https://github.com/AOMediaCodec/libavif/blob/main/doc/avifenc.1.md) (`-y`, and `-c` per build). ### libjpeg-turbo (`jpeg-turbo`) @@ -308,23 +310,23 @@ At matched 4:2:0, SVT-AV1 and aom are close on the photographs (−3% to +13%), | Image | libjpeg-turbo | mozjpeg | jpegli | | --- | --- | --- | --- | -| `kodim04.png` (photo) | 166.6 KB, 89.6 | 196.7 KB, 90.1 (+18%) | **127.6 KB**, 90.0 (−23%) | -| `kodim19.png` (photo) | 182.6 KB, 90.4 | 208.0 KB, 89.9 (+14%) | **143.6 KB**, 90.0 (−21%) | -| `kodim23.png` (photo) | 153.0 KB, 90.3 | 127.3 KB, 89.9 (−17%) | **94.8 KB**, 90.2 (−38%) | +| `cid22-friends.png` (photo, people) | 82.3 KB, 89.6 | 89.4 KB, 90.2 (+9%) | **62.9 KB**, 89.8 (−24%) | +| `cid22-church.png` (photo, detail) | 103.7 KB, 90.3 | 97.0 KB, 89.9 (−7%) | **76.4 KB**, 89.7 (−26%) | +| `cid22-blossoms.png` (photo, color) | 219.0 KB, 78.7 * | 160.4 KB, 90.2 | **133.8 KB**, 90.1 | | `forest-leaves.png` (texture) | 919.6 KB, 90.3 | 1093.7 KB, 89.6 (+19%) | **724.5 KB**, 89.9 (−21%) | | `corporate-memphis.png` (illustration) | 127.8 KB, 88.1 * | 97.8 KB, 89.6 | **78.5 KB**, 89.9 | | `report-ui.png` (UI, text) | 302.8 KB, 78.8 * | 213.3 KB, 90.2 | **165.5 KB**, 89.9 | | `pipeline-diagram.png` (diagram) | 177.5 KB, 75.7 * | 176.5 KB, 89.9 | **111.0 KB**, 89.3 * | -jpegli is the smallest on every image, 21–45% below libjpeg-turbo. libjpeg-turbo cannot reach the target on the three graphics images even at `-quality 100`. And mozjpeg, the "improved" encoder, is *larger* than plain libjpeg-turbo on three photographs and the texture. +jpegli is the smallest on every image, 21–45% below libjpeg-turbo. libjpeg-turbo cannot reach the target on the three graphics images and the blossoms photograph, even at its highest settings (`-quality 99`–`100`). And mozjpeg, the "improved" encoder, is *larger* than plain libjpeg-turbo on the people photograph (+9%) and the texture (+19%). -**Why mozjpeg loses on photographs: its defaults, not its compression.** libjpeg-turbo always subsamples color 4:2:0 by default. mozjpeg chooses by quality: 4:4:4 at `-quality` 90 and above, 4:2:2 (`2x1`) from 80, because "for some images chroma subsampling significantly degrades color quality, making it impossible to achieve high visual quality regardless of quality setting" ([`rdswitch.c`](https://github.com/mozilla/mozjpeg/blob/master/rdswitch.c)). Reaching ssimulacra2 90 takes `-quality` 95–99, so mozjpeg encodes 4:4:4, which reaches the target on text and graphics but costs bytes on photographs, where 4:2:0 is enough. Searched to the same target with the subsampling forced equal (`-sample 2x2` on both), mozjpeg is smaller than libjpeg-turbo on all seven images (−4% to −24%), so its encoder improvements are real; at 4:4:4 and these high settings the result is mixed (smaller on the illustration and the diagram, 15–28% larger on the photographs, the texture and the UI screenshot). +**Why mozjpeg loses on photographs: not its subsampling default.** libjpeg-turbo always subsamples color 4:2:0 by default. mozjpeg chooses by quality: 4:4:4 at `-quality` 90 and above, 4:2:2 (`2x1`) from 80, because "for some images chroma subsampling significantly degrades color quality, making it impossible to achieve high visual quality regardless of quality setting" ([`rdswitch.c`](https://github.com/mozilla/mozjpeg/blob/master/rdswitch.c)). Reaching ssimulacra2 90 takes `-quality` 94–99, so mozjpeg encodes 4:4:4. That choice is not what costs it bytes: libjpeg-turbo forced to 4:4:4 (`-sample 1x1`) is *smaller* than its own 4:2:0 default on the three images where both reach the target (−7% to −17%: the texture and the people and church photographs) and reaches the target on the other four, so at this quality full-resolution color pays for itself on every image here, photographs included. Searched to the same target with the subsampling forced equal, the difference is in the encoders: at 4:2:0 (`-sample 2x2`) mozjpeg is smaller than libjpeg-turbo on six of the seven images (−1.5% to −24%; +7% on the people photograph), but at 4:4:4 it is smaller only on the illustration and the diagram, and 5–28% larger on the photographs, the texture and the UI screenshot. mozjpeg's encoder improvements hold at 4:2:0; at 4:4:4 and these high settings, where its default lands, they do not. -**Takeaway.** For JPEG at high quality the subsampling default decides more than the encoder, so compare encoders at matched settings before drawing conclusions. jpegli wins at its defaults across all content here, and is about as fast as libjpeg-turbo (both under 40 ms per encode, against 80–550 ms for mozjpeg). +**Takeaway.** For JPEG at high quality, compare encoders at matched settings before drawing conclusions: the subsampling and how each encoder spends bits at that subsampling both move the result, and whether 4:2:0 is enough depends on the image's color detail, not on whether it is a photograph. jpegli wins at its defaults across all content here, and stays fast (10–90 ms per encode, libjpeg-turbo under 10 ms, against 100 ms–1.1 s for mozjpeg). **Using it.** `cjpeg -quality N` (add `-sample 1x1` for 4:4:4); libjpeg-turbo 3.2 reads PNG directly, earlier releases need PPM, BMP, GIF or Targa. -**References.** [libjpeg-turbo](https://libjpeg-turbo.org/) and its [`usage.txt`](https://github.com/libjpeg-turbo/libjpeg-turbo/blob/main/doc/usage.txt) (default subsampling 2x2, "4:2:0"); [mozjpeg](https://github.com/mozilla/mozjpeg) ("a strict superset of libjpeg-turbo's functionality") and its quality-dependent subsampling in [`rdswitch.c`](https://github.com/mozilla/mozjpeg/blob/master/rdswitch.c); [jpegli](https://github.com/google/jpegli) (adaptive dead-zone quantization). +**References.** [JPEG XL vs AVIF for web use-cases](https://jakearchibald.com/2026/jpeg-xl-vs-avif/) (on 4:2:0 as a default, for AVIF; the JPEG measurements here point the same way); [libjpeg-turbo](https://libjpeg-turbo.org/) and its [`usage.txt`](https://github.com/libjpeg-turbo/libjpeg-turbo/blob/main/doc/usage.txt) (default subsampling 2x2, "4:2:0"); [mozjpeg](https://github.com/mozilla/mozjpeg) ("a strict superset of libjpeg-turbo's functionality") and its quality-dependent subsampling in [`rdswitch.c`](https://github.com/mozilla/mozjpeg/blob/master/rdswitch.c); [jpegli](https://github.com/google/jpegli) (adaptive dead-zone quantization). ## Size-regression gate @@ -372,6 +374,6 @@ the closest bound (visible as `iters` hitting the max with the knob pinned). - Input is a PNG in sRGB. Ancillary chunks (ICC profile, chromaticities) are stripped before measuring, so wide-gamut inputs are treated as sRGB. -- The search target metric is ssimulacra2; dssim is reported alongside for reference. +- The search target metric is ssimulacra2; dssim is reported alongside for reference but does not drive the search. ssimulacra2 comes from Cloudinary (Jon Sneyers, also a JPEG XL author), "a perceptual image quality metric by the folks behind JPEG XL", as [JPEG XL vs AVIF for web use-cases](https://jakearchibald.com/2026/jpeg-xl-vs-avif/) puts it; it was tuned and validated on independent subjective scores (CID22, TID2013, KADID-10k, KonFiG-IQA), but a metric with no tie to any codec, such as [ColorVideoVDP](https://github.com/gfxdisp/ColorVideoVDP), is a useful cross-check where JPEG XL and another format are close. - Codec versions affect the operating point. Use the Docker image for reproducible numbers; a native install will differ with its tool versions. diff --git a/compare/demos/README.md b/compare/demos/README.md index 90d7e9d..d459c71 100644 --- a/compare/demos/README.md +++ b/compare/demos/README.md @@ -16,13 +16,17 @@ The set covers the kinds of content where codecs behave differently, so the repo | Image | Content | Size | Source | License | | --- | --- | --- | --- | --- | -| `kodim04.png`, `kodim19.png`, `kodim23.png` | Photographs | 768×512 | [Kodak Lossless True Color Image Suite](https://r0k.us/graphics/kodak/) | Released by Kodak for unrestricted use | +| `cid22-friends.png` | Photograph: two people, faces and skin | 512×512 | [CID22](https://cloudinary.com/labs/cid22) validation set, reference `5055743` | [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) | +| `cid22-church.png` | Photograph: fine detail (building, grass) | 512×512 | [CID22](https://cloudinary.com/labs/cid22) validation set, reference `2936831` | [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) | +| `cid22-blossoms.png` | Photograph: saturated color over a sky gradient | 512×512 | [CID22](https://cloudinary.com/labs/cid22) validation set, reference `5458393` | [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) | | `corporate-memphis.png` | Flat illustration | 1200×750 | [Corporate Memphis (2019)](https://commons.wikimedia.org/wiki/File:Corporate_Memphis_(2019).png) by Katharina Brenner, Wikimedia Commons | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | | `forest-leaves.png` | High-detail texture | 1024×1024 | [Forest Leaves 02](https://polyhaven.com/a/forest_leaves_02), Poly Haven (diffuse map, 1k) | [CC0](https://polyhaven.com/license) | | `report-ui.png` | UI screenshot with small text | 1200×800 | This project: a screenshot of `example-report.html` | MIT, as the repository | | `pipeline-diagram.png` | Diagram with thin lines and text | 1200×675 | This project: `own/pipeline-diagram.svg` | MIT, as the repository | -Every image is an 8-bit RGB PNG with no alpha channel, and each third-party original is lossless, so no codec is measured against artifacts a previous lossy encode left behind. +Every image is an 8-bit RGB PNG with no alpha channel, so no codec is measured against artifacts a previous lossy encode left behind in the file itself. + +The photographs come from CID22 (Cloudinary Image Dataset '22), the set of subjective quality scores that ssimulacra2, the metric this tool searches on, was tuned and validated against: Jon Sneyers, Elad Ben Baruch and Yaron Vaxman, "CID22: Large-Scale Subjective Quality Assessment for High Fidelity Image Compression" ([doi:10.36227/techrxiv.22659061](https://doi.org/10.36227/techrxiv.22659061)), licensed [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/). Its references are modern photographs downscaled to 512×512, which also hides any compression in the sources they were made from. `fetch.sh` downloads the 17 MB validation set without distorted images and extracts these three; a report that embeds them is an adaptation and carries the same license. Photographs of people show more than one person rather than a single-person portrait. The two own images are committed in `own/` (losslessly optimized with oxipng). Both were rendered with headless Chrome at a device scale factor of 1: diff --git a/compare/demos/baseline.json b/compare/demos/baseline.json index 3e6d122..9144e07 100644 --- a/compare/demos/baseline.json +++ b/compare/demos/baseline.json @@ -1,6 +1,84 @@ { "target": 90, "images": { + "cid22-blossoms.png": { + "JPEG XL": { + "bytes": 105038, + "ssimulacra2": 90.0777 + }, + "WebP": { + "bytes": 118162, + "ssimulacra2": 76.8704 + }, + "AVIF": { + "bytes": 128030, + "ssimulacra2": 89.7455 + }, + "jpegli": { + "bytes": 137045, + "ssimulacra2": 90.1094 + }, + "mozjpeg": { + "bytes": 164234, + "ssimulacra2": 90.2027 + }, + "HEIC": { + "bytes": 194761, + "ssimulacra2": 82.2173 + } + }, + "cid22-church.png": { + "JPEG XL": { + "bytes": 61171, + "ssimulacra2": 89.8528 + }, + "AVIF": { + "bytes": 68127, + "ssimulacra2": 89.7307 + }, + "HEIC": { + "bytes": 71021, + "ssimulacra2": 90.1109 + }, + "jpegli": { + "bytes": 78183, + "ssimulacra2": 89.7383 + }, + "WebP": { + "bytes": 94246, + "ssimulacra2": 88.906 + }, + "mozjpeg": { + "bytes": 99280, + "ssimulacra2": 89.8889 + } + }, + "cid22-friends.png": { + "JPEG XL": { + "bytes": 52793, + "ssimulacra2": 89.8345 + }, + "AVIF": { + "bytes": 54356, + "ssimulacra2": 89.9703 + }, + "jpegli": { + "bytes": 64424, + "ssimulacra2": 89.7992 + }, + "HEIC": { + "bytes": 79196, + "ssimulacra2": 90.0101 + }, + "WebP": { + "bytes": 87218, + "ssimulacra2": 89.4407 + }, + "mozjpeg": { + "bytes": 91561, + "ssimulacra2": 90.2512 + } + }, "corporate-memphis.png": { "AVIF": { "bytes": 19074, @@ -53,84 +131,6 @@ "ssimulacra2": 89.5984 } }, - "kodim04.png": { - "JPEG XL": { - "bytes": 120534, - "ssimulacra2": 90.2114 - }, - "jpegli": { - "bytes": 130656, - "ssimulacra2": 90.0199 - }, - "HEIC": { - "bytes": 141631, - "ssimulacra2": 89.7688 - }, - "AVIF": { - "bytes": 150019, - "ssimulacra2": 90.1069 - }, - "WebP": { - "bytes": 173748, - "ssimulacra2": 89.8091 - }, - "mozjpeg": { - "bytes": 201372, - "ssimulacra2": 90.1333 - } - }, - "kodim19.png": { - "JPEG XL": { - "bytes": 136558, - "ssimulacra2": 90.0209 - }, - "jpegli": { - "bytes": 147032, - "ssimulacra2": 90.0228 - }, - "AVIF": { - "bytes": 148498, - "ssimulacra2": 89.7449 - }, - "HEIC": { - "bytes": 155322, - "ssimulacra2": 89.9361 - }, - "WebP": { - "bytes": 173918, - "ssimulacra2": 89.9646 - }, - "mozjpeg": { - "bytes": 213006, - "ssimulacra2": 89.9248 - } - }, - "kodim23.png": { - "JPEG XL": { - "bytes": 74113, - "ssimulacra2": 89.5865 - }, - "jpegli": { - "bytes": 97039, - "ssimulacra2": 90.2479 - }, - "AVIF": { - "bytes": 111556, - "ssimulacra2": 89.7408 - }, - "mozjpeg": { - "bytes": 130349, - "ssimulacra2": 89.8537 - }, - "WebP": { - "bytes": 140374, - "ssimulacra2": 89.0397 - }, - "HEIC": { - "bytes": 157442, - "ssimulacra2": 90.2236 - } - }, "pipeline-diagram.png": { "AVIF": { "bytes": 62798, diff --git a/compare/demos/fetch.sh b/compare/demos/fetch.sh index 861c6ad..f7f6d58 100755 --- a/compare/demos/fetch.sh +++ b/compare/demos/fetch.sh @@ -13,31 +13,66 @@ sha256() { if command -v sha256sum > /dev/null; then sha256sum "$1" | cut -d' ' -f1; else shasum -a 256 "$1" | cut -d' ' -f1; fi } +# The photographs come from the CID22 validation set, published as one zip: it is +# downloaded once, checked, and only the chosen references are extracted. +cid22_url="https://cloudinary-marketing-res.cloudinary.com/raw/upload/v1682006008/CID22_validation_set_without_distorted.zip" +cid22_sha="daa95777dc8f2abd086449c1eb1e6636ce178bfc5f7346f4bfdb806375319f63" +# name, path inside the zip, sha256 +cid22=( + "cid22-friends.png CID22_validation_set/original/5055743.png a86da9a4a647648f74497bd3e0ca5987020b9d9abcde558e9a5f649b9b8bc0e8" + "cid22-church.png CID22_validation_set/original/2936831.png be4bfd85b5c7bcfca45090271b3750ec548b2c6f9bee0883c2374d1592d7e667" + "cid22-blossoms.png CID22_validation_set/original/5458393.png 1225af495f25c0f07d9c3479c8b9c029bf1ac9f463e498d81790ffac6764a746" +) + # name, url, sha256 sources=( - "kodim04.png https://r0k.us/graphics/kodak/kodak/kodim04.png e3b946107c5d3441c022f678d0c3caf1e224d81b1604ba840a4f88e562de61aa" - "kodim19.png https://r0k.us/graphics/kodak/kodak/kodim19.png b7450b264b1b0a411390d8931b112c27905a992520fc90569dc4b920aa32bbdc" - "kodim23.png https://r0k.us/graphics/kodak/kodak/kodim23.png e3111a2fd4da24af15d6459ef9eacfe54106b38e27b4a21821b75c3f5d2d5baf" "corporate-memphis.png https://upload.wikimedia.org/wikipedia/commons/0/04/Corporate_Memphis_%282019%29.png 72c3eb1cc07649692bf3ca9c636f32cf29b15c9453fc09f03f351bed350d602d" "forest-leaves.png https://dl.polyhaven.org/file/ph-assets/Textures/png/1k/forest_leaves_02/forest_leaves_02_diffuse_1k.png 09903abd88ae5a12932b79f03b0da15f7a952e9b43a4ad041eac31a3b925736d" ) +check() { + local name="$1" file="$2" expected="$3" actual + actual="$(sha256 "$file")" + if [ "$actual" != "$expected" ]; then + echo "checksum mismatch for $name: expected $expected, got $actual" >&2 + rm -f "$file" + exit 1 + fi +} + +have() { [ -s "$1" ] && [ "$(sha256 "$1")" = "$2" ]; } + +missing=0 +for entry in "${cid22[@]}"; do + read -r name _ expected <<< "$entry" + if have "$dir/$name" "$expected"; then echo "have $name"; else missing=1; fi +done +if [ "$missing" = 1 ]; then + zip="$(mktemp)" + trap 'rm -f "$zip"' EXIT + echo "fetching CID22 validation set (17 MB)" + curl -fsSL --max-time 120 "$cid22_url" -o "$zip" + check "CID22 zip" "$zip" "$cid22_sha" + for entry in "${cid22[@]}"; do + read -r name path expected <<< "$entry" + have "$dir/$name" "$expected" && continue + unzip -p "$zip" "$path" > "$dir/$name" + check "$name" "$dir/$name" "$expected" + echo "extracted $name" + done +fi + for entry in "${sources[@]}"; do read -r name url expected <<< "$entry" out="$dir/$name" - if [ -s "$out" ] && [ "$(sha256 "$out")" = "$expected" ]; then + if have "$out" "$expected"; then echo "have $name" continue fi echo "fetching $name" # Wikimedia asks for a descriptive User-Agent. curl -fsSL --max-time 60 -A "image-tools demo fetch (https://github.com/nucliweb/image-tools)" "$url" -o "$out" - actual="$(sha256 "$out")" - if [ "$actual" != "$expected" ]; then - echo "checksum mismatch for $name: expected $expected, got $actual" >&2 - rm -f "$out" - exit 1 - fi + check "$name" "$out" "$expected" done for own in "$here"/own/*.png; do