Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,17 @@ feed ever served.
contents of the layers, and the entry below is written against the published `preview.1` rather than
against an empty feed.

## [Unreleased]

### Added

- **Run-length images are decoded.** `RunLengthDecode` joins Flate, LZW and the two ASCII filters: a
bilevel or scan-style image that declares it now yields its samples — and `Gray8`, `Rgb24` or `Cmyk32`
pixels — rather than `ImageFilterUnsupported`, and a stream that ends without its end-of-data byte is
decoded as far as it goes. [`ADR-005`](docs/adr/0005-image-filter-decoders.md) scopes the remaining four
filters — `CCITTFaxDecode`, `DCTDecode`, `JPXDecode` and `JBIG2Decode` — with the evidence path each one
has to meet.

## [1.8.0] - 2026-11-14

### Added
Expand Down
82 changes: 82 additions & 0 deletions docs/adr/0005-image-filter-decoders.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# ADR-005: Decoding the image filters the reader passes through

- **Status**: **Accepted** (2026-11-14) — `RunLengthDecode` implemented; `CCITTFaxDecode`, `DCTDecode`,
`JPXDecode` and `JBIG2Decode` accepted in that order, one release slice each
- **Date**: 2026-11-14
- **Related**: [`docs/parity.md`](../parity.md) row 14,
[`docs/manual/src/08-inspection.md`](../../manual/src/08-inspection.md),
[`ADR-001`](0001-pdf-platform-boundary.md)

## Context

Row 14 of the parity matrix asks the reader to "decode image pixels when asked". It does that for
`FlateDecode` (with predictors), `LZWDecode`, `ASCIIHexDecode`, `ASCII85Decode` and, since this ADR,
`RunLengthDecode`. Every other filter is handed back as stored bytes with `ImageFilterUnsupported` rather
than decoded, which is correct for a filter nobody has written a decoder for — but four of those filters
are common enough that "the caller can read the JPEG itself" is not an answer for a *pixel* API:

| Filter | What it is | Why it matters |
| --- | --- | --- |
| `CCITTFaxDecode` | Group 3 and Group 4 fax coding for bilevel data | the shape a scanned page usually has |
| `DCTDecode` | JPEG (baseline, extended, progressive) | the shape a photograph or a screenshot usually has |
| `JPXDecode` | JPEG 2000 (JP2 or raw codestream) | mandated for PDF/A-2 and later, so archival documents carry it |
| `JBIG2Decode` | JBIG2 generic region, symbol dictionary, text region | the compact shape a scanned text page can have |

`CCITTFaxDecode`, `JPXDecode` and `JBIG2Decode` are **not** bytes a caller can hand to another reader: they
have no container a JPEG viewer accepts. `DCTDecode` is the exception — its bytes *are* a JPEG — which is
why it was the one pass-through this library documented.

## Decision

Each filter is **hand-written**, in this repository, against the specification, with no new package
reference. ADR-001 keeps the product free of a third-party PDF or imaging engine, and the trimming and
Native AOT claims rest on `src/` depending on the base class library alone. A decoder is added in
`PdfFilters` (a partial beside `PdfFilters.Lzw.cs`), wired into `PdfImageFilters.TryDecodeOne`, and exposed
through the same contract `ImagePixels.Of` already applies: `Gray8`, `Rgb24` or `Cmyk32`, a payload that
matches the geometry, and `null` for anything the decoder cannot state.

Shipping order and acceptance criteria:

| Filter | Variants in scope | Acceptance |
| --- | --- | --- |
| `RunLengthDecode` | — | **Done.** Literal and repeat runs, an optional end-of-data byte, a truncated stream decoded as far as it goes. `PdfImageFilterTests` and the `FilterChains` fixture pin it, and the image is exposed as `Gray8`. |
| `CCITTFaxDecode` | Group 3 one-dimensional (`/K` = 0), Group 3 two-dimensional (`/K` > 0), Group 4 (`/K` < 0); `/Columns`, `/Rows`, `/BlackIs1`, `/EncodedByteAlign`, `/EndOfLine` | The decoded bilevel bits equal an independent decoder's output on the same bytes. Evidence: Pillow's CCITT reader through `pikepdf` on the fixture, and a real fax sample in `tests/.../Assets/parity/images`. |
| `DCTDecode` | SOF0, SOF1, SOF2; restart markers; 1, 3 and 4 components; 4:4:4, 4:2:2, 4:2:0; YCbCr and CMYK/YCCK (Adobe APP14); EXIF orientation | Geometry and component count exact; every sample within **2 levels** of a reference decoder (an integer IDCT is allowed to differ by rounding). Evidence: Pillow on the `parity/images/jpeg-*.jpg` fixtures, compared pixel by pixel, with the per-image maximum recorded in `docs/mutation-testing.md`'s sibling measurement note. |
| `JPXDecode` | JP2 and raw J2K codestream; 5/3 reversible and 9/7 irreversible; tiles and tile-parts | Reversible path **bit-exact**; irreversible path within 2 levels. Evidence: `opj_decompress` on the same codestream. |
| `JBIG2Decode` | Generic region, symbol dictionary, text region, MMR | Bit-exact bilevel output. Evidence: `jbig2dec` on the same stream. |

Every slice also lands a manual section (chapter 8's filter table and the image chapter), a sentence in
parity row 14, a changelog entry, and a scoped Stryker run over the new decoder whose survivors are read
and recorded, the way this repository treats every other file.

## Rejected alternatives

- **A package** (`SkiaSharp`, `ImageSharp`, `Magick.NET`): breaks ADR-001 and the "depends on nothing but the
base class library" claim the AOT smoke app checks; it also puts a native load in the first call that
reads a page.
- **Shelling out** to `opj_decompress`, `jbig2dec` or `djpeg`: a library that needs an external process to
read a page is not the product.
- **Leaving `DCTDecode` as a pass-through**: a caller can already read the JPEG from `PdfImage.Data`; what
it cannot do is get *pixels*, which is what row 14 asks for.

## Consequences

- The order is by frequency: most scanned pages are CCITT G4 or JPEG, so the first two slices cover the
common case before the archival and text-optimised codecs follow.
- `JPXDecode` and `JBIG2Decode` are the largest. If a slice's acceptance criteria prove too broad (JPX
tiling, JBIG2 symbol refinement), that slice gets its own ADR rather than widening this one.
- A decoded JPEG or JP2 image stops reporting `IsDecodable=false`, so the `ImagePixels` budget rules apply:
a payload above the inline budget is measured rather than held, and `PixelFormat` stays `null` for it.
- `PdfFilters.Decode` (the strict route for object streams and attachments) is unchanged: a filter is added
there only when the specification allows it in an object stream, which `RunLengthDecode` and the four
image codecs do not.

## Status of the slices

| Slice | State |
| --- | --- |
| `RunLengthDecode` | **Implemented** (2026-11-14) |
| `CCITTFaxDecode` | Planned — the next slice |
| `DCTDecode` | Planned |
| `JPXDecode` | Planned |
| `JBIG2Decode` | Planned |
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ what it obliges later work to prove; it does not restate the implementation.
| [0002](0002-new-code-coverage-1.1.0.md) | Accepted | `1.1.0` ships with its new-code coverage measured at 77.76% against the 90% floor, recorded rather than waived; no test is written to move the number, and a named `1.2` task closes the gap on the parsers' refusal branches. |
| [0003](0003-per-letter-read-model.md) | Accepted | The per-letter read model (`PdfGlyph`, `PdfLetter`, `PdfRectangle`) stays a record graph in 1.x: the struct conversion was measured and raised allocations, because `PdfWord` stores its own copies. The 2.0 candidate is slices over one array plus structs, to be measured before adoption. |
| [0004](0004-buffer-model.md) | Accepted | The document model stays one `byte[]` and `System.IO.Pipelines`/`ReadOnlySequence<byte>` are not adopted in 1.x: the read path was measured at 1.00× input and a file-backed merge at 2.04×, because the application layer reads each source once to resolve the selection and the engine reads it again. The recommendation is to hand the inspected bytes to the engine (B1), which needs a maintainer's approval before any code moves. |
| [0005](0005-image-filter-decoders.md) | Accepted | The image filters the reader passes through are decoded in-house, one release slice each: `RunLengthDecode` (done, 2026-11-14), then `CCITTFaxDecode`, `DCTDecode`, `JPXDecode` and `JBIG2Decode`. No package reference is added, and each slice lands with an independent-decoder comparison and a scoped mutation record. |

New records use the next four-digit number and are added to this table in the same pull request. A
record is superseded, not rewritten: a later ADR states which decision replaces it and why.
7 changes: 4 additions & 3 deletions docs/manual/src/08-inspection.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,8 +232,9 @@ for four, the array form of `CalGray` or `CalRGB`, an `ICCBased` space whose pro
component counts, or an `Indexed` space over any of them. An image that carries no filter is the simplest
case rather than an exception — its payload is its samples — so an inline image, which producers rarely
compress, states its pixels the way a Flate one does. Nothing is left behind a filter either, because the
filters an image realistically uses are decoded here — Flate and LZW, each with its predictor, and the two
ASCII encodings — so an image that a viewer can show is an image whose samples this record states.
filters an image realistically uses are decoded here — Flate and LZW, each with its predictor, the two
ASCII encodings, and run-length — so an image that a viewer can show is an image whose samples this record
states.

Two things are unpacked on the way out rather than handed over as they lie in the file. Samples narrower than
a byte are widened to the same scale, so a bilevel image yields `0` and `255` rather than `0` and `1`, a
Expand All @@ -250,7 +251,7 @@ which one it is:
| Case | What the entry says | Why no pixels |
| --- | --- | --- |
| A `DCTDecode` image | `IsDecodable` true, `Filter` `DCTDecode`, `PixelFormat` null | The bytes are already a JPEG, which a caller can read; unpacking it here would mean shipping a second decoder. |
| A filter the library passes through — `JPXDecode`, `JBIG2Decode`, `CCITTFaxDecode`, `RunLengthDecode`, or a name it does not know | `IsDecodable` false, `Filter` names it, and the read warns with `ImageFilterUnsupported` | Nothing was decoded, so there are no samples to hand back. |
| A filter the library passes through — `JPXDecode`, `JBIG2Decode`, `CCITTFaxDecode`, or a name it does not know | `IsDecodable` false, `Filter` names it, and the read warns with `ImageFilterUnsupported` | Nothing was decoded, so there are no samples to hand back. [`ADR-005`](../../adr/0005-image-filter-decoders.md) schedules these decoders, in that order. |
| A `Separation`, `DeviceN` or `Lab` space, sixteen bits per sample, a palette whose samples run past its table or whose table holds nothing, a width or height of zero, or a geometry the document does not state | `BitsPerComponent` or `ColorSpace` states which | The first needs a function evaluated, the second a rounding rule, the third a decision the file does not support, and the last a rectangle that is not there. Pixels are eight bit and channel-ordered by contract, so converting or guessing would hide the mismatch from the caller. |

Neither a colour profile nor a tint transform is applied: the samples are what the document stores. That is
Expand Down
15 changes: 15 additions & 0 deletions docs/mutation-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -835,3 +835,18 @@ The thirteen that remain, read one by one:
layout fixtures in the suite are left-aligned, so `+` and `-` record the same operation. A centred or
indented layout is the fixture that would close it.

## RunLengthDecode — the first image codec (2026-11-14)

`RunLengthDecode` joined the decoded filters, and its decoder is a new file, so it gets its own reading:

| File | Score | Killed | Survived | No coverage | Mutants |
| --- | --- | --- | --- | --- | --- |
| `Pdf/PdfFilters.RunLength.cs` | **96.00%** | 22 | 1 | 2 | 31 |

The one survivor is `control < RunLengthEndOfData` mutated to `<=`: the `control == RunLengthEndOfData`
check above the branch already breaks on the end-of-data byte, so no input distinguishes the two
comparisons. Two earlier survivors were statements rather than missing tests — a `Math.Max` capacity hint
and a `count <= 0` guard whose empty write and zero step are the same behaviour — and both were **removed**
rather than pinned, which is what a survivor is for. `PdfImageFilters` gained the whitelist entry and the
dispatch case, and the RunLength tests exercise both through `Decode`.

Loading
Loading