[SizzleUnrlsd]

Closes #207, closes #210 (its line positions; the dependency maps were fixed in #216).

Findings in JSON, TOML and lock files were located by searching the text for the first line mentioning their key or name: the 33 HAR values of mitmproxy all pointed at the first "headers" of their file, and werkzeug's vulnerable-dependency pointed at flask's dependency list (uv.lock:620). The message also named werkzeug as its own requirer.

Design

• A position component apart from the rules (coretrace_python.source.positions). The detectors still judge the values json and tomllib decode; the component finds where a value is written from its structural path and checks that the text there holds that very value.
  • JSON is read with the position of every value (strings decoded as json decodes them), so repeated keys in different objects, nested arrays and escaped strings are told apart; a key repeated in one object keeps its last value, as json decodes it.
  • TOML: lines are counted by line feeds and read by a small lexer that skips strings and comments and follows arrays and inline tables opened by a value, so a line inside a multi-line string or array is never a header or a key; the key and its value are checked in the section the path names, the root, a [table], or the element of an array of tables that its [[header]] opens, with its [sub.tables]; keys and values are read with tomllib. An element is never taken by its rank alone: its key must hold the value.
  • What it cannot place (a dotted key, an inline table, a multi-line value, a document it cannot read) has no position.
• FileLocation (coretrace_python.source): the file, the JSON pointer (RFC 6901) of a value whose line is not established, and a digest of the value (sha256: and 16 hex digits), never the value itself. SourceSpan keeps its invariant (line ≥ 1); Finding.span is SourceSpan | FileLocation, and config_literals keeps its (value, name, location, function) shape.
  • Reports: JSON line/column/end_* are null with a pointer field; SARIF has no region, the pointer under the location's properties; text shows path: … without a line.
  • Inline suppressions apply only to findings at a verified line.
  • The baseline recognises a file location by its pointer and the digest of its value, so a new value at the same place is a new finding.
  • Cache format bumped to 14 for the new location encoding.
• Baseline schema 2: the fingerprint gains a fifth field, the pointer (empty for a verified line); for a file location, the fourth field, the text of a line otherwise, is the digest of the value. Files are written in schema 2. Schema 1 files are still read and their entries match exactly as they did. Before this change, a finding in a JSON, TOML or lock file was recorded with the text of its key's first line, its own only when the key was not repeated: such an entry still identifies that finding, and no other. A finding an old entry cannot identify with certainty is reported as new, never absorbed, and the check then asks on standard error for the baseline to be recorded again.
• vulnerable-dependency: lock-file findings point at another package's line and name the package as its own requirer #207: a locked package (uv.lock, poetry.lock) is located at the name of its own [[package]] entry, checked by the component, or given a file location (/package/<i>/name). Requirement gains locked, the version as the lock file writes it, and the message states the evidence: CVE-2026-27199: werkzeug 3.1.5, pinned in uv.lock, is in <3.1.6 (required by flask): …. Declared requirements keep their wording.

Validation

• New tests (no behaviour of an existing test changed):
  • tests/test_structured_positions.py (19): repeated keys, nested arrays, escaped strings, duplicate keys, mismatched values; TOML sections, arrays of tables with key and value checked, sub-tables of an element, quoted keys, values without a position; triple quotes in a comment, a one-line string or an escape, and quotes before a closing delimiter; an array element alone on its line; line feeds only (U+2028, CRLF).
  • tests/test_file_locations.py (8): two values of one key at their own lines; a file location; JSON, SARIF and text reports; inline suppression only on verified lines; baseline by pointer in schema 2; a changed value at a file location is a new finding, and the baseline does not contain the value; a separator such as U+2028 above a secret does not make the baseline record another line.
  • tests/test_baseline_schema_1.py (3): a new secret is never accounted for by an old entry (the scenario reproduced in review: it was absorbed before the fix); a repeated key is recognised only where its old entry was certain; Python findings match as before; the notice on standard error.
  • tests/test_line_numbering.py (12): one rule of line numbering (lines_of); a rotated secret below a U+2028, U+0085, form feed or lone \r in .env, YAML, INI and Python files is new; a suppression on the next line does not apply; requirements files and pyproject.toml requirements (PEP 621 array elements, Poetry keys, a file location when unplaceable).
  • tests/test_lock_positions.py (6): uv.lock and poetry.lock positions and message, a file location for an inline lock entry, the wording of a declared requirement.
• Existing tests: exactly three assertions of tests/test_baseline.py changed, with prior approval: the fingerprint's fifth field (empty pointer) in two expectations, and the written schema (1 → 2). They still check the same behaviours (line text not number, fallback without the file, counts).
• Full suite: 1922 passed, 1 failed (test_packaging::test_the_version_has_one_source_of_truth, a stale local install). ruff check, ruff format --check on new files, mypy and git diff --check pass.
• Regression snapshots: unchanged (read-only diff of every repository; MiniBookApiServer and NPL-IA have findings in JSON files, at the same lines).
• mitmproxy at be75825, enterprise plugins at ac35460, [email protected], no baseline: the same 75 results; 34 move. They are the 33 HAR values and werkzeug (uv.lock:620 → 1998), and their new positions are exactly the lines verified by hand in reports/mitmproxy-2026-10-02/triage.csv (34 of 34). Scan time unchanged (51 s, against 59 s on main, on a loaded machine).

Review

An independent review found no blocking defect and three to fix, all fixed and covered by tests: a file location was fingerprinted by its pointer alone, so a new secret at the same place was baselined (now a digest of the value is compared); triple quotes in a comment, a one-line string or an escape could put a value on a line inside a multi-line TOML string (now a lexer); TOML lines were counted by splitlines, which also breaks on U+2028 (now line feeds only). Each section's keys are now indexed once: a flat section of 3,000 keys is read in 0.04 s instead of 13.5 s.

A second review of those fixes found two more, fixed and covered by tests: the baseline read a finding's line with splitlines, so a U+2028 or U+0085 above a secret recorded another line's text and could absorb a new secret (the baseline now counts line feeds, as the parser and the component do); a TOML multi-line string closed by four or five quotes inverted the lexer, so a key inside a later string could be located (the quotes before the closing delimiter now belong to the string). FileLocation now requires its pointer and digest.

A third review found that line numbers had no single rule: the baseline counted line feeds while the .env, YAML, INI and requirements readers used splitlines and the parser also breaks on a lone \r, so a fingerprint or an inline suppression could refer to another line, and a rotated secret could be absorbed; and that pyproject.toml requirements kept guessed lines (first line containing their text, or line 1). lines_of (in coretrace_python.source) now counts lines as Python does, \n, \r\n or a lone \r, never U+2028 or a form feed, and every producer and consumer of a line number uses it; the position component finds the element of an array a key holds, so pyproject.toml requirements are located at their own element (PEP 621) or key (Poetry), or given a file location.

A fourth review found that an array element written over several lines was placed at the line it starts on, which does not hold it, so a changed multi-line secret was baselined, and that array elements were found again for every value (40 s for 4,000 elements). An element is now delimited by the end of its own value; one that spans lines gets a file location; each array is read once (0.04 s). Two older defects it found, outside this diff, are tracked in #223 (a secret in a multi-line Python string is recognised by its first line only) and #224 (accepted secrets at a line are recorded in the baseline in plain text).

A fifth review confirmed both fixes (fuzzed against tomllib on about 12,000 arrays: no returned position belongs to a multi-line element) and found that a datetime written with a space counted as two array elements; a bare scalar now ends at a separator or the end of its line. Closing a multi-line string no longer copies the rest of the text.

Not covered

• baseline: a secret in a multi-line Python string is recognised by its first line only #223, baseline: accepted secrets at a line are recorded in plain text #224: older baseline defects found in review (see above).

• requirements.txt lines are numbered as Python counts them; pip splits them with str.splitlines, so a form feed or U+2028 between two requirements is read as one malformed requirement here.

• JsonPositions gives up on a document nested beyond its recursion limit or using NaN/Infinity: every value of it then gets a file location (a safe degradation).

• Whether GitHub code scanning accepts a SARIF result whose location has no region is to be checked on an actual upload.

• YAML, INI, .env and properties files are read line by line, their positions exact by construction; YAML nested keys are not given structural paths.