[Wave 4] Low/info — Module sweep: Dev tooling: documentation, site & CLI-help CI guards (part 2/3) #671

Wave 4 · 20 finding(s) in 20 fix package(s). Fix these whenever a pull request touches this module ("clean the module you touch").

Findings

• CR-A17a-028 (code, Low) — Quadratic-time link regexes in docs guards contradict their 'backtracking-free' claim
  severity Low · reachability default · intent contract-violation · confidence confirmed · single reviewer · reproduced end to end
  Failure scenario. A fork-PR author (TC-U, IN-17) commits a docs/*.md or README.md with one 1 MB line of '[' characters. The anchor, usermanual, source-path and readme guards then burn tens of minutes of CI time (or a reviewer's local gauntlet) on that PR. The impact is availability of the contributor's own PR run only.
  Where. tools/check_anchor_resolution.py:52, tools/check_usermanual_self_contained.py:51, tools/check_source_path_refs.py:234, tools/check_readme_links.py:96
  Recommended fix. Cap line length before matching (skip or flag lines over about 2000 characters), or use a linear scanner. Correct the 'backtracking-free' comments and add a regex.md-style timing test.

• CR-A17a-029 (code, Low) — The catalog guard cannot detect removal of an audit emission whose name is also a webhook event
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · single reviewer · reproduced end to end
  Where. tools/check_audit_event_catalog.py:1-20, tools/check_audit_event_catalog.py:128-139, forgelm/webhook.py:593, forgelm/cli/_pipeline.py:1133
  Recommended fix. Reconcile the two vocabularies separately: audit emissions (log_event/_audit_event/_EVT) against the audit tables, and webhook event= against the webhook table.

• CR-A17a-030 (code, Low) — The api_version compatibility guard cannot detect nested or type-level changes and does not compare versions
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end
  Where. tests/test_library_api.py:758, tests/_data/api_signatures_1.2.0.json, tools/check_library_api_doc.py:40
  Recommended fix. Snapshot the full ForgeConfig JSON schema (nested fields, types, constraints) and the dataclass field types and defaults. Add a cross-version check that diffs the new snapshot against the previous version's and demands the matching bump class (MAJOR for removals or narrowing, MINOR for additions).

• CR-A17a-031 (code, Low) — Guide rendering trusts DOMParser as a sanitizer: raw HTML, javascript: links and unescaped heading ids from the user-manual build reach the live DOM, with no CSP
  severity Low · reachability undocumented · intent intended-but-harmful · confidence likely · single reviewer · reproduced (plumbing only)
  Failure scenario. A contributor (TC-U, fork PR) adds docs/usermanuals/en/

  /.md containing '' or 'Download' or a heading '## T {id="x" onmouseover="..."}'. A reviewer reading the rendered diff may not notice, because the guard suite turns green. After merge, site-deploy publishes the bundle and any visitor who opens that guide page (or clicks the link) runs the attacker's script in the forgelm.dev origin, which could rewrite the pip-install snippets on the…
  Where. tools/build_usermanuals.py:237-252, tools/build_usermanuals.py:295-297, site/js/guide.js:40-44 (confirmed), site/js/guide.js:243-249
  Recommended fix. Sanitise at build time: strip or allowlist raw HTML, reject non-http(s)/mailto/#/ hrefs, and escape h.id in guide.js. Alternatively pass the rendered html through an allowlist sanitiser (e.g. bleach or nh3) in render_markdown. Remove javascript: from the skip list in check_usermanual_self_contained.py and fail on it. Add a CSP meta tag to site/*.html. Correct the guide.js comment.

• CR-A17a-033 (code, Low) — build_usermanuals.py, update_site_version.py and check_no_analysis_refs.py have zero test coverage; all five requested mutants survive the full suite
  severity Low · reachability default · intent intended-but-harmful · confidence likely · independently confirmed · reproduced end to end
  Where. tools/build_usermanuals.py:277-289, tools/build_usermanuals.py:443-479, tools/update_site_version.py:88-197, tools/update_site_version.py:216-227
  Recommended fix. Add tests/test_build_usermanuals.py, test_update_site_version.py and test_check_no_analysis_refs.py in the existing test_check_* style. Cover parse_front_matter (valid, absent, malformed, non-dict), a stale-file build with --check, one fixture per rewrite pattern plus a --check drift test asserting exit 1, and a positive and negative fixture per branch of _PROHIBITED_RE including 'marketing'.

• CR-A17a-034 (code, Low) — A usermanuals page missing from _meta.yaml is silently left unpublished by the builder; the only detector is a path-filtered CI workflow
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end
  Where. tools/build_usermanuals.py:300-309, tools/build_usermanuals.py:306-309, .github/workflows/usermanuals-validate.yml:73-96
  Recommended fix. Move the orphan check into build_usermanuals.py (fail on any //.md not declared in meta), or add it as a tools/check*.py guard wired into ci.yml and the gauntlet, with a test. Do not leave the sole detector as an untested inline workflow script.

• CR-A17a-035 (code, Low) — None of the 8 verified false-pass classes is pinned by any test in tests/test_check_*.py
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. A contributor re-introduces or widens any of these 8 defects, or someone adds a fabricated YAML fence or junk subcommand to the docs. Every test in the 11 files stays green, because no test constructs the failing input.
  Where. tests/test_check_yaml_snippets.py:41-86, tests/test_check_usermanual_schema_drift.py:214-220, tests/test_check_cli_help_consistency.py:471, tests/test_check_site_claims.py:60
  Recommended fix. Add the one regression test per class that this section describes: a tmp_path doc with a list-indented fence; a doc with 'forgelm --fake-flag'; an assertion that 'for' and 'single' are not in discover_parser_surface().subcommands; check_artifacts() against an empty code set; _parse_blocks on an unparsable file; a '_to_int("on") is None' and 'fire on lifecycle events' test; a fenced-comment anchor…

• CR-A17a-036 (code, Low) — check_cli_help_consistency takes the parser surface from the cwd's (or installed) forgelm, not from --repo-root
  severity Low · reachability documented-opt-in · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end
  Failure scenario. Tree B = copy of the snapshot with an extra '--zzz-new-flag' on chat, plus a doc citing it. (1) cwd=pristine tree A, --repo-root B: false FAIL, 'flag --zzz-new-flag not in parser surface for subcommand chat', EXIT=1. (2) cwd=B, --repo-root=A, A's doc cites the flag that only B's parser has: 'OK: 1 forgelm invocation(s)', EXIT=0, so a ghost flag passes. (3) cwd without a forgelm/ dir plus the installed-wheel lane: surface comes from the installed wheel, not the checkout under review. This is the…
  Where. tools/check_cli_help_consistency.py:132-137, tools/check_cli_help_consistency.py:919-923
  Recommended fix. Pass cwd=repo_root to subprocess.run, and prepend repo_root to PYTHONPATH in the child env. Then assert that forgelm.file is under repo_root, as check_import_origin does. Fail with exit 1 if it is not. Make _run_help take repo_root.

• CR-A17a-037 (code, Low) — check_bilingual_code_blocks checks no content in non-YAML blocks, yet its docstring and localization.md say code blocks must be identical; a TR-only ingest command already ships
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Failure scenario. A maintainer changes a flag, value or subcommand in an EN bash/python example (for example a rename or a removed flag) and does not touch the TR mirror, or the reverse. CI passes because counts and languages still match, so one language keeps documenting a stale or absent command. The ingestion pair is already in this state: TR documents an extra command that EN lacks.
  Where. tools/check_bilingual_code_blocks.py:146-155, tools/check_bilingual_code_blocks.py:35-37, docs/standards/localization.md:61, docs/standards/localization.md:69-74
  Recommended fix. For shell-family and python blocks, compare normalised executable lines: strip comments and quoted literals, drop console output lines and the leading $ prompt, then compare forgelm subcommands, --flags with their values, paths, and python call names with their kwargs. Allowlist translated placeholders explicitly. Add a regression test with a flag-drift pair. Update localization.md to name the gu…

• CR-A17a-038 (code, Low) — check_site_claims validates four literal claims in static HTML only; translations.js and prose claims are never checked
  severity Low · reachability default · intent intended-but-harmful · confidence confirmed · independently confirmed · reproduced end to end
  Failure scenario. EN is corrected (as with home.hero.compliance.lead) and the other five locales keep or add an overclaim with no CI signal.
  Where. tools/check_site_claims.py:252-261, tools/check_site_claims.py:344-363, tools/check_site_claims.py:202-273 (confirmed), tools/check_site_claims.py:429-435 (confirmed)
  Recommended fix. Add a cheap guard that loads translations.js (node or a JS-literal parser) and asserts per-key number multisets match EN, including spelled-out small numbers in fr/es/zh/de/tr. Optionally add a denylist of overclaim terms per locale (conforme/uyumlu/manipulationssicher/infalsifiable).

• CR-A17a-039 (code, Low) — No link-check, HTML validity, accessibility, guide routing or search test for site/
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · not reproduced (code trace)
  Where. tools/check_anchor_resolution.py, tools/check_readme_links.py, tools/check_usermanual_self_contained.py, site/js/guide.js
  Recommended fix. Add an offline href resolver for site/*.html and a smoke test for guide.js routing.

• CR-A17a-040 (code, Low) — User-manual link guard approves #/section/page#anchor routes and unregistered pages that the SPA renders as 'not found'
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Where. tools/check_usermanual_self_contained.py:73-76, tools/check_usermanual_self_contained.py:166-167, site/js/guide.js:69-75, site/js/guide.js:166-174
  Recommended fix. Either make parseHash split off a trailing '#' (and scroll to it), or reject the anchor-suffixed form in the guard. Validate SPA routes against the _meta.yaml routes, not only the disk.

• CR-A17a-041 (code, Low) — Localization and user-manual build docs describe fallback mechanisms the build does not implement
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end
  Failure scenario. A maintainer adds a page with only en/tr titles; the build passes and the de/fr/es/zh navigation silently shows English titles, contrary to the 'required' rule.
  Where. tools/build_usermanuals.py:9-31 (confirmed), docs/standards/localization.md:13 (refuted), docs/standards/localization.md:155-161 (refuted), docs/usermanuals/_meta.yaml:4-7
  Recommended fix. Update the docstrings and standards text to match the code.

• CR-A17a-042 (code, Low) — update_site_version.py derives 'latest release' with its own brittle parser, so --check passes on a stale site
  severity Low · reachability default · intent intended-but-harmful · confidence likely · independently confirmed · reproduced end to end
  Failure scenario. CHANGELOG gets '## [0.12.0] — 2026-11-01 (security hotfix)' or '## [0.12.0] - 2026-11-01' on top. check_release_record_sync sees v0.12.0 as newest. update_site_version --check prints 'OK: site/ version literals match CHANGELOG latest release (0.11.0).' and exits 0, so the release ships with the site badge, JSON-LD and pip pins on 0.11.0, which is the drift the tool exists to prevent.
  Where. tools/update_site_version.py:68-80, tools/update_site_version.py:76-80, tools/update_site_version.py:138-142, tools/check_release_record_sync.py:257
  Recommended fix. Import check_release_record_sync's parse_changelog_releases/newest_release (or a shared helper) so both guards use one definition of 'latest released', and fail when the newest parsed heading does not match the site grammar. Anchor the literal patterns against a trailing pre-release suffix. Add tests/test_update_site_version.py covering these cases.

• CR-A17a-043 (code, Low) — Extends 'troubleshooting.md advises non-existent training.gradient_checkpointing': five more inline config keys on the same page (EN+TR) do not exist
  severity Low · reachability default · intent contract-violation · confidence likely · independently confirmed · reproduced end to end · already tracked: KNOWN-OPEN C31-DOC-SCHEMA-PROSE — docs/roadmap/phase-16-trust-surface-hardening.md:785-786
  Failure scenario. An operator applies a documented troubleshooting fix and gets a config error, or cannot find the knob, and the real cause stays unaddressed.
  Where. docs/usermanuals/en/operations/troubleshooting.md:74, docs/usermanuals/en/operations/troubleshooting.md:130, docs/usermanuals/en/operations/troubleshooting.md:155, docs/usermanuals/tr/operations/troubleshooting.md:74
  Recommended fix. Correct each key to the real field, or remove the advice. Consider extending check_usermanual_schema_drift to validate backticked dotted paths whose first segment is a ForgeConfig block.

• CR-A17a-044 (code, Low) — Neither YAML doc guard can see type-invalid values in usermanual fragments; the lora.md block also hides its first value behind duplicate keys
  severity Low · reachability default · intent contract-violation · confidence likely · single reviewer · reproduced end to end · already tracked: KNOWN-OPEN C31-DOC-SCHEMA-PROSE — docs/roadmap/phase-16-trust-surface-hardening.md:773,785-786
  Failure scenario. A usermanual fragment with a wrongly typed value (here a str for a List[str] field) ships while check_yaml_snippets.py --strict prints 'OK: 33 ForgeConfig-shaped YAML snippet(s) validated across 306 total yaml blocks' and check_usermanual_schema_drift.py --strict prints 'OK: 180 yaml block(s) under docs/usermanuals/ checked'. Both exit 0 on the snapshot.
  Where. tools/check_yaml_snippets.py:84, tools/check_yaml_snippets.py:139-149, tools/check_yaml_snippets.py:173-174, tools/check_usermanual_schema_drift.py:50-54
  Recommended fix. For every recognised top-level block in a fragment, instantiate the matching pydantic block model (LoraConfigModel, TrainingConfig, ...) instead of only walking keys, and load snippets with a duplicate-key-rejecting loader. Rewrite the lora.md example as separate alternatives (separate blocks, or the other forms in comments).

• CR-A17a-045 (code, Low) — No guard covers exit-contract enumerations outside _parser.py help; check_cli_exit_code_prose cannot see any of the stale sites
  severity Low · reachability default · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end · already tracked: KNOWN-OPEN C31-EXIT-CODE-PROSE — docs/roadmap/phase-16-trust-surface-hardening.md:782-785,790-792
  Failure scenario. The next new public code, or a new doc page copying '0/1/2/3/4', ships green. That already happened to 35 sites in three generations (0-4, 0-5, 0-6).
  Where. tools/check_cli_exit_code_prose.py:54, tools/check_cli_exit_code_prose.py:98, tools/check_cli_exit_code_prose.py:207, tools/check_cli_exit_code_prose.py:31
  Recommended fix. Add an enumerated-contract check: canonical set from _PUBLIC_EXIT_CODES (AST), scan for slash enumerations in the listed surfaces, fail when one differs unless allowlisted with a written scope justification. Optionally flag the dash form near 'exit'/'contract' with an allowlist.

• CR-A17a-046 (code, Low) — Lower-grade exit-code recipe defects in the cicd_pipeline guide (EN and TR) and operations/cicd.md: an EXIT_CODE that only ever records 0, case branches that never run under bash -e, forgelm | jq p…
  severity Low · reachability documented-opt-in · intent intended-but-harmful · confidence likely · single reviewer · reproduced with stubs
  Failure scenario. An operator builds a later step on $EXIT_CODE or on the case messages and never sees a non-zero value. Or they use the 'check if benchmark passed' jq line as a gate, and a crashed or failed run passes.
  Where. docs/guides/cicd_pipeline.md:61-64, docs/guides/cicd_pipeline-tr.md:67-70, docs/guides/cicd_pipeline.md:219-229, docs/guides/cicd_pipeline-tr.md:227-237
  Recommended fix. Use set +e; forgelm ...; rc=$?; set -e before recording or branching on the exit code. Add set -o pipefail (or shell: bash) to every forgelm | jq example, and run training once to a file before reading it. Give the GitLab train job allow_failure: exit_codes: [4] so deploy can run, and do not suggest CI self-approval.

• CR-A17a-047 (code, Info) — Performance notes for the docs guards (expected gains)
  severity Info · reachability unreachable · intent intended-and-acceptable · confidence likely · single reviewer · reproduced end to end
  Failure scenario. None: the only practical risk is a subprocess hang in check_cli_help_consistency (no timeout=), covered in the top-level-forms finding.
  Where. tools/check_cli_help_consistency.py:123-141,202-214, tools/check_doc_numerical_claims.py:231-247,386-387, tools/check_anchor_resolution.py:239-286, tools/check_source_path_refs.py:615-616
  Recommended fix. Optional: ThreadPool for the 20 help spawns plus subprocess timeout; cache parsed modules; read docs once.

• CR-A17a-048 (code, Info) — Site CLI and YAML claims are not validated by any guard (F-146 'site YAML/CLI claims validate against the code' overstates)
  severity Info · reachability default · intent intended-but-harmful · confidence likely · single reviewer · reproduced end to end
  Failure scenario. A flag is renamed or removed. docs/ is swept by the CLI guard, but site/quickstart.html and site/features.html keep advertising the dead command, and CI never notices.
  Where. tools/check_site_claims.py:429, tools/check_cli_help_consistency.py:299-302, tools/check_site_claims.py:297-435
  Recommended fix. Add site/*.html (tag-stripped) as a scope of the CLI-help guard, or reword the promise to name what check_site_claims actually verifies.

Done when

• Every finding above is fixed or closed with a reason (tick it here).
• Code fixes carry a test; doc fixes update the Turkish mirror in the same PR.

_{Found by the October 2026 ForgeLM code and documentation review against commit cab2947. Code links point at that commit; line numbers may have moved since. Finding IDs (CR-…, DOC-…, TECH-…) are stable references for this backlog.}