ci(health): bump pyscn to 1.32.4 and read its snake_case report

.github/workflows/code-health.yml

@@ -43,33 +43,46 @@ jobs:
           pip install -e .[metrics,health]
 
       - name: Structural analysis (pyscn)
-        # One output format per invocation, so run it twice: JSON to read the
-        # numbers, HTML to read the report. Never fails the job -- a non-zero
-        # exit here means "findings", which are the point.
+        # One run writes both the JSON report (the numbers) and the HTML report.
+        # Never fails the job -- a non-zero exit here means "findings", which
+        # are the point.
         continue-on-error: true
         run: |
-          pyscn analyze --json --no-open dataretrieval 2>&1 | tee pyscn-summary.txt
-          pyscn analyze --html --no-open dataretrieval >/dev/null
+          pyscn analyze --json --html --no-open dataretrieval 2>&1 | tee pyscn-summary.txt
 
       - name: Check the analysis actually resolved the package
-        # pyscn infers a project root, and when it guesses wrong it silently
-        # resolves only a fraction of the imports -- which *raises* the score,
-        # because most of what it grades is dependency-derived. A degraded run
-        # therefore looks like an improved one. Record the resolved edge count
-        # next to the score so that is visible rather than flattering.
+        # pyscn infers a project root. If it chose the wrong one, it would
+        # resolve only part of the package, which *raises* the score, because
+        # most of what it grades is dependency-derived. Its unresolved-import
+        # count does not show this: under a wrong root the package's own
+        # imports count as external. So compare the root with the checkout and
+        # the module count with the files on disk, and print each warning on
+        # its own. Missing report keys are named rather than raising, so a
+        # renamed key shows in the summary instead of dropping this line.
         if: always()
         continue-on-error: true
         run: |
-          python - <<'PY' > pyscn-sanity.txt
+          python - <<'PY' > pyscn-sanity.txt 2>&1
           import glob, json, os
           reports = sorted(glob.glob(".pyscn/reports/*.json"))
           if not reports:
               print("no pyscn JSON report found"); raise SystemExit
-          s = json.load(open(reports[-1]))["system"]["Summary"]
-          root, deps = s["ProjectRoot"], s["TotalDependencies"]
-          print(f"modules={s['TotalModules']} resolved_dependencies={deps} root={root}")
+          s = json.load(open(reports[-1])).get("system", {}).get("summary", {})
+          keys = ("project_root", "total_modules", "total_dependencies")
+          missing = [k for k in keys if k not in s]
+          if missing:
+              print(f"WARNING: the pyscn report has no system.summary {missing}; "
+                    "update this check for the pinned pyscn version.")
+              raise SystemExit
+          root, modules = s["project_root"], s["total_modules"]
+          files = len(glob.glob("dataretrieval/**/*.py", recursive=True))
+          print(f"modules={modules} files={files} "
+                f"dependencies={s['total_dependencies']} root={root}")
           if os.path.realpath(root) != os.path.realpath(os.getcwd()):
-              print(f"WARNING: project root {root!r} is not the checkout; "
+              print(f"WARNING: project root {root!r} is not the checkout; the "
+                    "scores above are not comparable to previous runs.")
+          if modules < 0.9 * files:
+              print(f"WARNING: pyscn analyzed {modules} modules of {files} files; "
                     "import resolution is probably degraded and the scores "
                     "above are not comparable to previous runs.")
           PY

.pyscn-known-clones.json

@@ -1,96 +1,95 @@
 {
   "_comment": [
     "Accepted pyscn clone groups. Purpose: make a NEW clone group a visible event.",
-    "The duplication sub-score is dominated by these five accepted families, so it",
-    "barely moves when real duplication is added -- this file is what the weekly",
-    "Code Health workflow diffs against instead.",
+    "The duplication sub-score is dominated by these accepted families, so it barely",
+    "moves when real duplication is added -- compare a weekly Code Health report",
+    "against this file instead.",
     "",
     "Groups are keyed by the SET OF FUNCTION NAMES they contain, not by line numbers,",
     "so ordinary edits above a getter do not churn this file. Regenerate only when",
     "adding or removing a getter from an accepted family, and say why in the commit.",
     "",
     "These are NOT a backlog. Collapsing them means **kwargs and the loss of the typed",
     "public surface tests/contracts/public_api_test.py freezes.",
-    "Verified 2026-08-09: at similarity_threshold 0.55 (default 0.65) the package still",
-    "reports exactly these five groups, so no latent clone sits below the bar.",
-    "Verified 2026-08-09: stripping docstrings makes duplication far WORSE (60 -> 0),",
-    "so suppressing them is not an option -- they are what keeps similarity below 0.86."
+    "Measured with pyscn 1.29.0 on 2026-08-09, which compared docstrings: at",
+    "similarity_threshold 0.55 (default 0.65) it reported the same groups, and stripping",
+    "docstrings made duplication far worse (60 -> 0). pyscn 1.32.4 skips docstrings by",
+    "default (skip_docstrings below), so neither measurement applies to the pinned version.",
+    "Regenerated with pyscn 1.32.4 on 2026-10-13. Its default grouping (complete_linkage,",
+    "was connected) and min_lines (8, was 10) differ from 1.29.0, so it no longer reports",
+    "the NGWMN, NWIS, and WQP getter families and reports two small validator pairs instead."
   ],
-  "pyscn_version": "1.29.0",
+  "pyscn_version": "1.32.4",
   "default_thresholds": {
     "similarity_threshold": 0.65,
-    "min_lines": 10,
-    "min_nodes": 20
+    "min_lines": 8,
+    "min_nodes": 20,
+    "group_mode": "complete_linkage",
+    "group_threshold": 0.65,
+    "skip_docstrings": true,
+    "clone_types": [
+      1,
+      2,
+      4
+    ]
   },
   "observed": {
-    "total_fragments": 283,
-    "cloned_fragments": 25,
+    "total_fragments": 351,
+    "cloned_fragments": 13,
     "groups": 5
   },
   "groups": [
     {
       "members": [
-        "dataretrieval/ngwmn.py::get_lithology",
-        "dataretrieval/ngwmn.py::get_providers",
-        "dataretrieval/ngwmn.py::get_well_construction"
+        "dataretrieval/_validation.py::reject_together",
+        "dataretrieval/_validation.py::require_any_of"
+      ],
+      "similarity": 0.85,
+      "clone_type": 4,
+      "rationale": "Shared argument checks (AGENTS.md, Error messages): each returns early when the arguments are valid and otherwise raises one message from the same parts. Each check owns its wording, so a common helper would only move the message apart from the check that states it."
+    },
+    {
+      "members": [
+        "dataretrieval/nwdc.py::_validate_county",
+        "dataretrieval/nwdc.py::_validate_huc"
       ],
-      "similarity": 0.831,
+      "similarity": 0.85,
       "clone_type": 2,
-      "rationale": "NGWMN collection getters over the OGC facade, one per collection."
+      "rationale": "NWDC selector validators: each strips a code, checks its digits and length, and raises a message naming that selector's own form and examples."
     },
     {
       "members": [
-        "dataretrieval/nwis.py::get_dv",
-        "dataretrieval/nwis.py::get_iv"
+        "dataretrieval/waterdata/time_series.py::get_stats_date_range",
+        "dataretrieval/waterdata/time_series.py::get_stats_por"
       ],
       "similarity": 0.85,
       "clone_type": 2,
-      "rationale": "Deprecated NWIS getters sharing the waterservices call shape; bodies already delegate to _get_json_values."
+      "rationale": "Water Data collection-family getters: ~30 explicitly typed parameters around a thin body. pyscn 1.32.4 skips docstrings, so the similarity is the parameter lists and bodies."
     },
     {
       "members": [
-        "dataretrieval/waterdata/measurements.py::get_channel",
         "dataretrieval/waterdata/measurements.py::get_field_measurements",
         "dataretrieval/waterdata/measurements.py::get_peaks",
-        "dataretrieval/waterdata/metadata.py::get_field_measurements_metadata",
-        "dataretrieval/waterdata/metadata.py::get_monitoring_locations",
-        "dataretrieval/waterdata/metadata.py::get_time_series_metadata",
-        "dataretrieval/waterdata/time_series.py::get_continuous",
         "dataretrieval/waterdata/time_series.py::get_daily",
         "dataretrieval/waterdata/time_series.py::get_latest_continuous",
         "dataretrieval/waterdata/time_series.py::get_latest_daily"
       ],
-      "similarity": 0.805,
+      "similarity": 0.842,
       "clone_type": 2,
-      "rationale": "Water Data collection-family getters: ~30 explicitly typed parameters and a worked-example docstring around a thin body."
+      "rationale": "Water Data collection-family getters: ~30 explicitly typed parameters around a thin body. pyscn 1.32.4 skips docstrings, so the similarity is the parameter lists and bodies."
     },
     {
       "members": [
-        "dataretrieval/waterdata/time_series.py::get_stats_date_range",
-        "dataretrieval/waterdata/time_series.py::get_stats_por"
-      ],
-      "similarity": 0.85,
-      "clone_type": 2,
-      "rationale": "Water Data collection-family getters: ~30 explicitly typed parameters and a worked-example docstring around a thin body."
-    },
-    {
-      "members": [
-        "dataretrieval/wqp.py::what_activities",
-        "dataretrieval/wqp.py::what_activity_metrics",
-        "dataretrieval/wqp.py::what_detection_limits",
-        "dataretrieval/wqp.py::what_habitat_metrics",
-        "dataretrieval/wqp.py::what_organizations",
-        "dataretrieval/wqp.py::what_project_weights",
-        "dataretrieval/wqp.py::what_projects",
-        "dataretrieval/wqp.py::what_sites"
+        "dataretrieval/waterdata/measurements.py::get_channel",
+        "dataretrieval/waterdata/metadata.py::get_monitoring_locations"
       ],
-      "similarity": 0.85,
-      "clone_type": 2,
-      "rationale": "WQP what_* wrappers: one shared query per service endpoint, differing only by the service path segment and the column each documents."
+      "similarity": 0.672,
+      "clone_type": 4,
+      "rationale": "Two Water Data getters matched as a semantic (type-4) clone at 0.672, just above the 0.65 threshold. Expect this pair to appear and disappear between runs as either getter is edited; that churn is not a new clone."
     }
   ],
   "sub_threshold_accepted": {
-    "_comment": "Families that are NOT reported at default thresholds (they sit under min_lines=10 / min_nodes=20) but were found by a deliberate scan at min_nodes=8 on 2026-08-09. Recorded so they are not rediscovered and re-litigated. They are NOT counted in `observed` above.",
+    "_comment": "Families that pyscn 1.29.0 did NOT report at its defaults (they sat under its min_lines=10 / min_nodes=20) but that a deliberate scan at min_nodes=8 found on 2026-08-09. Not re-scanned with 1.32.4, whose min_lines is 8. Recorded so they are not rediscovered and re-litigated. They are NOT counted in `observed` above.",
     "groups": [
       {
         "members": [

CONTRIBUTING.md

@@ -218,9 +218,10 @@ including for an agent working on this repo, which gets a whole-package
 structural overview from one command:
 
 ```bash
-pip install -e '.[health]'   # wheels: macOS ARM64, Linux x86-64, Windows x86-64
-                             # (no source distribution or other platform wheels)
-pyscn analyze dataretrieval  # HTML report, or --json for the numbers
+pip install -e '.[health]'   # wheels: macOS ARM64, Linux x86-64/ARM64 with
+                             # glibc 2.34+, Windows x86-64 (no source distribution
+                             # or other platform wheels)
+pyscn analyze dataretrieval  # HTML report; --json --html writes both
 ```
 
 Read its findings as suggestions, not conclusions. Its clone detector flags this

pyproject.toml

@@ -69,10 +69,11 @@ metrics = [
   "import-linter==2.13",
 ]
 # Advisory weekly deep sweep -- see .github/workflows/code-health.yml. Pyscn has
-# wheels only for macOS ARM64, Linux x86-64, and Windows x86-64, with no sdist,
-# so keep it separate from the extra used by merge gates and contributors.
+# wheels only for macOS ARM64, Linux x86-64 and ARM64 (glibc 2.34 or later), and
+# Windows x86-64, with no sdist, so keep it separate from the extra used by
+# merge gates and contributors.
 health = [
-  "pyscn==1.29.0",
+  "pyscn==1.32.4",
 ]
 test = [
   "pytest > 5.0.0",