fix: unify calendar validation and real-observation recovery

[stefan-jansen]

Closes #108. Closes #109. Part of #110, which remains open for the accepted Coin Metrics dataset-specific work.

Outcome

• Apply the shared calendar declaration throughout validation, storage, gap inspection and incremental range planning. Use authoritative sessions, not fixed US clocks. Preserve daily labels, overnight sessions, holidays, early closes, breaks, CME maintenance and nanosecond boundary precision.
• Report missing observations from stored datasets or bounded raw frames/Parquet files without network calls or writes. Include boundary gaps, entirely absent sessions and per-instrument gaps. Reuse the existing calendar grid; intraday counts are bars, not elapsed days.
• Remove automatic fabricated prices and volumes. Updates check gaps without filling them. Known null placeholders do not count as real observations; zero volume alone does not imply a missing observation.
• Separately recover actual observations through exact Databento timestamp requests. Recheck stale reports, preserve existing real rows/nulls and metadata, and report residual gaps. Reject mixed-instrument write-back, mismatched providers, unsupported schemas and symbol inputs that the SDK would expand into multiple instruments or ALL_SYMBOLS.
• Plan an incremental tail from the next expected observation, rather than advancing minute data by one day. Reuse the bounded report for legitimate exchange closures. Historical interior gaps remain the separate bounded-report workflow.

Contract and compatibility

Explicit calendar selection overrides declared feed/stored metadata, then NYSE applies. Explicit calendar=None disables schedule-derived handling; 24/7 remains a real schedule. Validation retains numerical checks and per-check opt-outs. Incremental row mode overlaps the latest real observation instead of inventing a next timestamp.

Explicit fill_gaps=True and synthetic SessionCompleter modes now raise before fetching or writing. Use check_gaps for inspection and explicitly invoke supported source recovery for downloads. Optional session-grid expansion exposes null placeholders only. Recovery success means processing succeeded, not that every expected observation was recoverable; inspect remaining_gaps. No-trade responses may legitimately leave absences.

One coherent maintainer-directed PR preserves the commits from owned drafts #114/#115 and the prepared validation branch. It includes implementation, regressions, maintained documentation and the source AGENTS.md batching rule. No contributor-owned work was rewritten. Data 0.2.1 remains immutable; no release or documentation deployment is performed.

Reproduction and review

Actual public APIs use real SyntheticProvider input, authoritative calendars and Flat/Hive storage. Only external SDK requests are substituted; no paid/live vendor requests or client data are used.

Workflow	Pinned baseline	Executed failure
Stored calendar report	9331160	38 failures, 6 controls, including newly supported contracts rather than 38 historical numerical bugs
No automatic imputation	9331160	12 API/CLI/config failures
Shared validation	721adee	6 original public behavior failures; expanded contract cases and native precision regressions retained
Actual source recovery	60f6591	18 configured public workflow failures
Incremental minute/session planning	d6076e5	12 failures across real Flat/Hive datasets, clean expected exit 1; immutable source export and source bytes verified
SDK symbol expansion	830b451	3 failures and 1 control; actual installed SDK parser independently confirms whitespace/comma expansion

The owning agent reviewed declaration propagation, timestamp precision, calendar/grid boundaries, identity preservation, stale/partial/no-trade responses, error reporting after publication, request expansion and compatibility. AI assisted code, tests and documentation. No independent human review is claimed; Copilot's quota notice is not a review.

Current qualification

Reviewed head: ca35907.

107 focused report/source regressions pass in 17.65s. The exact-head full suite passes 4199 tests with 292 default-marker deselections in 359.36s and 86.50% coverage. The separate ResourceWarning-error lane passes the same 4199 tests in 351.76s. Ruff check/format (100 columns), ty, normal/all-file hooks, actionlint and strict documentation pass. Both isolated distributions pass dependency, import, CLI and README checks; separately installed wheel and sdist public report, no-op recovery, incremental planner and calendar-validator workflows pass.

Candidate 0.2.2.dev32+gca3590707: wheel SHA256 3d0cc0da2778c012ed5a5d82ca3d2c772c57c7764b1280e07a4e5a2167cf4455; sdist SHA256 4493522da3142ee0a305793dbfb220254890d19d413c5df2151e378fead0af7d. These are verification artifacts, not a published release.

At the final refresh, all 27 hosted checks are complete: 26 successes and only the configured Python 3.15 dependency-wait skip. Its exception validator passes. Supported Python 3.12-3.14 qualification passes on Linux, macOS and Windows, including the corrected Windows regression. CI run39086478121, ecosystem run39086478516, security run39086478023 and documentation run39086478155 all pass. The PR is clean and mergeable at the reviewed head; no previous-head or skipped supported-platform checks substitute for qualification.

Coin Metrics support is not claimed. The remaining required dataset choice is market candles, asset metrics/reference rates, or both; the generic report and Databento recovery do not depend on that choice.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.