diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index 1140a6872..85df23b6d 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -1,6 +1,16 @@ ## 2026-11-04 — D-DRIVER-ZC-0: virtual access driver instead of 17,000 per-call copy audits **Status:** PROPOSED / DOCS ONLY. [Master](../plans/d-driver-zc-virtual-bus-contract-v1.md) · [staged PR charter](../plans/d-driver-zc-staged-pr-charter-v1.md) · [47 falsifiers](../plans/d-driver-zc-falsifier-matrix-v1.md) · [board entry](entries/2026-11-04-d-driver-zc-0-virtual-bus-contract.md) · [STATUS_BOARD](STATUS_BOARD.md) D-DRIVER-ZC · [EXO-154](https://linear.app/exo-red/issue/EXO-154) / [EXO-153](https://linear.app/exo-red/issue/EXO-153) / [EXO-50](https://linear.app/exo-red/issue/EXO-50). **Architecture move:** apply ndarray SIMD polyfill's facade-and-parity law to a single local/foreign/data-owner *address-access membrane*. Five virtual XX/XY/XZ/YY/ZZ lenses share one witnessed canonical SoA address; typed ClassView and Quack/mask-risc/ndarray execute without default copied values/row reconstruction. BIND→PROJECT→STREAM→EXECUTE→RELEASE; strict local borrow vs disk ingress vs remote message distinction; no new ISA/ALU/hash index or driver runtime. Co-addressed selections and once-per-frame routing first; foreign hops remain EXO-46/89/151/140-gated; D4/S-INGRESS and oracle-lifetime owners open. A 47-case matrix separates semantic results, representation bytes, reachability enforcement and end-to-end economics. #1953/#1955/#1956 already merged; the 8 ZC stages are independent, only ZC-0 documented now. "Thinking Hard Drive" is a proposed restricted theorem schema, not an assertion that disks perform computation or every query takes O(1). **Explicit defer:** no cognitive-shader-driver/Ladybug/P64/StreamDTO/BusDTO/PerturbationDTO stack investigation or consolidation during this PR. +## 2026-11-04 — D-CGMM-0: Codegraph Kythe/Datalog demand → ClassView FieldViewDef scratch → DT64 + +**Execution-ready docs addendum (2026-11-04):** [Detailed source→demand→native blueprint](../plans/d-cgmm-implementation-blueprint-v1.md), [EXO owner decision ledger](../plans/d-cgmm-exo-gates-v1.md), [independent fixture oracles](../plans/d-cgmm-executable-fixtures-v1.md); all new source/semantic/bench results still **NOT_RUN**. First lab should verify Ruff/Charon source evidence loss and identity, then demand-bound lazy Codegraph parity; do not skip straight to DT64 execution by treating candidate schema offers as permits. + + +**Status:** PROPOSED / DOCS ONLY. [Master architecture](../plans/d-cgmm-codegraph-metta-method-builder-v1.md) · [lazy-Codegraph evidence/demand separation](../plans/d-cgmm-lazy-codegraph-fact-demand-v1.md) · [ActionDef/OGIT many-to-many relation](../plans/d-cgmm-action-relation-schema-v1.md) · [staged PR charter](../plans/d-cgmm-staged-pr-charter-v1.md) · [46 falsifiers](../plans/d-cgmm-falsifier-matrix-v1.md) · [board entry](entries/2026-11-04-d-cgmm-0-metta-class-method-builder.md) · [EXO-155](https://linear.app/exo-red/issue/EXO-155) / [EXO-50](https://linear.app/exo-red/issue/EXO-50). + +**Operator's direction:** MeTTa becomes logical class/method schema glue rather than runtime. **Source AST/harvest→Kythe-shaped immutable evidence (not Codegraph Pos)→Crepe/Soufflé/XLOG compiler demand metadata→lazy Codegraph masks→witnessed ClassView/FieldViewDef with ephemeral MeTTa constraint scratch→separate OGIT LinkSpec M:N / OGAR ActionDef / DT64 MethodBinding admission→existing RelAlg/Quack/R2IL/mask-risc/ndarray execution.** Codegraph already binds joins lazily and uses graded masks, but eagerly builds six XLOG relations; lazy planner is a measurable hypothesis, not a shipped complete rewrite. No alternate hash index or owned AST/tree. Copy-free SoA hot path remains D-DRIVER-ZC's guarantee; field presence, tenant auth, Bag multiplicity, source grading and action effects are independent. Unsupported WHEN/K3, recursion, foreign tenant edge and effectful action are typed refusals, not a fallback interpreter. + +**Separate gates:** EXO-84/85 MeTTa, EXO-79/143 MORK, EXO-73 datafrog/recursion, EXO-108 Bag, EXO-95 enterprise source parity, EXO-121 ClassView, EXO-152 normalized method DTO, EXO-153/154 DR virtual driver/zero-copy. All future PRs require independent review; only docs stage opened now. No runtime implementation or D4 admission inferred. ## 2026-11-04 — D-DRISA decision memos after the local register proof → `.claude/plans/d-drisa-decision-memos-v1.md` diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 7ca5be70d..2558e5ca2 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -1,3 +1,24 @@ +## D-CGMM — Lazy Codegraph fact/demand + MeTTa class-method scratch (2026-11-04) + +**Implementation detail update (2026-11-04):** [Source→native blueprint](../plans/d-cgmm-implementation-blueprint-v1.md), [EXO gate ledger](../plans/d-cgmm-exo-gates-v1.md) and [fixed independent fixtures](../plans/d-cgmm-executable-fixtures-v1.md) refine the eight future stages. All executable fixtures are **NOT_RUN**, with expected pure COUNT=2 and tenant-A Set=3/Bag=4. The only authorized artifact is docs in draft PR #1958; owners of EXO-121/152/46/140/95/108/73 remain open gates. + + +**Status:** D-CGMM-0 PROPOSED / DOCS ONLY. [Master](../plans/d-cgmm-codegraph-metta-method-builder-v1.md) · [lazy Codegraph](../plans/d-cgmm-lazy-codegraph-fact-demand-v1.md) · [OGIT/OGAR edge-action addendum](../plans/d-cgmm-action-relation-schema-v1.md) · [PR charter](../plans/d-cgmm-staged-pr-charter-v1.md) · [46 falsifiers](../plans/d-cgmm-falsifier-matrix-v1.md) · [board entry](entries/2026-11-04-d-cgmm-0-metta-class-method-builder.md) · [EXO-155](https://linear.app/exo-red/issue/EXO-155) · [EXO-50](https://linear.app/exo-red/issue/EXO-50) append-only handoff. **No production source harvester, executable MeTTa/Datalog, new ClassId, schema, ISA, driver or automatic authorization.** + +| D-id | Proposed PR stage | State / named gate | +| --- | --- | --- | +| D-CGMM-0 | source-grounded architecture, owner crosswalk, 46 independent falsifiers | THIS DOCS PR, review only | +| D-CGMM-1 | Ruff/Charon→Kythe-shaped source identity, binary-lane effect/coverage census | FUTURE LAB, producer + Codegraph owner | +| D-CGMM-2 | Crepe/Soufflé/XLOG compile evidence and *lazy* Codegraph demand plans | FUTURE LAB, EXO-73/79/143 gates | +| D-CGMM-3 | ClassView-based FieldViewDef + disposable MeTTa-shaped constraint scratch | FUTURE LAB, EXO-121/152 | +| D-CGMM-4 | OGIT M:N edge roles, tenant permission and OGAR ActionDef offers | FUTURE LAB, EXO-95/108, OGAR/ontology | +| D-CGMM-5 | eligible same-row FieldViewDef → RelAlg pure polyfill | FUTURE LAB, Quack/R2IL owner | +| D-CGMM-6 | separately admitted immutable DT64 MethodBinding → R2IL native | GATED EXO-121/152, execution permission | +| D-CGMM-7 | prepared method/route memoization with source/version/scope fence | FUTURE LAB, D-DRISA/D-DRIVER-ZC gates | +| D-CGMM-8 | independent Datalog/MORK and provenance/caller coverage, NNUE ranking-only | FUTURE LAB, EXO-79/143/154/73 | + +**Architectural laws:** durable source identity ≠ Codegraph interned Pos ≠ DR local ordinal ≠ ClassId/MethodId. Source grades confirmed/possible/unknown retained. Crepe binding and Soufflé magic/stratification are **cold plan evidence**, not new runtimes. Codegraph partially lazy already, while six XLOG Cube relations are built at intake; the new proposal must measure whether demand-scoping actually wins. ClassView FieldMask means presence, not rule truth. MeTTa-shaped scratch is ephemeral; OGIT/OGAR/DT64 owners separately admit typed ontology and method declarations. RelAlg IR has no general M:N join/effect/WHEN executable path. No legacy cognitive-shader-driver/DTO consolidation is in scope. + ## D-DRIVER-ZC — one zero-copy access membrane and virtual bus (2026-11-04) **Status:** D-DRIVER-ZC-0 PROPOSED / DOCS ONLY. [Master](../plans/d-driver-zc-virtual-bus-contract-v1.md) · [staged PR charter](../plans/d-driver-zc-staged-pr-charter-v1.md) · [47 independent falsifiers](../plans/d-driver-zc-falsifier-matrix-v1.md) · [entry](entries/2026-11-04-d-driver-zc-0-virtual-bus-contract.md) · [EXO-154](https://linear.app/exo-red/issue/EXO-154) · [EXO-153](https://linear.app/exo-red/issue/EXO-153) · [EXO-50](https://linear.app/exo-red/issue/EXO-50). **No driver implementation or production access rule shipped.** diff --git a/.claude/board/entries/2026-11-04-d-cgmm-0-metta-class-method-builder.md b/.claude/board/entries/2026-11-04-d-cgmm-0-metta-class-method-builder.md new file mode 100644 index 000000000..fd850246e --- /dev/null +++ b/.claude/board/entries/2026-11-04-d-cgmm-0-metta-class-method-builder.md @@ -0,0 +1,45 @@ +# D-CGMM-0: Kythe-shaped facts, lazy Codegraph, MeTTa schema scratch and DT64 method building + +**2026-11-04. STATUS: DOCS / PRE-REGISTRATION ONLY.** This board entry is a proposed architecture, not an AST intake implementation, executable MeTTa/Datalog runtime, new ClassId/FieldRef/ActionDef API, source identity owner, vendor integration, NNUE certification or speedup claim. + +**Core operator insight:** Use MeTTa-shaped **logical class-method schema constraints**, not a second MeTTa interpreter. Codegraph should become **demand-scoped and lazy**, backed by compiler-evidence facts with Kythe-like stable source identity, with Crepe/Soufflé/XLOG compilation **metadata** guiding required relation/field views. A witnessed `ClassView` provides the stable field basis; proposed `FieldViewDef` offers are matched in temporary logical constraint scratch; OGIT/OGAR/DT64 admit ontology/ActionDef/method semantics; existing RelAlg/R2IL/mask-risc/ndarray executes eligible operations on borrowed SoA. + +**Architecture:** [D-CGMM master](../../plans/d-cgmm-codegraph-metta-method-builder-v1.md) · [Kythe/Datalog/lazy Codegraph](../../plans/d-cgmm-lazy-codegraph-fact-demand-v1.md) · [OGIT/OGAR ActionDef and M:N](../../plans/d-cgmm-action-relation-schema-v1.md) · [staged PR charter](../../plans/d-cgmm-staged-pr-charter-v1.md) · [46 falsifiers](../../plans/d-cgmm-falsifier-matrix-v1.md). +**Tracking:** [EXO-155](https://linear.app/exo-red/issue/EXO-155), [EXO-50](https://linear.app/exo-red/issue/EXO-50) message bus; [EXO-153](https://linear.app/exo-red/issue/EXO-153)/[EXO-154](https://linear.app/exo-red/issue/EXO-154) related virtual driver/zero-copy plans. + +## Existing proofs vs new proposal + +- Codegraph `graph.rs` currently interns symbols into `Pos`, builds a `MapTable` plus six XLOG `Cube` relations and masks; the reverse derived structures have `OnceCell`. `query.rs` uses bound joins and graded confirmed/possible/unknown masks. **Partially lazy today, not full demand-compiled Codegraph.** +- Ruff/Charon `.rows.bin`, FUNDECL and PROV are existing source-evidence intake; source effect/dispatch/receiver gaps remain **UNKNOWN**. Codegraph interned `Pos` is NOT Kythe durable source identity, ClassId, MethodId or DR ordinal. +- The [Crepe/Soufflé/XLOG archaeology](2026-11-02-coresearch-xlog-souffle-crepe-dr3-archaeology.md) documents binding/demand/index, magic-set/stratification and compile metadata losses. It expressly warns not to import a second runtime, and records `crepe_plan!` drift D1/D2. **Compiler metadata can be cold admission evidence only after falsification.** +- `ClassView` is already the meta lookup above SoA; `FieldMask`/`WideFieldMask` signify instance presence, NOT speculative rule truth. `lens_fields(full_classid)` is mandatory for behavior lens; unknown lens refuses. +- `lance-graph-ontology::MappingProposalKind::Edge{LinkSpec}` and `Cardinality::ManyToMany` already model typed ontology edges, **not** edge instances or Bag derivation counts. OGAR `ActionDef` owns richer behavior; `ActionSpec` is not equivalent. +- `lance-graph-relalg::Rel` supports one-population All/Empty/Plane/I32/U32 atom/And/Or/Not + Count/Sum/Min/Max/Keep. It does **NOT** execute an arbitrary M:N join, effect, guarded DT64 WHEN or MeTTa rewrite. +- `lance-graph-r2il::binding(_native)` admits only a restricted executable subset and explicitly refuses guarded WHEN/three-valued truth; `ExecPermit` is lab only. No automatic schema-to-code promotion. +- NNUE may rank candidate equivalence **offline only**. Source evidence, tenant permission and semantic effect order decide admission, not probability. + +## Independent future PR stages + +| ID | Purpose | State / owner | +| --- | --- | --- | +| D-CGMM-0 | docs, owner gates, 46 independent tests, board/Linear handoff | THIS DOCS PR | +| D-CGMM-1 | Ruff/Charon binary/Kythe fact identity and effect-loss census | FUTURE LAB; producer/Codegraph | +| D-CGMM-2 | compiled binding/demand metadata; lazy Codegraph against eager baseline | FUTURE LAB; XLOG/Codegraph/EXO-73 | +| D-CGMM-3 | ClassView-based FieldViewDef, disposable MeTTa constraint scratch | FUTURE LAB; EXO-121/152 | +| D-CGMM-4 | OGIT M:N edge tenant + OGAR ActionDef schema offers | FUTURE LAB, semantic/tenant owner gates | +| D-CGMM-5 | pure one-row RelAlg polyfill of eligible FieldViewDef | FUTURE LAB, Quack/R2IL owner | +| D-CGMM-6 | admitted DT64 method → native binding proof | GATED EXO-121/152, execution owner | +| D-CGMM-7 | memoized recipe/binding economics, no source-value cache | FUTURE LAB after 5/6 | +| D-CGMM-8 | independent datafrog/MORK oracle, NNUE-only candidate ranking and reachability | GATED EXO-79/143/73/154 | + +## Mandatory refusals + +Unknown effects ≠ no effects; possible dispatch ≠ confirmed; missing source producer ≠ known absence; ClassId concept ≠ app method equivalence; many-to-many ≠ Bag; ontology context ≠ tenant authorization; Codegraph `Pos` ≠ durable source ID; RelAlg GYO acyclicity ≠ a join executor; no recursive/cross-tenant/guarded action under a fallback interpreter. **Scratch is disposable and cannot issue ClassId/MethodId or mutate registered ontology.** + +**Related owner issues:** [EXO-84](https://linear.app/exo-red/issue/EXO-84), [EXO-85](https://linear.app/exo-red/issue/EXO-85), [EXO-79](https://linear.app/exo-red/issue/EXO-79), [EXO-143](https://linear.app/exo-red/issue/EXO-143), [EXO-101](https://linear.app/exo-red/issue/EXO-101), [EXO-73](https://linear.app/exo-red/issue/EXO-73), [EXO-108](https://linear.app/exo-red/issue/EXO-108), [EXO-95](https://linear.app/exo-red/issue/EXO-95), [EXO-121](https://linear.app/exo-red/issue/EXO-121), [EXO-152](https://linear.app/exo-red/issue/EXO-152), [EXO-153](https://linear.app/exo-red/issue/EXO-153), [EXO-154](https://linear.app/exo-red/issue/EXO-154). + +**Deferred:** any inspection or consolidation of cognitive-shader-driver / migrated Ladybug-rs / P64/StreamDTO/BusDTO/PerturbationDTO; no speculative production endpoint or new schema store. + +## Implementation packet / explicit EXO decisions (2026-11-04) + +New source-linked documents in the same **docs-only draft PR #1958**: [implementation blueprint](../../plans/d-cgmm-implementation-blueprint-v1.md), [EXO owner gate ledger](../../plans/d-cgmm-exo-gates-v1.md), [concrete independent fixture corpus](../../plans/d-cgmm-executable-fixtures-v1.md). The fixtures fix call-grade expectations, pure selection COUNT=2, M:N tenant-A Set=3/Bag=4, contradictory application-action effects and invalidation under source/scope change. **All NOT_RUN**. Explicit critical path: EXO-155 coordination; EXO-84/85 demand and MeTTa rules; EXO-121 FieldViewDef/ClassView; EXO-152 normalized native method; EXO-46/140 foreign binding; EXO-95 OGIT/OGAR/SAP/Odoo; EXO-108 Bag; EXO-73 recursion; EXO-79/143 MORK oracle; EXO-153/154 borrowed execution; EXO-104/105/109 oracle/copy firewall; EXO-50 append-only messaging. No production admission, new schema authority or automatic merge. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index ab4badb3c..16f155c44 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -576 entries, 2026-08-06 .. 2026-11-04. +577 entries, 2026-08-06 .. 2026-11-04. | date | entry id | finding | file | |---|---|---|---| @@ -33,6 +33,7 @@ exactly one of them, never both. | 2026-11-04 | `D-DRISA-3` | | [2026-11-04-d-drisa-3-alu-semantics.md](2026-11-04-d-drisa-3-alu-semantics.md) | | 2026-11-04 | `D-DRISA-1` | | [2026-11-04-d-drisa-1-register-proof.md](2026-11-04-d-drisa-1-register-proof.md) | | 2026-11-04 | `D-DRISA-0` | | [2026-11-04-d-drisa-0-remote-register-contract.md](2026-11-04-d-drisa-0-remote-register-contract.md) | +| 2026-11-04 | `D-CGMM-0` | | [2026-11-04-d-cgmm-0-metta-class-method-builder.md](2026-11-04-d-cgmm-0-metta-class-method-builder.md) | | 2026-11-03 | `D-XLOG-META-WIRE-P0` | | [2026-11-03-d-xlog-meta-wire-p0.md](2026-11-03-d-xlog-meta-wire-p0.md) | | 2026-11-03 | `D-GRAMMAR-WHEN-1` | | [2026-11-03-d-grammar-when-1.md](2026-11-03-d-grammar-when-1.md) | | 2026-11-03 | `D-GRAMMAR-CONSTRAINTS-0` | | [2026-11-03-d-grammar-constraints-0.md](2026-11-03-d-grammar-constraints-0.md) | diff --git a/.claude/plans/d-cgmm-action-relation-schema-v1.md b/.claude/plans/d-cgmm-action-relation-schema-v1.md new file mode 100644 index 000000000..565e7e1b5 --- /dev/null +++ b/.claude/plans/d-cgmm-action-relation-schema-v1.md @@ -0,0 +1,148 @@ +# D-CGMM-EDGE: OGIT link + OGAR ActionDef + ONNX-style alias + RelAlg evidence, in one logical schema scratch + +> **2026-11-04 · ADDENDUM to [D-CGMM-0](d-cgmm-codegraph-metta-method-builder-v1.md) · PROPOSED.** This is a *crosswalk of existing owned declarations*, **not** a new persistent schema registry, source parser, graph-store, join engine, ClassId minting algorithm, ONNX graph runtime, ActionDef implementation, or tenant authorization. [EXO-155](https://linear.app/exo-red/issue/EXO-155), [EXO-95](https://linear.app/exo-red/issue/EXO-95), [EXO-152](https://linear.app/exo-red/issue/EXO-152), [EXO-121](https://linear.app/exo-red/issue/EXO-121), [EXO-46](https://linear.app/exo-red/issue/EXO-46) remain owner gates. + +## 1. Operator's sharpened design + +**The logical schema scratch should be shaped like a *combination* of OGIT's object/edge meaning, OGAR's static ActionDef and the ONNX-native polyfill contract.** It should tell us not only "Rails, Odoo and SAP contain a time entry," but **which typed relation connects which roles, whether an edge is many-to-many, in whose tenant/context it is meaningful, what action is permitted on that edge, and which existing algebra answers a read.** + +The schema's *language* may be MeTTa-shaped facts/constraints because those are convenient for describing heterogeneous models and candidate rules, but the execution model is **not MeTTa**. The *scratch* temporarily composes already-harvested code and ontology facts; it does not own the resulting ontology, a source row, an admitted method or an execution permit. + +~~~text +OGAR Class / Association / ActionDef (static declared behavior) + + OGIT Entity / LinkSpec / MappingProposal (typed meaning) + + Codegraph source AST facts (source-grounded effects) + + external DTO rules (ONNX-like ALIAS / INLINE / REFUSE) + | + ephemeral MeTTa-shaped logical schema constraint scratch + [Role, Edge, Cardinality, TenantScope, Guard, Effect] + | + OFFER, REFUSE or ABSTAIN with evidence + / \ + ontology MappingProposal DT64 MethodBinding + [owner admits] [owner admits] + \ / + one coordinate/relationship identity; no copied values + | + Quack/XLOG/mask-risc/ndarray as admitted read execution + permitted effects through distinct Rubicon +~~~ + +**Ontological membership, relation traversal, method behavior, and physical execution are distinct axes.** They may share one conceptual declaration vocabulary, but must not become a second, catch-all engine. + +## 2. Exact existing owner crosswalk + +| Owner | Concrete verified surface | What it can legally contribute to scratch | +| --- | --- | --- | +| `lance-graph-ontology/src/lib.rs` | `OntologyRegistry` canonical OGIT TTL dictionary, bridges/contexts, `MappingProposal` intake | OGIT URI and namespace, bridge/public label, source/provenance/marking, ontology schema pointer; **not a new identity** | +| `lance-graph-ontology/src/proposal.rs` | `MappingProposalKind::{Entity{Schema}, Edge{LinkSpec}, Attribute}`; `MappingRow` stores `subject_type`/`object_type` and ontology context id | a versioned candidate typed edge, distinguished from entity/property; mapping confidence is not proof of semantics | +| `lance-graph-contract/src/property.rs` | `LinkSpec{subject_type,predicate,object_type,cardinality,codec_route}`; `Cardinality::{OneToOne,OneToMany,ManyToMany}` | typed schema-level subject/predicate/object and many-to-many *constraint*; **not an actual edge population**, unique instance or SQL multiplicity | +| same contract file | `ActionSpec{name,entity_type,target_predicate,trigger}` with Manual/Auto/Suggested | coarse UI/application action declaration, **not equivalent to OGAR ActionDef** | +| OGAR `ogar-vocab::Class` | cross-language structural model, associations, callback, computed fields, methods, validations, source language | static language-independent class shape, source-action candidates | +| OGAR `ogar-vocab::ActionDef` | action identity/predicate/object_class, default subject/temporal/modal, causal state guard, ordered workflow intent, optional state transition / timeout | richer **declared** behavioral semantics, not an executed action | +| OGAR `ActionInvocation` | dynamic action instance, tenant/company local context, provenance and lifecycle | **never** silently collapse a dynamic instance into a static ActionDef or execute it from scratch | +| `lance-graph-onnx/src/lib.rs` | aliases re-export native functions; INLINE adapters validate shape/dtype; unsupported ops REFUSE, no new runtime | pattern for *native method equivalence admission*, **not** a source for business effects | +| `lance-graph-relalg/src/ir.rs` | `Rel` and `Query` describe *one table's row selection* and `Measure`, no raw names or owned rows | normalize a supported single-row-population predicate/fold; **not** a many-to-many join/recursive relation executor | +| `lance-graph-xlog` / Quack | already-held Boolean/Bag relationship methods, constrained folds through mask-risc | eligible relationship query plan **only under existing coordinate, multiplicity and foreign-hop rules** | +| DT64 `MethodBinding` / OGAR vocabulary | immutable class/method/revision/ON×WHEN×ordered DO/ValueDemand | admitted method destination for a scratch *offer*, **not** source AST or new codebook | +| Codegraph `Graded` / residuals | source calls / possible dispatch / unknown edges | compiler-grade evidence; effect/order/transaction constraints stay unknown where harvest incomplete | + +**Source verified:** [ontology](../../crates/lance-graph-ontology/src/lib.rs), [proposal](../../crates/lance-graph-ontology/src/proposal.rs), [LinkSpec and ActionSpec](../../crates/lance-graph-contract/src/property.rs), [ONNX facade](../../crates/lance-graph-onnx/src/lib.rs), [RelAlg IR](../../crates/lance-graph-relalg/src/ir.rs), [OGAR IR source](https://github.com/AdaWorldAPI/OGAR/blob/main/crates/ogar-vocab/src/lib.rs), [OGAR ActionDef mapping](https://github.com/AdaWorldAPI/OGAR/blob/main/docs/OGAR-AST-CONTRACT.md). + +## 3. Relation-level scratch, without minting relation truth + +**Conceptual scratch fields (not a new Rust wire struct):** + +~~~text +RelationOffer { + predicate: proposed OGIT/OGAR predicate identity + ontology context + roles: [subject_role: typed ClassRef, object_role: typed ClassRef] # minimum binary + direction: declared and independently observed + cardinality: 1:1 | 1:N | M:N | unknown + multiplicity: Set | Bag | unknown # DISTINCT from cardinality + reflexive / symmetric / transitive / inverse: only if explicitly proved + edge_instance_identity: unknown | witnessed # for association attributes/duplicates + key_constraints: explicit uniqueness / foreign-key / deletion policy if observed + scope: tenant owner + named graph + principal/permission + inter-tenant permit + provenance: source artifact/line + ontology URI/rev + coverage/grade + candidate_behavior: ActionDef reference + guards/ordered effects/transaction policy + status: OFFER | REFUSE(reason) | ABSTAIN(unknown cause) +} +~~~ + +**Hard distinctions:** +- **Cardinality** (M:N) constrains allowable endpoint associations; **Bag multiplicity** counts derivations or repeated edges; **relation existence** is a Boolean support mask. One M:N edge can be a Set even if its association table permits multiple rows, or a Bag if duplicates matter. Never infer multiplicity from `Cardinality::ManyToMany`. +- **Edge identity** matters when the association itself has attributes, revisions, permissions or distinct repeated instances; a pure pair relation and a reified association entity are not interchangeable. +- **Tenant** is not just an ontology `bridge_id`. The existing registry indexes `(bridge_id,public_name)` and OGIT URI; `SchemaPtr::ontology_context_id` describes an ontology context. Neither alone is a per-row tenant permission. A cross-tenant link needs explicit two-endpoint authorization and a witnessed relation scope. Missing tenant policy => ABSTAIN or REFUSE, not global graph adjacency. +- **Direction** (A→B) does not imply inverse (B→A) or symmetric role exchange. `XX/XY/XZ` virtual bus names are *operand projections*, not relation schema semantics. +- **Many-to-many does not imply a Cartesian product.** The actual edge population determines support, and joining two populations still costs and requires admissible shared identity routing. +- **Recursion** is an external driver gate (EXO-73); `LinkSpec` alone does not license transitive closure. +- **Write effects** require a separate action-invocation/transaction/Rubicon admission. `MappingProposalKind::Edge` is not a command to insert an edge. + +## 4. Example: Billable work entry × project × approval × tenant + +**Illustrative semantics, not observed Rails/SAP source or runnable MeTTa syntax:** + +~~~text +OGIT entity: BillableWorkEntry; application-local: Odoo.account.analytic.line, + SAP.CATS, Rails.TimeEntry (Rails parity unverified). +Link candidate: (Worker, assigned_to, Project) [M:N] +Scoped role: a Worker in tenant A may be linked to a Project in tenant B + ONLY with explicit policy and source/relationship witness; otherwise REFUSE. + +Event: Approve(BillableWorkEntry) +OGAR ActionDef candidates can differ: + Odoo [dependency/callback, company scope; if harvested] + SAP [BAPI/CATS transaction; source trace required] + Rails [ActiveRecord callback/transaction; Ruby harvester required] + +DT64 offer: ClassId concepts, app-specific method refs, versioned ClassView, + ON / WHEN / ordered DO, declared truth/multiplicity, no executable effects yet. + +Read example: Count approved entries by project under one tenant/permission; + only if source/method/relationship witnesses and Bag semantics agree. +~~~ + +**Counterexample:** Tenant A and B share an OGIT URI and both use the label "approves", but A's method changes approval state, while B's triggers an accounting posting and rollback. The schema scratch must say **SameConcept / DifferentAction**, not normalize both into one executable method. + +## 5. Admission mapping: OGIT vs OGAR vs DT64 vs ONNX + +**No new universal schema owner.** Scratch should emit distinct *offers* to existing owners: +1. **Ontology OFFER:** a `MappingProposalKind::Edge{LinkSpec}` or entity/attribute mapping, with observed source and explicit tenant/context metadata. Only ontology's canonical registry may admit it. +2. **Behavior OFFER:** OGAR `ActionDef` or existing OGAR Class association/callback declarations, with provenance and effect/temporal/modal semantics. The OGAR codebook owner admits identity. +3. **Method OFFER:** DT64 `MethodBinding` using an existing ClassView basis, MethodId/SigRev, ON/WHEN/ordered effects. DT64 stores it; `ExecPermit`/execution ownership separately admits a limited native program. +4. **Native ALIAS offer:** like ONNX, exact semantic compatibility may select an existing native operation. A narrowly specified adapter may be needed (and separately approved), or REFUSE; no new handler invented to rescue a missing vocabulary. +5. **Relational query OFFER:** a non-effectful Set or Bag query on witnessed relationship planes under Quack/XLOG/RelAlg scope. `RelAlg::Rel` alone cannot represent arbitrary M:N joins, so it is not extended into another Datalog AST. + +## 6. Independent many-to-many falsifier ("edge tenant" test) + +**Independent fixture:** Tenant A has workers W1,W2 and projects P1,P2; tenant B has W3/P3. Relation edges: +- A: (W1,P1), (W1,P2), (W2,P2). +- B: (W3,P3). +- A duplicate (W1,P2) for Bag-control (same Boolean support, higher multiplicity). +- A deliberately forged cross-tenant (W2,P3) that is **present in a raw table** but not admitted by the tenant-permission graph. +- An "approval" method with the same label in A and B, but B has an additional side effect. + +**Independent oracles:** source rows keyed by actual global identity and tenant scope, enumerated outside Codegraph/Quack; OGIT type oracle; per-application method-effect and auth fixtures. Compare support query, Bag count, cardinality constraint, tenant authorization, inverse traversal and approval-method parity separately. + +**Six deliberate kills:** +1. Drop tenant check: cross-tenant link appears. +2. Replace Bag with Boolean: duplicate is lost. +3. Join on local position instead of global identity: wrong worker/project. +4. Reverse edge direction automatically: inverse false-positive. +5. Erase an ordered side effect: same-name methods falsely equivalent. +6. Turn a source effect UNKNOWN into absent: unsafe method admitted. + +**Expected result at CGMM-0:** only a test specification. Nothing executed. Future lab uses XLOG/Quack's existing relation orientation and source identities, never invents a relation engine. Attach exact source rows, expected counts, mutation results and owner gates to the PR. + +## 7. Why this unifies the architecture without creating a giant DTO + +The reusable abstraction is a **typed offer + constraint scratch + owner-specific admission**, rather than one gigantic persistent catch-all record for application classes, graph edges and effects. + +- The *same* scratch vocabulary can describe an entity, binary/many-to-many relationship or action behavior, but each output goes to its rightful owner. +- The *same* native compiled method may be reused across application-specific DTOs **only if** a full semantic witness (types, null/Bag, effects, permissions, source layout and versions) proves equivalence. +- NNUE may rank candidate structural equivalences offline, but a high score never asserts identity, permits cross-tenant access or makes an ActionDef executable. +- The hot loop remains borrowed SoA + mask-risc/ndarray; no MeTTa interpreter, ontology TTL walk or source AST decoding is in that loop. +- Because role/cardinality/tenant semantics are *explicit*, the same scratch engine may describe non-business graph tenants later without special casing Odoo, Rails or SAP. + +**The next discriminating step is not NNUE training.** Build one bounded lab where Codegraph's graded source facts plus ontology's known `LinkSpec::many_to_many` can generate a role-typed relation offer, and demonstrate a deliberate unknown effect or tenant policy blocks its conversion to executable DT64 behavior. That is the causal firewall between *description* and *authorization*. diff --git a/.claude/plans/d-cgmm-codegraph-metta-method-builder-v1.md b/.claude/plans/d-cgmm-codegraph-metta-method-builder-v1.md new file mode 100644 index 000000000..922de41f3 --- /dev/null +++ b/.claude/plans/d-cgmm-codegraph-metta-method-builder-v1.md @@ -0,0 +1,216 @@ +# D-CGMM-0 · Codegraph × MeTTa logical schema scratch × DT64 MethodBinding + +> **2026-11-04 · Architecture / research pre-registration · DOCS ONLY.** This does not implement a parser, new codegraph store, class registry, compiler, generic rewrite engine, NNUE scorer, new F### word, production authorization, or execute an unguarded/guarded DT64 method. [EXO-155](https://linear.app/exo-red/issue/EXO-155) tracks the program; [EXO-50](https://linear.app/exo-red/issue/EXO-50) is append-only coordination. [Staged PRs](d-cgmm-staged-pr-charter-v1.md), [test matrix](d-cgmm-falsifier-matrix-v1.md), [board](../board/entries/2026-11-04-d-cgmm-0-metta-class-method-builder.md). +> +> **Core idea:** MeTTa becomes the **logical DTO and class-method builder**, while **schema-constraint scratch** is the disposable staging area where source evidence from Codegraph is matched against MeTTa-style declarations, normalized and evaluated for *eligibility*. Only independently admitted declarations become immutable DT64 `MethodBinding`s. Admitted execution uses existing Quack/R2IL/mask-risc/ndarray; **no MeTTa evaluator**, duplicate code store, object-reconstructing hot path or unearned SIMD/Abakus speedup. +> +> The operator also wants a domain comparison between **Rails, Odoo, SAP**, not false cross-platform equivalence: same concept may have different callback timing, transaction semantics, authorization, field layouts and effects. Codegraph must represent those *differences* first. + +## 1. One pipeline, two directions, four authority levels + +~~~text +INPUTS (source evidence; not method authority) + Rust/Charon + Ruff subset Odoo/Python Rails/Ruby? SAP/ABAP? + | | | | + compiler-grounded artifact bounded harvest ADAPTER GAP ADAPTER GAP + (symbols, calls, spans, with provenance, until real until real + maybe-dispatch, residuals) not guessed types evidence evidence + +-------------+------------+----------------+-----------+ + | + CODEGRAPH FACT PLANES + (confirmed / possible / unknown / residual) + source + schema generations + | + [LOGICAL SCHEMA SCRATCH] + bounded temporary constraints / role candidates + identity, unification, compatibility, refusal + (MeTTa-shaped atoms, scoped typed relations) + | + candidate class/method declaration + evidence/refusal + | + DT64 / OGAR OWNER ADMISSION + ClassId / MethodId / SigRev / ClassView + ON × WHEN × ordered DO / ValueDemand + codebook + provenance + permit + | + COLD method preparation / bind / typed refusal + Quack/RelAlg OR optional R2IL native reading + | + mask-risc -> ndarray/Abakus if useful + borrowed canonical SoA/DR lens + | + witnessed result or refusal +~~~ + +**Backwards direction (audit):** For an already admitted DT64 `MethodRef`, Codegraph may locate and grade its *supporting source evidence*, map the declared behavior to source call/effect sites and identify stale or absent evidence. It **must not** synthesize compiler facts from the admitted method, infer that call absence proves no effect, or rewrite a source program because the schemas look similar. + +**Four different objects** must never be conflated: +- **Codegraph fact:** compiler or harvester observation with source/source-revision/span/grade/coverage; neither domain truth nor an execution permit. +- **Logical schema scratch candidate:** per-prepare, bounded, provisional constraint assignment; can be evicted with *no* change to admitted truth. +- **DT64 binding:** immutable, owner-validated class/schema/method identity and ON/WHEN/DO method contract. +- **Prepared native execution:** validated method/lens/opcode/route and executable operations under live source/frame/scope/permission, optionally reusable after all relevant witnesses are checked. + +## 2. Source-grounded compatibility and known missing data + +| Existing component | Verified/source-reported role | What must not be inferred | +| --- | --- | --- | +| `lance-graph-codegraph/src/artifact_bin.rs` | Versioned, validated `.rows.bin`; one owned bytes buffer; borrowed binary lanes including FUNDECL/PROV | Not all language AST, types, effects or borrows have been harvested | +| `lance-graph-codegraph/src/graph.rs` | Symbol interning to positions, DR `MapTable`, XLOG cube relations `CALLS/NODE/MAY/IMPL/RCONF/RPOSS` | Codegraph `Pos` is not ClassId, MethodId, OGAR codebook id or permanent DR global id | +| `lance-graph-codegraph/src/query.rs` | `confirmed/possible/unknown` masks; no tuple decode in hot queries | `possible` is not confirmed; `unknown` is not a negative proof | +| `lance-graph-codegraph/src/diag.rs` | PARTIAL, NOT_RUN, residual coverage; typed diagnosis | "No known gap" is not proof of complete harvest | +| Ruff/Charon source harvest | Compiler LLBC facts and limited Python intake; reports source effects and dispatch in some paths | Effects/type/borrow/control flow may be dropped before Codegraph; cannot certify from prose labels | +| [#1944 cognitive polyfill](https://github.com/AdaWorldAPI/lance-graph/pull/1944) | Ground, bounded conjunctive common subset, Set vs Bag and NAL differ; 12/12 lab tests | General MeTTa rewriting, unification, recursion, Atomese or symbol bind are **not implemented** | +| [MeTTa assimilation](metta-datalog-assimilation-v1.md) and [council v3](metta-datalog-assimilation-v1-council-v3.md) | MeTTa AST as declaration, symbols bound once, no value gather, Quack native semantics | Council forbids value-derived cross-lane gather; old proposals superseded | +| `lance-graph-dt64/src/lib.rs` | `MethodBinding {id,sig_rev,name,on,when,reads,effects,population,truth,demand}`; ordered effects and immutable per-class schema | Registration does not execute or authorize behavior | +| `lance-graph-r2il/src/binding.rs` | `prepare` binds eligible effects, `run` checks source/frame/scope and explicit `ExecPermit` | It explicitly refuses `GuardNotExecutable` and `ThreeValuedNotExecutable` | +| `lance-graph-r2il/src/binding_native.rs` | `prepare_native` compiles eligible fold programs; `run_native` rechecks and runs borrowed planes without allocating | Data-dependent fold chaining, changing shape or unadmitted method are not general-purpose execution | +| [#1955](https://github.com/AdaWorldAPI/lance-graph/pull/1955) | Lab address reuse and canonical in-memory SoA borrow | No source-backed production lease or remote/disk zero-copy | +| [#1956](https://github.com/AdaWorldAPI/lance-graph/pull/1956) | Abakus k-of-n exact but carry words 1.5–4.8× slower on tested shapes | No reason to choose carry words as default | +| [EXO-152](https://linear.app/exo-red/issue/EXO-152) | Native algebra/external DTO normalized-method capstone | No new normalized method registry may be unilaterally minted | +| [EXO-95](https://linear.app/exo-red/issue/EXO-95) | Existing SAP/Odoo cross-glove and business parity evidence | Rails semantics or SAP executable trace have **not** been established | + +The #1944 W-1 cold symbol→DR global→local binder and W-2 `Guards::check`/`FrameStore` owner gap remain open; those are not "solved" by a new scratch object. Existing `ClassView` and `AlgebraRef` positions are authoritative: string method names are labels, not IDs. + +## 3. The new central object: *logical schema-constraint scratch* + +**Proposed name:** `SchemaConstraintScratch`, purely conceptual, **NOT** an existing Rust type or API. A short-lived, bounded, typed **working set of hypotheses about class and method identities**. This should be built from Codegraph's already interned identities, compiler facts, external schema declarations and a versioned ClassView/OGAR vocabulary, ideally as bit masks over existing Codegraph/DR address coordinates. + +**It is not:** +- mask-risc `Scratch` (runtime register slots, reset by a program/epoch), +- MeTTa/Hyperon's `GroundingSpace` (another persistent trie/interpreter), +- MORK `PathMap` (external Set oracle/index, may materialize copies), +- an owned value dataset or an alternate ClassId/MethodId codebook, +- a cache of source ASTs, business objects or executable function pointers, +- an automatically inferred equivalence relation from names/NNUE similarity. + +### 3.1 Minimal conceptual constraint basis + +| Constraint atom | Cold scratch meaning | Can it be admitted from source alone? | +| --- | --- | --- | +| `HasField(ClassCandidate, FieldRole, Type, LayoutEvidence)` | Codegraph-observed or externally declared role/field candidate | Only with source + versioned field basis/owner decision | +| `HasMethod(ClassCandidate, MethodCandidate, Arity, Signature)` | Source decl and signature candidate, separate from MethodId | Only with unambiguous identity/shape | +| `Reads(Method, Field)` / `Writes(Method, Field)` | observed read/write effect with provenance and coverage grade | Missing effect evidence => UNKNOWN, not pure | +| `OnChange(Method, Field)` | callback or trigger from source framework | Framework/source-specific, cannot guess from call edge | +| `When(Method, Expr, Logic)` | guard candidate over typed field roles | DT64 can **store**, R2IL cannot generally **execute** | +| `Do(Method, OrderedEffects)` | sequence-preserving effect description | Executable only for supported native words/arity/effects | +| `EquivalentUnder(A,B,Semantics,Witness)` | proposed equivalence scoped by explicit assumptions | Never on names or NNUE alone; must survive oracle | +| `Incompatible(A,B,Reason)` | negative evidence (effect, type, auth, ordering etc) | Retained even if candidate ranking is high | +| `UnknownCause(Candidate,Gap)` | absent language harvest, unresolved dispatch, missing definition/effect | No promotion while decisive gap remains | + +Scratch construction may be memoized **only as disposable pre-admission optimization** keyed by immutable artifact generation, source snapshot, producer version, ClassView/schema/codebook revision, scope and semantic rule version. Cached inference is **never** execution authority. Eviction changes latency, not truth. + +### 3.2 Constraint processing: source → scratch → admission + +~~~text +FACTS: F = (codegraph generation, source artifact digests, coverage) +SCOPE: U = (candidate domain, ClassView basis, method lens, permissions) +SCRATCH: H = {field mappings, method mappings, guard constraints, + effect order, allowed algebra, negatives, abstentions} +RULES: H' = NormalizeAndConstrain(F, U, H) # COL D / PROPOSED +CHECK: admissibility(H') = ADMIT | REFUSE(kind) | ABSTAIN(cause) +OUTPUT: at most an immutable method OFFER and source witness, no execution +~~~ + +An **ABSTAIN** preserves uncertainty when a source producer is incomplete, a dispatch target is dynamic, an AST variant is unsupported or an NNUE proposal lacks proof. **REFUSE** means an independently verified incompatibility or policy violation. Neither is `COUNT=0` nor "method absent". Unresolved sources must remain visible in Codegraph's graded masks and residuals. + +Scope and lifetime are two layers: Codegraph scratch is **cold pre-admission**; mask-risc scratch is **hot transient evaluation**. Do not let scratch identities survive a ClassView rev, source revision, permission revocation or frame replacement. At the admission Rubicon, only *immutable DT64-certified binding metadata* may persist. Any cached compiled program has a separate source/layout/witness lifetime, and is not a durable schema fact. + +### 3.3 Example: same concept ≠ same method behavior + +This is **illustrative pseudo-declaration**, *not MeTTa grammar accepted by any existing parser*: + +~~~text +Concept: BillableWorkEntry +Candidate adapters: Odoo.WorkLog, SAP.CATS, Rails.TimeEntry +Logical fields: actor, project, interval, approval, billable +Method intent: ApproveEntry(entry) + +Cross-source constraints: + same_concept ≠ same_transaction_policy + Odoo: observed ORM dependency/callback evidence (versioned, if harvested) + SAP: observed BAPI/CATS semantics (requires actual source/trace) + Rails: ActiveRecord callback/transaction semantics (requires Ruby harvest) + match only with explicit field types, effect order, authorization + absent Rails/SAP AST/effect trace => ABSTAIN, never assume parity + +Proposed DT64 offer: ON(field positions), WHEN(approved/preconditions), + READS, DO(ordered allowed effects), Truth, ValueDemand, SigRev +Native executable result: only if admitted semantics exactly match allowed R2IL +~~~ + +The target is not "convert SAP into Odoo." It is **one normalized concept with distinct application-local renderings and lawful method bindings**, able to share a native fold *only when the algebraic behavior is proven identical*. + +## 4. MeTTa-shaped declaration semantics, MORK and datafrog boundaries + +**Use MeTTa as a declarative syntax / pattern vocabulary.** The admissible subset is finite, typed, bounded facts, constraints, structural joins and verified normalizations. No unconstrained `superpose` / `collapse` execution, arbitrary grounded callbacks, impure rewrite, recursive fixpoint, unification over unbounded nested terms, or term construction in the hot path. + +**Rewrite distinction:** A candidate rewrite is not a proven equivalence. Codegraph `CALLS` is a fact; MeTTa-like inference may derive *candidate patterns*, but `A=B` becomes an allowable method substitution only after algebra/type/effect/permission/source witnesses satisfy a separate law. An egglog congruence is not a Codegraph edge; a learned NNUE score is not an algebraic law. + +**MORK:** an external set-oriented structural oracle; its `PathMap` is not a second Lance identity authority. [EXO-79](https://linear.app/exo-red/issue/EXO-79) records license/usage gate. **MeTTa `BindingsSet` is a multiset**; MORK space is Set. Do not silently drop duplicates, and don't ask a Boolean mask to answer multiplicity. [EXO-143](https://linear.app/exo-red/issue/EXO-143) four-binding case and [EXO-108](https://linear.app/exo-red/issue/EXO-108) bag terminal remain gates. + +**datafrog:** independent oracle for applicable Datalog/join/fixpoint semantics. No automatic recursion lowering into mask-risc; [EXO-73](https://linear.app/exo-red/issue/EXO-73) owns a lawful recursive driver, and [EXO-101](https://linear.app/exo-red/issue/EXO-101) explicitly rules out prior gather-based semijoin results as a production basis. + +**NNUE:** an optional, *offline* approximate candidate prioritizer over codegraph-derived features (types, roles, call/effect provenance, structural positions). It may rank which cross-app method candidates the exact constraint solver tests first. It **cannot create facts, approve an absent source effect, override a refusal, grant execution, populate WORLD, or cause a learned representation to be treated as canonical DR addresses**. Report calibration, novelty, abstention, negative controls. A perfect NNUE score is not authority. + +## 5. DT64 binding contract, exactly where the execution seam stops + +A successful scratch result describes an OFFER to the owner of `ClassId`/`ClassView`/`BindStore`. On admission: + +- Class identity is issued by its existing codebook owner; application-local class identity and shared concept half are distinct. +- Resolve symbol labels to **DR global identities once, cold**, then source-specific local field positions. Never equate Codegraph `Pos`, Charon `FunDecl` or a symbolic atom with `MethodId` or a physical row address. +- Bind `ClassSchema {schema_rev,field_count,extent}` and `MethodBinding` under immutable signature revisions. ON and READS are masks; WHEN stores a guarded expression and truth mode; DO retains **effect order**. Do not reduce method semantics to a bag of calls. +- Reject unproven effect order, unknown field type, changed source schema, unclear permissions, missing guard semantics or unauthorized effect. **Prepared plan caching** must use its full binding and source/layout/scope witness, not the fact that a method name happens to match. +- The real `r2il::binding::prepare` explicitly refuses methods whose WHEN is present and three-valued truth. `binding_native::prepare_native` inherits eligibility conditions and refuses fold-result-dependent future folds, data-derived scalar after fold, shape changes and excess outputs. **These are honest stop conditions.** +- `ExecPermit` is lab-level only, not an authorization issuance mechanism. The source owning executor and the named operator must approve production admission. DT64's own `admit` does not grant execution. + +**Important inversion:** Do not try to "make Codegraph execute." Codegraph learns source structure; DT64 binds meaning; existing native executor executes eligible semantics. The connection is **source evidence → immutable admissible method**, not Codegraph → arbitrary AST interpreter. + +## 6. Memoization through the existing ndarray / Abakus fabric + +There are **three separate memoization levels**, with different validity: + +| Level | May be cached cold? | Keys and invalidations | Runtime meaning | +| --- | --- | --- | --- | +| Candidate constraint scratch result | optionally; *disposable* | artifact digest/generation, harvester version, provenance gaps, class/method schema, rule revision, scope | Proposal only; cache hit never grants permission | +| Admitted method recipe / native prepared plan | if owner permits | `MethodRef` + `SigRev` + `GuardWitness` + `ClassView` basis + OGAR vocabulary rev + IR/semantics version + source/layout/permission guards | Reuse compiled algebra with hot rechecks; no copied values | +| Local DR route / optional source locator receipt | D-DRISA / D-DRIVER-ZC gated | frame/map issue, source incarnation, layout/row-index seal, field scope, source lease | Data-local address reuse only, no business-object cache | + +**Abakus:** a potential bit-sliced arithmetic *execution consumer* for k-of-n or later significance-plane accumulation. #1956's measured carry path loses on eight tested shapes. The memoization gain must be independently measured as **compile/bind/re-route amortization or reusable significance-plane calculations**. Never promise that Abakus changes semantic-schema parsing cost or outperforms native arithmetic without an experiment. Use existing `ndarray::simd` API; no new ISA, cache backend, F### or CPU emulator. + +**Cold/hot split:** parse/harvest/constraint solve/DT64 registration/prepare all **cold**; hot path performs witness checks, scoped borrow, existing native program and result. Profiling must not hide cold preparation in a benchmark claiming zero allocation. + +## 7. What this can and cannot establish + +The *smallest meaningful demonstration* is not SAP ingestion or trained NNUE. + +1. Harvest one **compiler-grounded** source method and one independently defined synthetic application profile with differing method effects. +2. Derive a bounded set of candidate field/method constraints from Codegraph into ephemeral masks, including a **negative incompatible candidate**. +3. Produce one lawful, guard-free two-valued DT64 `MethodBinding` proposal and one refused guarded/dynamic/effectful proposal. +4. Owner-approved lab admission binds the valid proposal via the existing R2IL native seam, runs an independent scalar oracle, then executes 256 times without reconstructing values or reparsing source. +5. Change schema/source/scope/effect witness and prove the prepared binding or candidate scratch is invalidated appropriately. A wrong inferred ClassId or field ordinal must be refused. +6. Count scratch footprint, cold harvest and prepare, allocations, hot path, copied canonical value bytes, and status per evidence class. + +This proves a **class-method builder contract** in one constrained case, not general executable MeTTa, a Rails/SAP parser or semantic equivalence of enterprise systems. + +## 8. Source and owner crosswalk + +**Repository source:** +- [Codegraph](../../crates/lance-graph-codegraph/src/lib.rs), [graph](../../crates/lance-graph-codegraph/src/graph.rs), [query grades](../../crates/lance-graph-codegraph/src/query.rs), [coverage](../../crates/lance-graph-codegraph/src/diag.rs), [binary lanes](../../crates/lance-graph-codegraph/src/artifact_bin.rs). +- [DT64 MethodBinding](../../crates/lance-graph-dt64/src/lib.rs), [R2IL prepare/run](../../crates/lance-graph-r2il/src/binding.rs), [native prepare/run](../../crates/lance-graph-r2il/src/binding_native.rs). +- [MeTTa frontend](../../crates/lance-graph-metta/src/lib.rs), [#1944 cross-frontend probe](../probes/2026-11-03-d-cognitive-polyfill/REPORT.md), [MeTTa plan](metta-polyfill-v1.md), [council correction](metta-datalog-assimilation-v1-council-v3.md), [MORK Schnellverschluss](mork-pathmap-schnellverschluss-v1.md). +- [Semantic grammar recovery](../board/entries/2026-11-02-coresearch-semantic-grammar-recovery.md) particularly B1–B9 AST/effect/provenance harvest losses, source FACT → GUARD → TRANSFORMATION distinction. + +**Linear existing decision owners:** +[EXO-84](https://linear.app/exo-red/issue/EXO-84) MeTTa; [EXO-85](https://linear.app/exo-red/issue/EXO-85) assimilation; [EXO-79](https://linear.app/exo-red/issue/EXO-79) MORK; [EXO-143](https://linear.app/exo-red/issue/EXO-143) binding bag; [EXO-101](https://linear.app/exo-red/issue/EXO-101) datafrog; [EXO-73](https://linear.app/exo-red/issue/EXO-73) recursion; [EXO-108](https://linear.app/exo-red/issue/EXO-108) Bag; [EXO-121](https://linear.app/exo-red/issue/EXO-121) ClassView; [EXO-46](https://linear.app/exo-red/issue/EXO-46) cold symbol/DR and focus; [EXO-140](https://linear.app/exo-red/issue/EXO-140) foreign lookup; [EXO-152](https://linear.app/exo-red/issue/EXO-152) normalized method external dialects; [EXO-95](https://linear.app/exo-red/issue/EXO-95) enterprise business parity; [EXO-153](https://linear.app/exo-red/issue/EXO-153) native register; [EXO-154](https://linear.app/exo-red/issue/EXO-154) zero-copy driver. + +**Non-goals:** no new MeTTa execution, MORK code vendoring, SAP or Rails harvester invented from text, Codegraph → DT64 automatic promotion, neural authority, production WHEN interpreter, e-graph equality from graph adjacency, new ClassId allocator, second role/tuple store, or new cognitive-shader driver. + +## Update (2026-11-04): lazy Codegraph and FieldViewDef / Datalog separation + +The operator refined the class-method builder: **facts and identity remain Kythe-shaped source evidence, Crepe/Soufflé/XLOG supply only cold demand/binding/stratification plan evidence, and Codegraph becomes a demand-scoped graded view rather than a second owned AST object tree.** Its existing `Cube` / `Pos` / `OnceCell` paths are partially lazy already, not a complete demand-driven implementation. See the source-backed [D-CGMM-LAZY-0 addendum](d-cgmm-lazy-codegraph-fact-demand-v1.md) and the [staged plan](d-cgmm-staged-pr-charter-v1.md). + +**FieldViewDef** is a **proposed, immutable semantic offer on the existing ClassView bit basis** (including `lens_fields(full_classid)` when behavior is requested). A disposable MeTTa-shaped constraint scratch can produce typed OGIT `LinkSpec` / OGAR `ActionDef` / DT64 `MethodBinding` offers, subject to independent owner admission. **Pure, same-population** views may polyfill through existing RelAlg, but arbitrary M:N joins, guarded WHEN, effectful action bodies, recursive rules and unknown source effects remain typed refusals/ABSTAIN. `FieldMask` bits continue to mean instance presence only; no new ClassView method, codebook or hot-path MeTTa evaluator is authorized by this plan. + +## Implementation packet (2026-11-04) + +To execute this research as independently reviewed PRs, use [implementation blueprint](d-cgmm-implementation-blueprint-v1.md) for exact source→demand→FieldViewDef→RelAlg/DT64 contracts, [EXO decision ledger](d-cgmm-exo-gates-v1.md) for owner authorization boundaries, and [concrete fixture oracles](d-cgmm-executable-fixtures-v1.md) for nonvacuous expected results, negative controls and cold/hot accounting. All files are **unexecuted documentation**, not a general MeTTa/Datalog runtime or a production `FieldViewDef` API. [CGMM staged PR charter](d-cgmm-staged-pr-charter-v1.md) remains authoritative about sequencing. diff --git a/.claude/plans/d-cgmm-executable-fixtures-v1.md b/.claude/plans/d-cgmm-executable-fixtures-v1.md new file mode 100644 index 000000000..f750c25bd --- /dev/null +++ b/.claude/plans/d-cgmm-executable-fixtures-v1.md @@ -0,0 +1,133 @@ +# D-CGMM-FIXTURES-0: concrete independent source and semantic oracles + +> **2026-11-04 · UNEXECUTED TEST SPECIFICATION.** These fixtures define independent test inputs and expected results for future, separate CGMM PRs. They do not claim shipped AST harvest, real SAP/Rails semantics or native runtime authorization. [Blueprint](d-cgmm-implementation-blueprint-v1.md) · [EXO ledger](d-cgmm-exo-gates-v1.md) · [46 higher-level falsifiers](d-cgmm-falsifier-matrix-v1.md) · [EXO-155](https://linear.app/exo-red/issue/EXO-155). + +## 0. Oracle independence + +Reference data is authored BEFORE implementations. A correct oracle enumerates static facts/values by stable producer/global ID and may not call `Codegraph::partners`, candidate demand planner, FieldView scratch or native prepared fold. An optional pinned Crepe/datafrog/Soufflé/MORK comparator is a **separate oracle** with semantic/licensing gates. Every deliberate disable must turn a nonvacuous result or refusal check red. + +## F-A: Ruff/Charon/Kythe-shaped source fact and grade corpus + +~~~text +repo = sandbox-codeapp; revision = R1; producer = fixture-v1 +source node keys (stable strings scoped to repo/producer/rev): + f=Controller::run g=Service::read h=Store::save + i=DynImpl::process t=Trait::process k=Audit::record +facts, each with exact source span, revision and producer receipt: + CALLS_RESOLVED(f,g) + CALLS_RESOLVED(f,h) + MAY_DISPATCH(f,t) + IMPLEMENTS(i,t) + UNRESOLVED_CALL(f,?dynamic) + READS(g,amount) # SYNTHETIC typed producer only; not shipped intake + WRITES(h,status) # SYNTHETIC typed producer only; not shipped intake + effects(i)=UNKNOWN; extraction=PARTIAL +reference CALLEES(f): + confirmed={g,h}, sites=2 + possible={i} + unknown={?dynamic} +reference method purity: + h is not pure (known write); i is ABSTAIN(UNKNOWN_EFFECTS); + g pure only with COMPLETE certified read/write coverage +~~~ + +**Demand D1** only confirmed callees and source locations; **D2** additionally possible; **D3** additionally unknown. These MUST preserve grades, not promote MAY to confirmed. Baseline Codegraph still constructs six Cube relations at intake: measure, do not claim they vanished. + +**Kills:** promote MAY to CALLS; omit WRITES while claiming COMPLETE; equate two same-named nodes from source revisions R1/R2; accept placeholder `?` as durable identity; refresh malformed source and partially mutate committed cube. Any missing writer effect column yields **NOT_RUN / ABSTAIN**, not an invented result. + +## F-B: ClassView / FieldViewDef / RelAlg value corpus + +One synthetic source-owned SoA: source global IDs T1..T6, a class basis S1 with `status` U32 at position 0 and `amount` I32 at position 1, and a separate 109-field class with a test field at **position 100**. + +| Global ID | Tenant | status | amount | presence | status==2 AND amount>=100 | +| --- | --- | --- | --- | --- | --- | +| T1 | A | 2 | 120 | both present | TRUE | +| T2 | A | 2 | 90 | both present | FALSE | +| T3 | A | 1 | 150 | both present | FALSE | +| T4 | A | 2 | 101 | both present | TRUE | +| T5 | B | 0 | 0 | both present | FALSE | +| T6 | A | absent | 101 | amount only | UNKNOWN | + +**Expected:** For admitted *two-valued* domain T1..T5, output mask `{T1,T4}` and COUNT=**2**. With T6 under explicit K3 semantics, today's R2IL native eligibility must **refuse** three-valued execution. Any SQL WHERE UNKNOWN filtering requires independent declared semantics and separate admission, not hidden coalescing. + +**Kills:** swap field positions 0/1; same label from different ClassId; truncate position 100 to a 64-bit field mask or wrap position; try `lens_fields` under only the concept half; treat missing status as present zero; authorize using a stale schema basis. Scratch candidate masks are not field-presence bits. An alias/pointer check and materialization trap demonstrate no owned canonical status/amount vectors on admitted borrowed hot path. + +## F-C: OGIT M:N, tenant scope, Set and Bag + +Stable IDs: +- A: W1=1001, W2=1002; P1=1101, P2=1102. +- B: W3=2001; P3=2101. + +~~~text +edge-instance rows: +e1=(W1,P1,tenant=A) +e2=(W1,P2,tenant=A) +e3=(W1,P2,tenant=A) # distinct edge identity, duplicate pair +e4=(W2,P2,tenant=A) +e5=(W3,P3,tenant=B) +e6=(W2,P3,cross-tenant) # physically present, unauthorized +OGIT proposed link: Worker --assigned_to--> Project +Cardinality::ManyToMany +Tenant policy: same-tenant endpoints allowed; cross-tenant denied +~~~ + +**Expected:** Tenant A Boolean Set of legal pairs has cardinality **3**, Bag count **4**. A+B legal Set **4**, Bag **5**. e6 never appears in admitted results. Forward direction does **not** automatically permit Project→Worker inverse; LinkSpec schema is not edge-instance identity or an execution permit. + +**Kills:** disable tenant filter, deduplicate Bag count, join on unshared local ordinal, reverse edge direction, treat schema cardinality as actual pair population or enumerate cross-product. Each must be visibly wrong or explicitly refused. + +## F-D: OGAR ActionDef, Odoo/SAP/Rails equivalence + +~~~text +SYNTHETIC app profiles only, no asserted real ERP behavior: +Odoo ApproveEntry: condition=pending, ordered effects: + (1) status=approved; (2) log approval timestamp +SAP ApproveEntry: condition=pending, ordered effects: + (1) status=approved; (2) post accounting document; + (3) log approval timestamp +Rails ApproveEntry: Ruby harvester evidence missing +~~~ + +**Expected:** Same concept, **different method effects**, so the Odoo/SAP actions must **not** share an admitted executable binding. Rails yields `ABSTAIN(MISSING_PRODUCER)`, never inferred purity. OGAR ActionDef is a declaration, not ActionInvocation; DT64 can store a guarded method but current R2IL must refuse its execution. + +**Kills:** omit SAP posting, reorder effects, approve Rails without source, cache by common method name, collapse tenant identity. All must change the refusal/equivalence verdict. + +## F-E: independently versioned mutations + +~~~text +R1 → R2: source adds WRITE fact; human method labels unchanged +S1 → S2: T1.amount changes 120→80; DR identity mapping unchanged +M1 → M2: MethodBinding SigRev/ordered effects change; source static +P1 → P2: tenant-A execution permit revoked; class basis unchanged +L1 → L2: physical source SoA incarnation replaced; numeric version reused +scratch-cache eviction with no source/schema change +~~~ + +**Expected:** R2 invalidates fact/constraint scratch; S2 invalidates value-derived result mask, not necessarily DR map; M2 invalidates admitted method recipe; P2 invalidates authorization; L2 invalidates layout lease even with matching weak digest. Scratch eviction alone must not change truth. Reusing a method name or FNV digest alone is prohibited. + +## 1. Fair benchmark arms + +| Arm | Work included | Claim and counters | +| --- | --- | --- | +| B0 scalar oracle | Independent enumeration on static facts/value rows | Correctness only, separate time | +| B1 current Codegraph | Parse `.rows.bin`, intern positions, build 6 relations, run `partners/or_partners` | Cold ingestion + relation build, p50/p95/p99, bytes and binds | +| B2 demand-scoped lab | Same source/result/grades, prepare demand and query | Cold compile, demanded fact count, Q=1/2/8/16/64/256, cache and memory | +| B3 FieldView scratch | Candidate roles and logical constraints | OFFER/REFUSE/ABSTAIN, cold bytes/allocations | +| B4 owner-approved RelAlg/R2IL | Pure same-row query on borrowed SoA | Cold native prepare and hot repeated fold, no source-value copy | +| B5 witness-keyed memo | B4 with prepared recipe and frame/scope/source validation | Hit/miss/evict/invalidate, total query and permission checks | + +Corpus sizes: 20 for correctness; 256 and 4,096 symbols for cost; 65,536 DR rows is a **separate** source-layout experiment, not comparable with 65,536 harvested AST symbols. Vary dynamic-call rate, sparse/dense fact edges, 1 vs 16 methods and Q independently. If B2 loses after including cold cost, report **NO BENEFIT**. + +## 2. Stage acceptance gates + +| Proposed PR | Positive | Deliberate disable | +| --- | --- | --- | +| CGMM-1 | Stable source identity + preserved grades/effect gaps | Delete/write wrong effect -> purity admission rejected | +| CGMM-2 | Confirmed/possible/unknown parity + explicit cold/hot economy | MAY promoted; stale demand cache | +| CGMM-3 | S1/wide ClassView exact, scratch eviction stable | Wrong full_classid/position -> refusal | +| CGMM-4 | Tenant A Set=3, Bag=4; legal tenant scope; distinct method effects | Tenant filter or action posting removed | +| CGMM-5 | COUNT=2 on T1..T5 and native read parity | Missing T6 coerced zero; foreign M:N "join" silently accepted | +| CGMM-6 | Owner-issued guard-free method exact native parity | Guarded/K3 or stale permit refuses | +| CGMM-7 | Valid recipe caching, no copied SoA value columns | Value/source/scope revised but cache reused | +| CGMM-8 | Independent oracle + scoped Codegraph caller coverage | Insert copied-value bypass; CI fails inside claimed roots | + +**All gates NOT_RUN** in this documentation PR. Future PRs must report actual raw results, commit SHAs, source/owner decisions and EXO-50 handoff; no automatic production admission. diff --git a/.claude/plans/d-cgmm-exo-gates-v1.md b/.claude/plans/d-cgmm-exo-gates-v1.md new file mode 100644 index 000000000..22e297345 --- /dev/null +++ b/.claude/plans/d-cgmm-exo-gates-v1.md @@ -0,0 +1,71 @@ +# D-CGMM-EXO-0: Linear decision ledger and staged PR ownership crosswalk + +> **2026-11-04 · DOCS / DECISION REQUESTS ONLY.** This ledger records named issue owners and **decisions to request**, not permissions granted. The source of truth for implementation is [D-CGMM #1958](https://github.com/AdaWorldAPI/lance-graph/pull/1958), its [implementation blueprint](d-cgmm-implementation-blueprint-v1.md) and [staged charter](d-cgmm-staged-pr-charter-v1.md). [EXO-155](https://linear.app/exo-red/issue/EXO-155) tracks this program, [EXO-50](https://linear.app/exo-red/issue/EXO-50) is an append-only message bus. Linked issues are **not** implicit approvals. + +## 1. Critical path owner decisions: answer with a source and a test + +| Priority / stages | Existing Linear issue | Exact decision owner must make | Required evidence / what to refuse before ruling | +| --- | --- | --- | --- | +| **P0, CGMM-1** | [EXO-155](https://linear.app/exo-red/issue/EXO-155) (program); source producer owner still to be named | May Ruff/Charon writer extend typed source fact lanes for read/write/receiver/dispatch without breaking versioned `.rows.bin` consumer? Which source identity is stable? | Writer+reader contract/fixture and independent Kythe-like identity oracle. If producer lacks effects, keep UNKNOWN and stop automatic method equivalence. | +| **P0, CGMM-2** | [EXO-85](https://linear.app/exo-red/issue/EXO-85) MeTTa/Datalog assimilation; [EXO-84](https://linear.app/exo-red/issue/EXO-84) native polyfill | Is demand/binding metadata a read-only compiler artifact consumed by Codegraph and Quack, with no parallel interpreter? | Compare current Codegraph bound joins to demand-scoped masks and `crepe_plan!` D1/D2 against independent oracle. Do not import Soufflé/Crepe runtime. | +| **P0, CGMM-3** | [EXO-121](https://linear.app/exo-red/issue/EXO-121) ClassView capstone | Is `FieldViewDef` an immutable offer anchored to existing `FieldRef` and full behavior lens, or should an existing type be reused? Does it belong above zero-dep `ClassView`? | Same label different ClassId/schema, 109-field wide basis, full_classid behavior lens, scratch mask vs presence distinction. No trait mutation before decision. | +| **P0, CGMM-4** | [EXO-95](https://linear.app/exo-red/issue/EXO-95) business parity; ontology/OGAR owners | Which OGIT `LinkSpec` and OGAR `ActionDef` fields are authoritative for role, M:N cardinality, ordered effects and app-specific behavior? | Tenant A/B, duplicate edges, SameConcept/DifferentAction, unknown Rails/SAP evidence. A LinkSpec does not authorize edge instances or action execution. | +| **P0, CGMM-5/6** | [EXO-152](https://linear.app/exo-red/issue/EXO-152) normalized method DTO/native aliases | Can an eligible pure FieldViewDef map to an existing `RelAlg::Rel` and then an admitted DT64 MethodBinding without creating a second method identity? What semantics demand REFUSE? | Exact single-population read vs M:N refusal; guard-free/K3/WHEN refusal; pinned method version and native independent oracle. | +| **P0, CGMM-3/4/6** | [EXO-46](https://linear.app/exo-red/issue/EXO-46) counted global→focus binding | Who attests cold symbol→DR global/local and scope binding? What lookup costs are legal before borrowed fold? | No Codegraph `Pos` = DR ordinal or ClassId shortcut. Refuse unowned foreign bind; count preparation once and require source/frame witness. | +| **P0, CGMM-4/5** | [EXO-140](https://linear.app/exo-red/issue/EXO-140) admission B call-site classification | Which `Semijoin/EqU32Via/Gather` or foreign-hop shapes are lawful; who classifies each call site? | No value-derived cross-source gather masquerading as FieldViewDef; classify each with reader and source-order witnesses. | +| **P0, CGMM-4/5/8** | [EXO-108](https://linear.app/exo-red/issue/EXO-108) Set-vs-Bag terminal | How is count-distinct over projection distinguished from Bag fanout product and Boolean support? | Duplicate-edge fixture with different Set and Bag answers; no novel opcode absent insufficiency proof. | +| **P1, CGMM-5/8** | [EXO-73](https://linear.app/exo-red/issue/EXO-73) recursion driver | Where does a legal recursive driver and bounded/unbounded termination contract live? | CGMM first explicitly refuses recursion. An α-acyclic shape classifier and a bounded external Datalog oracle do not create an admitted runtime. | +| **P1, CGMM-8** | [EXO-79](https://linear.app/exo-red/issue/EXO-79) MORK PathMap/license, [EXO-143](https://linear.app/exo-red/issue/EXO-143) MeTTa four-binding case | Under which license/version may MORK run externally as differential oracle, and how is Set vs multiset binding checked? | External oracle pinned, 4-binding counterexample, no MORK code vendoring; no runtime authority. | +| **P1, CGMM-7/8** | [EXO-154](https://linear.app/exo-red/issue/EXO-154) zero-copy virtual driver, [EXO-153](https://linear.app/exo-red/issue/EXO-153) DR registers | May prepared plan/route receipts borrow canonical SoA without copying business values? What source/layout/permission epochs are checked? | #1955 local borrowed proof only; #1956 ALU speed loss; no disk/remote zero-copy inference, no new index. | +| **P1, CGMM-8** | [EXO-104](https://linear.app/exo-red/issue/EXO-104) tuple materialization, [EXO-105](https://linear.app/exo-red/issue/EXO-105) reference executor, [EXO-109](https://linear.app/exo-red/issue/EXO-109) oracle independence | Which functions are admissible only as independent test oracles, and how do we stop reference tuples reaching production? | Candidate and oracle cannot share the same execution core; compile/CI root-scoped bypass counters; no broad symbol grep presented as a full call-graph proof. | +| **P1, CGMM-6** | [EXO-155](https://linear.app/exo-red/issue/EXO-155) + DT64/R2IL execution owner to be explicitly named | Who issues production execution capability, validates WHEN/DO, body revision and tenant permission? | `ExecPermit` is lab-only; current `GuardNotExecutable` and `ThreeValuedNotExecutable` remain active. Do not treat `BindStore::admit` as an execute permit. | +| **P2, CGMM-8** | [EXO-101](https://linear.app/exo-red/issue/EXO-101) datafrog/semijoin parity | Which external Datalog fixture may be used as a credible separate reference for recursion/bag? | No unproven gather driver, no semantic cross-contamination or production import. | +| **P2, CGMM-8** | [EXO-50](https://linear.app/exo-red/issue/EXO-50) | Preserve append-only review, cross-agent disagreement and PR/source references | EXO-50 posts are communications, not admission or merge authorization. | + +**Note:** Issue names/statuses represent the 2026-11-04 source check, not a frozen guarantee that the owners have already approved a decision. A named issue is a decision queue; **the actual accepted ruling, commit, tests and scope must be recorded separately**. + +## 2. Stage-to-Linear traceability + +| Proposed PR | Main Linear tracker | Cross-owner gates | Definition of done at lab level | +| --- | --- | --- | --- | +| D-CGMM-0 (this docs PR) | EXO-155 | EXO-50, 121, 152 | Source maps, owner ledger, independent fixture matrix, no runtime change | +| D-CGMM-1 source facts | EXO-155 | Ruff/Charon producer owner, EXO-95 | Writer/reader contract, stable identity under revisions, detected effect-loss mutation | +| D-CGMM-2 lazy demand | EXO-155 | EXO-84, 85, 73 | Same graded mask as eager/oracle, query Q cost/copy evidence, stale plan refusal | +| D-CGMM-3 FieldViewDef scratch | EXO-155 | EXO-121, 152 | Correct ClassView basis + typed OFFER/REFUSE/ABSTAIN, no field-codebook mutation | +| D-CGMM-4 ontology/action/tenants | EXO-155 | EXO-95, 108, 46, 140 | M:N cardinality and Bag distinct, tenant and effect-order kills, independent references | +| D-CGMM-5 RelAlg polyfill | EXO-155 | EXO-152, 73, 108, Quack/R2IL owner | One legal pure fold and explicit unsupported foreign/action/WHEN refusals | +| D-CGMM-6 DT64 native bind | EXO-155 | EXO-121, 152, execution permit owner | Immutable method identity, native parity, stale witness/guard/K3 refusals | +| D-CGMM-7 memo economy | EXO-155 | EXO-153, 154, 46 | No copied payload, Q amortization + whole-query cost, correct cache invalidation | +| D-CGMM-8 evidence/callgraph | EXO-155 | EXO-79, 143, 101, 73, 104, 105, 109, 154 | Independent oracles, root-scoped coverage, uncertainty and external licensing tracked | + +**Approval dependency:** CGMM-1/2 may be performed as workspace-excluded *labs* only after the applicable source/editor access is established. CGMM-3/4 may draft schema offers without admitting them. **CGMM-5/6 require named RelAlg/DT64/OGAR owners before altering any production vocabulary or executing a generated MethodBinding.** CGMM-7/8 must not be used to backdoor an unadmitted performance optimization. + +## 3. Exact questions for EXO-155 owner review + +1. **Fact identity:** Which persistent source-node key actually survives Ruff/Charon export and schema refresh? Is it enough for cross-language provenance? If not, how is UNKNOWN retained without a new graph registry? +2. **Demand:** Is Codegraph permitted to keep demanded compile metadata in an excluded/lab struct, and should prior XLOG magic/polarity loss be patched *only* after a disable test proves observable semantic impact? +3. **FieldViewDef home:** Existing ClassView field/ontology object references versus a new small sibling typed offer; prohibit mutable scratch in zero-dependency ClassView trait. +4. **Tenant edges:** Do we propose OGIT LinkSpec and OGAR ActionDef independently rather than a giant normalized class schema? Which application policy owns intertenant permission? +5. **RelAlg boundary:** Does single-row RelAlg handle the exact pure offer, and when is Quack/XLOG/DT64 necessary? How do unsupported join/WHEN/recursion/Bag shapes refuse rather than degrade? +6. **Prepared native memo:** Which authority attests signature, semantic law, value revision, source, layout, ClassView, scope and permissions? Which of these can change independently? +7. **Negative evidence:** What coverage witness would permit closed-world conclusions about source effects? Under PARTIAL harvest the default remains ABSTAIN. +8. **Oracle independence:** What separate source fixture and external implementation can verify actual answers without calling candidate graph functions? +9. **CI gate:** Which declared migrated Codegraph/RelAlg/DT64 call roots can be sealed first so a deliberate value-copy or bypass turns CI red? Do not claim all historical calls. +10. **Product intent:** When is a candidate semantic DTO an application-specific class offer rather than a domain-common concept? Same vocabulary is insufficient to equate transaction effects. + +## 4. Evidence and status protocol + +~~~text +EXO-RULING-REQUEST: + issue: EXO- + from: D-CGMM- / GitHub PR / exact commit + current_source: path:line and tool/commit digest + proposed_semantic_contract: typed inputs, outputs, refusal and owner + counterexample: independent witness or mutation that fails + evidence_grade: SOURCE_VERIFIED | LAB_MEASURED | PROPOSED | GATED | NOT_COMPARABLE + decision: ACCEPT_NARROWLY | REFUSE | DEFER | REQUEST_PROBE + authorization: explicit owner/ruling reference, *not* an EXO-50 comment + open_dependencies: linked EXO IDs and affected future PRs +~~~ + +**A permitted proposal is not an admitted executor.** In particular, an inferred ON/WHEN/DO tuple cannot become executable simply because its source syntax was accepted, and a joined M:N view cannot become a source-owned foreign lookup because a path planner has an index hint. diff --git a/.claude/plans/d-cgmm-falsifier-matrix-v1.md b/.claude/plans/d-cgmm-falsifier-matrix-v1.md new file mode 100644 index 000000000..55402e24c --- /dev/null +++ b/.claude/plans/d-cgmm-falsifier-matrix-v1.md @@ -0,0 +1,97 @@ +# D-CGMM independent falsifier matrix: evidence, demand, field roles, actions and native execution + +> **2026-11-04. ALL NEW CASES PROPOSED/NOT RUN.** Historical #1944 MeTTa/XLOG/Crepe bounded parity, #1955 borrowed SoA and #1956 Abakus correctness are predecessor evidence, not new CGMM test results. [Stages](d-cgmm-staged-pr-charter-v1.md), [lazy Codegraph](d-cgmm-lazy-codegraph-fact-demand-v1.md), [Action/Edge](d-cgmm-action-relation-schema-v1.md), [EXO-155](https://linear.app/exo-red/issue/EXO-155). + +**Independent oracle rule:** source-of-truth fixture carries stable producer IDs, per-source revision and compiler-grounded facts. Oracle enumerates those original facts, tenant policies and typed value rows without calling Codegraph's `Cube::partners`, candidate Codegraph binding, scratch normalizer, RelAlg lowering, or issued method binding. At least one deliberate disable per stage must make the oracle/representation/refusal check red. An empty expected result is insufficient unless a distinct positive control also passes. + +**Source grades:** compiler-confirmed, possible dispatch, unresolved dynamic, missing producer, inferred candidate, owner-admitted. The candidate must not silently use a stronger grade than its input. + +## F. Source/Kythe identity and loss (F1–F8) + +| ID | Nonvacuous fixture and oracle | Falsifier / expected refusal | +| --- | --- | --- | +| F1 | Two identical symbol names in different crates, source IDs distinct; source-ID oracle gives two nodes | Intern by name only → wrong identity or typed conflict | +| F2 | One method declaration with source span, source revision and producer digest | Stale source span or swapped revision → refusal | +| F3 | Five CALLS, two MAY, one dynamic UNKNOWN; expected grade masks differ | Promote MAY to CALLS; grade oracle goes red | +| F4 | A method reads x, writes y and executes ordered effects (synthetic complete producer) | Drop WRITES/order while claiming complete coverage → fail | +| F5 | Producer omits a write-effect column and marks PARTIAL | Treat omission as pure/no-effect → incorrect admission; ABSTAIN required | +| F6 | Cross-language sources with verified node identity/provenance and one missing Ruby source | Infer Rails method effects from Odoo shape → fail | +| F7 | Codegraph symbol `Pos` changes on source refresh while stable source identity does not | Store `Pos` as durable MethodId → stale identity refusal | +| F8 | Missing/invalid binary schema or placeholder `?` symbol | Typed intake refusal; no partial mutation of committed cube | + +## D. Datalog demand / lazy Codegraph (D1–D10) + +| ID | Nonvacuous fixture and oracle | Falsifier / expected refusal | +| --- | --- | --- | +| D1 | Root f with two confirmed callees, five unrelated source edges | Demand only f confirmed; unrelated edges not projected or cached | +| D2 | Two query roots share one neighbor, repeated Q=1/2/16/256 | Result equality with eager query; measure route/bind count and costs separately | +| D3 | Same root with one MAY-dispatch target | Demand confirmed-only excludes MAY; include-possible includes it but grade stays POSSIBLE | +| D4 | Dynamic unbounded source call exists | Closed-world NOT/CALLS absence interpreted as proof of safety → refusal | +| D5 | Crepe plan `let` drift fixture against `crepe!` independent output | Unverified `crepe_plan!` index certificate must not be trusted | +| D6 | XLOG magic-set compile report for query with demand and negation | Lost polarity/stratification metadata yields NOT_COMPARABLE/REFUSE, not silently executable | +| D7 | Source generation changes between demand preparation and query | Stale demand memo/hot mask must refuse or recompute | +| D8 | Bound/free direction reversed in one relation | One result differs; deliberate direction reversal makes oracle red | +| D9 | Boolean Set versus counting Bag on duplicate edges | Distinct support stays, total count differs; no multiplicity collapse | +| D10 | Recursive SCC request and cycle with 3 nodes | No admitted recursion driver → UnsupportedRecursion, not guessed closure | + +## V. ClassView / FieldViewDef / constraint scratch (V1–V10) + +| ID | Nonvacuous fixture and oracle | Falsifier / expected refusal | +| --- | --- | --- | +| V1 | Two field names equal under different ClassId/schema bases | Scratch cannot equate field position by label | +| V2 | Full behavior classid lens known; concept half alone insufficient | Missing `lens_fields(full_classid)` must refuse, never fallback | +| V3 | 109-field Odoo-like wide class with one field at position 100 | Reusing 64-bit position or truncating → fail | +| V4 | FieldMask indicates present zero, source semantics preserve present zero | Treat presence as sign/value → fail | +| V5 | Scalar FieldViewDef borrowed canonical lane and same ClassView basis | Any copied source-value column in admitted read path triggers anti-copy checker | +| V6 | Independent type conflict: source I32, candidate U32 signed compare | Refuse instead of auto-coercing absent/negative values | +| V7 | Same schema, changed source effect facts | Scratch invalidated; admitted unchanged field basis remains stable | +| V8 | Scratch eviction and recomputation | Identical OFFER/REFUSE/ABSTAIN, no mutation of admitted DT64 | +| V9 | Two candidates overlap in source span and disagree on method identity | Preserve conflicting candidates, do not last-write-wins | +| V10 | Model/NNUE changes candidate order but not ground facts | Scores do not change admission or unknown grade | + +## R. OGIT/OGAR/RelAlg relationship/action semantics (R1–R10) + +| ID | Nonvacuous fixture and oracle | Falsifier / expected refusal | +| --- | --- | --- | +| R1 | M:N Worker W1→P1/P2, W2→P2 with independent global-ID edges | Join on local ordinal gives wrong answer | +| R2 | Duplicate W1→P2 edge | Boolean support and Bag count diverge as expected | +| R3 | Tenant A W1/W2/P1/P2, tenant B W3/P3, forged A→B edge | Dropped tenant check leaks edge; typed unauthorized refusal | +| R4 | Inverse traversal not explicitly declared | Reversing (W1,P2) into (P2,W1) unsupported unless independently admitted | +| R5 | Two ActionDef named Approve with different ordered side effects | Same name alone cannot share MethodBinding | +| R6 | Missing source write/effect for SAP/Rails comparison | ABSTAIN rather than equivalent or nonexistent | +| R7 | Verified U32 status==2 on one source lane | RelAlg U32 equality matches independent scalar oracle | +| R8 | Verified I32 amount>=100 with negative and absent values | RelAlg threshold matches independently for admitted two-valued domain; unknown refuses | +| R9 | RelAlg `join_body::is_alpha_acyclic` classifies acyclic M:N join | It must not be treated as executed join, only planning metadata | +| R10 | Source action has conditional WHEN guard, temporal/modal or data-dependent effect | RelAlg pure query cannot execute it; owner-only method OFFER or REFUSE | + +## X. Execution/admission/memo economics (X1–X8) + +| ID | Nonvacuous fixture and oracle | Falsifier / expected refusal | +| --- | --- | --- | +| X1 | Valid guard-free 2-valued method approved for lab and source scope | Native R2IL exact parity to scalar baseline with source-owned borrow | +| X2 | Guarded WHEN method and explicit Kleene K3 method | `GuardNotExecutable` / `ThreeValuedNotExecutable`; no alternate interpreter | +| X3 | Two methods with different MethodRef/SigRev but same method name | Prepared cache cannot reuse one for the other | +| X4 | Scope/permission revoked while source identity unchanged | Cache hit cannot bypass permission refusal | +| X5 | Source layout/value changed with unchanged class bit basis | Value-dependent derived mask invalidation independent from class basis | +| X6 | Q=1/2/8/16/64/256 cold + warm for memoized recipe | Whole-query cost measured; invalidate and cache-miss controls; no false speedup inference | +| X7 | Valid native Boolean vs Abakus k-of-n words on same workload | Compare exact results and cost; no forced carry default | +| X8 | Unadmitted M:N action or speculative candidate requests WORLD write | No owner permit or Rubicon → refusal, no write effect | + +**Total:** 8 F + 10 D + 10 V + 10 R + 8 X = **46 proposed independent gates**, none executed in CGMM-0. + +## Evaluation record for every future PR + +~~~text +CASE: D-CGMM/ +SOURCES: pinned source producer + artifact/revision + code/line +IDENTITIES: source node key, Codegraph Pos (local), ClassView basis, MethodRef if admitted +DEMAND: root/bound-free columns, semantics/grade/permissions, recursion policy +EXPECTED: independent source fact / result mask / explicit refusal +ANTI: deliberate mutation; observed test failure, no vacuous zero +COPIES: owned canonical value bytes, scratch, result and source ingest separated +COST: intake, derive, query, prepare/run, Q, bytes and p50/p95; eager vs lazy +STATUS: PASS | FAIL | PARTIAL | NOT_RUN | NOT_COMPARABLE | BLOCKED_OWNER +ADMISSION: source/ontology/DT64/execution owners, exact EXO links +~~~ + +**Strict refusal:** No external Datalog optimizer, MORK pathmap, ANN/NNUE candidate, MeTTa rule, class label or Kythe reference **by itself** grants DT64 execution permission or source data ownership. diff --git a/.claude/plans/d-cgmm-implementation-blueprint-v1.md b/.claude/plans/d-cgmm-implementation-blueprint-v1.md new file mode 100644 index 000000000..68c37f2cf --- /dev/null +++ b/.claude/plans/d-cgmm-implementation-blueprint-v1.md @@ -0,0 +1,181 @@ +# D-CGMM-IMPLEMENTATION-0: executable research blueprint, from source evidence to lawful native method + +> **2026-11-04 · DETAILED PLAN · DOCS ONLY · NOT AN IMPLEMENTATION.** This is an expansion of [D-CGMM-0](d-cgmm-codegraph-metta-method-builder-v1.md) in draft [PR #1958](https://github.com/AdaWorldAPI/lance-graph/pull/1958), not a new source graph or general-purpose MeTTa runtime. Tracking: [EXO-155](https://linear.app/exo-red/issue/EXO-155); [EXO-50](https://linear.app/exo-red/issue/EXO-50) is an append-only communications bus, not permission. Owner decisions and their exact exit artifacts are in the [EXO gate ledger](d-cgmm-exo-gates-v1.md); the [fixture matrix](d-cgmm-executable-fixtures-v1.md) defines independent tests. The existing [staged charter](d-cgmm-staged-pr-charter-v1.md) remains canonical for CGMM-0…8. +> +> **Operator objective:** make MeTTa-shaped logical schema constraints a **cold class-method builder** attached to witnessed ClassView/FieldViewDef roles. Separate **durable source facts** (Ruff/Charon/Kythe) from **cold Datalog demand analysis** (Crepe/Soufflé/XLOG compile artifacts) and **lazy Codegraph query masks**. Convert only semantically and permission-eligible, owner-approved declarations to immutable OGIT/OGAR/DT64 contracts, and reuse existing RelAlg/R2IL/mask-risc/ndarray execution. Never reconstruct canonical business objects or introduce a second execution engine. + +## 0. Status and unnegotiable source facts + +| Surface | What current source establishes | What this plan MUST NOT call shipped | +| --- | --- | --- | +| `crates/lance-graph-codegraph/src/artifact_bin.rs` | A `CrateFile` owns one validated `Vec`, accessors borrow decoded lanes/labels, FUNDECL/PROV section tags exist; writer is `ruff_charon::store_bin` | All read/write effects, receiver typing, Rails/SAP AST, or zero-copy across every producer | +| `crates/lance-graph-codegraph/src/graph.rs` | `Codegraph` owns lanes, name interner, local `Pos` table, `MapTable`, `Cube` of six relations, masks; staged refresh; `rconf`/`rposs` `OnceCell` | Stable Kythe source identity equaling `Pos`, a completely demand-lazy store, or append-only historical truth for arbitrary source updates | +| `crates/lance-graph-codegraph/src/query.rs` | `Graded { confirmed, possible, unknown }` masks, bound `partners` joins, `or_partners` loops over positions | Closed-world negative evidence from `MAY_DISPATCH`, implicit joins on foreign identity, zero-cost repeated binding | +| `crates/lance-graph-contract/src/class_view.rs` | `ClassView` above SoA, field-position basis, `FieldMask` pure presence, `WideFieldMask` and `lens_fields(full_classid)` | A mutable MeTTa scratch field inside the trait or a new ClassId issuer | +| `crates/lance-graph-ontology/src/proposal.rs` | `MappingProposalKind::Edge { LinkSpec }` and explicit ontology source context | A true tenant permission, an actual edge instance, or M:N multiplicity | +| `crates/lance-graph-contract/src/property.rs` | `LinkSpec`, `Cardinality::ManyToMany`, `ActionSpec` | OGAR `ActionDef` equivalence, a join engine, write permission | +| `crates/lance-graph-relalg/src/ir.rs` | Single-population `Rel`/`Query` mask/atom/selection/reduction vocabulary | General M:N join, recursive fixpoint, arbitrary method effects | +| `crates/lance-graph-relalg/src/join_body.rs` | GYO α-acyclicity analysis over variable sets | Executed Yannakakis/WCOJ plan or a proof of foreign-hop admission | +| `crates/lance-graph-r2il/src/binding.rs` and `binding_native.rs` | Validates method/binding, guard/truth restrictions; can prepare/run a subset of native mask-risc folds using borrowed planes | General executable WHEN, K3, fold-fed-fold, or production `ExecPermit` | +| [Crepe/Soufflé/XLOG source archaeology](../board/entries/2026-11-02-coresearch-xlog-souffle-crepe-dr3-archaeology.md) | Crepe bound/free/index information, Soufflé-like demand, XLOG compiler metadata and losses, discarded polarity/magic report, `crepe_plan!` D1/D2 drift | A proven compatible compiler plan or license to import external runtime | +| [#1944](https://github.com/AdaWorldAPI/lance-graph/pull/1944) | Bounded, ground relational common subset from separate frontends | General MeTTa unification/rewrite or a new Datalog engine | +| [#1955](https://github.com/AdaWorldAPI/lance-graph/pull/1955), [#1956](https://github.com/AdaWorldAPI/lance-graph/pull/1956) | Local borrowed register proof; correct but slower-than-native carry ALU in measured cases | Remote/storage source admission or universally superior Abakus | + +**Design target, not code:** `SourceFactLease → DemandEvidence → LazyCodeView → FieldViewDefOffer → ConstraintScratch → {OntologyOffer, ActionOffer, MethodOffer} → owner admission → PreparedNativePlan`. All names here are *descriptive*, not public Rust API. Preserve existing functions/types and request ownership before any API change. + +## 1. Source intake: fact identity must survive a lazy query + +A source harvester should produce *atomic, provenance-grounded facts*, rather than a monolithic serialized AST object. The minimal **conceptual** source receipt is: + +~~~text +SourceFactReceipt { + producer: (language, tool, version, artifact schema version), + source: (repository identity, commit/revision, file identity, span), + symbol: stable producer-scoped node key + source owner, + predicate: kind (defines / calls / may-dispatch / reads / writes / ...), + object: stable producer-scoped key OR typed scalar/role, + confidence: compiler-confirmed | possible | unresolved | missing-producer, + extraction: COMPLETE | PARTIAL + explicit residual reasons +} +~~~ + +**No new durable ID authority:** Link source-producer identities to Codegraph's local interner/`Pos` only under a per-artifact validated table. Never equate Kythe VName or Charon `FunDecl` with global DR identity, MethodId, ClassId or local ordinal. + +**Binary-lane policy:** Existing `.rows.bin` is versioned/validated by an independently implemented reader. To add effect/receiver/tenant facts, first prove that the producer actually emits them and identify how its writer/reader will evolve without breaking older must-understand sections. An optional sidecar in a lab may serve as a negative-control input; no production sidecar format is silently admitted. + +**Atomic refresh:** On parser refusal or broken provenance, `Codegraph::prepare_refresh`/commit semantics must leave the previously committed graph and source witness unchanged. Newly observed absence is not `FALSE` under a PARTIAL producer. + +**First falsifier:** two same-named methods in different source versions, one resolved and one dynamic call, one documented write whose producer drops it. An incorrect label-only interner or loss of the write fact must prevent a method-equivalence OFFER. + +## 2. Demand compiler: use Crepe/Soufflé insight but not their executor + +Demand planning is a *pure*, cold analysis of exactly which relations/columns need querying for a method-building question. Conceptual contract: + +~~~text +CodeDemandV0 { + roots: validated source-node IDs or admitted ClassView role refs, + relation_atoms: typed (predicate, bound/free coordinates, direction), + grade_floor: confirmed | include_possible | include_unknown, + output: graded_mask | source_span_mask | field_role_offer | method_offer, + semiring: B | N, unknown_mode: OpenWorld | CompleteWitnessed, + source_artifact_generation, producer_schema, ontology_class_revision, + scope/tenant/principal, recursion_policy, max_hops/max_atoms/max_bytes +} +CodeDemandPlanV0 { + demanded fact lanes, bound columns, legal ordering, proof obligations, + required coverage/strata/polarity witnesses, identity mapping, budgets +} +~~~ + +**Do not turn this into a new runtime planner**. Crepe bound/free metadata and Soufflé magic demand are *candidate compile records* only. XLOG's magic-set/polarity/stratum records must be source-confirmed and tested for loss before using them as admission facts. `crepe_plan!` vs `crepe!` D1/D2 drift explicitly prevents trusting a plan without a comparator. + +**Current baseline:** Codegraph already executes `Cube::scan→bind→join→support` lazily for a bound symbol but eagerly builds its six relations at intake and performs per-set-bit repeated `partners` inside `or_partners`. First benchmark this exact baseline. A demand-scoped version must not be credited for a lower cold cost that it merely moved into the caller. + +**Conservative first implementation:** start with a small *excluded lab or test fixture* that constructs a demand descriptor, resolves one bound symbol using the existing Codegraph query API, and measures source facts touched. If the data structure lacks selective construction, do not pretend fewer fact lanes were built. Measure masks and binding counts; introduce demand pruning only after its independent semantic oracle passes. + +**Logical errors are mandatory refusals:** partial source negation, non-stratified negation, unsupported recursive SCC, unsupported indexed foreign gather, stale source revision, mismatch of Bag vs Boolean Set or wrong binding direction. A demand compiler cannot turn UNKNOWN into FALSE to simplify a plan. + +## 3. FieldViewDef: exact semantic boundary + +**Proposed immutable offer, never a mutable ClassView trait extension:** + +~~~text +FieldViewDefV0 { + owner_class: ClassId + ClassView schema/behavior-lens revision, + role: ScalarField | EdgeSubject | EdgeObject | ActionOperand, + referent: FieldRef | OGIT LinkSpec role | OGAR ActionDef operand, + type: field dtype, signedness, null/absence, presence basis, + relation: optional (predicate, orientation, cardinality, + Set|Bag, edge-identity, source/tenant scopes), + admissible_reads: pure method candidates with source evidence, + evidence: source facts + artifact generation + confirmed/unknown masks +} +FieldConstraintScratchV0 { + declarations: bounded, borrowed FieldViewDef references, + candidate_masks: logical MATCH/CONTRADICTION/UNKNOWN over a distinct basis, + relational_constraints: typed Codegraph/Quack relation references, + rule_version, source_version, class_version, permission scope, + result: OFFER | REFUSE(reason) | ABSTAIN(missing_evidence) +} +~~~ + +**Strict basis rule:** `FieldMask` and `WideFieldMask` continue to encode instance presence ONLY. Candidate logical masks must use a distinct, typed logical basis, not borrow bit 0 and reinterpret it as "matching". For behavior lenses, require `lens_fields(full_classid)`; do not silently fall back to the u16 concept half. `FieldViewDef` is **not** OGAR's rendered `FieldView`. + +**Logical rule universe:** finite, typed, bounded MeTTa-shaped facts and equality constraints over source roles; no general MeTTa interpreter, MORK trie, executable rewrite, automatic transitive closure, AST copy or source-driven script execution. Scratch can be memoized with full source/ontology/class/rule/scope witness but remains disposable, *never an admitted MethodId*. + +**Two-way workflow:** Codegraph may provide source evidence supporting or contradicting an existing DT64 method; any mismatch produces a **diagnostic candidate** for review, not direct mutation of DT64 or recompiled production methods. + +## 4. Entity, edge, action: three distinct offers + +The same temporary constraint workspace supports three **typed destinations**: + +| Offer | Existing canonical owner | Minimum witness | Common refusal | +| --- | --- | --- | --- | +| Entity/field mapping | OGIT ontology `OntologyRegistry`, `ClassView` owner | URI/context + source type + source/source-revision + field identity | Field label match with type/layout conflict | +| Relationship edge | `MappingProposalKind::Edge { LinkSpec }` / RelationshipMap owner | subject/object ClassIds, predicate URI, orientation, cardinality, edge instance/multiplicity when needed, tenant authorization | M:N mistaken for Bag; cross-tenant link without permit | +| Action/method | OGAR `ActionDef` and DT64 `MethodBinding` owners, separately | signature/class/ON, ordered WHEN/DO/effect/transaction witnesses and scope | Same-name method with different ordered effects or missing source writes | + +A single ontology URI is **not** sufficient tenant authorization. An M:N declared association is **not** an observed edge population. Bag multiplicity depends on actual derivations, not `Cardinality::ManyToMany`. A reversed edge isn't automatically an inverse or symmetric relation. An event can have executable effects even if its source syntax resembles a read-only query. + +For Rails/Odoo/SAP examples, use synthetic labelled profiles unless independently harvested source and transaction traces exist. Specifically, an Odoo ORM callback and SAP approval transaction are not interchangeable because their user-facing names match. + +## 5. Native method lowering and refusal map + +After an owner separately approves a **pure, finite, typed** offer, a cold lowerer may reuse existing RelAlg IR for *one witnessed population*: + +| Schema/read shape | Candidate mapping to existing algebra | Admission outcome | +| --- | --- | --- | +| `FieldRef` U32 == known constant | Typed `Rel::Atom` U32-equality over an existing lane | OFFER after field, source, class and null witnesses | +| I32 amount ≥ bound | Existing I32 compare + `Rel::And` with allowed population | OFFER if exact signedness and absence handling | +| Keep support or count | Existing `Query::Measure` / native terminal | OFFER under one-population, Set/Bag interpretation | +| Binary same-source mask combination | `Rel::{And,Or,Not}` where supported | OFFER only for same coordinate universe and admitted truth law | +| OGIT M:N role traversal | Quack/XLOG relation planning outside single-row RelAlg | **REFUSE this lowerer;** owner-specific relation OFFER only | +| Cross-source foreign join | Admission-B/EXO-46/140 | **BLOCKED** until owned lookup/index and scope established | +| Conditional WHEN / K3 truth | DT64 can store, current R2IL prepare explicitly refuses | **GuardNotExecutable / ThreeValuedNotExecutable** | +| OGAR action with effects, writes or rollback | OGAR behavior owner and transaction owner | Declaration OFFER, **not executable read** | +| General MeTTa rewrite / recursion / eglog congruence | No admitted runtime analogue | **REFUSE**; external oracle only | + +**No hidden fallback** to a tuple interpreter, DataFusion, Crepe/Soufflé/datafrog evaluator or implicit value gather. `RelAlg::join_body::is_alpha_acyclic` is a structural classifier, not a many-to-many executor. `ExecPermit` is lab permission only. + +## 6. Three distinct memoization levels, with invalidation law + +1. **Compile/demand-plan memo:** recipe keyed by source artifact generation, producer version, typed query roots, bound/free pattern, stratification/polarity, semiring, compiler-rule version and permission scope. It may retain plan *metadata*, not copied source objects. +2. **Semantic constraint scratch memo:** optional cache of offers/rejections keyed by source fact digest, ClassView/ontology/OGAR revisions, tenant/permission/scope, rule basis and relevant evidence completeness. Its eviction/recompute changes no previously admitted truth. +3. **Admitted native method binding:** only after real ownership approval, reuse `MethodRef`/`SigRev`/`ClassView`/OGAR body revision/IR/semiring/source-frame-layout/scope witness, checking any value-derived registers against value revision. Hot execution borrows canonical SoA rather than re-parsing AST or constructing `PSCustomObject` rows. + +**No cache-by-name** and **no cache-by-FNV-identity alone**. Any source revision, codebook shift, method change, semantic law change, permission revocation or lease expiry must trigger revalidation or refusal. + +**Abakus is *not* the schema solver.** Its existing half/full-adder words may execute numeric/threshold operations under ndarray, but #1956's measured carry implementation lost to native words in eight tested shapes. Compare only where arithmetic semantics are required. No public CPU-layout emulator or learned-codebooks as automatic plan substitutes. + +## 7. Proof sequence, data collection and STOP rules + +**CGMM-1 (fact intake)** first checks the writer→reader edge and source identity; no producer-generated effects means **UNKNOWN**, not fabricated writes. Independent oracle is the original source fixture. Deliberately drop a WRITE fact and verify the converter refuses calling that method pure. + +**CGMM-2 (demand/lazy)** compares baseline Codegraph eager six relations and current `partners/or_partners` against one bounded demand query. Measure ingestion, cold plan, query Q=1/2/8/16/64/256, retained bytes, masks, graph refresh, physical CPU time, query-site count and traversal. Kill a `MAY_DISPATCH` truth upgrade and stale cache. Report NO BENEFIT if demand loses. + +**CGMM-3 (FieldViewDef)** uses source-backed ClassView positions and one 109-field wide class; prove different schemas with identical labels don't alias and behavior-lens full class ID required. Candidate masks differ from instance-presence masks, and scratch eviction changes no admitted schema. Kill a class schema basis mismatch. + +**CGMM-4 (OGIT/OGAR)** uses M:N tenant fixture plus duplicate edge and two same-name actions with different effect order. Independent oracle enumerates global IDs by tenant and ordered effects. Kill tenant filter, Bag support collapse, inverse assumption, an action effect; all must be detected. + +**CGMM-5 (RelAlg polyfill)** runs an actual same-universe scalar/measure query on borrowed source, with independent scalar reference. It must reject an M:N join, K3, effectful or guarded method. Compiler-produced scalar/mask values are not identical to ClassView metadata. + +**CGMM-6 (DT64 admission)** only when owners authorize: create immutable MethodBinding with correct method and ClassView basis; prepare through current R2IL native path. Compare exact result and refuse stale schema/source/scope and guarded WHEN. No new R2IL interpreter. + +**CGMM-7 (memo)** reports Q and cold vs hot economics including invalidation/eviction; compare native Boolean/byte and optional Abakus only where their semantics match. Never claim speedup from one benchmark without whole-query denominator. + +**CGMM-8 (audit)** uses independent oracle and Codegraph reachability to identify access-capability bypasses in chosen migrated roots, preserving dynamic unknown paths and scope. EXO-104/105/109 prohibit tuple-shaped oracle leakage into production. NNUE only proposes candidate order. + +**Common STOP:** Source facts not independently verifiable; missing source-effect harvest; proven incompatible class or tenant scope; emitted Codegraph local Pos treated as durable class identity; implicit primary-field or row reconstruction; unowned lookup/foreign gather; unsupported general rewriting or recursive evaluator; method admitted from name/NNUE score; CI falsely asserting global source completeness. + +## 8. Publication packet per future staged PR + +Each stage must include: +- **START:** base main SHA, relevant upstream tool/artifact revs, exact EXO issue and owner. +- **CONTRACT:** precise input/outputs/witnesses, obligations, refusal and fallback policy. +- **FIXTURE:** independent fact and value oracle, one positive/one adversarial instance; seeded reproducibility. +- **DISABLE:** deliberate mutation which turns at least one specific nonvacuous test red. +- **NUMBERS:** ingestion cost, cold bind, warm query, Q, bytes retained/copied, source/route accesses, p50/p95/p99 and variance; do not compare incomparable hot/cold arms. +- **VERDICT:** PASS | FAIL | NOT_COMPARABLE | NOT_RUN | BLOCKED_OWNER with source links and bounded claim. +- **HANDOFF:** PR + commit hash, board cross-links, append-only EXO-50 message; separate operator merge/admission decision. + +**First recommended executable PR:** CGMM-1 source fact loss and identity census in an excluded test workspace, *followed* by CGMM-2 demand-projection economics. The larger FieldViewDef DSL and method-binding program must not race ahead of demonstrably reliable compiler source facts. diff --git a/.claude/plans/d-cgmm-lazy-codegraph-fact-demand-v1.md b/.claude/plans/d-cgmm-lazy-codegraph-fact-demand-v1.md new file mode 100644 index 000000000..b26481cc9 --- /dev/null +++ b/.claude/plans/d-cgmm-lazy-codegraph-fact-demand-v1.md @@ -0,0 +1,175 @@ +# D-CGMM-LAZY-0: Kythe facts × Datalog demand × lazy Codegraph × FieldViewDef × DT64 + +> **2026-11-04 · ARCHITECTURE ADDENDUM · PROPOSED / NOT IMPLEMENTED.** The central task is to **separate durable source identity and facts from on-demand relational evaluation and the class/method semantic lens**. This does not adopt Crepe, Soufflé or MORK as a production executor; it does not introduce a second graph store, general-purpose rewrite engine, new F###, promoted WORLD facts, output row cache or remotely dereferenceable register. [D-CGMM master](d-cgmm-codegraph-metta-method-builder-v1.md) · [Action/Edge schema](d-cgmm-action-relation-schema-v1.md) · [staged PR charter](d-cgmm-staged-pr-charter-v1.md) · [falsifiers](d-cgmm-falsifier-matrix-v1.md) · [EXO-155](https://linear.app/exo-red/issue/EXO-155). + +## 0. The architectural inversion + +**Kythe-shaped facts are evidence, not a query engine. Datalog-shaped analysis is a compiler/demand vocabulary, not a second evaluator. Codegraph is a lazily resolved *view* of the requested evidence, not a duplicate AST object graph. FieldViewDef is a typed, ClassView-based semantic offer, not a new ontology. DT64 admits immutable method contracts, and only current native algebra executes them.** + +~~~text +Ruff / Charon / other admitted source harvesters + | + | source-specific nodes and edges + stable producer identity + v + Kythe-shaped compiler evidence and existing .rows.bin/FUNDECL/PROV lanes + | source identity, revision, provenance, coverage grades + | no guessed missing facts, no revived source AST objects + v + COLD DATALOG COMPILE / DEMAND ANALYSIS + Crepe-style bound/free index requirements, Soufflé-style magic demand + stratification/negation rules and XLOG compile metadata, *as DATA* + | required relations, bindings, polarity, semiring, termination + | explicit unsupported/refused shapes, no imported runtime + v + LAZY CODEGRAPH SELECTOR + bound relation/symbol queries -> typed support and evidence masks + incremental/epoch-scope result cache only if validated; otherwise recompute + | confirmed / possible / unknown / residual preserved + v + ClassView + FieldViewDef OFFER / disposable MeTTa-shaped scratch + finite source-to-concept mapping, roles, action/effect/tenant constraints + | | + | +---- OGIT LinkSpec M:N + OGAR ActionDef owner offers + v + DT64 MethodBinding: immutable ON × WHEN × ordered DO + | owner admit, source/class/method/scope/exec witness + v + Existing RelAlg / Quack / R2IL -> mask-risc -> ndarray + | source-owned borrowed SoA, no mandatory canonical value gather + v + scalar / relation mask / typed refusal, no automatic WORLD write +~~~ + +**What is actually lazy today:** `lance-graph-codegraph/src/graph.rs` already interns symbols into local `Pos`, has DR `MapTable` and XLOG `Cube` relations plus `rconf` and `rposs` `OnceCell` caches. `query.rs` performs bound joins on demand, returning graded masks rather than tuple objects. Yet intake eagerly constructs six core relations and row/vector masks; `or_partners` iterates input positions and invokes repeated bound joins. Claim *partial laziness*, not an already-completed lazy Codegraph engine. The first optimization may only need **demand metadata and fewer repeated bindings**, not a rewrite of the graph. + +## 1. Three stores must NOT be confused + +| Layer | Existing source/owner | Durable content | What is only a view | +| --- | --- | --- | --- | +| Source artifacts / Kythe-like evidence | Ruff, Charon, source producer, `.rows.bin` and provenance | Symbol identity, AST span, resolved/may/dynamic calls, typed effects where actually harvested, producer revision | Codegraph interned `Pos` and masks | +| Canonical relational register | DR `MapTable` / `Frame` / source-owning SoA and XLOG/Quack execution | Source/coordinate identities, admitted relationship planes and source values | A demand-scoped relation projection or semantic field lens | +| Semantic declarations | OGIT ontology/`LinkSpec`, OGAR class/action codebook, DT64 `BindStore` | Admitted ClassId, MethodId, schema and method revisions, `ActionDef` | `FieldViewDef` proposals, MeTTa constraint scratch, compiled temporary query plans | + +**Kythe VNames or equivalent stable symbol identity need their own namespace/producer/source revision.** Do not treat local Codegraph `Pos`, Charon function ID, human symbol name, DR local ordinal or DT64 ClassId as identical IDs. If a source producer does not emit stable identity or needed effect evidence, record `UNKNOWN_SOURCE_ID` / `UNKNOWN_EFFECT` rather than inventing a link. + +## 2. Take the *compile artifacts* from Crepe/Soufflé/XLOG, not their runtime + +[2026-11-02 source archaeology](../board/entries/2026-11-02-coresearch-xlog-souffle-crepe-dr3-archaeology.md) documents the traps: + +- **Crepe**: binding analysis (which relation arguments are bound or free) chooses index needs; `crepe_plan!` emits planning data but its recorded D1/D2 drifts from `crepe!` need a falsifier before trusting it. +- **Soufflé**: demand/magic rewriting constrains which facts/rules matter to a query; stratification, negation and recursion are substantial semantic constraints. Do not import its runtime, indexes or automatic recursion by analogy. +- **XLOG compiler**: source archaeology identifies already-derived SCC/stratum, magic-set reports, negation polarity, join ordering and selectivity metadata; some are dropped at compile boundaries. *Recover as named admission/planning evidence only if an independent probe demonstrates loss matters.* XLOG execution is not admitted by an AST import. +- **Kythe**: node and edge facts provide source-grounded anchors and cross-references. Facts that are only `MAY_DISPATCH` or unresolved are not negative knowledge. +- **AST**: syntax structure belongs to a producer; Codegraph consumes typed facts, and only fetches a bounded source AST fragment when a declared evidence gap requires it. Never rehydrate full AST trees as a standard query operation. +- **datafrog**: an independent finite Datalog/fixpoint oracle for a bounded case; EXO-73 owns recursion execution policy. Crepe, Soufflé and datafrog are *oracles/comparators*, not runtime owners. +- **MORK**: a set-oriented external structural oracle; differences from MeTTa multiset binding and Datalog Bag semantics must remain typed and measured. EXO-79 license gate remains. +- **NNUE**: optional offline ranking of candidate patterns; does not issue evidence, ClassIds or execution permits. + +**No universal closed-world negation:** absence of a `CALLS` fact under partial coverage cannot prove a method has no writes or caller. If a program needs stratified negation on complete facts, its required completeness witness must be explicit; otherwise the compiler returns `ABSTAIN(UNKNOWN_COVERAGE)`. + +## 3. Demand descriptor (conceptual, NOT Rust API) + +~~~text +CodeDemand { + roots: source symbol ids / owner-validated ClassView role ids + required_edges: [CALLS, MAY, IMPLEMENTS, READS, WRITES, HAS_FIELD, + EXTENDS, DECLARES_ACTION, LINK_ROLE, ...] + bind_pattern: per relation argument bound/free, direction and depth + evidence_floor: confirmed | allow_possible | include_unknown + semiring: BooleanSet | CountingBag (with overflow law) + logic: two-valued | Kleene (with complete-world declaration) + source: repository/language/artifact digest/generation/producer schema + ontology: OGIT/OGAR class and method revision (if relevant) + scope: owner/tenant/permission and cross-source admission + budget: max relations, symbols, hops, rounds, scratch bytes + terminal: support mask | graded mask | relation offer | method offer +} +DemandPlan { + required fact lanes, proven safe binding/index/order metadata + snapshot and identity witnesses, compilation evidence and refusals + no arbitrary executable native code and no new persisted index +} +~~~ + +**Lazy materialization policy:** a demand may prepare an *ephemeral, bounded support mask* or a few existing XLOG/DR relation views, but not source AST objects, newly owned source-value arrays, a separate always-on call graph or all tenant edges. The same semantic plan may be reused if every source, method, scope and compiler-version witness matches. Different roots can share an immutable admitted plan, while bound source positions and resulting masks remain **request scoped**. + +Performance must measure **cold demanded-fact binding + eager baseline + query cost**, not only a prepared fold that hides initialization. When query cost is dominated by index lookup, proposal is a *demand memo* on existing Plan/DR support, not automatic global hash-table construction. + +## 4. ClassView plus FieldViewDef: the narrow schema glue + +`lance-graph-contract/src/class_view.rs` explicitly defines the class as a meta lookup flying above SoA; `fields(class_id)` resolves a stable ordered field basis, while `lens_fields(full_classid)` resolves a behavior-lens basis. `FieldMask`/`WideFieldMask` mean **instance presence**, not semantic candidate truth, permissions or logical negation. + +**FieldViewDef** is a proposed *cold, immutable, typed offer referring to those existing fields*, not the existing `ogar-render-askama::field_view::FieldView`, which is a render-only addressed UI carrier. The MeTTa-shaped **FieldViewConstraintScratch** is a short-lived matcher of supported/contradicted/unresolved class, field, edge and ActionDef roles; it does not modify ClassView or create a new method codebook. + +### 4.1 One declaration, several owner-specific views + +~~~text +FieldViewDef (proposal only): + class: existing ClassId / full behavior lens + schema basis + role: scalar | relation_subject | relation_object | action_operand + reference: FieldRef or LinkSpec role or ActionDef operand (source-witnessed) + types/truth: dtype, absence, cardinality, semiring, tenant scope + provenance: compiler/ontology/source receipts and unresolved evidence + read_intent: optional pure selection/fold, no action executed + +MeTTa-shaped constraints (illustrative, not parser syntax): + HasField(Worker, project_id) + HasLink(Worker, assigned_to, Project, M:N, tenant=A) + HasAction(ApproveEntry, ordered_effects=[...], when=guard) + Writes(method, field) [SOURCE_CONFIRMED | UNKNOWN] + EquivalentUnder(methodA, methodB, typed-source-witness) [CANDIDATE] + +Scratch: OFFER | REFUSE(reason) | ABSTAIN(missing evidence) +~~~ + +A spaCy-like `Doc/Token/Span/Matcher` pipeline is a **useful cold annotation interface analogy**, not a second crate or text-sequence representation. A Codegraph edge or M:N link is not a contiguous `Span`. A source span can ground the edge's declaration, never stand in for its relationship population. + +### 4.2 Native polyfill boundary + +If a `FieldViewDef`'s read is confined to **one witnessed row universe** and existing numeric lanes, it may lower to existing `RelAlg::Rel::{All,Empty,Plane,Atom,And,Or,Not}` and `Query::Measure::{Count,Sum,Min,Max,Keep}`, then `RelAlg::run` → R2IL/mask-risc. RelAlg's IR has no strings, owns no source rows, and does not contain a general join or recursive execution program. + +**Examples:** +- `status == 2` as verified U32 lane -> `Rel::eq_u32(lane,2)`, with absence/truth checked. +- `amount >= 100` on a verified I32 lane -> `Rel::i32(lane,Cmp::Ge,100)`. +- typed Boolean intersections -> `Rel::And` / `Or` / `Not` on the *same* frame and truth law. +- M:N ontology link -> **relation OFFER only** to Quack/XLOG owners; `RelAlg::join_body::is_alpha_acyclic` is a structural classifier, not a join executor. +- OGAR effect/ActionDef or DT64 guarded WHEN -> declaration OFFER / execution REFUSE pending owner and vocabulary; `binding_native` does not generally execute these. +- source AST rewrite, arbitrary MeTTa unification, recursion, tenant-forbidden foreign hop -> typed REFUSE/ABSTAIN, never an alternative code path. + +## 5. Demand and memoization laws + +- **L1 Identity**: source Kythe/node identity ≠ Codegraph interned `Pos` ≠ ClassId/MethodId ≠ DR local ordinal. +- **L2 Evidence**: confirmed, possible and unknown stay distinct under projection, negation and derivation; partial extraction never proves absence. +- **L3 Lazy**: only requested facts/roles are projected, with bounded work; do not manufacture complete AST objects or an independent always-on graph. +- **L4 Semantics**: Set/Bag support, M:N cardinality, SQL Kleene and effects never collapse by name or mask width. +- **L5 Ownership**: Source owns AST/facts; Codegraph owns graded query view; ClassView owns basis; OGIT/OGAR/DT64 admit meaning; existing algebra executes. +- **L6 Cache**: snapshot/producer/ontology/class/method/semiring/scope versions form a complete invalidation witness. Scratch eviction cannot change truth. +- **L7 Economics**: track cold intake, eager-vs-demand preparation, allocations, retained memory, query latency, partial coverage and cache hit/miss separately. +- **L8 Failure**: no silently empty results on unsupported recursive, foreign, missing source, unknown effects or unauthorized tenant tasks. + +## 6. Smallest runnable falsifier + +**Fixture:** two crates with 20 source symbols. Include five confirmed `CALLS` edges, two `MAY_DISPATCH` edges, one unresolved dynamic edge, source spans, one field read and one write, one OGIT `LinkSpec(Worker,assigned_to,Project,ManyToMany)` declaration, two application-local method candidates with same name but different effects. Use independent global-ID source fixture and a separately specified scalar/Datalog oracle. + +**Demand D1:** From one known method, ask only confirmed callees and fields read. The candidate must **not** build unrelated M:N/ActionDef views, and return the exact confirmed mask and source spans. + +**Demand D2:** Request tenant-scoped Worker→Project M:N candidates and effect-sensitive method equivalence. The candidate must preserve duplicate-edge Bag vs Boolean support and **ABSTAIN** when required Rails/SAP source effects or tenant authorization is absent. + +**Demand D3:** Change only the source's write-effect fact or a dynamic target, without changing ClassView field positions. Derived constraint cache invalidates; admitted ClassView basis stays valid; previously optimistically matched methods must now refuse/abstain. + +**Independent oracles:** (a) simple Rust scalar reference enumerating immutable source facts by stable source ID; (b) optional Crepe/datafrog/Soufflé bounded query comparator with pinned version, license/semantic gates, and independent semantics; (c) Codegraph eager baseline over same fact corpus. A comparator using candidate's own `Cube::partners` is **not independent**. + +**Deliberate disables:** force eager full-graph materialization, silently mark `MAY` confirmed, equate two local symbol positions across source revisions, remove tenant guard, drop one ordered effect, and let a stale cache survive a source mutation. Each must fail a result, provenance or representation assertion. Measure all relevant costs and explicitly report when lazy demand **loses**. + +**Acceptance:** clear separation between durable facts, demand plan and temporary Codegraph view, exact graded semantics, at least one legitimate RelAlg-native read and one correctly refused effectful/M:N request. **Do not promote source or method authority or promise production latency from a small fixture.** + +## 7. Why this beats an eager universal schema DTO + +Codegraph never has to *become* Rails, Odoo, SAP, MeTTa, MORK, Soufflé or spaCy. It asks the smallest witnessed question about each source; ClassView/FieldViewDef turns evidence into candidate meaning; native relational algebra executes admitted computation. This is **a reusable demand-driven semantic boundary** instead of a second object world. + +## Sources and owners + +- [Codegraph graph source](../../crates/lance-graph-codegraph/src/graph.rs), [graded query source](../../crates/lance-graph-codegraph/src/query.rs), [binary lanes](../../crates/lance-graph-codegraph/src/artifact_bin.rs). +- [Crepe/Soufflé/XLOG archaeology](../board/entries/2026-11-02-coresearch-xlog-souffle-crepe-dr3-archaeology.md), [RelAlg IR](../../crates/lance-graph-relalg/src/ir.rs), [RelAlg compiler/executor](../../crates/lance-graph-relalg/src/lib.rs). +- [ClassView](../../crates/lance-graph-contract/src/class_view.rs), [OGIT LinkSpec](../../crates/lance-graph-contract/src/property.rs), [OGAR ActionDef](https://github.com/AdaWorldAPI/OGAR/blob/main/docs/OGAR-AST-CONTRACT.md), [DT64](../../crates/lance-graph-dt64/src/lib.rs), [R2IL native binding](../../crates/lance-graph-r2il/src/binding_native.rs). +- Existing decisions: [EXO-84](https://linear.app/exo-red/issue/EXO-84), [EXO-85](https://linear.app/exo-red/issue/EXO-85), [EXO-79](https://linear.app/exo-red/issue/EXO-79), [EXO-143](https://linear.app/exo-red/issue/EXO-143), [EXO-73](https://linear.app/exo-red/issue/EXO-73), [EXO-108](https://linear.app/exo-red/issue/EXO-108), [EXO-121](https://linear.app/exo-red/issue/EXO-121), [EXO-152](https://linear.app/exo-red/issue/EXO-152), [EXO-153](https://linear.app/exo-red/issue/EXO-153), [EXO-154](https://linear.app/exo-red/issue/EXO-154), [EXO-155](https://linear.app/exo-red/issue/EXO-155). diff --git a/.claude/plans/d-cgmm-staged-pr-charter-v1.md b/.claude/plans/d-cgmm-staged-pr-charter-v1.md new file mode 100644 index 000000000..a8e5162cb --- /dev/null +++ b/.claude/plans/d-cgmm-staged-pr-charter-v1.md @@ -0,0 +1,97 @@ +# D-CGMM staged PR charter: compiler facts → lazy Codegraph → FieldViewDef → DT64 native binding + +> **2026-11-04 · DOCS-ONLY PRE-REGISTRATION.** Only CGMM-0 is opened now. CGMM-1 through CGMM-8 are proposed *separately reviewed* PRs. No production code, new class codebook, general MeTTa runtime, Soufflé/Crepe/MORK executor, new F### word or unapproved source traversal is authorized. [Master](d-cgmm-codegraph-metta-method-builder-v1.md) · [Lazy Codegraph](d-cgmm-lazy-codegraph-fact-demand-v1.md) · [OGIT/OGAR edge/action](d-cgmm-action-relation-schema-v1.md) · [Falsifiers](d-cgmm-falsifier-matrix-v1.md) · [EXO-155](https://linear.app/exo-red/issue/EXO-155). + +## The dependency contract + +~~~text +CGMM-0 docs, exact source ownership and refusal matrix + | + +-- CGMM-1 source-fact intake census and typed Kythe identity/coverage (LAB) + | | + | +-- CGMM-2 demand/binding-plan and lazy Codegraph view (LAB) + | | | + | | +-- CGMM-3 FieldViewDef + MeTTa-shaped constraint scratch (LAB) + | | | + | | +-- CGMM-4 OGIT M:N/tenant + OGAR ActionDef offers (LAB) + | | +-- CGMM-5 exact pure RelAlg polyfill (LAB) + | | | | + | | | CGMM-6 owner-admitted DT64 -> R2IL native + | | | | + | | +------- CGMM-7 prepared plan/memo economics + | | | + | +-- CGMM-8 reachability / provenance / NNUE ranking audit + | + +-- EXO-79,84,85,95,101,108,121,140,143,152,153,154 + are named gates, not automatic permissions +~~~ + +## CGMM-0. Architecture + Linear + board (this PR) + +**Deliverables:** [master](d-cgmm-codegraph-metta-method-builder-v1.md), [Action/Relation addendum](d-cgmm-action-relation-schema-v1.md), [Lazy Codegraph addendum](d-cgmm-lazy-codegraph-fact-demand-v1.md), this charter, [falsifier matrix](d-cgmm-falsifier-matrix-v1.md), board entry, board status/integration/entry index, EXO-155 issue and append-only EXO-50 handoff. + +**Acceptance:** Codegraph source facts separated from derivation plan and lazy projection; ClassView and OGIT/OGAR/DT64 ownership preserved; exact refusal at current R2IL `GuardNotExecutable`/three-valued boundary; no ambiguity of FieldViewDef with render-only OGAR FieldView; no production code or new ISA. No merge without review. + +## CGMM-1. Intact Ruff/Charon/Kythe source fact intake (future lab) + +**Goal:** map the actual source artifact to stable source node/edge IDs and independent coverage grades without an eager copied AST object graph. Follow the current `.rows.bin` FUNDECL/PROV reader and Codegraph interner. Produce an explicit FACT-LOSS register: receiver types, field read/write, dynamic dispatch, edge polarity, source span, effect ordering and negative statements. A codegraph position is not a stable Kythe ID. + +**Oracles:** independent source facts by producer source ID; single method with writes, reads, possible trait dispatch and unresolved call; source revision changes; falsely confident missing effects. Source producer modification is a separate owner review; do not auto-fill unknown columns. + +**Metrics:** intake bytes and allocations, source→binary loss, provenance completeness, duplicate facts, AST materializations, cold/hot refresh. **Stop** if a fact is unavailable or a schema extension would break the existing binary contract. + +## CGMM-2. Crepe/Soufflé-style demand artifact, lazy Codegraph (future lab) + +**Goal:** expose compiler-like **binding-pattern and demand metadata** to existing Codegraph/XLOG/DR queries, without importing Crepe/Soufflé as a live executor or changing the global map. Compare existing eager six-relation intake and repeated `or_partners` queries with a bounded demand-scoped relation/mask plan. + +**Required evidence:** query roots, bound/free columns, source graph generation, grade floor, semiring, source/permission scope, recursion/negation/termination policy and unsupported constructs. Crepe `crepe_plan!` D1/D2 drift and XLOG polarity/magic report loss from the 2026-11-02 archaeology are independent falsifiers, not trusted compile data yet. + +**Positive:** one anchored call graph query touches only requested symbol relations after admissible indexing; repeated Q=1/2/16/256 amortization, bounded retained masks. **Negative:** `MAY` upgraded to confirmed; missing dynamic call interpreted as absence; stale source-epoch cache; unsupported recursion silently executed; demanded query slower than eager for all cases (report NO BENEFIT). + +## CGMM-3. FieldViewDef / ClassView-attached constraint scratch (future lab) + +**Goal:** one immutable **FieldViewDef OFFER** over an existing ClassView field or behavior-lens basis; per-session, disposable MeTTa-shaped constraint scratch returns `OFFER | REFUSE | ABSTAIN`. Roles include scalar, relation subject/object and action operand, with no physical data copy. Do not change the zero-dependency ClassView trait's `FieldMask` or `WideFieldMask` purpose. + +**Tests:** same field labels under different source/class version, full behavior-lens ID check, schema revision/position invalidation, partial source effect and incompatible method pair. Compile-fail or negative guard rejects mutable class basis and trait additions; scratch eviction does not modify an admitted method. + +## CGMM-4. OGIT LinkSpec + OGAR ActionDef M:N / tenant profile (future lab) + +**Goal:** use one scratch vocabulary to propose **separate owner-admitted** ontology edge, OGAR action and DT64 method contracts. Cross-app Rails/Odoo/SAP scenarios are *synthetic* unless independently harvested. No automatic equivalence from label or NNUE. + +**Fixture:** A tenant has W1/W2 and P1/P2; B tenant W3/P3. A relation includes duplicate (W1,P2) for Bag and a forged cross-tenant edge (W2,P3). A and B share an "Approve" label but have ordered effects that differ. + +**Tests:** cardinality vs multiplicity; relation orientation vs inverse; tenant auth vs ontology context; edge-instance identity vs endpoints; unknown source writes → ABSTAIN; one effect intentionally removed causes equivalence to be rejected. No relation insert or ActionInvocation. + +## CGMM-5. RelAlg polyfill for exact eligible field views (future lab) + +**Goal:** a finite, typed, pure same-universe read `FieldViewDef` lowers via existing `RelAlg::Rel::{All,Empty,Plane,Atom,And,Or,Not}` / `Measure::{Count,Sum,Min,Max,Keep}` to R2IL/mask-risc. Source field label is resolved cold to typed lane slot, never string inside hot IR. + +**Tests:** exact I32/U32 comparisons, support vs Bag counts, missing/zero, SQL UNKNOWN refusal when unsupported, same-frame masks, independently computed oracle. **Refuse** cross-population M:N hop, arbitrary recursion, effects, guarded WHEN, dynamic data-dependent folds. `join_body::is_alpha_acyclic` is a classifier, not an executable join. + +## CGMM-6. DT64 MethodBinding owner admission and native execute proof (future, gated) + +**Owner decisions:** EXO-121/152, OGAR codebook, DT64 execution owner; source lease and scope; existing `ExecPermit` lab-only. A scratch offer is not a ClassId or automatic MethodRef; owner allocates proper identity and accepts exact signature/effects/guard/truth semantics. + +**First runnable consumer:** one guard-free two-valued fold is admitted to lab `r2il::binding_native::prepare_native`/`run_native` and compared against independent source-value oracle. Explicit `GuardNotExecutable` and `ThreeValuedNotExecutable` outcomes for other offers are positive refusals. No new method body interpreter. + +## CGMM-7. Compile/plan/route memoization economics (future lab) + +**Goal:** reuse immutable normalized pure method recipes under all source/artifact/schema/method/ontology/rule/permission/DR frame witnesses without caching source AST values or copying canonical SoA. Keep **cold harvest/constraint/prepare** and **hot execute** separate. Q=1/2/8/16/64/256, scope/source invalidation, cache eviction and negative-control reroute. + +**ALU:** compare existing native Boolean/byte path against shipped Abakus carry words only when semantic arithmetic is required; #1956 measured 1.5–4.8× carry loss for its eight shapes, so no forced Abakus winner. + +## CGMM-8. Independent oracle and reachability firewall (future lab) + +**Goal:** two independent oracles (scalar source-enumeration and bounded Crepe/datafrog/Soufflé/MORK where pinned and licensed) confirm the graded result and refuse unsupported semantics. NNUE can reorder **candidates**, never grant facts or permits. Link the graduated intake to D-DRIVER-ZC caller-coverage proof, reporting unknown/dynamic call paths explicitly. + +**Stop:** missing artifact/producer, same function used for oracle and candidate, copy of source value cache, generic Datalog/MeTTa runtime adoption, caller context not attested or claimed complete without reachability proof. + +## PR governance + +**One stage per PR**, exact commit SHAs, source-backed grades, a deliberate disable that turns a nonvacuous assertion red, cold/hot/representation metrics, separate board entry and EXO-50 append-only handoff. No auto-merge and no reclassifying a research adapter as a production API. All future work gated on independent owner decisions; CGMM-0 does not authorize CGMM-1–8. + +**Explicitly out of scope:** migrated Ladybug/cognitive-shader-driver and P64/StreamDTO/BusDTO/PerturbationDTO consolidation; the operator deferred that study. + +## Implementation-ready release packet and exact EXO gates + +The [detailed blueprint](d-cgmm-implementation-blueprint-v1.md), [EXO gate ledger](d-cgmm-exo-gates-v1.md) and [executable fixture specification](d-cgmm-executable-fixtures-v1.md) **refine** CGMM-1…8 without authorizing them. They require **source fact fidelity before lazy demand optimization**, correct grades and ClassView basis before FieldViewDef offers, and verified pure RelAlg lowering before any DT64 native execution. Cold/hot costs, Bag/Set counts, cross-tenant authorization and deliberately wrong source effects have fixed independent expected answers. If an owner gate is open, the stage may only produce a scoped lab or decision memo, not a shipped method.