Stack 2 0

.github/workflows/ci.yml

@@ -35,14 +35,8 @@ jobs:
           --password postgres \
           --database-name test_db \
           --test-only false
-        uv run omop-config databases add test_cdm_db \
-          --kind cdm \
-          --connection pg_test \
-          --schema-name public
-        uv run omop-config databases add test_vector_db \
-          --kind generic \
-          --connection pg_test \
-          --schema-name public
+        uv run omop-config databases add cdm test_cdm_db --connection pg_test --cdm-schema public
+        uv run omop-config databases add generic test_vector_db --connection pg_test --schema-name embeddings
         uv run omop-config vector-stores add test_embeddings \
           --backend-type pgvector \
           --database test_vector_db

.gitignore

@@ -19,4 +19,5 @@ settings.json
 site/
 _knowledge/
 data/
-.env
+.env
+*.db

README.md

@@ -19,6 +19,7 @@ Start with the [documentation home](docs/index.md), or choose a path directly:
 - [Initial local setup](docs/from-scratch.md) walks through a fresh CDM-backed deployment.
 - [Integrations](docs/usage/integrations.md) shows MCP, REST, and Python calls.
 - [Configuration](docs/usage/configuration.md) explains shared-stack references and optional capabilities.
+- [Upgrading from 0.6](docs/usage/upgrading-from-0.6.md) lists the 0.7 configuration and runtime changes.
 - [Tools overview](docs/tools/overview.md) maps jobs to the discoverable MCP surface.
 
 ## Quick start

config/groundworkers.example.toml

@@ -1,4 +1,4 @@
-# groundworkers example config — oa-configurator 1.x StackConfig format
+# groundworkers example config — oa-configurator 2.0 StackConfig format
 #
 # Copy to a path selected by the setup console and fill in your values.
 # Full reference: https://oa-configurator.readthedocs.io (or run omop-config --help)
@@ -33,19 +33,19 @@ database_name = "cava"
 # Databases — logical roles over those connections
 # ---------------------------------------------------------------------------
 
-# kind = "cdm" bundles the CDM, vocabulary, and results roles. vocab_connection
-# and vocab_schema fall back to the primary connection and schema when unset.
+# kind = "cdm" bundles the CDM, vocabulary, and results roles. The role schema
+# fields fall back to the primary schema when unset. Omit these fields for SQLite.
 [databases.cdm_db]
 kind         = "cdm"
 connection   = "cdm_main"
-schema_name  = "public"
+cdm_schema   = "public"
 vocab_schema = "public"
 
-# The embedding store's database. kind = "generic": no CDM role-splitting.
+# The embedding store's database. Keep its schema distinct from the CDM schema.
 [databases.emb_db]
 kind        = "generic"
 connection  = "cdm_main"
-schema_name = "public"
+schema_name = "groundworkers"
 
 # ---------------------------------------------------------------------------
 # Providers and models — endpoints, then the named models served through them

docs/adapters/cdm.md

@@ -1,32 +1,30 @@
 # CDM Adapter
 
-`CDMAdapter` holds the SQLAlchemy engine and session factory for a single CDM database connection. It is the shared connection layer used by both `VocabService` and `OmopGraphAdapter`.
+`CDMAdapter` holds the primary engine, vocabulary engine, and routed session factory for a resolved CDM database. It is the shared connection layer used by `VocabService` and `OmopGraphAdapter`.
 
 ## What it is
 
 `CDMAdapter` is a thin connection-management object. It does not issue queries itself. Its purpose is to:
 
-- hold a `sqlalchemy.Engine` pointed at the CDM database
-- expose a `sessionmaker` for callers that need scoped sessions
-- be constructed once per CDM database in `build_application()` and shared across the
-  components that need it
+- hold the primary and vocabulary `sqlalchemy.Engine` objects
+- expose a routed session factory for callers that query role-tagged OMOP tables
+- be constructed once per CDM database in `build_application()` and shared across the components that need it
 
 ## Who uses it
 
 | Component | How |
 |---|---|
 | `VocabService` | Takes a `CDMAdapter` as its only constructor argument. Uses the session factory to open per-call sessions for vocabulary table queries. |
-| `OmopGraphAdapter` | Shares the same engine (no separate connection pool) so omop-graph and vocabulary queries use the same connection pool. |
+| `OmopGraphAdapter` | Uses the routed session factory so vocabulary and primary-role graph tables reach their configured databases. |
 
 The adapter is created once per CDM database at application startup and wired through `build_application()`. It is not per-request.
 
 ## Construction
 
-`CDMAdapter` is config-agnostic: it takes an already-constructed `sqlalchemy.Engine`, not a config object. Only `build_application()` reads config and calls `sqlalchemy.create_engine()`. This means:
+`CDMAdapter` is config-agnostic. It accepts an engine and optional routed factory, while `build_application()` creates them through `omop_alchemy.config.create_cdm_engines()` and `omop_alchemy.cross_database.cdm_sessionmaker()`:
 
-- the adapter is testable by passing a SQLite engine or an in-memory engine directly
-- changing the connection source means changing `build_application()` only;
-  `CDMAdapter` and every service that uses it are untouched
+- the adapter remains testable by passing a SQLite engine directly
+- changing the connection source means changing `build_application()` only;`CDMAdapter` and every service that uses it are untouched
 
 ## Usage from `build_application()`
 
@@ -48,6 +46,6 @@ app.services.vocab  # backed by app.adapters.cdm
 
 ## Lifecycle
 
-`CDMAdapter` connects lazily: the engine is constructed at adapter creation time, but the first database connection is deferred until the first query. `is_available()` performs a lightweight probe (`SELECT 1`) to confirm connectivity without blocking unrelated tool calls.
+`build_app_config()` creates role engines through `create_cdm_engines(..., register_claims=False)`. Startup fails fast when either database role is unreachable and prints the scrubbed reason with a database recovery hint. `CDMAdapter.is_available()` also performs a lightweight Core query (`SELECT 1`).
 
-`close()` disposes the engine and releases the connection pool. Call it explicitly in teardown (for example in a test fixture `yield`/`finally` block) when you need to ensure connections are returned promptly. Process-level startup and shutdown wiring belongs in the application or transport layer, not in the adapter itself.
+`close()` disposes both role engines. Call it explicitly in teardown (for example in a test fixture `yield`/`finally` block) when you need to ensure connections are returned promptly. Process-level startup and shutdown wiring belongs in the application or transport layer, not in the adapter itself.

docs/adapters/omop_emb.md

@@ -10,15 +10,15 @@ The adapter is responsible for:
 - optional on-the-fly query encoding through an embedding client
 - exposing backend availability and registered model metadata
 
-The adapter does **not** resolve stack config itself. `build_application(...)` constructs it from the already-resolved `omop-emb` package config and any required engines.
+The adapter does **not** resolve stack config itself. `build_application(...)` constructs it from Groundworkers' `[tools.groundworkers]` `embedding_model_name` and `vector_store_name` references.
 
 ## Backends
 
 `groundworkers` supports the same primary storage backends it wires from `omop-emb`:
 
-| Backend | omop-emb config |
+| Backend | Groundworkers stack config |
 |---|---|
-| `sqlitevec` | `backend = "sqlitevec"` plus `sqlite_path` |
+| `sqlitevec` | a `[vector_stores.*]` entry with `backend_type = "sqlitevec"` and a database reference |
 | `pgvector` | a `[vector_stores.*]` entry with `backend_type = "pgvector"` and a `database` reference |
 
 FAISS remains a sidecar acceleration layer rather than a standalone primary backend.
@@ -34,7 +34,7 @@ No live model API is required.
 
 ### Text search mode
 
-Requires an embedding client configured through `omop-emb` package settings.
+Requires `embedding_model_name` and `vector_store_name`, plus a configured embedding provider/model.
 
 - `embedding_search` encodes a text query on the fly
 - `embedding_search_batch` encodes and searches multiple text queries with one provider/index batch, returning rows aligned to input order

docs/development/integration.md

@@ -18,7 +18,8 @@ app = build_application(config)
 |---|---|
 | `groundworkers` | `GroundworkersConfig` |
 | `cdm_database` | `ResolvedCDMDatabase` |
-| `cdm_engine`, `vocabulary_engine` | `sqlalchemy.Engine` |
+| `cdm_engine`, `cdm_vocab_engine` | `sqlalchemy.Engine` |
+| `cdm_session_factory` | Routed SQLAlchemy session factory for primary, vocabulary, and results roles |
 | `embedding_model` | `ResolvedModel \| None` |
 | `vector_store` | `ResolvedVectorStore \| None` |
 
@@ -42,6 +43,8 @@ services.source_planning
 
 A CDM-only configuration is fully supported. Graph availability follows the resolved CDM database — there is no separate section to enable.
 
+The engines are created by `omop_alchemy.config.create_cdm_engines(..., register_claims=False)`, and `cdm_session_factory` comes from `omop_alchemy.cross_database.cdm_sessionmaker`. Use the session factory for queries that may touch more than one OMOP role. On PostgreSQL, engine creation checks role schema claims eagerly, so the configured databases must be reachable while `build_app_config()` runs.
+
 ## Grounding free text
 
 ```python
@@ -127,6 +130,6 @@ operation = (
 controller = ConfigWizardController(cdm_setup_workflow(operation), service)
 ```
 
-The same shape covers `MODEL_SETUP_TARGET` / `model_setup_workflow` and `LLM_SETUP_TARGET` / `llm_setup_workflow`. The provider decides create-versus-update from the current file, owns revision checking and redaction, and returns `conflicted` rather than clobbering another writer. `model_discoverer` is the injected seam for live model inventory; supply your own to avoid network calls in tests.
+The same shape covers `MODEL_SETUP_TARGET` / `model_setup_workflow`, `LLM_SETUP_TARGET` / `llm_setup_workflow`, and `VECTOR_STORE_SETUP_TARGET` / `vector_store_setup_workflow`. The provider decides create-versus-update from the current file, owns revision checking and redaction, and returns `conflicted` rather than clobbering another writer. `model_discoverer` is the injected seam for live model inventory; supply your own to avoid network calls in tests.
 
 This requires the `tui` extra for Groundskeeping, but not Textual — the provider and controller are host-agnostic.

docs/development/plugins.md

@@ -59,7 +59,7 @@ For example, a plugin recommending comparator cohorts might reason like this.
 
 | Use case | What you need | How you get it |
 |---|---|---|
-| Look up existing OMOP concepts | Read access to the CDM vocabulary connection Groundworkers already opened | `context.cdm_engine` / `context.resolver` |
+| Look up existing OMOP concepts | Read access through Groundworkers' routed vocabulary session | `context.cdm_session_factory` / `context.resolver` |
 | Rank or filter by similarity to concepts Groundworkers already embedded | The concept embeddings Groundworkers already populated, via omop-emb | `context.vector_store` + `context.embedding_backend_factory` |
 | Ask an LLM to help decide something | The chat model Groundworkers already configured, via omop-llm | `context.chat_backend_factory` |
 
@@ -76,12 +76,14 @@ The concept-embedding row and the free-text-embedding row look similar but are n
 
 ### Assumptions this table is making
 
-- **Vocabulary only, so far.** Every example above reads OMOP *vocabulary* i.e. operations over concepts, concept relationships, concept embeddings, or the host's shared grounding cascade. Nothing about `context.cdm_engine` restricts a plugin to vocabulary tables specifically, however. These tables are available over the same CDM connection groundworkers itself uses, and a plugin *could* query clinical/person-level tables (`condition_occurrence`, `drug_exposure`, etc.) through it just as easily, subject to whatever environmental and procedural access controls actually govern that data.
+- **Vocabulary only, so far.** Every example above reads OMOP vocabulary: concepts, concept relationships, concept embeddings, or the host's shared grounding cascade. `context.cdm_session_factory` routes vocabulary and primary roles, including on split-vocabulary deployments. Clinical/person-level queries need a database role explicitly configured for them.
 - **Shared mapping access.** `PluginContext.mapping_context` is an optional,read-only capability façade over the host's mapping and grounding policy. Use its operations when a plugin needs candidate bundles, text grounding, or concept context. `concept_shared_standard_ancestor(concept_ids, max_depth=5)` returns the unique nearest shared standard ancestor when one exists; raise `max_depth` to widen the hierarchy search. The underlying vocabulary, graph, and service objects are deliberately not exposed. Do not construct a second `omop-graph` or vocabulary adapter from `context.cdm_engine`. The field is `None` only when the host did not compose services for the plugin context.
 
 ## Build and register a new plugin
 
-`build()` receives `PluginContext`. The context provides the resolved CDM database and engine, vector store, shared lazy model backend factories, the optional read-only `mapping_context`, and a deliberately narrow resolver for independent named OA resources. Return `None` when a prerequisite is unavailable. Groundworkers then keeps serving its core capabilities and reports why the plugin was not activated.
+`build()` receives `PluginContext`. The context provides the resolved CDM database and primary engine, routed `cdm_session_factory`, vocabulary engine, vector store, shared lazy model backend factories, the optional read-only `mapping_context`, and a deliberately narrow resolver for independent named OA resources. Return `None` when a prerequisite is unavailable. Groundworkers then keeps serving its core capabilities and reports why the plugin was not activated.
+
+Use `context.cdm_session_factory` for role-tagged OMOP queries. `context.cdm_engine` remains available for primary-role access. On split-vocabulary stacks, plugins must use the routed factory; an Engine-based OmopGraphAdapter or DomainNameResolver cannot route vocabulary queries and will fail with upgrade guidance.
 
 `register()` adds MCP tools, prompts, or resources against the state returned by `build()`. The shared MCP registration wrapper supplies the same safe error translation used by core tools. Code-defined prompts can continue to derive their arguments from the callable signature. A plugin publishing data-defined prompts can pass explicit string metadata with `PromptArgument`; this keeps dynamic pack fields out of Python closure signatures:
 
@@ -158,7 +160,8 @@ Schemas requiring live discovery or a conditional flow (and therefore not fittin
 ```python
 from typing import Annotated, ClassVar
 
-from oa_configurator import PackageConfigBase, Sensitive
+from oa_configurator import PackageConfigBase
+from oa_configurator.refs import Sensitive, Secret
 from pydantic import Field
 
 
@@ -167,10 +170,23 @@ class ExampleConfig(PackageConfigBase):
     api_token: Annotated[str | None, Sensitive()] = Field(default=None)
 ```
 
-The `oa_configurator.Secret` alias is equivalent to `Annotated[str | None, Sensitive()]`. Prefer a `RefTo` field when a plugin can reuse an existing database, model, provider, or vector-store entry. The plugin then stores only the logical reference name; sensitivity remains declared on the referenced schema, such as `ConnectionConfig.password` or `ProviderConfig.api_key`, and the generic recursive configuration workflow preserves those declarations.
+The `Secret` alias is equivalent to `Annotated[str | None, Sensitive()]`. Prefer a `RefTo` field when a plugin can reuse an existing database, model, provider, or vector-store entry. The plugin then stores only the logical reference name; sensitivity remains declared on the referenced schema, such as `ConnectionConfig.password` or `ProviderConfig.api_key`, and the generic recursive configuration workflow preserves those declarations.
 
 This protects display and logging surfaces; it does not encrypt persisted configuration. Plugin documentation must describe the storage expectations for any secret it introduces.
 
+## Upgrading plugins to Groundworkers 0.7
+
+1. Require `oa-configurator>=2,<3` and `groundworkers>=0.7,<0.8` in the plugin package.
+2. Import `Sensitive` and `Secret` from `oa_configurator.refs`, and load files with `load_stack_config(path)`.
+3. Create CDM role engines with `omop_alchemy.config.create_cdm_engines(resolved, register_claims=False)` for read-only plugin access; use `context.cdm_session_factory` when Groundworkers provides the database.
+4. Remove schema fields from SQLite database entries. For PostgreSQL CDM entries, use `cdm_schema`; vector-store database entries continue to use `schema_name`.
+5. Keep accepting `context.cdm_engine` for integrations that need the primary SQLAlchemy engine. The routed factory handles vocabulary, primary, and results roles for ORM queries.
+6. Treat `context.embedding_backend_factory()` as a read-only omop-emb 3.0 `EmbeddingStoreReader`. It is shared with Groundworkers tools and status; reader methods no longer accept `metric_type`, and model enumeration is `get_registered_models()`. Do not use it to register models, create tables, or write vectors. Use a plugin-owned store and explicit setup operation for plugin data.
+7. Pass `context.cdm_session_factory` when constructing `OmopGraphAdapter`; Engine mode only works for colocated databases. `context.cdm_engine` is the primary-role engine and cannot query vocabulary on a split deployment.
+8. Use the current public error codes: `LEGACY_REGISTRY`, `MISPLACED_REGISTRY`, `MISSING_STORAGE_TABLE`, `MODEL_REGISTRATION_CONFLICT`, and `READ_ONLY_STORE`.
+9. Update `describe()` consumers for the `llm_model`, `stack`, `groundworkers`, and `app_name` keys and `cdm_schema` / `vocab_schema` names. The old `base.sql` schema/engine helpers have been removed; use oa-configurator's `physical_schema_of`, `qualified`, `supports_schemas`, and `schema_for_role` APIs.
+10. Update direct `AppConfig` construction for the required `stack`, `groundworkers`, `cdm_database`, `cdm_engine`, `cdm_vocab_engine`, `cdm_session_factory`, `embedding_model`, `llm_model`, `vector_store`, and `knowledge_root` fields; prefer `build_app_config()` in plugins and tests.
+
 ## Setup steps
 
 Plugin data preparation belongs in registered setup steps. A plugin may expose a `setup_steps` tuple of `PluginSetupStep` instances. Each step declares a stable key, operator copy, typed arguments, and a `build_plan(values, config_path)` callable that returns a `MaintenancePlan`.

docs/index.md

@@ -69,6 +69,7 @@ The output includes the active tools, prompts, resources, plugins, redacted conf
 - [Initial local setup](from-scratch.md) for a fresh local deployment
 - [Integrations](usage/integrations.md) for MCP, REST, and direct Python usage
 - [Configuration](usage/configuration.md) for the shared-stack config model and runtime combinations
+- [Upgrading from 0.6](usage/upgrading-from-0.6.md) for the 0.7 configuration and plugin changes
 - [Tools overview](tools/overview.md) for the discoverable MCP surface
 
 ## Relationship to groundcrew