diff --git a/.changeset/22307-cold-boot-catalog-refusal.md b/.changeset/22307-cold-boot-catalog-refusal.md index 627ed100d2d..370b80256f4 100644 --- a/.changeset/22307-cold-boot-catalog-refusal.md +++ b/.changeset/22307-cold-boot-catalog-refusal.md @@ -6,7 +6,7 @@ feat(objectql)!: a cold boot refuses a package-held position or permission-set n Clause-②: no - + **BREAKING** — an accept-set narrowing at boot, shipped as `major` on the v18 pre-release line (`.changeset/pre.json` is in `next` pre mode on `main`). A deployment whose environment catalog holds a position or permission-set name that a configured package also declares booted before this release and is refused at boot after it. @@ -16,16 +16,16 @@ Clause-②: no **What an operator sees.** The kernel reports `Plugin com.objectstack.engine.objectql failed to start`, and the cause is the package door's envelope: `code: 'NAMESPACE_CONFLICT'` (`NAMESPACE_CONFLICT_CODE` is exported), `status: 422`, and `conflicts[]` with `{ catalogType, name, incomingPackageId, existingHolder: { kind: 'environment' } }`. The message names the package that declares each name and the environment catalog that holds it. -**The upgrade shape.** A deployment fails to boot after this release when an active, environment-wide `sys_metadata` row of type `permission` or `position` (or the legacy plural `permissions` / `positions`, which the boot's load folds to the same types) has the name of a permission set or position that a configured package declares. That includes a row saved over a package-held name before the packaged locks refused such saves, whether or not the row was bound to the package, and a row over one of the platform security plugin's own permission sets (`member_default`, `admin_full_access` and the rest it declares). +**The upgrade shape.** A deployment fails to boot after this release when an active, environment-wide `sys_metadata` row of type `permission` or `position` (or the legacy plural `permissions` / `positions`, which the boot's load folds to the same types) has the name of a permission set or position that a configured package declares. That includes a row saved over a package-held name before the packaged locks refused such saves, whether or not the row was bound to the package, and a row over one of the platform security plugin's own permission sets (`member_default`, `admin_full_access` and the rest it declares). **Run `os migrate security-catalog-overlays` before the first v18 boot**, with the flags and environment the deployment boots with: it lists exactly these rows, and with `--apply` deletes them. -**The one-line fix: rename the item in the package, or rename or delete the environment's item, then restart.** +**The one-line fix: before the first v18 boot, run `os migrate security-catalog-overlays` to list the rows, then `os migrate security-catalog-overlays --apply` to delete them; or rename the item in the package.** - **Before upgrading, for a permission set.** On the release you run now, a boot whose environment catalog overlays a package-declared permission set logs at `kernel:ready`: `[security] N package-declared permission set(s) are being shadowed by an environment overlay`, with the set names. Those are the permission sets this release refuses at boot. The audited **Discard Overlay** action on the set's record in Setup (`POST /api/v1/security/permission-sets//discard-overlay`, documented under "Declared ≠ enforced" on the Permission Sets page) removes the overlay and resyncs the set to the package's definition. So does `DELETE /api/v1/meta/permission/`, which answers "Customization overlay deleted … reset to artifact default". Either works for the platform security plugin's own sets too, and neither needs direct database access. Positions have no such reading and no such action. -- **After upgrading, for a package you can leave out.** Boot once without the package in the configuration, rename or delete the environment's item through the metadata API (`DELETE /api/v1/meta/permission/`, `DELETE /api/v1/meta/position/`), then add the package back. -- **After upgrading, for a name the platform security plugin declares, or for any row the metadata API does not reach.** Back the database up, then delete the row in it. The rows that refuse the boot are the active, environment-wide ones of that name: `organization_id IS NULL` and `state = 'active'`, whatever their `package_id`, under the type or its legacy plural: `DELETE FROM sys_metadata WHERE organization_id IS NULL AND state = 'active' AND type IN ('permission', 'permissions') AND name = '';` (for a position, `type IN ('position', 'positions')`). A draft row and an organization-scoped row are not loaded at boot and do not refuse it. +- **After upgrading, for any refused name.** The same step: `os migrate security-catalog-overlays`, then `--apply`. It needs no server, so a deployment whose boot is refused can run it, and it covers permission sets and positions, a name the platform security plugin declares, and a row stored under a legacy plural. +- **What the step deletes.** The rows that refuse the boot are the active, environment-wide ones of a package-held name: `organization_id IS NULL` and `state = 'active'`, whatever their `package_id`, under the type or its legacy plural. In SQL, for one permission-set name, the step's deletion is `DELETE FROM sys_metadata WHERE organization_id IS NULL AND state = 'active' AND type IN ('permission', 'permissions') AND name = '';` (for a position, `type IN ('position', 'positions')`). The step runs it through the metadata write path, so each deleted row leaves a `sys_metadata_history` tombstone. A draft row and an organization-scoped row are not loaded at boot and do not refuse it. -`DELETE /api/v1/meta/permission/` and `DELETE /api/v1/meta/position/` reach a row stored under `permission` or `position` only, one row per call. A row stored under the legacy plural `permissions` / `positions` is not reached: the call answers `200` that nothing was found and removes nothing. Where a name has two active rows, for example one bound to no package and one bound to the package, each call removes one. A plural-typed row is removed by Discard Overlay before upgrading (a permission set), or by the SQL above after upgrading, for any name. +`DELETE /api/v1/meta/permission/` and `DELETE /api/v1/meta/position/` reach a row stored under `permission` or `position` only, one row per call. A row stored under the legacy plural `permissions` / `positions` is not reached: the call answers `200` that nothing was found and removes nothing. Where a name has two active rows, for example one bound to no package and one bound to the package, each call removes one. A plural-typed row is removed by Discard Overlay before upgrading (a permission set), or by `os migrate security-catalog-overlays --apply`, for any name. -No `os` command deletes a `sys_metadata` row offline: `os meta delete` and `os data delete` call a running server. Nothing renames or removes either item automatically. +`os migrate security-catalog-overlays` is the one `os` command that deletes these rows with no server running; `os meta delete` and `os data delete` call a running server. Nothing renames or removes either item automatically: the step deletes only under `--apply`, and adopts nothing. **What is NOT refused.** A stored definition under a built-in position name (`org_admin`, `everyone` and the other four): the platform declares its built-in positions itself, after this check, and the stored definition keeps answering first. The same package restarting with its own names. A package whose names the environment catalog does not hold, booting beside the environment's own items. diff --git a/.changeset/22371-security-catalog-overlays-step.md b/.changeset/22371-security-catalog-overlays-step.md new file mode 100644 index 00000000000..4cdae48695e --- /dev/null +++ b/.changeset/22371-security-catalog-overlays-step.md @@ -0,0 +1,24 @@ +--- +"@objectstack/cli": minor +"@objectstack/objectql": minor +"@objectstack/core": minor +"@objectstack/runtime": minor +"@objectstack/spec": minor +--- + +feat(cli): `os migrate security-catalog-overlays` lists, and with `--apply` deletes, the environment-wide rows a v18 cold boot refuses — run it before the first v18 boot + +Clause-②: yes (widening) + +- **What it is for.** A v18 cold boot refuses to start when the environment catalog holds a permission-set or position name that a configured package also declares (ADR-0048 N.3). A deployment upgraded from 17.x can carry such rows: a set or position saved in the environment over a package's name before the packaged locks refused that, or over one of the platform security plugin's own sets (`member_default`, `admin_full_access` and the rest). The server cannot clear them, because it does not start. This step is the offline remedy. +- **What it lists.** Every active, environment-wide `sys_metadata` row (`organization_id IS NULL`, `state = 'active'`) of type `permission` or `position`, the legacy plurals `permissions` and `positions` included, whose name a configured package holds. That is the population the cold boot refuses: the list is computed from the engine's own reading of who holds a name, not from a copy of it. A draft row and an organization-scoped row are not loaded at boot, so they are never listed. Each row is shown with its stored type, its `package_id` binding and the package that holds its name. +- **How it gets there without the refusal.** It composes the deployment as `os serve` does: the host config's plugins, the application or the compiled artifact, and the security plugin behind `serve`'s auth gate. It runs the kernel's first phase for them only, and boots without reading `sys_metadata` back into the registry, so the cold-boot check meets nothing and the boot comes up. No server starts. The security plugin's shipped sets are package-held names only where `serve` composes that plugin, so the step reads the environment `serve` reads. With `OS_AUTH_SECRET` (or `AUTH_SECRET` / `BETTER_AUTH_SECRET`) set, or on a development boot, the plugin is composed and its sets are held, unless the `auth` tier is off. The step takes `os serve`'s `--preset` and `--dev` with `serve`'s meaning, read through the rules `serve` reads them by: `--preset` names the tier preset the auth tier falls back to, and `--dev` composes the config's `devPlugins` and makes the boot a development one (as does `NODE_ENV=development`). The report says which way it went. **Run the step with the flags and environment the deployment boots with.** +- **Preview by default.** The preview boots read-only, like `os migrate plan`. It writes nothing, creates no database file, and exits `1` when it lists a row: the next boot would be refused. `--apply` deletes the listed rows, after a `[y/N]` prompt or `--yes`, and prints one audit line per row: what was deleted, through which write path, or why not. It exits `0` when every listed row is gone. `--json` prints one document (`listed`, `deleted`, `failed`, `rows[]` with `outcome`, `securityPlugin` and `serveFlags`). `--force` gets past a busy SQLite file. `--database-url` / `$OS_DATABASE_URL` name the target. +- **How it deletes.** A row stored under `permission` or `position` goes through the metadata protocol's own delete, the one `DELETE /api/v1/meta/:type/:name` uses. That writes a `sys_metadata_history` tombstone and a `sys_metadata_audit` row, with actor `os migrate security-catalog-overlays`. A row stored under a legacy plural is out of that door's reach, so it goes through the `sys_metadata` repository beneath it. That delete is addressed by the row's stored type, name, `package_id` and checksum, and writes the same history tombstone. +- **The refusal names it.** The cold boot's `NAMESPACE_CONFLICT` message now points at `os migrate security-catalog-overlays` for the environment's rows, in place of "through the metadata API on a boot that leaves the package out of the configuration, or in the database". The envelope (`code`, `status`, `conflicts[]`) is unchanged. +- **What it never does.** It adopts nothing: `managed_by` and `package_id` are never rewritten, and no row is renamed. A row the environment needs under its own name must be re-created under a name no package holds. A host config that exists and cannot be loaded is refused before any row is read, because the held names would be incomplete. +- **New exports it is built on.** + - `@objectstack/objectql` exports `findPackageHeldSecurityCatalogNames(registry)`: every package-held permission-set and position name, with its holder packages. The cold-boot check now reads the same list. + - `@objectstack/core` exports the auth gate `os serve` and the step share. `resolvePlatformAuthComposition` answers whether the platform composes `AuthPlugin` and the security plugin beside it, or why not. With it come `resolveStackTiers`, `STACK_TIER_PRESETS`, `CAPABILITY_TO_TIER`, `resolveAuthSecret`, `isDevelopmentBoot`, `DEV_AUTH_SECRET_FALLBACK`, `stackSuppliesAuthPlugin`, `isHostKernelComposition` and the `PlatformAuthComposition` / `PlatformAuthSkipReason` types. `Serve.TIER_PRESETS` and `Serve.CAPABILITY_TO_TIER` are now handles over the core declarations, and `os serve` composes exactly what it composed before. + - `@objectstack/runtime`'s `createStandaloneStack` accepts `hydrateMetadataFromDb: false`, which skips the `sys_metadata` read-back. The default stays on. + - `@objectstack/spec` registers the ADR-0087 D3 entry `security-catalog-environment-overlay-refused` in its migration registry and `spec-changes.json`, so `os migrate meta --from 17` names the refusal and this step as its remedy. Its projection is the entry under protocol 18 in the upgrade guide (`docs/protocol-upgrade-guide.md`). diff --git a/content/docs/deployment/cli.mdx b/content/docs/deployment/cli.mdx index f2cd2ff5225..fa1a29b4f61 100644 --- a/content/docs/deployment/cli.mdx +++ b/content/docs/deployment/cli.mdx @@ -937,6 +937,7 @@ written. | `os migrate apply` | **Refuses** (exit 1, `error: database_busy` under `--json`). Stop the other process, or pass `--force` | | `os migrate files-to-references --apply` | **Refuses** likewise — it rewrites rows, so a concurrent writer is at least as dangerous | | `os migrate meta --stored --apply` | **Refuses** likewise — it rewrites `sys_metadata` rows, and a live process saving metadata is exactly the collision | +| `os migrate security-catalog-overlays --apply` | **Refuses** likewise — it deletes `sys_metadata` rows, and a live process saving metadata is exactly the collision | The check applies to SQLite only: Postgres and MySQL take their own server-side locks. Only same-user processes are visible without elevated privileges, and a @@ -1114,6 +1115,7 @@ where the data lives. | `os migrate value-shapes` | Scan stored reference and structured-JSON field values against the platform's value contract, and record the deployment's migration flag when clean | | `os migrate summary-nulls` | Backfill roll-up `count` / `sum` columns still stored as `NULL` on parent rows created before the insert-time seed. Repairs values; no flag, nothing depends on it having run | | `os migrate meta --stored` | Replay the metadata conversion chain over this deployment's `sys_metadata` rows and rewrite the ones still carrying a pre-protocol shape. Hygiene, not a gate — nothing depends on it having run | +| `os migrate security-catalog-overlays` | List the environment-wide permission-set and position rows (the legacy plurals `permissions` / `positions` included) stored over a name a configured package holds — the rows a v18 cold boot refuses — each with the package that holds its name; `--apply` deletes them, one history tombstone per row. Run it before the first v18 boot, with the flags and environment the deployment boots with: the platform security plugin's sets are held only where `os serve` composes it (an auth secret set, and the `auth` tier on), and the step takes `serve`'s `--preset` and `--dev` with `serve`'s meaning. Adopts nothing; exits 1 while it lists a row | | `os migrate duplicates` | Report business identifiers already minted twice across the organization partitions, and the rows blocking the boot-time NULL-safe index tightenings — a read-only inventory as JSON on stdout. Renumbers nothing and writes no row and no schema; run it before the boot-time tenancy repair, which overwrites part of the evidence | **The boot itself writes no row and no schema you did not ask for.** Each of these commands boots diff --git a/docs/protocol-upgrade-guide.md b/docs/protocol-upgrade-guide.md index efd68ba1b04..645bd0a066d 100644 --- a/docs/protocol-upgrade-guide.md +++ b/docs/protocol-upgrade-guide.md @@ -1272,6 +1272,9 @@ This is a RUNTIME registration API, not stored metadata, so — like `hook-regis - **`screen-field-lookup-reference-required`** — `The `reference` key of a `type: 'lookup'` field on a `screen` node — `flows[].nodes[].config.fields[]` where the node `type` is `screen` and the field `type` is `lookup` (`ScreenFieldConfigSchema`). Nothing is renamed, retired or re-typed and the key set does not move: `reference` was already declared and already optional in the shape. What narrows is the ACCEPT SET for one value of the sibling `type` — a `lookup` field with no `reference`, or with a blank one, parsed before this major and is refused now. Every other widget hint is untouched, and a `lookup` field that already names its target parses byte-identically.` → Name the object whose records the picker offers, beside the type: `{ name: 'resolved_by_article', type: 'lookup', reference: 'crm_knowledge_article' }`. The value is an object NAME (the canonical id — same string `FieldSchema.reference` carries), not a label and not a record id. ⚠️ There is deliberately no default and no inference: a picker pointed at the wrong object is worse than one that refuses to load, because it offers a human a plausible list of the wrong records and the flow stores the id it is given. Where the field genuinely has no target object — the author was using `lookup` to mean "type an id here" — the fix is the other direction: change `type` to `'text'`, which is what that field actually was, and keep the prose that asked for an id in `inlineHelpText`. - Why not automatic: Maintainer ruling A′, 2026-09-13, verbatim, untranslated: 「同意」. ADR-0078 forbids metadata that parses, carries no marking and does nothing — and its own worked example of that state is a `lookup` with no `reference`: the field renders a picker, the picker has no object to query, and nothing anywhere says so. The key shipped OPTIONAL on this surface one release earlier, on the argument that flows declaring a bare `lookup` already exist; the ruling reversed that, holding that a degraded shape which ships is not a reason to bend the contract to it. ⛔ NOT losslessly convertible, and the reason is the same one `schedule-flow-acting-organization-required` gives: the remedy is a value the artifact does not contain. A bare lookup records the field name and nothing about its intended object, so `objectstack migrate meta` can identify every site but can answer none of them — and a conversion that guessed (the first object with a matching-looking name, the flow's trigger object) would write an authoritative wrong answer into metadata a human then trusts. Registered under ADR-0087 D3 rather than left silent because the change DOES carry a prescription a human can execute, which is what D3 says a structured TODO is for. - Done when: Every `type: 'lookup'` field on every `screen` node in the stack declares a non-empty `reference`, and the stack parses: `ScreenFieldConfigSchema` refuses the bare form with a message addressed to `reference` (`SCREEN_FIELD_LOOKUP_REFERENCE_REQUIRED`), so a full metadata parse — `os lint`, or any publish — reports one issue per unfixed site and names the FLOW and the FIELD in its path. Work the list to empty rather than sampling it: a flow whose screen never reaches that node in testing is refused at publish just the same. For each site, answer which object the picker was meant to offer — the declaration is the answer, and where there is no such object the field was never a lookup (retype it `'text'`). ⚠️ Runs SUSPENDED at a screen before the upgrade rehydrate their `ScreenSpec` from stored context, so an in-flight run parked on an unfixed screen carries the old shape: drain or re-drive those rather than assuming the fix reaches them retroactively. +- **`security-catalog-environment-overlay-refused`** — `the cold boot of a deployment whose sys_metadata holds an active, environment-wide row of type permission or position (or the legacy plural permissions or positions) under the name of a permission set or position a configured package declares, the platform security plugin's shipped sets included` → before the first boot on this major, run `os migrate security-catalog-overlays` with the flags and environment the deployment boots with (`--preset` and `--dev` mean what they mean to `os serve`): it lists exactly those rows, each with the package that holds its name. Then run `os migrate security-catalog-overlays --apply` to delete them through the metadata write path (a history tombstone per row). Or rename the item in the package. Nothing is adopted: an item the environment needs under its own name is re-created under a name no package holds + - Why not automatic: Positions, permission sets and capabilities hold one name per deployment, and a package registering a name the environment catalog already holds was already refused on a hot install. A cold boot now refuses it too, right after the stored rows load: the stored row used to be served in place of the package's definition, with only a collision warning. Rows like this exist on deployments that saved over a package-held name before the packaged locks refused such saves, and over the security plugin's own sets, which it declares only where the boot composes it behind the auth gate (an auth secret set, or a development boot). The refused deployment cannot start, so no in-server action can clear the rows, and the metadata API reaches no legacy-plural row at all; the offline step can. No conversion applies: the rows are the environment's own work, and whether to drop or rename one is the operator's call, which the step's preview puts in front of them. ADR-0048, ADR-0087. + - Done when: On the deployment's database and configuration, with its boot flags and environment, `os migrate security-catalog-overlays` lists no row (exit 0). The listing covers permission sets and positions and the legacy plural spellings, and never an organization-scoped or a draft row. After `--apply`, the next boot of the same database and configuration comes up, where before it was refused with NAMESPACE_CONFLICT (422) naming the environment catalog as the holder. - **`send-template-input-org-retired`** — `contracts.emailService.sendTemplate input.org` → (removed — never implemented; delete the key from the call. It is NOT replaced by `organizationId`: that member is the delivery row's tenant stamp (`sys_email.organization_id` pass-through, added so the email writer stamps a delivery row's organization at the source) and opts into no template overlay resolution) - Why not automatic: ADR-0049 enforce-or-remove. `SendTemplateInput.org` was declared as "Tenant id for org-overlay resolution (when supported)" and no implementation ever read it: `@objectstack/plugin-email` — the only IEmailService implementation — resolves templates on `(name, locale)` only, so a caller passing `org` got no org-overlay resolution and no error; the "(when supported)" hedge was the declaration admitting the gap. After the delivery-row stamp landed `organizationId` beside it, the input carried two org-shaped keys of which one did nothing — exactly the shape that invites an AI author to pick the wrong one. There is no behaviour to preserve and nothing stored to rewrite: the key only ever appeared in a call-time input bag (the `data.engine.update options.upsert` precedent), which is why this is a D3 semantic entry with no D2 conversion — no metadata seam ever runs on it. Org-overlay template resolution, if it ever earns a measured business pull, is a new capability with its own ruling — not this key revived. - Done when: No caller passes `org` to `IEmailService.sendTemplate()`. The enforcement channel is the compiler: `SendTemplateInput` is a programmatic contracts interface with no Zod surface, so authoring `org` is an excess-property `tsc` error (pinned in `packages/spec/src/contracts/email-service.test.ts`). Runtime behaviour is deliberately UNCHANGED: nothing ever read the member, so removing it removes no behaviour — a JavaScript caller still passing `org` keeps its exact pre-removal outcome (the key is carried inert and ignored). Template resolution still keys on `(name, locale)`, and `organizationId` still stamps `sys_email.organization_id` without acquiring any overlay semantics. diff --git a/packages/cli/src/commands/migrate/security-catalog-overlays.integration.test.ts b/packages/cli/src/commands/migrate/security-catalog-overlays.integration.test.ts new file mode 100644 index 00000000000..648955c5dfc --- /dev/null +++ b/packages/cli/src/commands/migrate/security-catalog-overlays.integration.test.ts @@ -0,0 +1,375 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * `os migrate security-catalog-overlays` against a real SQLite database: the + * offline step that lists, and with `--apply` deletes, the environment-wide + * rows a v18 cold boot refuses (maintainer ruling A′ on #22371, record + * 6073500921, item 1, as amended by ruling letter B, record 6074838935). + * + * ## The fixture + * + * A compiled artifact (`dist/objectstack.json`, the shape a deployed app boots + * from) whose package declares two positions and one permission set, over a + * database an older release left behind with: + * + * - LISTED: a `permission` row and a `position` row over the package's names, + * one bound to no package and one bound to the package itself, and a + * legacy-plural `positions` row over the package's other position; + * - LISTED ONLY WITH AUTH ON: a legacy-plural `permissions` row over + * `member_default`, a set the platform security plugin declares — and that + * plugin is composed only behind `os serve`'s auth gate; + * - CONTROLS, never listed: an environment-wide row whose name no package + * holds, an organization-scoped row and a draft row over a held name. + * + * ## What each case measures + * + * The command runs in-process through oclif, as the binary runs it. Its list is + * compared with THE COLD BOOT'S OWN REFUSAL over the same database and + * configuration: the same composition booted with `sys_metadata` hydration on, + * which is what the engine's check judges — once with `OS_AUTH_SECRET` set and + * once without, because the security plugin's names join only behind the gate + * (measured on #22371: 3 names against 2). After `--apply` the same cold boot + * comes up, and the step lists nothing. + * + * `os serve`'s `--preset` and `--dev` move that gate, and the step takes them + * with `serve`'s meaning: `--preset minimal` with a secret set composes no + * security plugin (no `auth` tier), and `--dev` with no secret composes it (the + * development fallback). Each is compared with the cold boot of the same + * composition — the same flags — so the list stays that boot's refusal. + */ + +import { describe, it, expect, beforeAll, afterAll, beforeEach, afterEach, vi } from 'vitest'; +import { copyFileSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { SqlDriver } from '@objectstack/driver-sql'; +import type { IObjectQLEngine } from '@objectstack/spec/contracts'; +import { NAMESPACE_CONFLICT_CODE } from '@objectstack/objectql'; +import { SECURITY_PLUGIN_ID } from '@objectstack/plugin-security'; +import { isExitSignal } from '../../utils/format.js'; +import { bootSchemaStack } from '../../utils/schema-migrate.js'; +import MigrateSecurityCatalogOverlays from './security-catalog-overlays.js'; + +// [#10126] Pay the first transform of these dist-resolved workspace deps at +// MODULE LOAD: the command reaches them through dynamic `import()`s inside +// `run()`, which vitest clocks (`scripts/check-test-source-alias.mjs`). +import '@objectstack/runtime'; +import '@objectstack/metadata-protocol'; +import '@objectstack/platform-objects/plugin'; + +const HERE = dirname(fileURLToPath(import.meta.url)); +const CLI_ROOT = resolve(HERE, '..', '..', '..'); +const CASE_TIMEOUT_MS = 180_000; +const SYSTEM = { context: { isSystem: true } } as const; + +const PKG = 'com.example.m22371'; +const ARTIFACT = { + manifest: { id: PKG, name: 'm22371', version: '1.0.0', type: 'app' }, + positions: [{ name: 'm22371_lead', label: 'Lead' }, { name: 'm22371_old_lead', label: 'Old lead' }], + permissions: [{ name: 'm22371_rep', label: 'Rep', objects: {} }], +}; + +interface SeedRow { type: string; name: string; organization_id: string | null; package_id: string | null; state: string } +const SEED: SeedRow[] = [ + { type: 'permission', name: 'm22371_rep', organization_id: null, package_id: null, state: 'active' }, + { type: 'position', name: 'm22371_lead', organization_id: null, package_id: PKG, state: 'active' }, + { type: 'positions', name: 'm22371_old_lead', organization_id: null, package_id: null, state: 'active' }, + { type: 'permissions', name: 'member_default', organization_id: null, package_id: null, state: 'active' }, + // Controls. + { type: 'permission', name: 'm22371_free', organization_id: null, package_id: null, state: 'active' }, + { type: 'permission', name: 'm22371_rep', organization_id: 'org_m22371', package_id: null, state: 'active' }, + { type: 'permission', name: 'm22371_rep', organization_id: null, package_id: null, state: 'draft' }, +]; +const bodyOf = (row: SeedRow) => + (row.type.startsWith('permission') ? { name: row.name, label: 'Saved in the environment', objects: {} } : { name: row.name, label: 'Saved in the environment' }); + +/** The listed rows, as the command reports them: `type/name@binding heldBy`. */ +const LISTED_AUTH_OFF = [ + `position/m22371_lead@${PKG} ${PKG}`, + `positions/m22371_old_lead@- ${PKG}`, + `permission/m22371_rep@- ${PKG}`, +]; +const LISTED_AUTH_ON = [ + ...LISTED_AUTH_OFF, + `permissions/member_default@- ${SECURITY_PLUGIN_ID}`, +]; + +const CONTROLS = [ + 'permission/m22371_free/env/active', + 'permission/m22371_rep/org_m22371/active', + 'permission/m22371_rep/env/draft', +]; + +let dir: string; +let templateDb: string; +const savedCwd = process.cwd(); +const ENV_KEYS = ['OS_ARTIFACT_PATH', 'OS_DATABASE_URL', 'DATABASE_URL', 'OS_DATABASE_DRIVER', 'OS_HOME', 'NODE_ENV', + 'OS_AUTH_SECRET', 'AUTH_SECRET', 'BETTER_AUTH_SECRET', 'OS_ENVIRONMENT_ID'] as const; +const savedEnv: Record = {}; + +function setAuth(on: boolean): void { + if (on) process.env.OS_AUTH_SECRET = 'm22371-secret-'.padEnd(40, 'x'); + else delete process.env.OS_AUTH_SECRET; +} + +/** A fresh copy of the template database for one case. */ +let caseSeq = 0; +function caseDb(): string { + caseSeq += 1; + const file = join(dir, 'cases', `${caseSeq}.db`); + mkdirSync(dirname(file), { recursive: true }); + copyFileSync(templateDb, file); + return file; +} + +/** Every `sys_metadata` row, `type/name/org/state`, read on a connection of our own. */ +async function storedRows(dbFile: string): Promise { + const driver = new SqlDriver({ client: 'better-sqlite3', connection: { filename: dbFile }, useNullAsDefault: true }); + try { + const rows = await (driver as any).knex('sys_metadata').select('type', 'name', 'organization_id', 'state'); + return rows.map((r: any) => `${r.type}/${r.name}/${r.organization_id ?? 'env'}/${r.state}`).sort(); + } finally { + await driver.disconnect(); + } +} + +/** The history and audit rows the deletions wrote. */ +async function trail(dbFile: string): Promise<{ history: string[]; audit: string[] }> { + const driver = new SqlDriver({ client: 'better-sqlite3', connection: { filename: dbFile }, useNullAsDefault: true }); + try { + const k = (driver as any).knex; + const history = await k('sys_metadata_history').select('type', 'name', 'operation_type', 'recorded_by', 'organization_id'); + const audit = await k('sys_metadata_audit').select('type', 'name', 'operation', 'outcome', 'actor'); + return { + history: history.filter((r: any) => r.operation_type === 'delete') + .map((r: any) => `${r.type}/${r.name} ${r.operation_type} by ${r.recorded_by} org=${r.organization_id ?? 'env'}`).sort(), + audit: audit.filter((r: any) => r.operation === 'delete') + .map((r: any) => `${r.type}/${r.name} ${r.operation} ${r.outcome} by ${r.actor}`).sort(), + }; + } finally { + await driver.disconnect(); + } +} + +/** + * THE COLD BOOT: the step's own composition — `os serve`'s, by the shared rules + * — booted with hydration ON, which is what the engine's check judges. Answers + * its refusal's `conflicts[]` as `catalogType/name holder` lines, or `[]` when + * the boot came up. + */ +async function coldBootConflicts( + dbFile: string, + serveFlags?: { dev?: boolean; preset?: string }, +): Promise { + let stack; + try { + stack = await bootSchemaStack({ + jsonOutput: false, + databaseUrl: `file:${dbFile}`, + projectRoot: dir, + composeHostStack: true, + composeAuthGatedSecurity: true, + ...(serveFlags ? { serveFlags } : {}), + deferSchemaDdl: true, + readOnlyProbe: true, + }); + } catch (e) { + const refusal = ((e as { cause?: unknown }).cause ?? e) as { code?: string; status?: number; conflicts?: any[] }; + expect(refusal.code, String(e)).toBe(NAMESPACE_CONFLICT_CODE); + expect(refusal.status).toBe(422); + return (refusal.conflicts ?? []).map((c) => `${c.catalogType}/${c.name} ${c.incomingPackageId}`).sort(); + } + await stack.shutdown(); + return []; +} + +interface RunResult { payload: any; exitCode: number } + +/** Run the command through oclif and capture its one JSON document. */ +async function runJson(argv: string[]): Promise { + const savedExit = process.exitCode; + const swallow = ((_chunk: unknown, ...rest: unknown[]) => { + const cb = rest.find((a) => typeof a === 'function') as (() => void) | undefined; + if (cb) cb(); + return true; + }) as typeof process.stdout.write; + const stdout = vi.spyOn(process.stdout, 'write').mockImplementation(swallow); + vi.spyOn(process.stderr, 'write').mockImplementation(swallow); + vi.spyOn(console, 'log').mockImplementation(() => {}); + vi.spyOn(console, 'warn').mockImplementation(() => {}); + vi.spyOn(console, 'error').mockImplementation(() => {}); + let thrownExit: number | undefined; + try { + try { + await MigrateSecurityCatalogOverlays.run(argv, { root: CLI_ROOT }); + } catch (error) { + if (!isExitSignal(error)) throw error; + thrownExit = (error as { oclif?: { exit?: number } }).oclif?.exit; + } + const out = stdout.mock.calls.map((c) => String(c[0])).join('').trim(); + let payload: any; + try { payload = JSON.parse(out); } catch { payload = { unparsed: out.slice(-400) }; } + return { payload, exitCode: thrownExit ?? (process.exitCode as number | undefined) ?? 0 }; + } finally { + process.exitCode = savedExit; + vi.restoreAllMocks(); + } +} + +const listedOf = (payload: any): string[] => + (payload.rows ?? []).map((r: any) => `${r.type}/${r.name}@${r.packageId ?? '-'} ${r.heldBy.join(',')}`); +const asConflicts = (payload: any): string[] => + [...new Set((payload.rows ?? []).flatMap((r: any) => r.heldBy.map((p: string) => `${r.catalogType}/${r.name} ${p}`)))] + .sort() as string[]; + +beforeAll(async () => { + for (const key of ENV_KEYS) savedEnv[key] = process.env[key]; + for (const key of ENV_KEYS) delete process.env[key]; + process.env.NODE_ENV = 'production'; + dir = mkdtempSync(join(tmpdir(), 'os-22371-')); + mkdirSync(join(dir, 'dist'), { recursive: true }); + process.env.OS_HOME = join(dir, 'home'); + process.env.OS_ARTIFACT_PATH = join(dir, 'dist', 'objectstack.json'); + writeFileSync(process.env.OS_ARTIFACT_PATH, JSON.stringify(ARTIFACT)); + + // The database an older release left: tables provisioned, then the rows + // written at the driver the way that release stored them. Seeded on a boot + // that hydrates nothing — a hydrating one is refused over these rows. + templateDb = join(dir, 'template.db'); + const seed = await bootSchemaStack({ + jsonOutput: false, databaseUrl: `file:${templateDb}`, projectRoot: dir, hydrateMetadata: false, + }); + try { + const ql = seed.kernel.getService('objectql') as IObjectQLEngine; + const now = new Date().toISOString(); + for (const row of SEED) { + await ql.insert('sys_metadata', { + ...row, version: 1, checksum: null, created_at: now, updated_at: now, metadata: JSON.stringify(bodyOf(row)), + }, SYSTEM); + } + } finally { + await seed.shutdown(); + } +}, CASE_TIMEOUT_MS); + +afterAll(() => { + process.chdir(savedCwd); + for (const [key, value] of Object.entries(savedEnv)) { + if (value === undefined) delete process.env[key]; + else process.env[key] = value; + } + try { rmSync(dir, { recursive: true, force: true }); } catch { /* ignore */ } +}); + +// The command takes `process.cwd()` as the project root, as a real invocation does. +beforeEach(() => { process.chdir(dir); }); +afterEach(() => { process.chdir(savedCwd); setAuth(false); }); + +describe('os migrate security-catalog-overlays — the preview lists exactly what the cold boot refuses', () => { + it.each([ + { auth: false, flags: [], listed: LISTED_AUTH_OFF, securityPlugin: { composed: false, reason: 'no-secret' } }, + { auth: true, flags: [], listed: LISTED_AUTH_ON, securityPlugin: { composed: true } }, + // `serve --preset minimal` has no `auth` tier: with a secret set it composes + // no security plugin, so its sets are not held and their row is not listed. + { + auth: true, flags: ['--preset', 'minimal'], listed: LISTED_AUTH_OFF, + securityPlugin: { composed: false, reason: 'auth-tier-off' }, + }, + // `serve --dev` with no secret falls back to the development one and + // composes it (NODE_ENV is `production` throughout this file). + { auth: false, flags: ['--dev'], listed: LISTED_AUTH_ON, securityPlugin: { composed: true } }, + ])('auth $auth, flags $flags: the held rows are listed, the controls are not, and the list is the cold boot\'s conflicts', async ({ auth, flags, listed, securityPlugin }) => { + setAuth(auth); + const db = caseDb(); + const before = await storedRows(db); + const serveFlags = { + dev: flags.includes('--dev'), + ...(flags.includes('--preset') ? { preset: flags[flags.indexOf('--preset') + 1] } : {}), + }; + + // Over the compiled artifact that declares the held names, the cold boot + // of the same composition — the same flags — refuses; and the step, which + // hydrates nothing, runs. + const refused = await coldBootConflicts(db, serveFlags); + expect(refused.length, 'the cold boot is refused over these rows').toBeGreaterThan(0); + + const { payload, exitCode } = await runJson(['--database-url', `file:${db}`, '--json', ...flags]); + expect(payload.error, JSON.stringify(payload).slice(0, 600)).toBeUndefined(); + expect(payload.apply).toBe(false); + expect(payload.serveFlags).toEqual({ preset: serveFlags.preset ?? 'default', dev: serveFlags.dev }); + expect(payload.securityPlugin).toEqual(securityPlugin); + expect(listedOf(payload)).toEqual(listed); + expect(payload.rows.every((r: any) => r.outcome === 'listed')).toBe(true); + // The consistency the ruling names: the step's list is the refusal's conflicts. + expect(asConflicts(payload)).toEqual(refused); + // Rows listed: the boot would be refused, so the preview exits 1. + expect(exitCode).toBe(1); + // A preview writes nothing. + expect(await storedRows(db)).toEqual(before); + for (const control of CONTROLS) expect(before).toContain(control); + }, CASE_TIMEOUT_MS); +}); + +describe('os migrate security-catalog-overlays --apply — deletes exactly the listed rows, and the cold boot then comes up', () => { + it('auth on: every listed row deleted through the write path, one audit line each; the controls stay', async () => { + setAuth(true); + const db = caseDb(); + const before = await storedRows(db); + + const { payload, exitCode } = await runJson(['--apply', '--yes', '--database-url', `file:${db}`, '--json']); + expect(payload.error, JSON.stringify(payload).slice(0, 600)).toBeUndefined(); + expect(exitCode).toBe(0); + expect(listedOf(payload)).toEqual(LISTED_AUTH_ON); + expect(payload.deleted).toBe(LISTED_AUTH_ON.length); + expect(payload.failed).toBe(0); + // One outcome per row, naming the write path it took. + expect(payload.rows.map((r: any) => `${r.type}/${r.name} ${r.outcome} ${r.door}`)).toEqual([ + 'position/m22371_lead deleted protocol.deleteMetaItem', + 'positions/m22371_old_lead deleted sys-metadata-repository', + 'permission/m22371_rep deleted protocol.deleteMetaItem', + 'permissions/member_default deleted sys-metadata-repository', + ]); + + // Exactly the listed rows are gone; every control is still stored. + const after = await storedRows(db); + expect(before.filter((r) => !after.includes(r)).sort()).toEqual([ + 'permission/m22371_rep/env/active', + 'permissions/member_default/env/active', + 'position/m22371_lead/env/active', + 'positions/m22371_old_lead/env/active', + ]); + for (const control of CONTROLS) expect(after).toContain(control); + + // The engine's trail: a history tombstone per row, and the ADR-0010 audit + // row for each one the protocol's door deleted. + const { history, audit } = await trail(db); + expect(history).toEqual([ + 'permission/m22371_rep delete by os migrate security-catalog-overlays org=env', + 'permissions/member_default delete by os migrate security-catalog-overlays org=env', + 'position/m22371_lead delete by os migrate security-catalog-overlays org=env', + 'positions/m22371_old_lead delete by os migrate security-catalog-overlays org=env', + ]); + expect(audit).toEqual([ + 'permission/m22371_rep delete allowed by os migrate security-catalog-overlays', + 'position/m22371_lead delete allowed by os migrate security-catalog-overlays', + ]); + + // The cold boot of the same database and configuration that was refused now + // comes up, and a second preview lists nothing (exit 0). + expect(await coldBootConflicts(db)).toEqual([]); + const again = await runJson(['--database-url', `file:${db}`, '--json']); + expect(again.payload.listed).toBe(0); + expect(again.exitCode).toBe(0); + }, CASE_TIMEOUT_MS * 2); + + it('auth off: the security plugin\'s set is left alone — the boot without it never refused that row', async () => { + const db = caseDb(); + const { payload, exitCode } = await runJson(['--apply', '--yes', '--database-url', `file:${db}`, '--json']); + expect(payload.error, JSON.stringify(payload).slice(0, 600)).toBeUndefined(); + expect(exitCode).toBe(0); + expect(payload.deleted).toBe(LISTED_AUTH_OFF.length); + expect(await storedRows(db)).toContain('permissions/member_default/env/active'); + expect(await coldBootConflicts(db)).toEqual([]); + }, CASE_TIMEOUT_MS * 2); +}); diff --git a/packages/cli/src/commands/migrate/security-catalog-overlays.ts b/packages/cli/src/commands/migrate/security-catalog-overlays.ts new file mode 100644 index 00000000000..e950a8b6ec3 --- /dev/null +++ b/packages/cli/src/commands/migrate/security-catalog-overlays.ts @@ -0,0 +1,354 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { Command, Flags } from '@oclif/core'; +import { createInterface } from 'node:readline'; +import chalk from 'chalk'; +import { STACK_TIER_PRESETS } from '@objectstack/core'; +import { + printHeader, + printSuccess, + printWarning, + printError, + printInfo, + printStep, + createTimer, + emitJson, + errorCodeFields, + isExitSignal, +} from '../../utils/format.js'; +import { bootSchemaStack } from '../../utils/schema-migrate.js'; +import { describeSecurityPluginComposition, describeUnloadableHostConfig } from '../../utils/schema-migration-plugins.js'; +import { absentTableReads } from '../../utils/absent-table-reads.js'; +import { OCCUPANCY_HINT, probeMigrationTarget } from '../../utils/migrate-occupancy-gate.js'; +import { describeOccupancy } from '../../utils/sqlite-occupancy.js'; +import { + deleteSecurityCatalogOverlayRows, + listSecurityCatalogOverlayRows, + type SecurityCatalogOverlayOutcome, + type SecurityCatalogOverlayRow, +} from '../../utils/security-catalog-overlays.js'; + +async function confirm(question: string): Promise { + if (!process.stdin.isTTY) return false; // non-interactive → require --yes + const rl = createInterface({ input: process.stdin, output: process.stdout }); + try { + const answer: string = await new Promise((res) => rl.question(question, res)); + return /^y(es)?$/i.test(answer.trim()); + } finally { + rl.close(); + } +} + +/** One row as the report shows it, in both faces. */ +function describeRow(row: SecurityCatalogOverlayRow): string { + const label = row.catalogType === 'permission' ? 'permission set' : 'position'; + const spelling = row.storedType === row.catalogType ? '' : ` (stored under the legacy type '${row.storedType}')`; + const binding = row.packageId ? `bound to ${row.packageId}` : 'bound to no package'; + return `${label} "${row.name}"${spelling}, ${binding}, row ${row.id} — held by ${row.heldBy.join(', ')}`; +} + +function jsonRow(row: SecurityCatalogOverlayRow, outcome?: SecurityCatalogOverlayOutcome): Record { + return { + id: row.id, + type: row.storedType, + catalogType: row.catalogType, + name: row.name, + packageId: row.packageId, + heldBy: row.heldBy, + outcome: outcome?.outcome ?? 'listed', + ...(outcome?.outcome === 'deleted' ? { door: outcome.door } : {}), + ...(outcome?.outcome === 'failed' ? { error: outcome.error } : {}), + }; +} + +/** + * `os migrate security-catalog-overlays` — list, and with `--apply` delete, the + * environment-wide `sys_metadata` rows a v18 cold boot refuses: an active, + * environment-wide `permission` / `position` row (the legacy plurals included) + * whose name a configured package holds (ADR-0048 N.3). + * + * ## Why an offline step + * + * A deployment upgraded from 17.x can carry such rows — a permission set or a + * position saved in the environment over a package's name before the packaged + * locks refused that, or over one of the platform security plugin's own sets. + * v18's cold boot refuses to start over any of them, so nothing that needs the + * server can clear them; the metadata API's delete reaches no legacy-plural row + * at all. This step is the upgrade's supported remedy (maintainer ruling A′ on + * #22371, record 6073500921, item 1, as amended by ruling letter B, record + * 6074838935): run it before the first v18 boot, or after a refused one. + * + * ## How it answers exactly what the cold boot refuses + * + * It composes the deployment the way `os serve` does — the host config's + * plugins, the application or the compiled artifact, and the security plugin + * behind `serve`'s auth gate (`resolvePlatformAuthComposition`, + * `@objectstack/core`, the one rule both ask) — for the kernel's first phase + * only, and boots it WITHOUT `sys_metadata` hydration. The cold-boot check then + * meets an empty environment half and the boot comes up. The package-held names + * are read off the registry through the engine's own holder reading + * (`findPackageHeldSecurityCatalogNames`), and met with the stored rows. See + * `../../utils/security-catalog-overlays.ts` for the population and the write + * path. The security plugin's shipped sets join only when `serve` composes it, + * so the step reads the environment `serve` reads — `OS_AUTH_SECRET` above all + * — and says which way the gate went. + * + * `--preset` and `--dev` are `serve`'s two flags that move that composition, + * with `serve`'s meaning, read through the rules `serve` reads them by: + * `--preset` names the tier preset the auth gate's tiers fall back to + * (`resolveStackTiers`), and `--dev` merges the config's `devPlugins` + * (`stackBootPlugins`) and makes the boot a development one for the auth + * secret's fallback (`isDevelopmentBoot`). Pass the flags the deployment boots + * with. They move the composition only: this boot stays a one-shot boot (no + * dev schema self-heal, `NODE_ENV` untouched, no `.env*` file loaded), so the + * environment is the operator's to export as the deployment's. + * + * ## The family's conventions + * + * Preview by default; `--apply` is the only writing mode, confirmed with + * `[y/N]` or `--yes`; `--force` past a busy SQLite file; `--json` for one + * document; `--database-url` / `$OS_DATABASE_URL` for the target. The preview + * boots read-only (deferred DDL, no seed, no file brought into existence), and + * `--apply` boots plain, as `os migrate meta --stored --apply` does. + * + * Exit status: 0 when nothing is listed or everything listed was deleted; 1 + * when the preview lists a row (the boot would be refused), when a deletion + * failed, or when the composition could not be read. + */ +export default class MigrateSecurityCatalogOverlays extends Command { + static override description = + 'List, and with --apply delete, the environment-wide permission-set and position rows (legacy plurals ' + + 'included) over a name a configured package holds — the rows a v18 cold boot refuses. Run before the first ' + + 'v18 boot.'; + + static override examples = [ + '$ os migrate security-catalog-overlays', + '$ os migrate security-catalog-overlays --json', + '$ os migrate security-catalog-overlays --preset minimal', + '$ os migrate security-catalog-overlays --dev', + '$ os migrate security-catalog-overlays --apply', + '$ os migrate security-catalog-overlays --apply --yes --json', + ]; + + static override flags = { + 'database-url': Flags.string({ + description: 'Database to examine (defaults to $OS_DATABASE_URL / the project DB)', + env: 'OS_DATABASE_URL', + }), + apply: Flags.boolean({ + description: 'Delete the listed rows (default is a read-only preview)', + default: false, + }), + yes: Flags.boolean({ + char: 'y', + description: 'Skip the --apply confirmation prompt', + default: false, + }), + force: Flags.boolean({ + description: 'Apply even when another process is using the database (SQLite occupancy check)', + default: false, + }), + json: Flags.boolean({ description: 'Output the machine-readable result as JSON.' }), + // [#22371] `os serve`'s two flags that move the composition, with its meaning. + preset: Flags.string({ + description: 'Compose as `os serve --preset` does: the tier preset the auth tier falls back to when the stack ' + + 'declares no tiers. Pass what the deployment boots with.', + options: Object.keys(STACK_TIER_PRESETS), + }), + dev: Flags.boolean({ + description: 'Compose as `os serve --dev` does: the config\'s devPlugins, and a development boot for the auth ' + + 'secret\'s fallback. Pass it when the deployment boots with --dev (`os dev`).', + }), + }; + + async run(): Promise { + const { flags } = await this.parse(MigrateSecurityCatalogOverlays); + const timer = createTimer(); + const apply = flags.apply; + if (!flags.json) printHeader('Migrate · security-catalog-overlays'); + + // Occupancy gate — probed BEFORE boot (afterwards our own pool is what the + // probe finds) and before the prompt, so nobody confirms something we then + // refuse. The same gate `os migrate meta --stored --apply` keeps. + const occupancy = await probeMigrationTarget(flags['database-url']); + if (occupancy.status === 'busy' && apply && !flags.force) { + if (flags.json) { + await emitJson({ + error: 'database_busy', + database: occupancy.filename, + signal: occupancy.signal, + detail: occupancy.detail, + hint: OCCUPANCY_HINT, + }, 0, { compact: true }); + this.exit(1); + return; + } + printError(describeOccupancy(occupancy)); + printWarning(OCCUPANCY_HINT); + this.exit(1); + return; + } + if (occupancy.status === 'busy' && !flags.json) { + printWarning(apply + ? `--force: ${describeOccupancy(occupancy)} Deleting anyway — the live process may write metadata mid-run.` + : `${describeOccupancy(occupancy)} The preview below writes nothing.`); + } + if (occupancy.status === 'unknown' && !flags.json) { + printWarning(`Could not check whether the database is in use — ${occupancy.detail}`); + } + + if (apply && !flags.yes) { + if (flags.json || !process.stdin.isTTY) { + if (flags.json) { + await emitJson({ error: 'confirmation_required', hint: 'pass --yes' }, 0, { compact: true }); + this.exit(1); + return; + } + printWarning( + 'Apply mode deletes the listed sys_metadata rows — each through the metadata write path, with a ' + + 'history tombstone. Re-run with --yes to confirm, or run without --apply to preview.', + ); + this.exit(1); + return; + } + const ok = await confirm( + chalk.bold('\nDelete the environment-wide rows over package-held permission-set and position names? [y/N] '), + ); + if (!ok) { + printInfo('Aborted — nothing deleted.'); + return; + } + } + + if (!flags.json) { + printStep(apply ? 'Composing the deployment (APPLY mode, no hydration)…' : 'Composing the deployment (preview only, no hydration)…'); + } + + let stack; + try { + // The deployment's composition, as `os serve` composes it, for its first + // phase — and NO hydration, so the cold-boot check this step clears does + // not stop the boot. The preview boots read-only, like `os migrate plan`. + stack = await bootSchemaStack({ + jsonOutput: flags.json, + ...(flags['database-url'] ? { databaseUrl: flags['database-url'] } : {}), + composeHostStack: true, + composeAuthGatedSecurity: true, + serveFlags: { dev: flags.dev === true, ...(flags.preset ? { preset: flags.preset } : {}) }, + hydrateMetadata: false, + ...(apply ? {} : { deferSchemaDdl: true, readOnlyProbe: true }), + }); + } catch (error: any) { + if (flags.json) { await emitJson({ error: error.message, ...errorCodeFields(error) }, 0, { compact: true }); this.exit(1); return; } + printError(error.message || String(error)); + this.exit(1); + return; + } + + // Collected rather than thrown: `this.exit()` raises an oclif ExitError. + // Decide the code here, exit after the stack is down. + let exitCode = 0; + try { + exitCode = await this.report(stack, { + apply, + json: flags.json === true, + serveFlags: { preset: flags.preset ?? 'default', dev: flags.dev === true }, + }, timer); + } catch (error: any) { + if (isExitSignal(error)) throw error; + exitCode = 1; + if (flags.json) await emitJson({ error: error.message, ...errorCodeFields(error) }, 0, { compact: true }); + else printError(error.message || String(error)); + } finally { + await stack.shutdown(); + } + if (exitCode !== 0) this.exit(exitCode); + } + + /** List, delete under `--apply`, and report on the booted stack; answers the exit status. */ + private async report( + stack: Awaited>, + opts: { apply: boolean; json: boolean; serveFlags: { preset: string; dev: boolean } }, + timer: { elapsed: () => number; display: () => string }, + ): Promise { + const { apply, json, serveFlags } = opts; + const composition = stack.composition; + // A host config that exists and could not be read leaves the held set + // incomplete: listing would miss names and the boot would still refuse. + // Refused before any row is read or deleted. + const unloadable = describeUnloadableHostConfig(composition); + if (unloadable !== null) { + if (json) await emitJson({ error: 'host_config_unloadable', message: unloadable }, 0, { compact: true }); + else printError(unloadable); + return 1; + } + + // The preview's read-only boot measured whether `sys_metadata` exists; a + // table that does not exist stores no row (the `meta --stored` reading). + const reads = absentTableReads(stack); + const rows = reads.absent('sys_metadata') ? [] : await listSecurityCatalogOverlayRows(stack.kernel); + const outcomes = apply && rows.length > 0 ? await deleteSecurityCatalogOverlayRows(stack.kernel, rows) : []; + const deleted = outcomes.filter((o) => o.outcome === 'deleted').length; + const failed = outcomes.filter((o) => o.outcome === 'failed').length; + const exitCode = (apply ? failed > 0 : rows.length > 0) ? 1 : 0; + + if (json) { + reads.notice(true); + await emitJson({ + database: stack.dbLabel, + apply, + hostConfig: composition.hostConfigPath, + serveFlags, + securityPlugin: composition.securityPlugin ?? null, + listed: rows.length, + deleted, + failed, + rows: rows.map((row, i) => jsonRow(row, outcomes[i])), + duration: timer.elapsed(), + }); + return exitCode; + } + + printInfo(`Database: ${chalk.white(stack.dbLabel)}`); + if (composition.hostConfigPath) printInfo(`Host config: ${chalk.white(composition.hostConfigPath)}`); + printInfo(`Composed as: ${chalk.white(`os serve --preset ${serveFlags.preset}${serveFlags.dev ? ' --dev' : ''}`)}`); + reads.notice(false); + // What decided whether the platform security plugin's sets are held — the + // one composition fact that moves this list with the boot environment. + if (composition.securityPlugin) printInfo(describeSecurityPluginComposition(composition.securityPlugin)); + console.log(''); + if (rows.length === 0) { + printSuccess( + 'No environment-wide permission-set or position row is stored over a package-held name — ' + + `nothing here refuses the cold boot ${chalk.dim(`(${timer.display()})`)}`, + ); + return exitCode; + } + if (!apply) { + console.log(chalk.bold(`${rows.length} row(s) the cold boot refuses:`)); + for (const row of rows) console.log(` • ${describeRow(row)}`); + console.log(''); + printWarning( + 'A v18 boot of this deployment is refused while these rows are stored. Re-run with --apply to delete ' + + 'them (each through the metadata write path, with a history tombstone). Nothing is adopted: a row the ' + + 'environment needs under its own name must be re-created under a name no package holds.', + ); + return exitCode; + } + // One audit line per row: what was deleted, through which door, or why not. + for (const outcome of outcomes) { + if (outcome.outcome === 'deleted') { + console.log(` ${chalk.green('✓')} deleted ${describeRow(outcome.row)} ${chalk.dim(`(via ${outcome.door}, actor 'os migrate security-catalog-overlays')`)}`); + } else { + console.log(` ${chalk.red('✗')} not deleted: ${describeRow(outcome.row)} — ${outcome.error}`); + } + } + console.log(''); + if (failed > 0) { + printError(`${failed} of ${rows.length} row(s) could not be deleted; the cold boot still refuses them.`); + } else { + printSuccess(`Deleted ${deleted} row(s) — the cold boot no longer refuses them ${chalk.dim(`(${timer.display()})`)}`); + } + return exitCode; + } +} diff --git a/packages/cli/src/commands/serve.ts b/packages/cli/src/commands/serve.ts index d5e3bd6de32..d9b10ec232d 100644 --- a/packages/cli/src/commands/serve.ts +++ b/packages/cli/src/commands/serve.ts @@ -30,7 +30,19 @@ import { readEnvWithDeprecation, resolveTenancyPosture, resolveAllowDegradedTena import { PLATFORM_CAPABILITY_TOKENS, PLATFORM_ALWAYS_ON_CAPABILITIES, RETIRED_PLATFORM_CAPABILITY_GUIDANCE } from '@objectstack/spec/kernel'; // [#22301] The `requires` token → provider table and its exact identity match, // shared with `@objectstack/verify`'s `bootStack` — see `Serve.CAPABILITY_PROVIDERS`. -import { CAPABILITY_PROVIDERS, providesCapability, materializeStackPlugin, type CapabilitySpec } from '@objectstack/core'; +import { + CAPABILITY_PROVIDERS, + providesCapability, + materializeStackPlugin, + STACK_TIER_PRESETS, + CAPABILITY_TO_TIER as SHARED_CAPABILITY_TO_TIER, + resolveStackTiers, + resolveAuthSecret, + resolvePlatformAuthComposition, + stackSuppliesAuthPlugin, + isDevelopmentBoot, + type CapabilitySpec, +} from '@objectstack/core'; // The posture vocabulary, read from the package that DEFINES it (#5359) — the // boot gate's fix list enumerates the accepted values, and a second literal // list would be free to drift the day a posture is added. @@ -110,6 +122,7 @@ import { bundleDeclaresTranslations, resolveStackCollection, stackDeclaredCapabilities, + stackBootPlugins, } from '../utils/stack-collections.js'; import { redactConnectionUrl, describeDriverConnection, describeDriverSqliteFile } from '../utils/connection-display.js'; import { captureServedDatabaseFile, watchServedDatabaseFile } from '../utils/served-database-file.js'; @@ -1193,7 +1206,9 @@ export default class Serve extends Command { prebuilt: Flags.boolean({ description: 'Skip esbuild/bundle-require — load config as native ESM (production mode)', default: false }), preset: Flags.string({ description: 'Plugin tier preset: minimal | default | full (overridden by config.tiers if set)', - options: ['minimal', 'default', 'full'], + // [#22371] The keys of the table `resolveStackTiers` reads — the same + // options `os migrate security-catalog-overlays --preset` offers. + options: Object.keys(STACK_TIER_PRESETS), }), 'log-level': Flags.string({ description: 'Kernel logger level. Defaults to $OS_LOG_LEVEL / $LOG_LEVEL, else `warn` so flow/hook execution failures surface (ADR-0032). Boot-phase warnings are replayed under the startup banner; `debug`/`info` stream the whole boot live instead. Use `silent` to fully quiet the runtime.', @@ -1323,12 +1338,13 @@ export default class Serve extends Command { * Auto-registered plugin tiers. Plugins explicitly listed in * `config.plugins` are always loaded — tiers only gate the optional * auto-registration blocks below (AIService, I18n, UI portals, etc.). + * + * [#22371] A handle over `STACK_TIER_PRESETS` (`@objectstack/core`), where the + * tier rule now lives beside the auth gate it feeds: `os migrate + * security-catalog-overlays` answers that gate as this command does. One + * declaration, several readers — ⛔ never a second copy here. */ - static readonly TIER_PRESETS: Record = { - minimal: ['core'], - default: ['core', 'i18n', 'ui', 'ai', 'auth'], - full: ['core', 'i18n', 'ui', 'ai', 'auth'], - }; + static readonly TIER_PRESETS: Readonly> = STACK_TIER_PRESETS; /** * True when a dynamic `import()` / `require.resolve()` failed because the @@ -1566,16 +1582,11 @@ export default class Serve extends Command { * `requires`. These have no CAPABILITY_PROVIDERS entry — their loading is * resolved by dedicated blocks in run() (`ai`/`ai-studio` by the intent-driven * AI block (#1597), `i18n`/`ui`/`auth` by their tier blocks). + * + * [#22371] A handle over `CAPABILITY_TO_TIER` (`@objectstack/core`), for the + * reason `TIER_PRESETS` gives. */ - static readonly CAPABILITY_TO_TIER: Record = { - ai: 'ai', - // `ai-studio` (AI-driven authoring) rides on the base AI service, so - // requiring it opens the same `ai` tier (#1597). - 'ai-studio': 'ai', - i18n: 'i18n', - ui: 'ui', - auth: 'auth', - }; + static readonly CAPABILITY_TO_TIER: Readonly> = SHARED_CAPABILITY_TO_TIER; /** * Is one of `identities` ALREADY loaded — i.e. did the app supply this @@ -2158,7 +2169,10 @@ export default class Serve extends Command { } const tenancyPosture = postureGate.posture; - const isDev = flags.dev || process.env.NODE_ENV === 'development'; + // [#22371] `--dev`, or NODE_ENV exactly `development` — `isDevelopmentBoot` + // (`@objectstack/core`), the reading the auth secret's development fallback + // is decided by here and in `os migrate security-catalog-overlays --dev`. + const isDev = isDevelopmentBoot(flags.dev); // Resolves the config path AND anchors every host-anchored load in this // file at the app that owns it (#11185) — one call, so the anchor cannot be @@ -2654,7 +2668,6 @@ export default class Serve extends Command { // and shadow any capability resolution (i.e. an explicit // instance wins over the auto-loader). const presetName = flags.preset ?? (isDev ? 'default' : 'default'); - const presetTiers = Serve.TIER_PRESETS[presetName] ?? Serve.TIER_PRESETS.default; // Dedupe `requires` (Set keeps first-seen order). The deprecated // `aiStudio`/`aiSeat` alias canonicalization was removed in framework#3308 // — legacy spellings are now unknown tokens (warned below, rejected at @@ -2732,17 +2745,17 @@ export default class Serve extends Command { // in `requires`. Capabilities NOT in that map (e.g. `automation`, // `analytics`, `audit`) bypass tier gating and are loaded directly by // the capability-resolver block further down. - const CAPABILITY_TO_TIER = Serve.CAPABILITY_TO_TIER; - const requiredTiers = requires - .map((c) => CAPABILITY_TO_TIER[c]) - .filter((t): t is string => typeof t === 'string'); + // // [#22288] `tiers` is package-owned like `requires`, and no boot path // lays it over the top level (`createStandaloneStack` does not carry it), // so a package's own `tiers` were ignored on `os serve`, `os dev` and // `os start` alike. Same rule as `requires` above. const declaredTiers = resolveStackCollection(config, 'tiers') as string[]; - const baseTiers = declaredTiers.length > 0 ? declaredTiers : presetTiers; - const tiers: Set = new Set([...baseTiers, ...requiredTiers]); + // [#22371] The precedence itself — declared tiers, else the preset's, plus + // the tier each `requires` token opens — is `resolveStackTiers` + // (`@objectstack/core`), the rule the auth gate below is fed by in this + // command and in `os migrate security-catalog-overlays` alike. + const tiers: Set = resolveStackTiers({ declaredTiers, requires, preset: presetName }); const tierEnabled = (t: string) => tiers.has(t); const requiresCapability = (c: string) => requires.includes(c); @@ -3009,13 +3022,10 @@ export default class Serve extends Command { if (sduiManifestLine) console.warn(chalk.yellow(` ⚠ ${sduiManifestLine}`)); } - // Load plugins from configuration - let plugins = config.plugins || []; - - // Merge devPlugins if in dev mode - if (flags.dev && config.devPlugins) { - plugins = [...plugins, ...config.devPlugins]; - } + // Load plugins from configuration, and merge devPlugins if in dev mode — + // `stackBootPlugins` [#22371], the list `os migrate + // security-catalog-overlays --dev` composes too. + let plugins: any[] = stackBootPlugins(config, flags.dev); // 1. Auto-register ObjectQL Plugin if objects define but plugins missing // [#15006] The whole gate — the `objects` read AND the already-composed @@ -3794,19 +3804,31 @@ export default class Serve extends Command { // @objectstack/plugin-auth. Without this block, running // `objectstack dev` on a vanilla user stack would 404 on // login/register flows. - const hasAuthPlugin = plugins.some( - (p: any) => p?.name === 'com.objectstack.auth' || p?.constructor?.name === 'AuthPlugin' - ); + // + // [#22371] WHETHER this block composes is ONE rule, + // `resolvePlatformAuthComposition` (`@objectstack/core`), which + // `os migrate security-catalog-overlays` asks too: the security plugin + // paired below declares its shipped permission sets on its manifest, so + // this gate decides which names the cold boot's catalog check holds + // against the environment. The gate, in order: the stack mounts no + // AuthPlugin of its own, the `auth` tier is on, the composition is not a + // host kernel, and an auth secret resolves. The first two skip this block + // silently — the `if` below asks them through the rule's own predicate + // and tier set — and the last two are reported inside it, as before; HOW + // the plugins are constructed stays this command's. + const hasAuthPlugin = stackSuppliesAuthPlugin(plugins); + const authComposition = resolvePlatformAuthComposition({ + plugins, + tiers, + // In dev, the rule falls back to a stable local secret so users don't + // have to set OS_AUTH_SECRET just to try the login/register flow. + secret: resolveAuthSecret({ isDev }), + }); if (!hasAuthPlugin && tierEnabled('auth')) { try { const authPkg = '@objectstack/plugin-auth'; const { AuthPlugin } = await import(/* webpackIgnore: true */ authPkg); - // In dev, fall back to a stable local secret so users don't have - // to set OS_AUTH_SECRET just to try the login/register flow. - const secret = readEnvWithDeprecation('OS_AUTH_SECRET', ['AUTH_SECRET', 'BETTER_AUTH_SECRET'], { silent: true }) - ?? (isDev ? 'dev-only-insecure-secret-change-me-in-production' : undefined); - // Guard: in cloud-connected runtime mode (e.g. objectos worker) // the host kernel is a pure routing shell. Auth is owned by each // per-project kernel (`ArtifactKernelFactory` injects an @@ -3821,18 +3843,15 @@ export default class Serve extends Command { // reliable signal — a regular `objectstack dev` app may set it // just to enable the marketplace proxy yet still want its own // local AuthPlugin. - const isHostKernel = plugins.some( - (p: any) => p?.name === 'com.objectstack.runtime.objectos-environment' - || p?.constructor?.name === 'ObjectOSEnvironmentPlugin' - ); - if (isHostKernel) { + if (!authComposition.composes && authComposition.reason === 'host-kernel') { console.warn(chalk.yellow( ' ⚠ AuthPlugin skipped on host kernel — runtime mode (ObjectOSEnvironmentPlugin detected).\n' + ' Auth is owned per-project by ArtifactKernelFactory in the cloud distribution.' )); - } else if (!secret) { + } else if (!authComposition.composes) { console.warn(chalk.yellow(' ⚠ AuthPlugin skipped — set OS_AUTH_SECRET to enable authentication in production')); } else { + const secret = authComposition.secret; // Resolution is UNCHANGED (same chain, same precedence) — the seam // additionally reports where the value came from and whether it // parses, so an unusable one can be said out loud instead of diff --git a/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts b/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts index 905837cdaa4..fbc15c72fe7 100644 --- a/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts +++ b/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts @@ -65,6 +65,7 @@ import MigrateMultiValueColumns from '../commands/migrate/multi-value-columns.js import MigratePlan from '../commands/migrate/plan.js'; import MigrateRecordedBy from '../commands/migrate/recorded-by.js'; import MigrateResume from '../commands/migrate/resume.js'; +import MigrateSecurityCatalogOverlays from '../commands/migrate/security-catalog-overlays.js'; import MigrateSummaryNulls from '../commands/migrate/summary-nulls.js'; import MigrateUnmappedColumns from '../commands/migrate/unmapped-columns.js'; import MigrateValueShapes from '../commands/migrate/value-shapes.js'; @@ -207,6 +208,16 @@ const CALLERS: Record = { // unknown. The boot is what this case is about. write: [{ label: 'migrate resume --run', argv: ['--run', 'run_absent_21391', '--yes', '--database-url', '@DB@', '--json'] }], }, + 'commands/migrate/security-catalog-overlays.ts': { + run: invoke(MigrateSecurityCatalogOverlays), + // The fixture's app declares no catalog name, so the run lists nothing and + // `--apply` deletes nothing — the boot is what these cases are about. + noWrite: [{ label: 'migrate security-catalog-overlays', argv: ['--database-url', '@DB@', '--json'] }], + write: [{ + label: 'migrate security-catalog-overlays --apply', + argv: ['--apply', '--yes', '--database-url', '@DB@', '--json'], + }], + }, 'commands/migrate/summary-nulls.ts': { run: invoke(MigrateSummaryNulls), noWrite: [{ label: 'migrate summary-nulls', argv: ['--database-url', '@DB@', '--json'] }], diff --git a/packages/cli/src/utils/schema-migrate.ts b/packages/cli/src/utils/schema-migrate.ts index e981dfe615c..d18361861d5 100644 --- a/packages/cli/src/utils/schema-migrate.ts +++ b/packages/cli/src/utils/schema-migrate.ts @@ -429,6 +429,41 @@ export async function bootSchemaStack( * whatever directory the test runner happens to be standing in (#4065). */ projectRoot?: string; + /** + * [#22371] Boot WITHOUT reading `sys_metadata` back into the registry. + * Default `true` — every caller before this one diffs and scans the object + * set the serving boot registers, and that includes what it hydrates. + * + * `false` is the boot `os migrate security-catalog-overlays` takes: it + * lists the environment-wide rows the cold boot's catalog check refuses + * (ADR-0048 N.3), and that check judges what hydration wrote, so a boot of + * such a deployment that hydrates is refused before the command can run. + * Hydration off, the check meets an empty environment half and the boot + * comes up; the command reads the rows itself (maintainer ruling letter B + * on #22371, record 6074838935). + */ + hydrateMetadata?: boolean; + /** + * [#22371] With {@link composeHostStack}, also compose the security plugin + * when `os serve` would — behind its auth gate (`resolvePlatformAuthComposition` + * in `@objectstack/core`, the one rule both answer) — for its declarations + * only. Its `init()` declares the platform's shipped permission sets on its + * own manifest, which is what makes them package-held names at boot. + * + * Off by default and opted into at the call site, like `composeHostStack`: + * `os migrate plan` / `apply` compose no tier-gated plugin, and this does + * not change that. See `buildSchemaMigrationPlugins`'s `authGatedSecurity`. + */ + composeAuthGatedSecurity?: boolean; + /** + * [#22371] With {@link composeHostStack}: the `os serve` flags the + * composition answers for — `--dev` (the config's `devPlugins`, and a + * development boot for the auth gate) and `--preset` (the tiers the gate + * falls back to), with `serve`'s meaning. See `buildSchemaMigrationPlugins`'s + * `serveFlags`. Only for the composition: the data stack itself is booted as + * every one-shot command boots it — no `dev` key, so no dev schema self-heal. + */ + serveFlags?: { readonly dev?: boolean; readonly preset?: string }; }, ): Promise { // Taken BEFORE the first line the boot can print. `createStandaloneStack` @@ -486,6 +521,8 @@ export async function bootSchemaStack( // repaired. `duplicates.integration.test.ts` pins this end of it: boot // included, the run must leave the database byte-identical. runPlatformMigrations: false, + // [#22371] Off only when the caller asks; see the option. + ...(opts.hydrateMetadata === false ? { hydrateMetadataFromDb: false } : {}), }); // No HTTP, no cluster — this is a one-shot schema operation. @@ -508,6 +545,12 @@ export async function bootSchemaStack( cwd: opts.projectRoot ?? process.cwd(), // The same answer the standalone stack got above: never on this boot. skipSeedData: true, + // [#22371] The compiled artifact's `requires`, as `serve`'s merge lays + // them over the config's, for the auth gate's tier reading. + ...(opts.composeAuthGatedSecurity === true + ? { authGatedSecurity: { artifactRequires: stack.requires } } + : {}), + ...(opts.serveFlags ? { serveFlags: opts.serveFlags } : {}), }) : { plugins: [], hostConfigPath: null, hostConfigLoaded: false, hostConfigError: null, diff --git a/packages/cli/src/utils/schema-migration-plugins.test.ts b/packages/cli/src/utils/schema-migration-plugins.test.ts index c59c74f20dd..d266b911d19 100644 --- a/packages/cli/src/utils/schema-migration-plugins.test.ts +++ b/packages/cli/src/utils/schema-migration-plugins.test.ts @@ -246,6 +246,37 @@ describe('buildSchemaMigrationPlugins', () => { expect(out.notes.join(' ')).toContain('declarations only'); }); + it('[#22371] --dev composes the config\'s devPlugins, as `serve --dev` registers them — and its auth gate reads them', async () => { + const dir = tempProject(); + writeFileSync( + join(dir, 'objectstack.config.ts'), + [ + "class DemoPlugin { name = 'com.example.demo'; async init() {} }", + // A dev-only plugin under the name `serve`'s auth gate looks for: present, + // the stack supplies its own AuthPlugin and the platform composes none. + "class DevAuth { name = 'com.objectstack.auth'; async init() {} }", + 'export default { plugins: [new DemoPlugin()], devPlugins: [new DevAuth()] };', + '', + ].join('\n'), + ); + + const plain = await buildSchemaMigrationPlugins({ basePlugins: [], cwd: dir, authGatedSecurity: {} }); + expect(plain.plugins.map((p: any) => p?.name)).not.toContain('com.objectstack.auth'); + expect(plain.securityPlugin).not.toEqual({ composed: false, reason: 'stack-supplies-auth' }); + + const dev = await buildSchemaMigrationPlugins({ + basePlugins: [], cwd: dir, authGatedSecurity: {}, serveFlags: { dev: true }, + }); + expect(dev.plugins.map((p: any) => p?.name)).toEqual([ + WRITE_GUARD, + 'com.example.demo', + 'com.objectstack.auth', + 'com.objectstack.platform-objects', + ]); + expect(dev.securityPlugin).toEqual({ composed: false, reason: 'stack-supplies-auth' }); + expect(dev.notes.join(' ')).toContain('2 plugin(s) (1 of them its devPlugins, as under --dev)'); + }, 60_000); + it('reports a config it could not load rather than pretending it composed one', async () => { const dir = tempProject(); writeFileSync( diff --git a/packages/cli/src/utils/schema-migration-plugins.ts b/packages/cli/src/utils/schema-migration-plugins.ts index 24a52ccaca5..543e562ac6a 100644 --- a/packages/cli/src/utils/schema-migration-plugins.ts +++ b/packages/cli/src/utils/schema-migration-plugins.ts @@ -2,8 +2,21 @@ import path from 'node:path'; import fs from 'node:fs'; +import { + isDevelopmentBoot, + resolveAuthSecret, + resolvePlatformAuthComposition, + resolveStackTiers, + type PlatformAuthSkipReason, +} from '@objectstack/core'; import { isAppPluginLike } from './graft-runtime-hooks.js'; -import { stackDeclaredCapabilities, stackDeclaresMetadata } from './stack-collections.js'; +import { isHostConfig } from './plugin-detection.js'; +import { + resolveStackCollection, + stackBootPlugins, + stackDeclaredCapabilities, + stackDeclaresMetadata, +} from './stack-collections.js'; /** * The object set a SCHEMA migration is planned against (#12938). @@ -61,6 +74,14 @@ import { stackDeclaredCapabilities, stackDeclaresMetadata } from './stack-collec * posture ({@link resolveRequiredProviders}). Without it every config that * lists a connector in `plugins` and `automation` in `requires` — the blank * template and the showcase among them — could not boot this command. + * - **The security plugin behind `serve`'s auth gate — only when the caller + * asks** (`authGatedSecurity`, #22371). `os migrate + * security-catalog-overlays` needs the permission-set names the deployment's + * boot holds, and the security plugin declares its shipped sets only where + * `serve`'s auth gate composes it. The gate is the one rule `serve` asks + * (`resolvePlatformAuthComposition`, `@objectstack/core`), so this is not an + * invented object set: it is the one this deployment's boot has. `plan` and + * `apply` do not ask, and compose nothing tier-gated, as before. * * ## Phase 1 only for host plugins — and why that is the contract, not a dodge * @@ -1241,6 +1262,16 @@ export interface SchemaMigrationComposition { * kernel bootstrap returns. */ lifecycle?: DeclarationBootLifecycle; + /** + * [#22371] What the auth gate answered when the caller asked for the + * security plugin `os serve` composes behind it (`authGatedSecurity`) — + * `undefined` when it did not ask. `composed: false` names the gate's reason, + * or `config-unloadable` when the host config exists and could not be read, + * so the gate had nothing to read. + */ + securityPlugin?: + | { readonly composed: true } + | { readonly composed: false; readonly reason: PlatformAuthSkipReason | 'config-unloadable' }; } const NOTHING_COMPOSED: SchemaMigrationComposition = Object.freeze({ @@ -1415,6 +1446,28 @@ export async function buildSchemaMigrationPlugins(opts: { basePlugins: readonly unknown[]; cwd?: string; skipSeedData?: boolean; + /** + * [#22371] Also compose the security plugin `os serve` composes behind its + * auth gate, for its declarations only — see {@link composeAuthGatedSecurity}. + * `artifactRequires` is the compiled artifact's `requires` as the standalone + * stack surfaced it. Unset (every caller but `os migrate + * security-catalog-overlays`): nothing tier-gated is composed, as before. + */ + authGatedSecurity?: { artifactRequires?: readonly string[] }; + /** + * [#22371] The `os serve` flags this composition answers for, with `serve`'s + * meaning — `os migrate security-catalog-overlays`' `--dev` and `--preset`: + * + * - `dev`: the host config's `devPlugins` are composed after its `plugins` + * (`stackBootPlugins`, the merge `serve` registers), and the boot is a + * development one for the auth gate's secret fallback + * (`isDevelopmentBoot`, `@objectstack/core`); + * - `preset`: the tier preset the auth gate's tiers fall back to when the + * stack declares none (`resolveStackTiers`). + * + * Unset: `serve` with neither flag. Every other caller passes none. + */ + serveFlags?: { readonly dev?: boolean; readonly preset?: string }; }): Promise { const cwd = opts.cwd ?? process.cwd(); const hostConfigPath = findHostConfig(cwd); @@ -1424,7 +1477,10 @@ export async function buildSchemaMigrationPlugins(opts: { // to mirror, and the five-table data stack is the honest answer. Returning // early — rather than composing the platform floor anyway — is what keeps an // artifact-less, config-less run byte-identical to the one before this card. - if (!hostConfigPath && !hasArtifactApp) return NOTHING_COMPOSED; + // [#22371] Unless the caller asked for the auth-gated security plugin: a + // bare `os serve` composes it on such a project too, and its shipped sets + // are package-held names there as anywhere. + if (!hostConfigPath && !hasArtifactApp && !opts.authGatedSecurity) return NOTHING_COMPOSED; // #13332 — armed FIRST, so its `init()` is ordered ahead of every host // plugin's: `resolvePluginOrder` is a DFS in registration order, and a host @@ -1441,6 +1497,9 @@ export async function buildSchemaMigrationPlugins(opts: { // #21732 — the config's capability declarations, read off the same loaded // config the plugins came from. See {@link resolveRequiredProviders}. let loadedRequires: unknown; + // [#22371] The loaded config and its own plugins, for the auth gate below. + let loadedConfig: unknown; + let loadedHostPlugins: readonly unknown[] = []; if (hostConfigPath) { try { @@ -1456,7 +1515,11 @@ export async function buildSchemaMigrationPlugins(opts: { // top-level read left a plugin's hard dependency on that provider // unordered and the plan exited 1. loadedRequires = stackDeclaredCapabilities(config); - const hostPlugins: unknown[] = Array.isArray(config?.plugins) ? config.plugins : []; + // [#22371] `serve`'s list — the config's `plugins`, and its `devPlugins` + // under `--dev` — by `serve`'s own merge. + const hostPlugins: unknown[] = stackBootPlugins(config, opts.serveFlags?.dev); + loadedConfig = config; + loadedHostPlugins = hostPlugins; for (const plugin of hostPlugins) { if (plugin && typeof plugin === 'object') plugins.push(composeForDeclarations(plugin, lifecycle)); } @@ -1490,9 +1553,12 @@ export async function buildSchemaMigrationPlugins(opts: { // is only known once the boot is over, so that sentence belongs to the // guard's disarm() note — which drops it if a raw execute() went // through (#14053 review, R1) — and must not be pre-claimed here. + const devPluginCount = hostPlugins.length - stackBootPlugins(config, false).length; notes.push( `Composed the host stack from ${path.relative(cwd, hostConfigPath) || hostConfigPath}: ` - + `${hostPlugins.length} plugin(s), registered for their declarations only ` + + `${hostPlugins.length} plugin(s)` + + `${devPluginCount > 0 ? ` (${devPluginCount} of them its devPlugins, as under --dev)` : ''}, ` + + 'registered for their declarations only ' + '(init runs, start does not), with the contract\'s row writes refused at the ' + 'driver for the whole boot.', ); @@ -1550,9 +1616,132 @@ export async function buildSchemaMigrationPlugins(opts: { notes.push(...resolved.notes); } + // [#22371] The security plugin `serve` pairs with AuthPlugin, behind the + // same gate — composed ahead of the host plugins, where `serve` registers it, + // so a config's own instance of the same name supersedes it here as there. + let securityPlugin: SchemaMigrationComposition['securityPlugin']; + if (opts.authGatedSecurity) { + if (hostConfigPath && !hostConfigLoaded) { + securityPlugin = { composed: false, reason: 'config-unloadable' }; + } else { + const gated = await composeAuthGatedSecurity({ + config: loadedConfig, + hostPlugins: loadedHostPlugins, + basePlugins: opts.basePlugins, + artifactRequires: opts.authGatedSecurity.artifactRequires, + ...(opts.serveFlags ? { serveFlags: opts.serveFlags } : {}), + lifecycle, + }); + securityPlugin = gated.status; + if (gated.plugin) plugins.splice(1, 0, gated.plugin); + notes.push(gated.note); + } + } + return { plugins, hostConfigPath, hostConfigLoaded, hostConfigError, notes, coverage: null, writeGuard, lifecycle, + ...(securityPlugin ? { securityPlugin } : {}), + }; +} + +/** + * [#22371] The security plugin `os serve` composes for this deployment, for its + * declarations only — or why it composes none. + * + * `serve` composes it in its auth step, paired with `AuthPlugin`, behind the + * gate `resolvePlatformAuthComposition` (`@objectstack/core`) states: the stack + * mounts no `AuthPlugin` of its own, the `auth` tier is on, the composition is + * not a host kernel, and an auth secret resolves. Its `init()` declares the + * platform's shipped permission sets on its own manifest, so behind the gate + * they are package-held names, and the cold boot's catalog check refuses an + * environment row over one of them; outside it they are not. Measured on + * #22371: one database and config were refused on 3 names with + * `OS_AUTH_SECRET` set and on 2 without it. So the gate is read from what + * `serve` reads, in `serve`'s own spelling: + * + * - the plugins: the host config's and the standalone stack's (`serve`'s + * `plugins` is one or the other — a host config's own, or the stack merged + * over an app config — and the gate asks only for two names among them); + * - the plugins under `--dev`: the config's `devPlugins` too, merged by + * `serve`'s own rule (`stackBootPlugins`) before they reach this gate; + * - the tiers: the config's declared `tiers` (no boot carries an artifact's), + * and the `requires` `serve` reads — a host config's own; otherwise the + * compiled artifact's when it declares any, which `serve`'s merge lays over + * the config's, else the config's — over the `--preset` the caller passed + * (`resolveStackTiers`, `serve`'s own precedence; the default preset when + * none). Of `serve`'s `requires` additions (`email`, `mcp`, the always-on + * slate, `queue` / `job`) none maps to the `auth` tier, so the gate's + * answer does not move with them; + * - the secret and the development fallback: `isDevelopmentBoot`, `serve`'s + * `isDev` — `--dev`, or `NODE_ENV=development`. + * + * `AuthPlugin` itself is not composed: it holds no catalog name, and its + * `init()` is not this boot's to run. The security plugin is wrapped by + * {@link composeForDeclarations}, like a host plugin: its `start()` seeds the + * shipped sets into `sys_permission_set`, which this boot must not do. + */ +async function composeAuthGatedSecurity(input: { + config: unknown; + hostPlugins: readonly unknown[]; + basePlugins: readonly unknown[]; + artifactRequires?: readonly string[]; + serveFlags?: { readonly dev?: boolean; readonly preset?: string }; + lifecycle: DeclarationBootLifecycle; +}): Promise<{ + plugin?: unknown; + status: NonNullable; + note: string; +}> { + const config = input.config ?? {}; + const requires = !isHostConfig(config) && Array.isArray(input.artifactRequires) + ? input.artifactRequires.filter((t): t is string => typeof t === 'string') + : stackDeclaredCapabilities(config); + const declaredTiers = resolveStackCollection(config, 'tiers').filter((t): t is string => typeof t === 'string'); + const decision = resolvePlatformAuthComposition({ + plugins: [...input.basePlugins, ...input.hostPlugins], + tiers: resolveStackTiers({ + declaredTiers, + requires, + ...(input.serveFlags?.preset !== undefined ? { preset: input.serveFlags.preset } : {}), + }), + secret: resolveAuthSecret({ isDev: isDevelopmentBoot(input.serveFlags?.dev) }), + }); + if (!decision.composes) { + const status = { composed: false, reason: decision.reason } as const; + return { status, note: describeSecurityPluginComposition(status) }; + } + const { SecurityPlugin, appSecurityPluginOptions } = await import('@objectstack/plugin-security'); + const status = { composed: true } as const; + return { + plugin: composeForDeclarations(new SecurityPlugin(appSecurityPluginOptions(config)), input.lifecycle), + status, + note: describeSecurityPluginComposition(status), + }; +} + +/** + * [#22371] What {@link SchemaMigrationComposition.securityPlugin} says, as an + * operator reads it — the line the composition notes carry, exported so the + * command that asked for it can print the same sentence. + */ +export function describeSecurityPluginComposition( + status: NonNullable, +): string { + if (status.composed) { + return 'Composed the security plugin as `os serve` composes it behind its auth gate (an auth secret ' + + 'resolves), for its declarations only: its shipped permission sets are package-held names on this ' + + 'deployment\'s boot.'; + } + const why: Record = { + 'stack-supplies-auth': 'the stack mounts its own AuthPlugin', + 'auth-tier-off': 'the `auth` tier is off: the declared `tiers`, else the preset\'s, carry none, and no ' + + '`requires` token opens it', + 'host-kernel': 'a host kernel, whose auth is per project', + 'no-secret': 'no OS_AUTH_SECRET, and not a development boot: no --dev, and NODE_ENV is not development', + 'config-unloadable': 'the host config could not be loaded, so the gate had nothing to read', }; + return `Did not compose the security plugin: \`os serve\` composes none here (${why[status.reason]}), so its ` + + 'shipped permission sets are not package-held names on this deployment\'s boot.'; } /** diff --git a/packages/cli/src/utils/security-catalog-overlays.ts b/packages/cli/src/utils/security-catalog-overlays.ts new file mode 100644 index 00000000000..cf852a3fef4 --- /dev/null +++ b/packages/cli/src/utils/security-catalog-overlays.ts @@ -0,0 +1,268 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The rows `os migrate security-catalog-overlays` lists and deletes, and how it + * deletes them (maintainer ruling letter B on #22371, record 6074838935, + * amending item 1 of ruling A′, record 6073500921). + * + * ## The population + * + * Every active, environment-wide `sys_metadata` row (`organization_id IS NULL`, + * `state = 'active'`) of type `permission` or `position` — the legacy plurals + * `permissions` / `positions` included, which the boot's load folds to the same + * types — whose name a configured package holds. That is exactly what the cold + * boot's catalog check refuses (ADR-0048 N.3): the check meets the package-held + * names with what `sys_metadata` hydration put in the registry's bare slot, and + * hydration loads precisely these rows. This module meets the SAME package-held + * reading, exported by the engine for this purpose + * (`findPackageHeldSecurityCatalogNames`), with the rows read directly, on a boot + * that hydrated nothing — so it lists what the cold boot would refuse, and it + * holds no second reading of "who holds a name". + * + * ⛔ A draft row and an organization-scoped row are not loaded at boot and do + * not refuse it, so they are never listed. ⛔ Nothing is adopted: `managed_by` + * and `package_id` are never rewritten (the Discard Overlay scope ruled + * 2026-08-20); a listed row is deleted, or left alone. + * + * ## The deletion: the engine's audited write path + * + * A row stored under the canonical type goes through the metadata protocol's + * own delete, `deleteMetaItem` — the door `DELETE /api/v1/meta/:type/:name` + * takes: one transaction that removes the row and appends a + * `sys_metadata_history` tombstone, a `sys_metadata_audit` row, the mutation + * projector and the watch event. That door addresses a row by `(type, name)` + * and removes one row per call; where two rows share the name (one bound to no + * package, one bound to the package), each call removes one, and which row went + * is read back by id rather than assumed. + * + * A row stored under a legacy plural is out of that door's reach: it folds the + * type it is given to the canonical one before reading, and the audit trail + * refuses a non-canonical type by design. Such a row goes through the layer + * beneath it, the `SysMetadataRepository` delete, addressed by its stored + * spelling, its name, its `package_id` and its checksum: the same transaction + * and the same history tombstone (under the spelling the row carried), without + * the audit row. Each outcome names the door it took. + */ + +import type { SecurityCatalogType } from '@objectstack/core'; + +/** The command's own name, the `actor` its deletions record. */ +export const SECURITY_CATALOG_OVERLAYS_ACTOR = 'os migrate security-catalog-overlays'; + +/** The stored spellings of each catalog type the boot's load folds together. */ +const STORED_TYPE_SPELLINGS: ReadonlyArray = [ + ['position', 'position'], + ['positions', 'position'], + ['permission', 'permission'], + ['permissions', 'permission'], +]; + +/** One listed row. */ +export interface SecurityCatalogOverlayRow { + /** The `sys_metadata` row id. */ + readonly id: string; + /** The type as stored: `permission` / `position`, or a legacy plural. */ + readonly storedType: string; + /** The catalog type the boot folds it to. */ + readonly catalogType: 'permission' | 'position'; + readonly name: string; + /** The row's own package binding (`package_id`), `null` when it has none. */ + readonly packageId: string | null; + /** The configured packages that hold the name — what the cold boot names as the incoming side. */ + readonly heldBy: readonly string[]; + /** The stored checksum, the optimistic lock a repository delete is taken under. */ + readonly checksum: string | null; +} + +/** What a deletion did with one listed row. */ +export type SecurityCatalogOverlayOutcome = + | { + readonly row: SecurityCatalogOverlayRow; + readonly outcome: 'deleted'; + /** The write path the deletion took (module header). */ + readonly door: 'protocol.deleteMetaItem' | 'sys-metadata-repository'; + } + | { readonly row: SecurityCatalogOverlayRow; readonly outcome: 'failed'; readonly error: string }; + +const SYSTEM = { context: { isSystem: true } } as const; + +interface EngineLike { + registry: unknown; + find(object: string, query: Record, options?: Record): Promise; +} + +function engineOf(kernel: { getService(name: string): unknown }): EngineLike { + const engine = kernel.getService('objectql') as EngineLike | undefined; + if (!engine || typeof engine.find !== 'function' || !engine.registry) { + throw new Error( + 'No ObjectQL engine on this stack, so the package-held names cannot be read. ' + + 'Run this from the project root whose stack registers the ObjectQL engine.', + ); + } + return engine; +} + +/** Every active environment-wide row of one stored spelling, ordered by name then id. */ +async function storedRows(engine: EngineLike, storedType: string): Promise { + const rows = await engine.find( + 'sys_metadata', + { where: { type: storedType, organization_id: null, state: 'active' } }, + SYSTEM, + ); + return [...rows].sort((a, b) => + String(a.name).localeCompare(String(b.name)) || String(a.id).localeCompare(String(b.id))); +} + +/** + * The listed rows (module header, "The population"), positions first, then by + * name, then by stored spelling (canonical first). + * + * @param kernel a kernel booted with the deployment's composition and WITHOUT + * `sys_metadata` hydration, its packages registered. + */ +export async function listSecurityCatalogOverlayRows( + kernel: { getService(name: string): unknown }, +): Promise { + const engine = engineOf(kernel); + const { findPackageHeldSecurityCatalogNames } = await import('@objectstack/objectql'); + const held = new Map(); + for (const entry of findPackageHeldSecurityCatalogNames(engine.registry as never)) { + held.set(`${entry.catalogType}|${entry.name}`, entry.packageIds); + } + if (held.size === 0) return []; + + const listed: SecurityCatalogOverlayRow[] = []; + for (const [storedType, catalogType] of STORED_TYPE_SPELLINGS) { + for (const row of await storedRows(engine, storedType)) { + const name = String(row.name); + const heldBy = held.get(`${catalogType}|${name}`); + if (!heldBy) continue; + listed.push({ + id: String(row.id), + storedType, + catalogType, + name, + packageId: typeof row.package_id === 'string' && row.package_id !== '' ? row.package_id : null, + heldBy, + checksum: typeof row.checksum === 'string' ? row.checksum : null, + }); + } + } + const typeOrder: Record = { position: 0, permission: 1, capability: 2 }; + return listed.sort((a, b) => + typeOrder[a.catalogType] - typeOrder[b.catalogType] + || a.name.localeCompare(b.name) + || Number(a.storedType !== a.catalogType) - Number(b.storedType !== b.catalogType) + || a.id.localeCompare(b.id)); +} + +/** Which of `ids` are still stored. */ +async function stillStored(engine: EngineLike, storedType: string, name: string, ids: readonly string[]): Promise> { + const rows = await engine.find( + 'sys_metadata', + { where: { type: storedType, name, organization_id: null, state: 'active' } }, + SYSTEM, + ); + const present = new Set(rows.map((r: any) => String(r.id))); + return new Set(ids.filter((id) => present.has(id))); +} + +function messageOf(error: unknown): string { + return error instanceof Error ? error.message : String(error); +} + +/** + * Delete `rows` through the engine's audited write path (module header, "The + * deletion"), one outcome per row, in the order given. Never throws for a row: + * a refusal is that row's `failed` outcome, and the next row is still tried. + */ +export async function deleteSecurityCatalogOverlayRows( + kernel: { getService(name: string): unknown }, + rows: readonly SecurityCatalogOverlayRow[], +): Promise { + const engine = engineOf(kernel); + const protocol = kernel.getService('protocol') as { + deleteMetaItem?: (request: Record) => Promise<{ success?: boolean; reset?: boolean; message?: string }>; + } | undefined; + const outcomes = new Map(); + + // Canonical spellings: the protocol's delete, one call per row of a name. + const canonical = new Map(); + for (const row of rows) { + if (row.storedType !== row.catalogType) continue; + const key = `${row.catalogType}|${row.name}`; + canonical.set(key, [...(canonical.get(key) ?? []), row]); + } + for (const group of canonical.values()) { + const { catalogType, name } = group[0]!; + const pending = new Map(group.map((r) => [r.id, r])); + if (typeof protocol?.deleteMetaItem !== 'function') { + for (const row of group) { + outcomes.set(row.id, { row, outcome: 'failed', error: 'No metadata protocol on this stack, so the row cannot be deleted through it.' }); + } + continue; + } + for (let attempt = 0; attempt < group.length && pending.size > 0; attempt++) { + let refusal: string | undefined; + try { + const receipt = await protocol.deleteMetaItem({ + type: catalogType, + name, + state: 'active', + actor: SECURITY_CATALOG_OVERLAYS_ACTOR, + }); + if (receipt?.reset !== true) refusal = receipt?.message ?? 'the delete removed no row'; + } catch (error) { + refusal = messageOf(error); + } + const remaining = await stillStored(engine, catalogType, name, [...pending.keys()]); + for (const [id, row] of [...pending]) { + if (remaining.has(id)) continue; + outcomes.set(id, { row, outcome: 'deleted', door: 'protocol.deleteMetaItem' }); + pending.delete(id); + } + if (refusal !== undefined) { + for (const row of pending.values()) outcomes.set(row.id, { row, outcome: 'failed', error: refusal }); + pending.clear(); + break; + } + } + for (const row of pending.values()) { + outcomes.set(row.id, { row, outcome: 'failed', error: 'the delete reported success, and the row is still stored' }); + } + } + + // Legacy plural spellings: the repository beneath that door, row by row. + const plural = rows.filter((row) => row.storedType !== row.catalogType); + if (plural.length > 0) { + const { SysMetadataRepository } = await import('@objectstack/metadata-protocol'); + const repository = new SysMetadataRepository({ engine: engine as never, organizationId: null, orgLabel: 'env' }); + for (const row of plural) { + try { + await repository.delete( + { type: row.storedType, name: row.name, org: 'env' } as never, + { + // The repository's own lock: the stored checksum, and a null one + // against a row that carries none (`lockAccepts`). + parentVersion: row.checksum as string, + actor: SECURITY_CATALOG_OVERLAYS_ACTOR, + source: SECURITY_CATALOG_OVERLAYS_ACTOR, + message: `Deleted the environment-wide ${row.storedType}/${row.name} row over a name ` + + `${row.heldBy.join(', ')} holds (ADR-0048 N.3)`, + intent: 'override-artifact', + state: 'active', + packageId: row.packageId, + }, + ); + const remaining = await stillStored(engine, row.storedType, row.name, [row.id]); + outcomes.set(row.id, remaining.has(row.id) + ? { row, outcome: 'failed', error: 'the delete returned, and the row is still stored' } + : { row, outcome: 'deleted', door: 'sys-metadata-repository' }); + } catch (error) { + outcomes.set(row.id, { row, outcome: 'failed', error: messageOf(error) }); + } + } + } + + return rows.map((row) => outcomes.get(row.id) ?? { row, outcome: 'failed', error: 'not attempted' }); +} diff --git a/packages/cli/src/utils/stack-collections.test.ts b/packages/cli/src/utils/stack-collections.test.ts index 95503636b81..e14aa4ae68a 100644 --- a/packages/cli/src/utils/stack-collections.test.ts +++ b/packages/cli/src/utils/stack-collections.test.ts @@ -20,6 +20,7 @@ import { resolveStackCollection, shouldAutoRegisterObjectQL, shouldAutoRegisterStorageDriver, + stackBootPlugins, stackDeclaresMetadata, } from './stack-collections.js'; @@ -246,3 +247,24 @@ describe('#15006 — the union author-time rule input', () => { .toEqual(['probe_account', 'probe_order']); }); }); + +describe('#22371 — stackBootPlugins, `os serve`\'s `plugins` and its `--dev` merge', () => { + const own = [{ name: 'own' }]; + const dev = [{ name: 'dev-only' }]; + + it('without --dev: the stack\'s own array itself, devPlugins left out', () => { + expect(stackBootPlugins({ plugins: own, devPlugins: dev }, false)).toBe(own); + expect(stackBootPlugins({ plugins: own, devPlugins: dev }, undefined)).toBe(own); + }); + + it('with --dev: the devPlugins after the own plugins; none declared, the own array itself', () => { + expect(stackBootPlugins({ plugins: own, devPlugins: dev }, true)).toEqual([...own, ...dev]); + expect(stackBootPlugins({ devPlugins: dev }, true)).toEqual(dev); + expect(stackBootPlugins({ plugins: own }, true)).toBe(own); + }); + + it('no plugins at all: an empty list', () => { + expect(stackBootPlugins({}, false)).toEqual([]); + expect(stackBootPlugins(undefined, true)).toEqual([]); + }); +}); diff --git a/packages/cli/src/utils/stack-collections.ts b/packages/cli/src/utils/stack-collections.ts index 4916edb2d18..f00cd5707b3 100644 --- a/packages/cli/src/utils/stack-collections.ts +++ b/packages/cli/src/utils/stack-collections.ts @@ -140,6 +140,21 @@ function hasStorageDriverPlugin(plugins: readonly unknown[]): boolean { }); } +/** + * [#22371] `os serve`'s `plugins`: the stack's own, and under `--dev` its + * `devPlugins` after them — the list `serve` registers, and the one its auth + * gate reads. `os migrate security-catalog-overlays --dev` composes the same + * list, so the merge is declared once, here. Keyed on the FLAG, as on `serve`: + * `NODE_ENV=development` alone merges nothing. + * + * With nothing to merge, the answer is the stack's own array itself. + */ +export function stackBootPlugins(stack: unknown, dev: boolean | undefined): unknown[] { + const bag = asBag(stack); + const plugins = (bag?.plugins || []) as unknown[]; + return dev && bag?.devPlugins ? [...plugins, ...(bag.devPlugins as unknown[])] : plugins; +} + /** * `os serve` step 1 — should this boot auto-register the ObjectQL engine? * diff --git a/packages/cli/test/normalized-call-sites.test.ts b/packages/cli/test/normalized-call-sites.test.ts index de6830f4c7a..c57f5215171 100644 --- a/packages/cli/test/normalized-call-sites.test.ts +++ b/packages/cli/test/normalized-call-sites.test.ts @@ -440,6 +440,10 @@ const READS: Readonly> = { evidence: [{ in: 'commands/i18n/extract.ts', code: 'extractTranslations(authoringRuleUnionStack(normalized' }], why: 'its caller hands it the union (#22238).', }, + 'utils/schema-migrate.ts :: stack.requires': { + reads: 'top-level', + why: 'the `createStandaloneStack` RESULT, not an authored stack: its top-level `requires` is the one the runtime already read off the resolved artifact (`resolveArtifactCollections`, ADR-0130 D4 option B), so on a two-package artifact it is every body\'s tokens. Handed to the auth gate of `os migrate security-catalog-overlays` as the field `serve`\'s merge lays over the config (#22371).', + }, 'utils/scaffold-wiring.ts :: config.requires': { reads: 'top-level', why: 'a presence probe only: `declaredCapabilities` answers `[]` rather than `null` for a stack whose top level declares an empty `requires`, so the printed line says it replaces that key. The tokens come from `stackDeclaredCapabilities` (#22288).', diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 500be3c30ae..ebca4d0e295 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -28,6 +28,10 @@ export * from './capability-providers.js'; // [#22301] And the rule for the app's own `plugins` array, which both boots // mount: what a string, bundle or instance entry becomes. Same reason. export * from './stack-plugins.js'; +// [#22371] And the rule that decides whether a boot composes the platform auth +// family (and the security plugin paired with it): `os serve` and +// `os migrate security-catalog-overlays` answer it alike. Same reason. +export * from './stack-auth.js'; export * from './lite-kernel.js'; export * from './types.js'; export * from './logger.js'; diff --git a/packages/core/src/stack-auth.test.ts b/packages/core/src/stack-auth.test.ts new file mode 100644 index 00000000000..752d6567999 --- /dev/null +++ b/packages/core/src/stack-auth.test.ts @@ -0,0 +1,120 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * #22371 — the ONE rule for whether a boot composes the platform auth family + * (and the security plugin paired with it), read by `os serve` and by + * `os migrate security-catalog-overlays`. One case per clause of the rule, in + * its order; the environment reads (the secret, NODE_ENV) are driven through `process.env` + * and restored. + */ + +import { afterEach, describe, expect, it } from 'vitest'; +import { + CAPABILITY_TO_TIER, + DEV_AUTH_SECRET_FALLBACK, + STACK_TIER_PRESETS, + isDevelopmentBoot, + isHostKernelComposition, + resolveAuthSecret, + resolvePlatformAuthComposition, + resolveStackTiers, + stackSuppliesAuthPlugin, +} from './stack-auth.js'; + +const ENV_VARS = ['OS_AUTH_SECRET', 'AUTH_SECRET', 'BETTER_AUTH_SECRET', 'NODE_ENV'] as const; +const saved = Object.fromEntries(ENV_VARS.map((k) => [k, process.env[k]])); +afterEach(() => { + for (const k of ENV_VARS) { + if (saved[k] === undefined) delete process.env[k]; + else process.env[k] = saved[k]; + } +}); +const clearSecrets = () => { for (const k of ENV_VARS) delete process.env[k]; }; + +describe('resolveStackTiers — declared tiers, else the preset, plus the tiers `requires` opens', () => { + it('no declared tiers: the default preset, which carries `auth`', () => { + expect([...resolveStackTiers({ declaredTiers: [], requires: [] })]).toEqual([...STACK_TIER_PRESETS.default]); + expect(resolveStackTiers({ declaredTiers: [], requires: [] }).has('auth')).toBe(true); + }); + + it('an unknown preset name falls back to the default; `minimal` has no `auth`', () => { + expect(resolveStackTiers({ declaredTiers: [], requires: [], preset: 'nope' }).has('auth')).toBe(true); + expect(resolveStackTiers({ declaredTiers: [], requires: [], preset: 'minimal' }).has('auth')).toBe(false); + }); + + it('declared tiers replace the preset, and a `requires` token reopens its tier', () => { + expect(resolveStackTiers({ declaredTiers: ['core'], requires: [] }).has('auth')).toBe(false); + expect(resolveStackTiers({ declaredTiers: ['core'], requires: ['auth'] }).has('auth')).toBe(true); + expect(resolveStackTiers({ declaredTiers: ['core'], requires: ['ai-studio'] }).has('ai')).toBe(true); + // A token with no tier (resolved by the capability providers instead) opens none. + expect([...resolveStackTiers({ declaredTiers: ['core'], requires: ['automation'] })]).toEqual(['core']); + expect(CAPABILITY_TO_TIER.automation).toBeUndefined(); + }); +}); + +describe('resolveAuthSecret — OS_AUTH_SECRET, its legacy spellings, else the development fallback', () => { + it('the preferred spelling wins; a legacy spelling answers when it is absent', () => { + clearSecrets(); + process.env.BETTER_AUTH_SECRET = 'legacy'; + expect(resolveAuthSecret({ isDev: false })).toBe('legacy'); + process.env.OS_AUTH_SECRET = 'preferred'; + expect(resolveAuthSecret({ isDev: false })).toBe('preferred'); + }); + + it('none set: the fallback in development only', () => { + clearSecrets(); + expect(resolveAuthSecret({ isDev: true })).toBe(DEV_AUTH_SECRET_FALLBACK); + expect(resolveAuthSecret({ isDev: false })).toBeUndefined(); + }); +}); + +describe('isDevelopmentBoot — `--dev`, or NODE_ENV exactly `development`', () => { + it('the flag answers alone; without it only NODE_ENV=development is a development boot', () => { + delete process.env.NODE_ENV; + expect(isDevelopmentBoot(true)).toBe(true); + expect(isDevelopmentBoot(false)).toBe(false); + expect(isDevelopmentBoot(undefined)).toBe(false); + process.env.NODE_ENV = 'production'; + expect(isDevelopmentBoot(true)).toBe(true); + expect(isDevelopmentBoot(undefined)).toBe(false); + process.env.NODE_ENV = 'development'; + expect(isDevelopmentBoot(undefined)).toBe(true); + }); +}); + +describe('resolvePlatformAuthComposition — the gate, in `serve`\'s order', () => { + const on = new Set(['core', 'auth']); + const authPlugin = { name: 'com.objectstack.auth', init: async () => {} }; + class AuthPlugin { init = async () => {}; } + const hostKernel = { name: 'com.objectstack.runtime.objectos-environment', init: async () => {} }; + + it('composes, with the secret it runs on, when every clause holds', () => { + expect(resolvePlatformAuthComposition({ plugins: [], tiers: on, secret: 's3cret' })) + .toEqual({ composes: true, secret: 's3cret' }); + }); + + it('a stack mounting AuthPlugin itself (by name or class) composes none — first, whatever else holds', () => { + expect(stackSuppliesAuthPlugin([authPlugin])).toBe(true); + expect(stackSuppliesAuthPlugin([new AuthPlugin()])).toBe(true); + expect(resolvePlatformAuthComposition({ plugins: [new AuthPlugin(), hostKernel], tiers: new Set(), secret: undefined })) + .toEqual({ composes: false, reason: 'stack-supplies-auth' }); + }); + + it('the auth tier off composes none, ahead of the host-kernel and secret clauses', () => { + expect(resolvePlatformAuthComposition({ plugins: [hostKernel], tiers: new Set(['core']), secret: undefined })) + .toEqual({ composes: false, reason: 'auth-tier-off' }); + }); + + it('a host kernel composes none, ahead of the secret clause', () => { + expect(isHostKernelComposition([hostKernel])).toBe(true); + expect(resolvePlatformAuthComposition({ plugins: [hostKernel], tiers: on, secret: undefined })) + .toEqual({ composes: false, reason: 'host-kernel' }); + }); + + it('no secret composes none — an empty one included', () => { + expect(resolvePlatformAuthComposition({ plugins: [], tiers: on, secret: undefined })) + .toEqual({ composes: false, reason: 'no-secret' }); + expect(resolvePlatformAuthComposition({ plugins: [], tiers: on, secret: '' })) + .toEqual({ composes: false, reason: 'no-secret' }); + }); +}); diff --git a/packages/core/src/stack-auth.ts b/packages/core/src/stack-auth.ts new file mode 100644 index 00000000000..929337d3065 --- /dev/null +++ b/packages/core/src/stack-auth.ts @@ -0,0 +1,184 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { readEnvWithDeprecation } from '@objectstack/types'; + +/** + * The ONE rule that decides whether a boot composes the platform's auth family + * — `AuthPlugin` and, paired with it, the security plugin — for a stack that + * does not mount `AuthPlugin` itself. Read by `os serve` (`@objectstack/cli`, + * its "5d. Auto-register AuthPlugin (and paired Security/Audit)" step) and by + * `os migrate security-catalog-overlays`, which composes what `serve` composes + * for the first phase, under the same `--preset` and `--dev`. + * + * ## Why this lives in `@objectstack/core` + * + * The auth gate decides which permission-set names a deployment's packages hold + * at boot: the security plugin declares its shipped sets (`member_default`, + * `admin_full_access` and the rest) on its own manifest in `init()`, and it is + * composed only behind this gate. The cold-boot check refuses an environment + * row over a package-held name, so the offline step that lists those rows must + * answer the gate exactly as `serve` does — measured on #22371: one database + * and one config were refused on 3 names with `OS_AUTH_SECRET` set and on 2 + * without it. Maintainer ruling letter B on #22371 (record 6074838935): the + * step composes the deployment "by `serve`'s own rules (the configuration's + * plugins, the application or the compiled artifact, the security plugin under + * `serve`'s auth gating)". So the gate is one declaration with two readers, in + * the package both already depend on — the reason `stack-plugins.ts` and + * `capability-providers.ts` give for their own home. ⛔ Never a second copy of + * it beside a reader. + * + * ## The rule + * + * The tiers first ({@link resolveStackTiers}): the stack's declared `tiers` + * when it declares any, otherwise the preset's; plus the tier every `requires` + * token opens ({@link CAPABILITY_TO_TIER}). Then the gate + * ({@link resolvePlatformAuthComposition}), in `serve`'s order: + * + * 1. the stack mounts `AuthPlugin` itself → the platform composes none; + * 2. the `auth` tier is off → none; + * 3. the composition is a host kernel (`ObjectOSEnvironmentPlugin`, the cloud + * runtime: auth belongs to each per-project kernel there) → none; + * 4. no auth secret ({@link resolveAuthSecret}: `OS_AUTH_SECRET`, its two legacy + * spellings, or — on a development boot ({@link isDevelopmentBoot}) — the + * development fallback) → none; + * 5. otherwise the platform composes `AuthPlugin`, and the security plugin + * beside it. + * + * What each boot does with the answer stays with that boot: `serve` constructs + * `AuthPlugin` with its origins, providers and cookies, and warns on 3 and 4; + * the offline step composes the security plugin for its declarations alone. + * + * Pure apart from the environment reads of {@link resolveAuthSecret} and + * {@link isDevelopmentBoot}: importing this module loads nothing. + */ + +/** + * The tier presets `os serve --preset` names — and `os migrate + * security-catalog-overlays --preset`, the same flag over the same table. Each + * command's flag lists these keys as its options. `default` is the fallback + * for an unknown or absent name. + */ +export const STACK_TIER_PRESETS: Readonly> = Object.freeze({ + minimal: Object.freeze(['core']), + default: Object.freeze(['core', 'i18n', 'ui', 'ai', 'auth']), + full: Object.freeze(['core', 'i18n', 'ui', 'ai', 'auth']), +}); + +/** + * The `requires` tokens that open a tier when listed. A token not in this map + * (`automation`, `analytics`, `audit` …) bypasses tier gating: its provider is + * loaded by the capability resolver ({@link CAPABILITY_PROVIDERS}) instead. + */ +export const CAPABILITY_TO_TIER: Readonly> = Object.freeze({ + ai: 'ai', + // `ai-studio` (AI-driven authoring) rides on the base AI service, so + // requiring it opens the same `ai` tier (#1597). + 'ai-studio': 'ai', + i18n: 'i18n', + ui: 'ui', + auth: 'auth', +}); + +/** + * The tiers a boot enables. Precedence: the stack's declared `tiers` when it + * declares any, otherwise the named preset's (`default` for an unknown or + * absent name); every `requires` token that maps to a tier adds it. + */ +export function resolveStackTiers(input: { + readonly declaredTiers: readonly string[]; + readonly requires: readonly string[]; + readonly preset?: string; +}): Set { + const presetTiers = STACK_TIER_PRESETS[input.preset ?? 'default'] ?? STACK_TIER_PRESETS.default; + const baseTiers = input.declaredTiers.length > 0 ? input.declaredTiers : presetTiers; + const requiredTiers = input.requires + .map((token) => CAPABILITY_TO_TIER[token]) + .filter((tier): tier is string => typeof tier === 'string'); + return new Set([...baseTiers, ...requiredTiers]); +} + +/** Does the stack mount `AuthPlugin` itself? By registered name or class name, as `serve` has always asked. */ +export function stackSuppliesAuthPlugin(plugins: readonly unknown[]): boolean { + return plugins.some((p) => { + const plugin = p as { name?: unknown; constructor?: { name?: unknown } } | null | undefined; + return plugin?.name === 'com.objectstack.auth' || plugin?.constructor?.name === 'AuthPlugin'; + }); +} + +/** + * Is this composition a host kernel — the cloud runtime's routing shell, marked + * by `ObjectOSEnvironmentPlugin`? `OS_CLOUD_URL` alone is not the signal: a + * regular app may set it for the marketplace proxy and still want its own auth. + */ +export function isHostKernelComposition(plugins: readonly unknown[]): boolean { + return plugins.some((p) => { + const plugin = p as { name?: unknown; constructor?: { name?: unknown } } | null | undefined; + return plugin?.name === 'com.objectstack.runtime.objectos-environment' + || plugin?.constructor?.name === 'ObjectOSEnvironmentPlugin'; + }); +} + +/** + * The auth secret a development boot falls back to, so trying the login flow + * locally does not need `OS_AUTH_SECRET`. Never used outside development. + */ +export const DEV_AUTH_SECRET_FALLBACK = 'dev-only-insecure-secret-change-me-in-production'; + +/** + * Is this a development boot? `--dev` was passed, or `NODE_ENV` is exactly + * `development` — `os serve`'s `isDev`, read where the flag is parsed. An + * unset `NODE_ENV` is not development (`serve` defaults it to `production`). + * It decides the auth secret's development fallback ({@link resolveAuthSecret}), + * which is why `os migrate security-catalog-overlays --dev` reads it here + * rather than spelling it again. + */ +export function isDevelopmentBoot(devFlag: boolean | undefined): boolean { + return devFlag === true || process.env.NODE_ENV === 'development'; +} + +/** + * The auth secret this boot runs with: `OS_AUTH_SECRET`, else its legacy + * spellings `AUTH_SECRET` / `BETTER_AUTH_SECRET` (read silently), else — on a + * development boot only ({@link isDevelopmentBoot}) — + * {@link DEV_AUTH_SECRET_FALLBACK}; `undefined` when none. + */ +export function resolveAuthSecret(input: { readonly isDev: boolean }): string | undefined { + return readEnvWithDeprecation('OS_AUTH_SECRET', ['AUTH_SECRET', 'BETTER_AUTH_SECRET'], { silent: true }) + ?? (input.isDev ? DEV_AUTH_SECRET_FALLBACK : undefined); +} + +/** Why a boot composes no platform auth family. */ +export type PlatformAuthSkipReason = + /** The stack mounts `AuthPlugin` in its own `plugins`. */ + | 'stack-supplies-auth' + /** The `auth` tier is not enabled. */ + | 'auth-tier-off' + /** A host kernel: auth is owned per project there. */ + | 'host-kernel' + /** No auth secret, and not a development boot. */ + | 'no-secret'; + +/** The gate's answer: compose the platform auth family (with the secret it runs on), or why not. */ +export type PlatformAuthComposition = + | { readonly composes: true; readonly secret: string } + | { readonly composes: false; readonly reason: PlatformAuthSkipReason }; + +/** + * Does this boot compose the platform auth family — `AuthPlugin`, and the + * security plugin paired with it? The rule is this module's header; the checks + * run in `serve`'s order, so the reason is the first one that holds. + */ +export function resolvePlatformAuthComposition(input: { + /** The stack's own `plugins`, after the boot's merge with the standalone stack. */ + readonly plugins: readonly unknown[]; + /** The tiers the boot enabled ({@link resolveStackTiers}). */ + readonly tiers: ReadonlySet; + /** The secret the boot resolved ({@link resolveAuthSecret}). */ + readonly secret: string | undefined; +}): PlatformAuthComposition { + if (stackSuppliesAuthPlugin(input.plugins)) return { composes: false, reason: 'stack-supplies-auth' }; + if (!input.tiers.has('auth')) return { composes: false, reason: 'auth-tier-off' }; + if (isHostKernelComposition(input.plugins)) return { composes: false, reason: 'host-kernel' }; + if (!input.secret) return { composes: false, reason: 'no-secret' }; + return { composes: true, secret: input.secret }; +} diff --git a/packages/objectql/src/index.ts b/packages/objectql/src/index.ts index 3a2ede5e0e6..b8c18c12874 100644 --- a/packages/objectql/src/index.ts +++ b/packages/objectql/src/index.ts @@ -84,6 +84,16 @@ export { OBJECT_OWNERSHIP_CONFLICT_CODE, } from './registry.js'; +// [#22371] The package half of the cold-boot check's holder reading (ADR-0048 +// N.3): every package-held position and permission-set name, with its holders. +// Exported for `os migrate security-catalog-overlays`, which boots the +// deployment's composition without hydrating `sys_metadata` and meets this +// list with the stored rows — the population the cold boot refuses — so the +// CLI holds no second reading of "who holds a name" (maintainer ruling letter B +// on #22371, record 6074838935). The environment half and the refusal itself +// stay private to the engine. +export { findPackageHeldSecurityCatalogNames } from './registry.js'; + // [#14553] The navigation-contribution group diagnostic (ADR-0029 D7, // ADR-0112 D6c). Exported because `os build` is the SECOND door that has to diff --git a/packages/objectql/src/protocol-boot-hydration-scoped.test.ts b/packages/objectql/src/protocol-boot-hydration-scoped.test.ts index 1129f4cb55a..1a46821732b 100644 --- a/packages/objectql/src/protocol-boot-hydration-scoped.test.ts +++ b/packages/objectql/src/protocol-boot-hydration-scoped.test.ts @@ -23,7 +23,7 @@ import { describe, it, expect, afterEach } from 'vitest'; import { ObjectStackProtocolImplementation } from '@objectstack/metadata-protocol'; import { MetadataManager } from '@objectstack/metadata'; import { createSecurityCatalogReader, ObjectKernel, type Plugin, type PluginContext } from '@objectstack/core'; -import { SchemaRegistry, NAMESPACE_CONFLICT_CODE } from './registry.js'; +import { SchemaRegistry, NAMESPACE_CONFLICT_CODE, findPackageHeldSecurityCatalogNames } from './registry.js'; import { assertEngineUpdateDispatch } from './engine-update-dispatch.js'; import { assertEngineFindOnePredicate } from './engine-findone-predicate.js'; import { ObjectQLPlugin } from './plugin.js'; @@ -459,4 +459,49 @@ describe('cold boot — a package-held catalog name the environment catalog hold expect(refusal).toBeUndefined(); expect(ql.registry.getItem<{ label?: string }>('position', 'org_admin')?.label).toBe('saved in the environment'); }); + + /** + * [#22371, ruling letter B, record 6074838935] The package half of the + * reading, exported for `os migrate security-catalog-overlays`, which boots + * without hydration and meets it with the stored rows instead of the bare + * slot. One reading, so the step lists what the cold boot refuses. + */ + describe('the package half of the reading, exported (findPackageHeldSecurityCatalogNames)', () => { + it('lists every package-held name with its holder, positions first; built-ins and the environment\'s own names are not in it', async () => { + const { refusal, ql } = await boot( + pkgOf({ position: ['coldboot_held_position'], permission: ['coldboot_held_set_b', 'coldboot_held_set_a'] }), + [storedRow('permission', 'coldboot_env_only_set')], + (engine) => { + engine.registry.registerItem('position', { name: 'org_admin', label: 'Organization Admin' }, 'name', 'com.objectstack.plugin-security'); + }, + ); + expect(refusal).toBeUndefined(); + expect(findPackageHeldSecurityCatalogNames(ql.registry)).toEqual([ + { catalogType: 'position', name: 'coldboot_held_position', packageIds: [PKG] }, + { catalogType: 'permission', name: 'coldboot_held_set_a', packageIds: [PKG] }, + { catalogType: 'permission', name: 'coldboot_held_set_b', packageIds: [PKG] }, + ]); + }); + + it('the cold boot\'s conflicts are exactly that list met with the stored names, no more and no fewer', async () => { + const stored = [ + storedRow('permission', 'coldboot_meet_set'), + storedRow('position', 'coldboot_meet_position', PKG), + storedRow('permission', 'coldboot_meet_env_only'), + ]; + const { refusal, ql } = await boot( + pkgOf({ position: ['coldboot_meet_position'], permission: ['coldboot_meet_set', 'coldboot_meet_unstored'] }), + stored, + ); + const storedNames = new Set(stored.map((r) => `${r.type}|${r.name}`)); + const met = findPackageHeldSecurityCatalogNames(ql.registry) + .filter((h) => storedNames.has(`${h.catalogType}|${h.name}`)) + .flatMap((h) => h.packageIds.map((packageId) => ({ + catalogType: h.catalogType, name: h.name, incomingPackageId: packageId, existingHolder: { kind: 'environment' }, + }))); + expect(met).toHaveLength(2); + expect(refusal?.code).toBe(NAMESPACE_CONFLICT_CODE); + expect(refusal?.conflicts).toEqual(met); + }); + }); }); diff --git a/packages/objectql/src/registry.ts b/packages/objectql/src/registry.ts index f3b9e430e66..4649c161d79 100644 --- a/packages/objectql/src/registry.ts +++ b/packages/objectql/src/registry.ts @@ -1629,8 +1629,8 @@ function coldBootConflictMessage(conflicts: readonly SecurityCatalogNameConflict `assignment names a position or a permission set by its bare name, so with two holders the environment's ` + `stored item would be served in place of the package's definition. The environment catalog loads from ` + `sys_metadata before this check, so the boot is refused. Rename the item in the package, or rename or ` + - `delete the environment's item (its environment-wide sys_metadata row: through the metadata API on a boot ` + - `that leaves the package out of the configuration, or in the database), then restart. ` + + `delete the environment's item (its environment-wide sys_metadata row: \`os migrate security-catalog-overlays\` ` + + `lists every such row with no server running, and with --apply deletes them), then restart. ` + `See ADR-0048.` ); } @@ -1722,6 +1722,32 @@ export function findEnvironmentHeldSecurityCatalogNames(registry: SchemaRegistry return registry['environmentHeldSecurityCatalogConflicts'](); } +/** + * Every position and permission-set name a package holds, with the packages + * that hold it, read off `registry` — the PACKAGE half of the reading the + * cold-boot check refuses with ({@link findEnvironmentHeldSecurityCatalogNames} + * is this list met with the environment catalog's bare slot, and nothing else). + * + * Public for the one consumer that has the other half somewhere the registry + * does not: `os migrate security-catalog-overlays` (`@objectstack/cli`) boots + * the deployment's composition with `sys_metadata` hydration off, so the cold + * boot's environment half is empty there, and meets this list with the stored + * rows instead (maintainer ruling letter B on #22371, record 6074838935: one + * reading of "who holds a name", never a second one beside the cold boot's). + * + * The rule each entry obeys is the cold boot's, because it is the same private + * reading: a package holds a name through an item registered under it (its + * composite slot) or its install claim, never through the bare slot; only the + * types the environment catalog can hold are read; built-in names are skipped. + * Sorted by type (positions first), then name, as the cold-boot refusal lists + * its conflicts. Read-only: it records nothing. + */ +export function findPackageHeldSecurityCatalogNames( + registry: SchemaRegistry, +): Array<{ readonly catalogType: SecurityCatalogType; readonly name: string; readonly packageIds: readonly string[] }> { + return registry['packageHeldSecurityCatalogNames'](); +} + /** * [ADR-0130 D1] What the install gate is told about the artifact now installing. * @@ -2490,28 +2516,61 @@ export class SchemaRegistry { * * Sorted by type, then name, so two boots of one database report alike. * - * Private, like the rest of this rule's registry half, so the public surface - * does not grow; the engine plugin, which owns the boot sequence it is asked - * in, reaches it through the module-level {@link findEnvironmentHeldSecurityCatalogNames}, - * which the package entries do not re-export. + * Computed as {@link packageHeldSecurityCatalogNames} met with the bare slot, + * so the package half is ONE reading: the cold boot asks it here, and + * `os migrate security-catalog-overlays` asks it through the exported + * {@link findPackageHeldSecurityCatalogNames} and meets it with the stored rows + * (maintainer ruling letter B on #22371, record 6074838935). + * + * Private, like the rest of this rule's registry half; the engine plugin, + * which owns the boot sequence it is asked in, reaches it through the + * module-level {@link findEnvironmentHeldSecurityCatalogNames}, which the + * package entries do not re-export. */ private environmentHeldSecurityCatalogConflicts(): SecurityCatalogNameConflict[] { const conflicts: SecurityCatalogNameConflict[] = []; - for (const type of ENVIRONMENT_HELD_SECURITY_CATALOG_TYPES) { - const collection = this.metadata.get(type); - if (!collection) continue; - const environmentNames = [...collection.keys()] - .filter((key) => !key.includes(':') && !BUILT_IN_SECURITY_CATALOG_NAMES[type].has(key)) - .sort(); - for (const name of environmentNames) { - for (const packageId of this.securityCatalogPackageHolders(type, name)) { - conflicts.push({ catalogType: type, name, incomingPackageId: packageId, existingHolder: { kind: 'environment' } }); - } + for (const { catalogType, name, packageIds } of this.packageHeldSecurityCatalogNames()) { + // The environment's item: the bare slot, whatever `_packageId` it wears. + // A held name never contains `:` and is never a built-in, so this is the + // bare-slot reading the method header states. + if (!this.metadata.get(catalogType)?.has(name)) continue; + for (const packageId of packageIds) { + conflicts.push({ catalogType, name, incomingPackageId: packageId, existingHolder: { kind: 'environment' } }); } } return conflicts; } + /** + * Every position and permission-set name a package holds, with its holders — + * the package half of {@link environmentHeldSecurityCatalogConflicts}, and the + * reading {@link findPackageHeldSecurityCatalogNames} exports. + * + * The candidate names are every composite slot's name and every install + * claim of the two types the environment catalog can hold + * ({@link ENVIRONMENT_HELD_SECURITY_CATALOG_TYPES}); each one's holders are + * {@link securityCatalogPackageHolders}'s answer, the one predicate. Built-in + * names are skipped, as the cold boot skips them (see the method above). + * Sorted by type in that list's order, then name. + */ + private packageHeldSecurityCatalogNames(): Array<{ catalogType: SecurityCatalogType; name: string; packageIds: string[] }> { + const held: Array<{ catalogType: SecurityCatalogType; name: string; packageIds: string[] }> = []; + for (const type of ENVIRONMENT_HELD_SECURITY_CATALOG_TYPES) { + const candidates = new Set(); + for (const key of this.metadata.get(type)?.keys() ?? []) { + const sep = key.lastIndexOf(':'); + if (sep > 0) candidates.add(key.slice(sep + 1)); + } + for (const name of this.securityCatalogClaims.get(type)?.keys() ?? []) candidates.add(name); + for (const name of [...candidates].sort()) { + if (name === '' || BUILT_IN_SECURITY_CATALOG_NAMES[type].has(name)) continue; + const packageIds = this.securityCatalogPackageHolders(type, name); + if (packageIds.length > 0) held.push({ catalogType: type, name, packageIds }); + } + } + return held; + } + /** * The packages holding `(type, name)` through an item registered under them * (a composite `:` slot) or an install claim — the package diff --git a/packages/runtime/src/standalone-stack-hydrate-metadata.test.ts b/packages/runtime/src/standalone-stack-hydrate-metadata.test.ts index 240fbc2d274..38a07b897ea 100644 --- a/packages/runtime/src/standalone-stack-hydrate-metadata.test.ts +++ b/packages/runtime/src/standalone-stack-hydrate-metadata.test.ts @@ -183,4 +183,39 @@ describe('[#20071] createStandaloneStack declares sys_metadata hydration instead const rows: Array<{ title?: unknown }> = await engine2.find(OBJECT, {}); expect(rows.map((r) => r.title)).toEqual(['survives the restart']); }, BOOT_TIMEOUT); + + // [#22371] The one way to turn it off, for the offline step that clears what + // the cold boot's catalog check refuses: that check judges what hydration + // wrote, so the step boots with hydration off. Declared on the plugin, and in + // effect on a real restart; the default boot over the same file is the + // control that the row is there to hydrate. + it('`hydrateMetadataFromDb: false` turns it off — declared on the plugin, and nothing stored comes back', async () => { + const off = await createStandaloneStack({ databaseUrl: ':memory:', hydrateMetadataFromDb: false }); + expect(hydrationFlag(off.plugins)).toEqual({ environmentId: 'env_local', hydrateMetadataFromDb: false }); + + const dir = tempDir('off'); + const databaseUrl = `file:${join(dir, 'hydrate-off.db')}`; + const stackConfig = { projectRoot: dir, databaseUrl, skipSeedData: true, runPlatformMigrations: false } as const; + const first = await boot((await createStandaloneStack(stackConfig)).plugins); + kernels.push(first); + await (first.getService('protocol') as any).saveMetaItem({ type: 'app', name: APP, item: { name: APP, label: 'Hydration Console' } }); + await first.shutdown(); + kernels.splice(kernels.indexOf(first), 1); + + const unhydrated = await boot((await createStandaloneStack({ ...stackConfig, hydrateMetadataFromDb: false })).plugins); + kernels.push(unhydrated); + const engineOff = unhydrated.getService('objectql'); + expect(engineOff.registry.getItem('app', APP), `app ${APP} is NOT hydrated with the option off`).toBeUndefined(); + const stored: unknown[] = await engineOff.find('sys_metadata', { where: { type: 'app', name: APP } }); + expect(stored, 'the row is still stored — only the read-back is off').toHaveLength(1); + await unhydrated.shutdown(); + kernels.splice(kernels.indexOf(unhydrated), 1); + + const control = await boot((await createStandaloneStack(stackConfig)).plugins); + kernels.push(control); + expect( + control.getService('objectql').registry.getItem<{ name?: string }>('app', APP)?.name, + 'CONTROL — the default boot over the same file hydrates it', + ).toBe(APP); + }, BOOT_TIMEOUT); }); diff --git a/packages/runtime/src/standalone-stack.ts b/packages/runtime/src/standalone-stack.ts index 48d47530d88..ec6d82723f3 100644 --- a/packages/runtime/src/standalone-stack.ts +++ b/packages/runtime/src/standalone-stack.ts @@ -297,6 +297,29 @@ export const StandaloneStackConfigSchema = z.object({ * an empty report and reads nothing. */ armLifecycleSweep: z.boolean().optional(), + /** + * [#22371] Does this boot read `sys_metadata` back into the registry + * (`ObjectQLPlugin`'s `hydrateMetadataFromDb`)? + * + * Defaults to `true`, the answer every serving boot needs (see the note at + * the `ObjectQLPlugin` construction below: an object authored at runtime + * must survive a restart). + * + * Set `false` for a boot that must see the deployment's PACKAGES and + * nothing the environment stored over them. One caller does today: + * `os migrate security-catalog-overlays`, the offline step that lists and + * deletes the environment-wide rows the cold boot's catalog check refuses + * (ADR-0048 N.3). That check judges what hydration wrote, so a boot of such + * a deployment that hydrates is refused before the step can run; one that + * hydrates nothing meets an empty environment half and comes up, and the + * step reads the rows itself (maintainer ruling letter B on #22371, record + * 6074838935: "without a server, without hydration, and without the boot + * the refusal stops"). + * + * ⛔ Only `false` changes anything: the option's own caution is about + * turning hydration ON, and this function keeps it on unless told. + */ + hydrateMetadataFromDb: z.boolean().optional(), }); export type StandaloneStackConfig = z.input; @@ -898,16 +921,21 @@ export async function createStandaloneStack(config?: StandaloneStackConfig): Pro // a control-plane proxy. // // Both facts are properties of THIS function, not of its caller, so no - // caller can make either clause false — which is why it is a literal - // and not a config field. The one-shot `os migrate *` / `os meta *` + // caller can make either clause false — which is why turning it ON is + // not a caller's choice. The one-shot `os migrate *` / `os meta *` // funnel (`bootSchemaStack`) wants it too: those commands diff and scan // the object set the serving boot registers, and the hydration read // writes nothing (a boot that defers DDL still defers the tables of // what it hydrated). + // + // [#22371] A caller may turn it OFF (`hydrateMetadataFromDb: false`), + // which no clause above forbids: the one that does is the offline step + // that clears what the cold boot's catalog check refuses, and needs a + // boot that check does not stop. See the config key. new ObjectQLPlugin({ environmentId, runPlatformMigrations: cfg.runPlatformMigrations ?? true, - hydrateMetadataFromDb: true, + hydrateMetadataFromDb: cfg.hydrateMetadataFromDb !== false, // [#21391] Only a `false` is passed through, so a serving boot // hands the plugin exactly the options it always did. ...(cfg.armLifecycleSweep === false ? { lifecycle: { enabled: false } } : {}), diff --git a/packages/spec/spec-changes.json b/packages/spec/spec-changes.json index da6f3068d73..1803b2c3756 100644 --- a/packages/spec/spec-changes.json +++ b/packages/spec/spec-changes.json @@ -2885,6 +2885,13 @@ "toMajor": 18, "rationale": "Maintainer ruling A′, 2026-09-13, verbatim, untranslated: 「同意」. ADR-0078 forbids metadata that parses, carries no marking and does nothing — and its own worked example of that state is a `lookup` with no `reference`: the field renders a picker, the picker has no object to query, and nothing anywhere says so. The key shipped OPTIONAL on this surface one release earlier, on the argument that flows declaring a bare `lookup` already exist; the ruling reversed that, holding that a degraded shape which ships is not a reason to bend the contract to it. ⛔ NOT losslessly convertible, and the reason is the same one `schedule-flow-acting-organization-required` gives: the remedy is a value the artifact does not contain. A bare lookup records the field name and nothing about its intended object, so `objectstack migrate meta` can identify every site but can answer none of them — and a conversion that guessed (the first object with a matching-looking name, the flow's trigger object) would write an authoritative wrong answer into metadata a human then trusts. Registered under ADR-0087 D3 rather than left silent because the change DOES carry a prescription a human can execute, which is what D3 says a structured TODO is for." }, + { + "surface": "the cold boot of a deployment whose sys_metadata holds an active, environment-wide row of type permission or position (or the legacy plural permissions or positions) under the name of a permission set or position a configured package declares, the platform security plugin's shipped sets included", + "replacement": "before the first boot on this major, run `os migrate security-catalog-overlays` with the flags and environment the deployment boots with (`--preset` and `--dev` mean what they mean to `os serve`): it lists exactly those rows, each with the package that holds its name. Then run `os migrate security-catalog-overlays --apply` to delete them through the metadata write path (a history tombstone per row). Or rename the item in the package. Nothing is adopted: an item the environment needs under its own name is re-created under a name no package holds", + "migrationId": "security-catalog-environment-overlay-refused", + "toMajor": 18, + "rationale": "Positions, permission sets and capabilities hold one name per deployment, and a package registering a name the environment catalog already holds was already refused on a hot install. A cold boot now refuses it too, right after the stored rows load: the stored row used to be served in place of the package's definition, with only a collision warning. Rows like this exist on deployments that saved over a package-held name before the packaged locks refused such saves, and over the security plugin's own sets, which it declares only where the boot composes it behind the auth gate (an auth secret set, or a development boot). The refused deployment cannot start, so no in-server action can clear the rows, and the metadata API reaches no legacy-plural row at all; the offline step can. No conversion applies: the rows are the environment's own work, and whether to drop or rename one is the operator's call, which the step's preview puts in front of them. ADR-0048, ADR-0087." + }, { "surface": "contracts.emailService.sendTemplate input.org", "replacement": "(removed — never implemented; delete the key from the call. It is NOT replaced by `organizationId`: that member is the delivery row's tenant stamp (`sys_email.organization_id` pass-through, added so the email writer stamps a delivery row's organization at the source) and opts into no template overlay resolution)", @@ -6481,6 +6488,13 @@ "toMajor": 18, "rationale": "Maintainer ruling A′, 2026-09-13, verbatim, untranslated: 「同意」. ADR-0078 forbids metadata that parses, carries no marking and does nothing — and its own worked example of that state is a `lookup` with no `reference`: the field renders a picker, the picker has no object to query, and nothing anywhere says so. The key shipped OPTIONAL on this surface one release earlier, on the argument that flows declaring a bare `lookup` already exist; the ruling reversed that, holding that a degraded shape which ships is not a reason to bend the contract to it. ⛔ NOT losslessly convertible, and the reason is the same one `schedule-flow-acting-organization-required` gives: the remedy is a value the artifact does not contain. A bare lookup records the field name and nothing about its intended object, so `objectstack migrate meta` can identify every site but can answer none of them — and a conversion that guessed (the first object with a matching-looking name, the flow's trigger object) would write an authoritative wrong answer into metadata a human then trusts. Registered under ADR-0087 D3 rather than left silent because the change DOES carry a prescription a human can execute, which is what D3 says a structured TODO is for." }, + { + "surface": "the cold boot of a deployment whose sys_metadata holds an active, environment-wide row of type permission or position (or the legacy plural permissions or positions) under the name of a permission set or position a configured package declares, the platform security plugin's shipped sets included", + "replacement": "before the first boot on this major, run `os migrate security-catalog-overlays` with the flags and environment the deployment boots with (`--preset` and `--dev` mean what they mean to `os serve`): it lists exactly those rows, each with the package that holds its name. Then run `os migrate security-catalog-overlays --apply` to delete them through the metadata write path (a history tombstone per row). Or rename the item in the package. Nothing is adopted: an item the environment needs under its own name is re-created under a name no package holds", + "migrationId": "security-catalog-environment-overlay-refused", + "toMajor": 18, + "rationale": "Positions, permission sets and capabilities hold one name per deployment, and a package registering a name the environment catalog already holds was already refused on a hot install. A cold boot now refuses it too, right after the stored rows load: the stored row used to be served in place of the package's definition, with only a collision warning. Rows like this exist on deployments that saved over a package-held name before the packaged locks refused such saves, and over the security plugin's own sets, which it declares only where the boot composes it behind the auth gate (an auth secret set, or a development boot). The refused deployment cannot start, so no in-server action can clear the rows, and the metadata API reaches no legacy-plural row at all; the offline step can. No conversion applies: the rows are the environment's own work, and whether to drop or rename one is the operator's call, which the step's preview puts in front of them. ADR-0048, ADR-0087." + }, { "surface": "contracts.emailService.sendTemplate input.org", "replacement": "(removed — never implemented; delete the key from the call. It is NOT replaced by `organizationId`: that member is the delivery row's tenant stamp (`sys_email.organization_id` pass-through, added so the email writer stamps a delivery row's organization at the source) and opts into no template overlay resolution)", diff --git a/packages/spec/src/migrations/entries/semantic/18.security-catalog-environment-overlay-refused.ts b/packages/spec/src/migrations/entries/semantic/18.security-catalog-environment-overlay-refused.ts new file mode 100644 index 00000000000..f87576d903e --- /dev/null +++ b/packages/spec/src/migrations/entries/semantic/18.security-catalog-environment-overlay-refused.ts @@ -0,0 +1,47 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import type { SemanticMigration } from '../../types.js'; + +// ADR-0048 addendum N.3 — a runtime narrowing at boot with no authorable key: +// a cold boot refuses an environment-wide permission-set or position row over a +// name a configured package holds. Registered because an upgrading deployment +// can carry such rows and must clear them before its first boot on this major, +// with the offline step that exists for that (the migration prescription the +// ruling on the step orders onto this change's note). +export const entry: SemanticMigration = { + id: 'security-catalog-environment-overlay-refused', + // No backticks in `surface` — build-upgrade-guide.ts renders it inside a + // code span AND a table cell. + surface: + 'the cold boot of a deployment whose sys_metadata holds an active, environment-wide row of type ' + + 'permission or position (or the legacy plural permissions or positions) under the name of a ' + + 'permission set or position a configured package declares, the platform security plugin\'s ' + + 'shipped sets included', + replacement: + 'before the first boot on this major, run `os migrate security-catalog-overlays` with the ' + + 'flags and environment the deployment boots with (`--preset` and `--dev` mean what they mean ' + + 'to `os serve`): it lists exactly those rows, each with the package ' + + 'that holds its name. Then run `os migrate security-catalog-overlays --apply` to delete them ' + + 'through the metadata write path (a history tombstone per row). Or rename the item in the ' + + 'package. Nothing is adopted: an item the environment needs under its own name is re-created ' + + 'under a name no package holds', + reason: + 'Positions, permission sets and capabilities hold one name per deployment, and a package ' + + 'registering a name the environment catalog already holds was already refused on a hot ' + + 'install. A cold boot now refuses it too, right after the stored rows load: the stored row ' + + 'used to be served in place of the package\'s definition, with only a collision warning. ' + + 'Rows like this exist on deployments that saved over a package-held name before the packaged ' + + 'locks refused such saves, and over the security plugin\'s own sets, which it declares only ' + + 'where the boot composes it behind the auth gate (an auth secret set, or a development boot). ' + + 'The refused deployment cannot start, so no in-server action can clear the rows, and the ' + + 'metadata API reaches no legacy-plural row at all; the offline step can. No conversion ' + + 'applies: the rows are the environment\'s own work, and whether to drop or rename one is the ' + + 'operator\'s call, which the step\'s preview puts in front of them. ADR-0048, ADR-0087.', + acceptanceCriteria: + 'On the deployment\'s database and configuration, with its boot flags and environment, ' + + '`os migrate security-catalog-overlays` lists no row (exit 0). The listing covers permission ' + + 'sets and positions and the legacy plural spellings, and never an organization-scoped or a ' + + 'draft row. After `--apply`, the next boot of the same database and configuration comes up, ' + + 'where before it was refused with NAMESPACE_CONFLICT (422) naming the environment catalog as ' + + 'the holder.', +}; diff --git a/packages/spec/src/migrations/registry.ts b/packages/spec/src/migrations/registry.ts index 57bd7409cab..0e512f201f0 100644 --- a/packages/spec/src/migrations/registry.ts +++ b/packages/spec/src/migrations/registry.ts @@ -18872,6 +18872,49 @@ const step18: MigrationStep = { + 'shape: drain or re-drive those rather than assuming the fix reaches them ' + 'retroactively.', }, + // ADR-0048 addendum N.3 — a runtime narrowing at boot with no authorable key: + // a cold boot refuses an environment-wide permission-set or position row over a + // name a configured package holds. Registered because an upgrading deployment + // can carry such rows and must clear them before its first boot on this major, + // with the offline step that exists for that (the migration prescription the + // ruling on the step orders onto this change's note). + { + id: 'security-catalog-environment-overlay-refused', + // No backticks in `surface` — build-upgrade-guide.ts renders it inside a + // code span AND a table cell. + surface: + 'the cold boot of a deployment whose sys_metadata holds an active, environment-wide row of type ' + + 'permission or position (or the legacy plural permissions or positions) under the name of a ' + + 'permission set or position a configured package declares, the platform security plugin\'s ' + + 'shipped sets included', + replacement: + 'before the first boot on this major, run `os migrate security-catalog-overlays` with the ' + + 'flags and environment the deployment boots with (`--preset` and `--dev` mean what they mean ' + + 'to `os serve`): it lists exactly those rows, each with the package ' + + 'that holds its name. Then run `os migrate security-catalog-overlays --apply` to delete them ' + + 'through the metadata write path (a history tombstone per row). Or rename the item in the ' + + 'package. Nothing is adopted: an item the environment needs under its own name is re-created ' + + 'under a name no package holds', + reason: + 'Positions, permission sets and capabilities hold one name per deployment, and a package ' + + 'registering a name the environment catalog already holds was already refused on a hot ' + + 'install. A cold boot now refuses it too, right after the stored rows load: the stored row ' + + 'used to be served in place of the package\'s definition, with only a collision warning. ' + + 'Rows like this exist on deployments that saved over a package-held name before the packaged ' + + 'locks refused such saves, and over the security plugin\'s own sets, which it declares only ' + + 'where the boot composes it behind the auth gate (an auth secret set, or a development boot). ' + + 'The refused deployment cannot start, so no in-server action can clear the rows, and the ' + + 'metadata API reaches no legacy-plural row at all; the offline step can. No conversion ' + + 'applies: the rows are the environment\'s own work, and whether to drop or rename one is the ' + + 'operator\'s call, which the step\'s preview puts in front of them. ADR-0048, ADR-0087.', + acceptanceCriteria: + 'On the deployment\'s database and configuration, with its boot flags and environment, ' + + '`os migrate security-catalog-overlays` lists no row (exit 0). The listing covers permission ' + + 'sets and positions and the legacy plural spellings, and never an organization-scoped or a ' + + 'draft row. After `--apply`, the next boot of the same database and configuration comes up, ' + + 'where before it was refused with NAMESPACE_CONFLICT (422) naming the environment catalog as ' + + 'the holder.', + }, { id: 'send-template-input-org-retired', surface: 'contracts.emailService.sendTemplate input.org',