feat(cli): os migrate security-catalog-overlays — list, and with --apply delete, the environment rows a v18 cold boot refuses

.changeset/22307-cold-boot-catalog-refusal.md

@@ -1,3 +1,3 @@
 ---
 '@objectstack/objectql': major
 ---
@@ -6,7 +6,7 @@
 
 Clause-②: no
 
-<!-- adr-0087: not-required (no-migration-prescription) the refusal removes no key, export or field and changes the shape of no stored body; what an operator does about a refused name is rename or remove one of the two items, which no conversion can choose for them -->
+<!-- adr-0087: registered security-catalog-environment-overlay-refused -->
 
 **BREAKING** — an accept-set narrowing at boot, shipped as `major` on the v18 pre-release line (`.changeset/pre.json` is in `next` pre mode on `main`). A deployment whose environment catalog holds a position or permission-set name that a configured package also declares booted before this release and is refused at boot after it.
 
@@ -16,16 +16,16 @@
 
 **What an operator sees.** The kernel reports `Plugin com.objectstack.engine.objectql failed to start`, and the cause is the package door's envelope: `code: 'NAMESPACE_CONFLICT'` (`NAMESPACE_CONFLICT_CODE` is exported), `status: 422`, and `conflicts[]` with `{ catalogType, name, incomingPackageId, existingHolder: { kind: 'environment' } }`. The message names the package that declares each name and the environment catalog that holds it.
 
-**The upgrade shape.** A deployment fails to boot after this release when an active, environment-wide `sys_metadata` row of type `permission` or `position` (or the legacy plural `permissions` / `positions`, which the boot's load folds to the same types) has the name of a permission set or position that a configured package declares. That includes a row saved over a package-held name before the packaged locks refused such saves, whether or not the row was bound to the package, and a row over one of the platform security plugin's own permission sets (`member_default`, `admin_full_access` and the rest it declares).
+**The upgrade shape.** A deployment fails to boot after this release when an active, environment-wide `sys_metadata` row of type `permission` or `position` (or the legacy plural `permissions` / `positions`, which the boot's load folds to the same types) has the name of a permission set or position that a configured package declares. That includes a row saved over a package-held name before the packaged locks refused such saves, whether or not the row was bound to the package, and a row over one of the platform security plugin's own permission sets (`member_default`, `admin_full_access` and the rest it declares). **Run `os migrate security-catalog-overlays` before the first v18 boot**, with the flags and environment the deployment boots with: it lists exactly these rows, and with `--apply` deletes them.
 
-**The one-line fix: rename the item in the package, or rename or delete the environment's item, then restart.**
+**The one-line fix: before the first v18 boot, run `os migrate security-catalog-overlays` to list the rows, then `os migrate security-catalog-overlays --apply` to delete them; or rename the item in the package.**
 
 - **Before upgrading, for a permission set.** On the release you run now, a boot whose environment catalog overlays a package-declared permission set logs at `kernel:ready`: `[security] N package-declared permission set(s) are being shadowed by an environment overlay`, with the set names. Those are the permission sets this release refuses at boot. The audited **Discard Overlay** action on the set's record in Setup (`POST /api/v1/security/permission-sets/<id>/discard-overlay`, documented under "Declared ≠ enforced" on the Permission Sets page) removes the overlay and resyncs the set to the package's definition. So does `DELETE /api/v1/meta/permission/<name>`, which answers "Customization overlay deleted … reset to artifact default". Either works for the platform security plugin's own sets too, and neither needs direct database access. Positions have no such reading and no such action.
-- **After upgrading, for a package you can leave out.** Boot once without the package in the configuration, rename or delete the environment's item through the metadata API (`DELETE /api/v1/meta/permission/<name>`, `DELETE /api/v1/meta/position/<name>`), then add the package back.
-- **After upgrading, for a name the platform security plugin declares, or for any row the metadata API does not reach.** Back the database up, then delete the row in it. The rows that refuse the boot are the active, environment-wide ones of that name: `organization_id IS NULL` and `state = 'active'`, whatever their `package_id`, under the type or its legacy plural: `DELETE FROM sys_metadata WHERE organization_id IS NULL AND state = 'active' AND type IN ('permission', 'permissions') AND name = '<name>';` (for a position, `type IN ('position', 'positions')`). A draft row and an organization-scoped row are not loaded at boot and do not refuse it.
+- **After upgrading, for any refused name.** The same step: `os migrate security-catalog-overlays`, then `--apply`. It needs no server, so a deployment whose boot is refused can run it, and it covers permission sets and positions, a name the platform security plugin declares, and a row stored under a legacy plural.
+- **What the step deletes.** The rows that refuse the boot are the active, environment-wide ones of a package-held name: `organization_id IS NULL` and `state = 'active'`, whatever their `package_id`, under the type or its legacy plural. In SQL, for one permission-set name, the step's deletion is `DELETE FROM sys_metadata WHERE organization_id IS NULL AND state = 'active' AND type IN ('permission', 'permissions') AND name = '<name>';` (for a position, `type IN ('position', 'positions')`). The step runs it through the metadata write path, so each deleted row leaves a `sys_metadata_history` tombstone. A draft row and an organization-scoped row are not loaded at boot and do not refuse it.
 
-`DELETE /api/v1/meta/permission/<name>` and `DELETE /api/v1/meta/position/<name>` reach a row stored under `permission` or `position` only, one row per call. A row stored under the legacy plural `permissions` / `positions` is not reached: the call answers `200` that nothing was found and removes nothing. Where a name has two active rows, for example one bound to no package and one bound to the package, each call removes one. A plural-typed row is removed by Discard Overlay before upgrading (a permission set), or by the SQL above after upgrading, for any name.
+`DELETE /api/v1/meta/permission/<name>` and `DELETE /api/v1/meta/position/<name>` reach a row stored under `permission` or `position` only, one row per call. A row stored under the legacy plural `permissions` / `positions` is not reached: the call answers `200` that nothing was found and removes nothing. Where a name has two active rows, for example one bound to no package and one bound to the package, each call removes one. A plural-typed row is removed by Discard Overlay before upgrading (a permission set), or by `os migrate security-catalog-overlays --apply`, for any name.
 
-No `os` command deletes a `sys_metadata` row offline: `os meta delete` and `os data delete` call a running server. Nothing renames or removes either item automatically.
+`os migrate security-catalog-overlays` is the one `os` command that deletes these rows with no server running; `os meta delete` and `os data delete` call a running server. Nothing renames or removes either item automatically: the step deletes only under `--apply`, and adopts nothing.
 
 **What is NOT refused.** A stored definition under a built-in position name (`org_admin`, `everyone` and the other four): the platform declares its built-in positions itself, after this check, and the stored definition keeps answering first. The same package restarting with its own names. A package whose names the environment catalog does not hold, booting beside the environment's own items.

.changeset/22371-security-catalog-overlays-step.md

@@ -0,0 +1,24 @@
+---
+"@objectstack/cli": minor
+"@objectstack/objectql": minor
+"@objectstack/core": minor
+"@objectstack/runtime": minor
+"@objectstack/spec": minor
+---
+
+feat(cli): `os migrate security-catalog-overlays` lists, and with `--apply` deletes, the environment-wide rows a v18 cold boot refuses — run it before the first v18 boot
+
+Clause-②: yes (widening)
+
+- **What it is for.** A v18 cold boot refuses to start when the environment catalog holds a permission-set or position name that a configured package also declares (ADR-0048 N.3). A deployment upgraded from 17.x can carry such rows: a set or position saved in the environment over a package's name before the packaged locks refused that, or over one of the platform security plugin's own sets (`member_default`, `admin_full_access` and the rest). The server cannot clear them, because it does not start. This step is the offline remedy.
+- **What it lists.** Every active, environment-wide `sys_metadata` row (`organization_id IS NULL`, `state = 'active'`) of type `permission` or `position`, the legacy plurals `permissions` and `positions` included, whose name a configured package holds. That is the population the cold boot refuses: the list is computed from the engine's own reading of who holds a name, not from a copy of it. A draft row and an organization-scoped row are not loaded at boot, so they are never listed. Each row is shown with its stored type, its `package_id` binding and the package that holds its name.
+- **How it gets there without the refusal.** It composes the deployment as `os serve` does: the host config's plugins, the application or the compiled artifact, and the security plugin behind `serve`'s auth gate. It runs the kernel's first phase for them only, and boots without reading `sys_metadata` back into the registry, so the cold-boot check meets nothing and the boot comes up. No server starts. The security plugin's shipped sets are package-held names only where `serve` composes that plugin, so the step reads the environment `serve` reads. With `OS_AUTH_SECRET` (or `AUTH_SECRET` / `BETTER_AUTH_SECRET`) set, or on a development boot, the plugin is composed and its sets are held, unless the `auth` tier is off. The step takes `os serve`'s `--preset` and `--dev` with `serve`'s meaning, read through the rules `serve` reads them by: `--preset` names the tier preset the auth tier falls back to, and `--dev` composes the config's `devPlugins` and makes the boot a development one (as does `NODE_ENV=development`). The report says which way it went. **Run the step with the flags and environment the deployment boots with.**
+- **Preview by default.** The preview boots read-only, like `os migrate plan`. It writes nothing, creates no database file, and exits `1` when it lists a row: the next boot would be refused. `--apply` deletes the listed rows, after a `[y/N]` prompt or `--yes`, and prints one audit line per row: what was deleted, through which write path, or why not. It exits `0` when every listed row is gone. `--json` prints one document (`listed`, `deleted`, `failed`, `rows[]` with `outcome`, `securityPlugin` and `serveFlags`). `--force` gets past a busy SQLite file. `--database-url` / `$OS_DATABASE_URL` name the target.
+- **How it deletes.** A row stored under `permission` or `position` goes through the metadata protocol's own delete, the one `DELETE /api/v1/meta/:type/:name` uses. That writes a `sys_metadata_history` tombstone and a `sys_metadata_audit` row, with actor `os migrate security-catalog-overlays`. A row stored under a legacy plural is out of that door's reach, so it goes through the `sys_metadata` repository beneath it. That delete is addressed by the row's stored type, name, `package_id` and checksum, and writes the same history tombstone.
+- **The refusal names it.** The cold boot's `NAMESPACE_CONFLICT` message now points at `os migrate security-catalog-overlays` for the environment's rows, in place of "through the metadata API on a boot that leaves the package out of the configuration, or in the database". The envelope (`code`, `status`, `conflicts[]`) is unchanged.
+- **What it never does.** It adopts nothing: `managed_by` and `package_id` are never rewritten, and no row is renamed. A row the environment needs under its own name must be re-created under a name no package holds. A host config that exists and cannot be loaded is refused before any row is read, because the held names would be incomplete.
+- **New exports it is built on.**
+  - `@objectstack/objectql` exports `findPackageHeldSecurityCatalogNames(registry)`: every package-held permission-set and position name, with its holder packages. The cold-boot check now reads the same list.
+  - `@objectstack/core` exports the auth gate `os serve` and the step share. `resolvePlatformAuthComposition` answers whether the platform composes `AuthPlugin` and the security plugin beside it, or why not. With it come `resolveStackTiers`, `STACK_TIER_PRESETS`, `CAPABILITY_TO_TIER`, `resolveAuthSecret`, `isDevelopmentBoot`, `DEV_AUTH_SECRET_FALLBACK`, `stackSuppliesAuthPlugin`, `isHostKernelComposition` and the `PlatformAuthComposition` / `PlatformAuthSkipReason` types. `Serve.TIER_PRESETS` and `Serve.CAPABILITY_TO_TIER` are now handles over the core declarations, and `os serve` composes exactly what it composed before.
+  - `@objectstack/runtime`'s `createStandaloneStack` accepts `hydrateMetadataFromDb: false`, which skips the `sys_metadata` read-back. The default stays on.
+  - `@objectstack/spec` registers the ADR-0087 D3 entry `security-catalog-environment-overlay-refused` in its migration registry and `spec-changes.json`, so `os migrate meta --from 17` names the refusal and this step as its remedy. Its projection is the entry under protocol 18 in the upgrade guide (`docs/protocol-upgrade-guide.md`).

content/docs/deployment/cli.mdx

@@ -937,6 +937,7 @@ written.
 | `os migrate apply` | **Refuses** (exit 1, `error: database_busy` under `--json`). Stop the other process, or pass `--force` |
 | `os migrate files-to-references --apply` | **Refuses** likewise — it rewrites rows, so a concurrent writer is at least as dangerous |
 | `os migrate meta --stored --apply` | **Refuses** likewise — it rewrites `sys_metadata` rows, and a live process saving metadata is exactly the collision |
+| `os migrate security-catalog-overlays --apply` | **Refuses** likewise — it deletes `sys_metadata` rows, and a live process saving metadata is exactly the collision |
 
 The check applies to SQLite only: Postgres and MySQL take their own server-side
 locks. Only same-user processes are visible without elevated privileges, and a
@@ -1114,6 +1115,7 @@ where the data lives.
 | `os migrate value-shapes` | Scan stored reference and structured-JSON field values against the platform's value contract, and record the deployment's migration flag when clean |
 | `os migrate summary-nulls` | Backfill roll-up `count` / `sum` columns still stored as `NULL` on parent rows created before the insert-time seed. Repairs values; no flag, nothing depends on it having run |
 | `os migrate meta --stored` | Replay the metadata conversion chain over this deployment's `sys_metadata` rows and rewrite the ones still carrying a pre-protocol shape. Hygiene, not a gate — nothing depends on it having run |
+| `os migrate security-catalog-overlays` | List the environment-wide permission-set and position rows (the legacy plurals `permissions` / `positions` included) stored over a name a configured package holds — the rows a v18 cold boot refuses — each with the package that holds its name; `--apply` deletes them, one history tombstone per row. Run it before the first v18 boot, with the flags and environment the deployment boots with: the platform security plugin's sets are held only where `os serve` composes it (an auth secret set, and the `auth` tier on), and the step takes `serve`'s `--preset` and `--dev` with `serve`'s meaning. Adopts nothing; exits 1 while it lists a row |
 | `os migrate duplicates` | Report business identifiers already minted twice across the organization partitions, and the rows blocking the boot-time NULL-safe index tightenings — a read-only inventory as JSON on stdout. Renumbers nothing and writes no row and no schema; run it before the boot-time tenancy repair, which overwrites part of the evidence |
 
 **The boot itself writes no row and no schema you did not ask for.** Each of these commands boots