docs: edit the guides for new readers and fix inaccuracies

docs/guides/commands.md

@@ -1,27 +1,28 @@
 # Commands
 
-Every command works in two modes:
+`setup`, `apply`, `pull`, `push`, `cleanup` and `call` work in two modes:
 
 - **Interactive** — run without arguments, get prompted for org and resources
-- **Direct** — pass an org slug and flags for scripting / CI
-
-| Command | Interactive | Direct | One-liner |
-| --- | --- | --- | --- |
-| `npm run setup` | ✅ | — | First-time org wizard — creates `.env.<org>` and `resources/<org>/`. |
-| `npm run validate` | — | `npm run validate -- <org>` | Schema-check local YAML/MD with no network call. **Run before every `apply`.** |
-| `npm run audit` | — | `npm run audit -- <org> [--type <t>]` | Read-only drift detector — orphan local YAML, state ghosts, UUID collisions, content-identical clusters, sibling base-slug clusters, dashboard orphans, assistants with inline `model.tools`. Exit 1 on any finding; safe to wire into CI. |
-| `npm run promote` | — | `npm run promote -- --pipeline <name> --from <org> --to <org> [--apply]` | Plan or apply a forward-only, dependency-aware promotion defined by `promotion.yml`. |
-| `npm run apply` | ✅ | `npm run apply -- <org> [--force]` | **Default deploy verb.** Pull → merge → push in one safe pass; resilient against dashboard drift. |
-| `npm run pull` | ✅ | `npm run pull -- <org> [flags]` | Fetch remote state into local files / state file. Local-first by default — won't clobber local edits. |
-| `npm run push` | ✅ | `npm run push -- <org> [flags]` | Raw push without a pre-pull. Refuses by default when local YAML files lack state entries (orphan-YAML gate); pass `--allow-new-files` to bypass after confirming intent. **Skip unless you just ran `pull` and are certain state is fresh** — otherwise prefer `apply`. |
-| `npm run cleanup` | ✅ | `npm run cleanup -- <org> [--force --confirm <org>]` | Inspect (default) or delete orphaned remote resources. Destructive run requires `--confirm <org>`. |
-| `npm run rollback` | — | `npm run rollback -- <org> --list` or `--to <ISO>` | Restore from a snapshot in `.vapi-state.<org>.snapshots/` (one is written before every push/apply). |
-| `npm run call` | ✅ | `npm run call -- <org> -a <name>` or `-s <squad>` | Start an interactive WebSocket call against an assistant or squad. |
-| `npm run sim` | — | `npm run sim -- <org> --suite <name> --target <name> [--timeout <min>]` | Run a simulation suite (or specific simulations) against a deployed assistant/squad. Prints the run link; exits 0 passed, 1 failed, 3 incomplete (timeout, Ctrl-C, missing results). |
-| `npm run check` | — | `npm run check -- <check>\|--all [--dry-run] [--changed-since <ref>] [--budget-minutes <n>] [--json <path>]` | Run the `vapi-checks.yml` simulation checks against the files on disk: each target is built inline (tools, handoffs, judges, personalities; tools mocked fail-closed, servers dead-ended) and run in one simulation run per target, so nothing is deployed. Posts `Vapi Evals` commit statuses when run by the PR workflow. `--dry-run` builds the payloads offline (no key, nothing sent; `--print-payload` writes them). Exits 0 passed, 1 failed, 2 config or build error, 3 incomplete (timeout, interrupt, budget, billing). |
-| `npm run migrate` | — | `npm run migrate` | One-time, all orgs at once: slim legacy state files to pure `name → uuid` and seed the per-developer `.vapi-state-hash/` baseline store from the old hashes. Required once after upgrading to the hash-store engine — `pull`/`push`/`apply` refuse legacy-shaped state until it runs. Idempotent. |
-| `npm run build` | — | — | Type-check the codebase (`tsc --noEmit`). |
-| `npm test` | — | — | Run regression tests (`node:test`). |
+- **Direct** — pass an org name and flags, for scripts and CI
+
+The other commands are direct only.
+
+| Command | Usage | What it does |
+| --- | --- | --- |
+| `npm run setup` | `npm run setup [-- <org>]` | Connect an org: creates `.env.<org>` and `resources/<org>/`. |
+| `npm run validate` | `npm run validate -- <org>` | Check resource files offline. Run it before every `apply`. |
+| `npm run apply` | `npm run apply -- <org> [types or paths]` | **The default deploy:** pull, merge, then push. See [workflows](workflows.md). |
+| `npm run pull` | `npm run pull -- <org> [--force] [--bootstrap]` | Sync platform changes down; never overwrites local edits unless `--force`. |
+| `npm run push` | `npm run push -- <org> [--dry-run]` | Push without pulling first. Prefer `apply`. |
+| `npm run rollback` | `npm run rollback -- <org> --list` or `--to <ISO>` | Restore a pre-deploy snapshot from `.vapi-state.<org>.snapshots/`. |
+| `npm run cleanup` | `npm run cleanup -- <org> [--force --confirm <org>]` | List platform resources with no file; delete them only with both flags. |
+| `npm run audit` | `npm run audit -- <org> [--type <type>]` | Report drift between files, state and the platform. Exits 1 on any finding, so it can run in CI. |
+| `npm run call` | `npm run call -- <org> -a <assistant>` or `-s <squad>` | Talk to an assistant or squad from your terminal. |
+| `npm run sim` | `npm run sim -- <org> --suite <name> --target <name>` | Run a simulation suite against deployed resources. Exits 0 passed, 1 failed, 3 incomplete. |
+| `npm run check` | `npm run check -- <check>` or `--all`, `[--dry-run]` | Run [PR checks](pr-checks.md) against local files, nothing deployed. Exits 0 passed, 1 failed, 2 config or build error, 3 incomplete. |
+| `npm run promote` | `npm run promote -- --pipeline <p> --from <org> --to <org> [--apply]` | Plan, or apply, a [promotion](promotion.md) between orgs. |
+| `npm run build` | `npm run build` | Type-check the code and tests. |
+| `npm test` | `npm test` | Run the test suite. |
 
 ## Interactive Mode
 
@@ -86,3 +87,17 @@ npm run call -- my-org -a my-assistant
 # Call a squad
 npm run call -- my-org -s my-squad
 ```
+
+## Upgrading from an older version
+
+Repositories created before the state-file format changed need a one-time
+migration. `pull`, `push` and `apply` refuse to run until it's done:
+
+```bash
+npm run migrate
+```
+
+It rewrites every org's `.vapi-state.<org>.json` to the current
+`{ "name": { "uuid": … } }` format and seeds your local
+`.vapi-state-hash/` drift baseline from the old file. It's safe to run more
+than once. Commit the rewritten state files.

docs/guides/configuration.md

@@ -1,10 +1,44 @@
 # Configuration
 
-## Environment Variables
+## Per-org settings (`.env.<org>`)
 
-| Variable        | Required | Description                                      |
-| --------------- | -------- | ------------------------------------------------ |
-| `VAPI_PRIVATE_API_KEY`  | ✅       | Vapi private API key from [Private API Keys](https://dashboard.vapi.ai/org/api-keys). The legacy name `VAPI_TOKEN` is still accepted. |
-| `VAPI_BASE_URL` | ❌       | API base URL (defaults to `https://api.vapi.ai`) |
+`npm run setup` creates one `.env.<org>` file per org. These files are
+gitignored: never commit them.
 
-These are stored in `.env.<org>` files, one per configured organization.
+| Variable | Required | Description |
+| --- | --- | --- |
+| `VAPI_PRIVATE_API_KEY` | ✅ | The org's private API key, from [Private API Keys](https://dashboard.vapi.ai/org/api-keys). The older name `VAPI_TOKEN` is also accepted. |
+| `VAPI_BASE_URL` | | API base URL. Defaults to `https://api.vapi.ai`; EU orgs use `https://api.eu.vapi.ai` (`npm run setup -- <org> --region eu` sets it). |
+| `VAPI_CREDENTIAL_<NAME>`, `VAPI_PHONE_NUMBER_<NAME>` | | Generated by `setup` and `pull` inside a marked block: the org's credential and phone-number IDs, by name. Values you set outside the block are kept and take precedence. |
+
+A variable already set in your shell or CI environment takes precedence over
+the file. `.env.<org>.local` and `.env.local` are also read, after
+`.env.<org>`, and only fill in variables that are still unset.
+
+## CI and workflow settings
+
+Set these in GitHub under **Settings → Secrets and variables → Actions**.
+
+| Name | Kind | Used by | Description |
+| --- | --- | --- | --- |
+| `VAPI_PROMOTION_TOKENS` | Secret | [Promotion](promotion.md) | JSON map of org name to private API key, for example `{"dev":"…","prod":"…"}`. |
+| `VAPI_PROMOTION_ENABLED` | Variable | Promotion | `true` to promote automatically after changes land on `main`. |
+| `VAPI_CHECK_TOKENS` | Secret | [PR checks](pr-checks.md) | JSON map of org name to private API key, for checks that run in several orgs or a CI org. |
+| `VAPI_PRIVATE_API_KEY` | Secret | PR checks | A single org's key, when every check runs in one org. |
+| `VAPI_CHECKS_ENABLED` | Variable | PR checks | `true` to run PR checks. The workflow does nothing without it. |
+
+## Configuration files
+
+| File | Purpose |
+| --- | --- |
+| `promotion.yml` | Promotion pipelines between orgs. Start from `promotion.example.yml`. |
+| `vapi-checks.yml` | PR checks. Start from `vapi-checks.example.yml`. |
+| `resources/<org>/.vapi-ignore` | Platform resources this repo shouldn't manage, as glob patterns. See `resources/.vapi-ignore.example` and [YAML conventions](../learnings/yaml-conventions.md). |
+
+## Other environment variables
+
+| Variable | Description |
+| --- | --- |
+| `CI=true` | Suppresses interactive warnings, such as the one `pull --force` prints. Set automatically by most CI systems. |
+| `VAPI_CALL_DEBUG=1` | Makes `npm run call` log message types it would otherwise ignore. |
+| `VAPI_GITOPS_ROOT` | Overrides the repository root that `npm run promote` and `npm run check` read from. Mainly for tests. |

docs/guides/how-it-works.md

@@ -1,8 +1,8 @@
 # How the engine works
 
-## Organization-Based Structure
+## One folder per org
 
-Resources are scoped by organization (not fixed `dev`/`stg`/`prod` names). Each org gets:
+Resources are scoped by organization, with names you choose (not fixed `dev`/`stg`/`prod`). Each org gets:
 
 - `.env.<org>` — private API key and base URL
 - `.vapi-state.<org>.json` — resource name ↔ UUID mappings (nothing else — committed)
@@ -40,7 +40,7 @@ files
 
 **`pull`** — downloads platform state. Detects locally modified files and skips them (your work is preserved). Use `--force` to overwrite everything.
 
-**`push`** — reads local files and syncs them to the platform. Handles creates, updates, and deletions.
+**`push`** — reads local files and syncs them to the platform: creates and updates. It deletes platform resources whose files you removed only when you pass `--force`.
 
 **`apply`** — runs `pull` then `push` in sequence.
 
@@ -110,62 +110,20 @@ Tracks resource ID ↔ Vapi UUID mappings per org:
 
 Every resource type has a section. Keys are sorted, so diffs stay readable.
 
-## Project Structure
-
-```
-vapi-gitops/
-├── docs/
-│   ├── Vapi Prompt Optimization Guide.md
-│   ├── changelog.md
-│   └── learnings/                      # Gotchas, recipes, troubleshooting per area
-│       ├── assistants.md
-│       ├── tools.md
-│       ├── squads.md
-│       ├── simulations.md
-│       └── ...
-├── src/
-│   ├── setup.ts               # Setup wizard (interactive) + non-interactive setup
-│   ├── setup-args.ts          # `npm run setup` argument parsing
-│   ├── interactive.ts          # Interactive pull/push/apply/call/cleanup flows
-│   ├── searchableCheckbox.ts   # Custom multi-select prompt component
-│   ├── pull.ts                 # Pull platform state
-│   ├── push.ts                 # Push local state to platform
-│   ├── apply.ts                # Orchestrator: pull → merge → push
-│   ├── call.ts                 # WebSocket call script
-│   ├── cleanup.ts              # Orphan cleanup
-│   ├── pull-cmd.ts             # Entry point: interactive or direct pull
-│   ├── push-cmd.ts             # Entry point: interactive or direct push
-│   ├── apply-cmd.ts            # Entry point: interactive or direct apply
-│   ├── call-cmd.ts             # Entry point: interactive or direct call
-│   ├── cleanup-cmd.ts          # Entry point: interactive or direct cleanup
-│   ├── types.ts                # TypeScript interfaces
-│   ├── config.ts               # Environment & configuration
-│   ├── api.ts                  # Vapi HTTP client
-│   ├── state.ts                # State file management
-│   ├── resources.ts            # Resource loading (YAML, MD, TS)
-│   ├── resolver.ts             # Reference resolution
-│   ├── credentials.ts          # Credential resolution (name ↔ UUID)
-│   ├── delete.ts               # Deletion & orphan checks
-│   └── check-cmd.ts            # Entry point: PR simulation checks (check-*.ts)
-├── resources/
-│   └── <org>/                  # One directory per configured org
-│       ├── assistants/
-│       ├── tools/
-│       ├── squads/
-│       ├── structuredOutputs/
-│       ├── evals/
-│       └── simulations/
-│           ├── personalities/
-│           ├── scenarios/
-│           ├── tests/
-│           └── suites/
-├── tests/
-│   ├── credentials.test.ts     # Credential walker scoping (P0-1 regression suite)
-│   ├── clean-resource.test.ts  # null-preservation in pull (P0-3 regression suite)
-│   ├── path-matching.test.ts   # Short-form path matching (P0-7 regression suite)
-│   ├── cleanup-safety.test.ts  # --confirm + empty-state gates (P0-4 regression suite)
-│   └── cli-arg-parsing.test.ts # Bare-id refusal, --confirm pass-through (P0-7)
-├── vapi-checks.example.yml     # Copy to vapi-checks.yml for PR simulation checks
-├── .env.<org>                  # Private API key per org (gitignored)
-└── .vapi-state.<org>.json      # State file per org
-```
+## Where things live
+
+| Path | What it is |
+| --- | --- |
+| `resources/<org>/` | Your resources, one folder per org |
+| `.vapi-state.<org>.json` | Name → UUID mappings per org (committed) |
+| `.env.<org>` | API key and generated binding IDs (gitignored) |
+| `promotion.yml`, `vapi-checks.yml` | Promotion pipelines and PR checks (copy from the `*.example.yml` files) |
+| `resources/<org>/.vapi-ignore` | Platform resources this repo shouldn't manage (see `resources/.vapi-ignore.example`) |
+| `src/` | The engine; `package.json` scripts name each command's entry point |
+| `tests/` | The test suite (`npm test`) |
+| `docs/guides/` | These guides |
+| `docs/learnings/` | The Vapi field guide |
+| `docs/changelog.md` | A template for your own deployment's change log |
+| `examples/` | Copyable examples; never loaded by the engine |
+| `.github/workflows/` | CI, PR checks and promotion workflows |
+| `AGENTS.md`, `CLAUDE.md` | Instructions for coding agents |

docs/guides/troubleshooting.md

@@ -11,18 +11,22 @@ The referenced resource doesn't exist. Check:
 
 ## "Cannot delete resource - still referenced"
 
-1. Find which resources reference it (shown in error)
-2. Remove the references
-3. Push again
-4. Then delete the resource file
+1. Find which resources reference it (shown in the error)
+2. Remove those references and deploy with `npm run apply -- <org>`
+3. Then delete the resource file and deploy again
 
 ## Resource not updating
 
-Check the state file has correct UUID:
+The file may be mapped to the wrong platform resource, or to one that no
+longer exists.
 
-1. Open `.vapi-state.<org>.json`
-2. Find the resource entry
-3. If incorrect, delete entry and re-run push
+Run `npm run audit -- <org>`. It reports state entries that point at a
+missing resource, or several files that point at the same one, with a
+suggested fix for each.
+
+Don't delete a state entry by hand: the next deploy would treat the file as
+new, and stop at the new-file check (or create a duplicate if the check is
+bypassed).
 
 ## "Credential with ID not found" errors
 
@@ -34,7 +38,13 @@ The credential UUID doesn't exist in the target org. Fix:
 
 ## "property X should not exist" API errors
 
-Some properties can't be updated after creation. Add them to `UPDATE_EXCLUDED_KEYS` in `src/config.ts`.
+Some properties can't be changed after a resource is created, so the API
+rejects them on update. Please [open an issue](https://github.com/VapiAI/gitops/issues/new/choose)
+with the resource type and property, so the engine stops sending it.
+
+As a stopgap you can add the property to `UPDATE_EXCLUDED_KEYS` in
+`src/config.ts`. That's an engine change, so expect to resolve it when you
+next pull in updates.
 
 ## "Refusing to run destructive cleanup" errors
 
@@ -62,11 +72,11 @@ The interactive `npm run cleanup` flow handles both gates for you (it shows
 the dry-run preview, asks you to confirm, and forwards `--force --confirm
 <org>` automatically when you say yes).
 
-## "Unrecognized argument" / push appears to do nothing
+## "Unrecognized argument" errors
 
-If you typed `npm run push -- my-org foo` (a bare resource id with no folder
-or extension), the CLI now refuses with `Unrecognized argument: foo` rather
-than silently running a full apply. Pass either:
+A bare resource ID (`npm run push -- my-org foo`, with no folder or
+extension) is rejected with `Unrecognized argument: foo`, rather than
+falling through to a full deploy. Pass either:
 
 - a resource type — `npm run push -- my-org assistants`, or
 - a path — `npm run push -- my-org assistants/foo.yml` (short form)