Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/vapi-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Vapi checks
# Runs the simulation checks in vapi-checks.yml against the PR branch's own
# files: each target is built inline and sent in one simulation run, so
# nothing is deployed. Opt in with the repository variable
# VAPI_CHECKS_ENABLED=true; see "PR checks" in the README.
# VAPI_CHECKS_ENABLED=true; see docs/guides/pr-checks.md.
#
# Never `pull_request_target`: the branch's code runs here, so it must only
# ever get this repository's secrets when the branch is this repository's.
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1032,4 +1032,4 @@ When transferring to human:
2. Create scenarios (what the simulated caller says + evaluation criteria)
3. Create simulations (pair personality + scenario)
4. Create suites (batch simulations together)
5. Run against the deployed resources with `npm run sim`, or against the local files (nothing deployed) with `npm run check` — see "PR Checks" in the README. The PR workflow runs `npm run check` on every affected PR when `VAPI_CHECKS_ENABLED=true`
5. Run against the deployed resources with `npm run sim`, or against the local files (nothing deployed) with `npm run check` — see [docs/guides/pr-checks.md](docs/guides/pr-checks.md). The PR workflow runs `npm run check` on every affected PR when `VAPI_CHECKS_ENABLED=true`
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Commit messages and PR titles follow [Conventional Commits](https://www.conventi
| --- | --- |
| A Vapi platform gotcha, recipe or troubleshooting guide | `docs/learnings/<topic>.md`, plus a row in [`docs/learnings/README.md`](docs/learnings/README.md) and the table in `AGENTS.md` for a new file |
| A sync-engine pain point and its fix (pull, push, state, cleanup) | `improvements.md`, in its Problem → Current behavior → Risk → Current mitigation → Possible fix → Status format |
| Setup or orientation for new users | `README.md` (keep it short; link to a guide for depth) |
| Setup or orientation for new users | `README.md` (keep it short; put depth in `docs/guides/`) |
| An example users can copy | `examples/`. Snippets in the docs that start with `# examples/<path>` must match the file exactly; `npm test` checks this. |

Don't edit `docs/changelog.md` here. It's a template for your own
Expand Down
1,308 changes: 202 additions & 1,106 deletions README.md

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,6 @@ acknowledge the report and keep you updated as we investigate.
module, including in `validate`, `promote` plans and PR checks. Review them
like code, and only run them from trusted branches.
- **PR checks run with your key on same-repository branches.** Forked PRs get
a dry run with no secrets. See "Cost and safety" in the README's
PR Checks section for what a check still sends to real providers.
a dry run with no secrets. See "Cost and safety" in
[docs/guides/pr-checks.md](docs/guides/pr-checks.md#cost-and-safety) for
what a check still sends to real providers.
88 changes: 88 additions & 0 deletions docs/guides/commands.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Commands

Every command works in two modes:

- **Interactive** — run without arguments, get prompted for org and resources
- **Direct** — pass an org slug and flags for scripting / CI

| Command | Interactive | Direct | One-liner |
| --- | --- | --- | --- |
| `npm run setup` | ✅ | — | First-time org wizard — creates `.env.<org>` and `resources/<org>/`. |
| `npm run validate` | — | `npm run validate -- <org>` | Schema-check local YAML/MD with no network call. **Run before every `apply`.** |
| `npm run audit` | — | `npm run audit -- <org> [--type <t>]` | Read-only drift detector — orphan local YAML, state ghosts, UUID collisions, content-identical clusters, sibling base-slug clusters, dashboard orphans, assistants with inline `model.tools`. Exit 1 on any finding; safe to wire into CI. |
| `npm run promote` | — | `npm run promote -- --pipeline <name> --from <org> --to <org> [--apply]` | Plan or apply a forward-only, dependency-aware promotion defined by `promotion.yml`. |
| `npm run apply` | ✅ | `npm run apply -- <org> [--force]` | **Default deploy verb.** Pull → merge → push in one safe pass; resilient against dashboard drift. |
| `npm run pull` | ✅ | `npm run pull -- <org> [flags]` | Fetch remote state into local files / state file. Local-first by default — won't clobber local edits. |
| `npm run push` | ✅ | `npm run push -- <org> [flags]` | Raw push without a pre-pull. Refuses by default when local YAML files lack state entries (orphan-YAML gate); pass `--allow-new-files` to bypass after confirming intent. **Skip unless you just ran `pull` and are certain state is fresh** — otherwise prefer `apply`. |
| `npm run cleanup` | ✅ | `npm run cleanup -- <org> [--force --confirm <org>]` | Inspect (default) or delete orphaned remote resources. Destructive run requires `--confirm <org>`. |
| `npm run rollback` | — | `npm run rollback -- <org> --list` or `--to <ISO>` | Restore from a snapshot in `.vapi-state.<org>.snapshots/` (one is written before every push/apply). |
| `npm run call` | ✅ | `npm run call -- <org> -a <name>` or `-s <squad>` | Start an interactive WebSocket call against an assistant or squad. |
| `npm run sim` | — | `npm run sim -- <org> --suite <name> --target <name> [--timeout <min>]` | Run a simulation suite (or specific simulations) against a deployed assistant/squad. Prints the run link; exits 0 passed, 1 failed, 3 incomplete (timeout, Ctrl-C, missing results). |
| `npm run check` | — | `npm run check -- <check>\|--all [--dry-run] [--changed-since <ref>] [--budget-minutes <n>] [--json <path>]` | Run the `vapi-checks.yml` simulation checks against the files on disk: each target is built inline (tools, handoffs, judges, personalities; tools mocked fail-closed, servers dead-ended) and run in one simulation run per target, so nothing is deployed. Posts `Vapi Evals` commit statuses when run by the PR workflow. `--dry-run` builds the payloads offline (no key, nothing sent; `--print-payload` writes them). Exits 0 passed, 1 failed, 2 config or build error, 3 incomplete (timeout, interrupt, budget, billing). |
| `npm run migrate` | — | `npm run migrate` | One-time, all orgs at once: slim legacy state files to pure `name → uuid` and seed the per-developer `.vapi-state-hash/` baseline store from the old hashes. Required once after upgrading to the hash-store engine — `pull`/`push`/`apply` refuse legacy-shaped state until it runs. Idempotent. |
| `npm run build` | — | — | Type-check the codebase (`tsc --noEmit`). |
| `npm test` | — | — | Run regression tests (`node:test`). |

## Interactive Mode

When you run a command without arguments, you get a fully interactive experience:

```bash
npm run push
# → Select org (if multiple configured)
# → All resources / Let me pick…
# → Searchable multi-select with git status indicators
# → Confirm and execute

npm run pull
# → Select org
# → All resources / Let me pick…
# → Shows which resources are already local (✔)
# → "Overwrite locally modified files?" — defaults to NO (local-first)
# → Confirm and execute

npm run cleanup
# → Select org
# → Dry-run preview of what would be deleted
# → "Proceed with actual deletion?" — defaults to NO
# → Destructive run is gated by both your confirm AND --confirm <org>
```

Navigation:
- **Type** to search/filter resources
- **Space** to toggle the focused row (or toggle the whole group when the cursor is on a header)
- **Ctrl+A** to select/deselect all currently-visible rows
- **Ctrl+G** to toggle every item in the focused group
- **→ / ←** (right / left arrow) to expand or collapse the focused group
- **Enter** to confirm
- **Esc** to clear the search; press again to step back to the previous prompt

## Direct Mode

Pass an org slug as the first argument to skip interactive prompts:

```bash
# Pull everything for an org
npm run pull -- my-org

# Force pull (overwrite local changes)
npm run pull -- my-org --force

# Push only assistants
npm run push -- my-org assistants

# Push a single file
npm run push -- my-org resources/my-org/assistants/my-agent.md

# Pull with bootstrap (state only, no files written)
npm run pull -- my-org --bootstrap

# Pull a single resource by UUID
npm run pull -- my-org --type assistants --id <uuid>

# Call an assistant
npm run call -- my-org -a my-assistant

# Call a squad
npm run call -- my-org -s my-squad
```
10 changes: 10 additions & 0 deletions docs/guides/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Configuration

## Environment Variables

| Variable | Required | Description |
| --------------- | -------- | ------------------------------------------------ |
| `VAPI_PRIVATE_API_KEY` | ✅ | Vapi private API key from [Private API Keys](https://dashboard.vapi.ai/org/api-keys). The legacy name `VAPI_TOKEN` is still accepted. |
| `VAPI_BASE_URL` | ❌ | API base URL (defaults to `https://api.vapi.ai`) |

These are stored in `.env.<org>` files, one per configured organization.
183 changes: 183 additions & 0 deletions docs/guides/file-formats.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
# File Formats

Every snippet below is a file from [`examples/starter/`](../../examples/starter/), a
small dental-clinic front desk with two assistants, tools, a handoff and a
simulation suite. CI checks that each snippet matches its file and that the
example passes `validate`, so you can copy from here safely.

A resource's ID is its path under the type folder, without the extension
(`tools/lookup-patient.yml` is `lookup-patient`). Reference other resources
by that ID, never by UUID: the engine resolves IDs to UUIDs per org.

## Assistants (`.md` or `.yml`)

Markdown with YAML frontmatter: the frontmatter is the assistant config and the
body is its system prompt.

```markdown
<!-- examples/starter/resources/starter/assistants/receptionist.md -->
---
name: Receptionist
firstMessage: Thanks for calling Bright Smile Dental. How can I help?
model:
provider: openai
model: gpt-4.1
temperature: 0.3
toolIds:
- lookup-patient
- handoff-to-scheduler
tools:
- type: endCall
voice:
provider: 11labs
voiceId: sarah
artifactPlan:
structuredOutputIds:
- call-summary
---

# Identity

You are the receptionist for Bright Smile Dental, 123 Main St. The clinic is
open Monday to Friday, 8am to 5pm.

# Flow

1. Ask for the caller's phone number and call `lookup_patient` with it.
2. If they want to book, change or check an appointment, hand off to the
Scheduler with `handoff_to_scheduler`. Don't book anything yourself.
3. Answer general questions (hours, address) briefly yourself.
```

## Tools (`.yml`)

```yaml
# examples/starter/resources/starter/tools/lookup-patient.yml
type: function
function:
name: lookup_patient
description: Look up the caller's patient record by phone number.
parameters:
type: object
properties:
phone:
type: string
description: The caller's phone number
required:
- phone
server:
url: https://example.com/vapi/lookup-patient
```

Handoffs between assistants are tools too. Give each one an explicit
`function.name` if your prompts mention it by name:

```yaml
# examples/starter/resources/starter/tools/handoff-to-scheduler.yml
type: handoff
function:
name: handoff_to_scheduler
destinations:
- type: assistant
assistantId: scheduler
description: Books, changes and checks appointments.
```

## Structured Outputs (`.yml`)

```yaml
# examples/starter/resources/starter/structuredOutputs/call-summary.yml
name: call-summary
type: ai
description: Summarizes the call for the front-desk log.
schema:
type: object
properties:
summary:
type: string
booked:
type: boolean
```

## Squads (`.yml`)

```yaml
# examples/starter/resources/starter/squads/front-desk.yml
name: Front Desk
members:
- assistantId: receptionist
- assistantId: scheduler
```

Members hand off to each other through handoff tools on the assistants, as
above. Prefer them over the legacy `assistantDestinations` field.

## Evals (`.yml`)

An eval file is the body of the [Evals API](https://docs.vapi.ai/api-reference/evals)
create request, written as YAML.

## Simulations

**Personality** (`simulations/personalities/`): the simulated caller, as an
assistant config.

```yaml
# examples/starter/resources/starter/simulations/personalities/calm-caller.yml
name: Calm caller
assistant:
model:
provider: openai
model: gpt-4.1-mini
messages:
- role: system
content: >
You are a patient calling a dental clinic. Follow your scenario,
answer questions briefly, and don't invent details.
```

**Scenario** (`simulations/scenarios/`): what the caller does, how the call is
judged (at least one evaluation), and mock results for the tools it calls.

```yaml
# examples/starter/resources/starter/simulations/scenarios/books-cleaning.yml
name: Books a cleaning
instructions: >
You are Jordan Lee, phone 206-555-0142, an existing patient. Book a teeth
cleaning for next Tuesday morning and accept the first slot offered. Once
the booking is confirmed, say thanks and goodbye.
evaluations:
- structuredOutputId: booking-confirmed
comparator: "="
value: true
required: true
toolMocks:
- toolName: lookup_patient
result: '{"found": true, "patientId": "P-1001"}'
- toolName: book_appointment
result: '{"success": true, "date": "next Tuesday", "time": "09:00"}'
```

**Simulation** (`simulations/tests/`): a personality paired with a scenario.

```yaml
# examples/starter/resources/starter/simulations/tests/books-cleaning-calm.yml
name: Books a cleaning (calm caller)
personalityId: calm-caller
scenarioId: books-cleaning
```

**Simulation Suite** (`simulations/suites/`):

```yaml
# examples/starter/resources/starter/simulations/suites/core.yml
name: Core
simulationIds:
- books-cleaning-calm
```

## TypeScript resources (`.ts`)

Any resource can also be a `.ts` file whose default export is the resource
object, useful for generating config. It is executed when loaded, so treat
`.ts` resources like code in review.
Loading
Loading