Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ Run them all on one port with `npx @copilotkit/aimock --config aimock.json`, or
- **HTTP route catalog** — `GET /__aimock/openapi.json` returns OpenAPI 3.1; `GET /__aimock/routes` returns `{ count, routes }`. For programmatic inspection, import `buildOpenApiDocument`, `CATALOG_ROUTES` and `ROUTE_DEFINITIONS` (with `CatalogRoute`/`RouteDefinition` types) from `@copilotkit/aimock`. The catalog describes HTTP operations; streaming responses include media and framing metadata, not a complete-stream JSON body.
- **Request IDs** — every response carries `X-Request-Id`, echoing the caller's header or minting one; the journal stores it on each entry and `GET /__aimock/journal?requestId=<id>` returns that request. A minted id is never forwarded upstream in record/proxy mode
- **Per-Request Strict Mode** — `X-AIMock-Strict` header overrides the server-level `--strict` flag per request (`true`/`1` = strict, `false`/`0` = lenient)
- **[Required Header Policy](https://aimock.copilotkit.dev/aimock-cli#required-header-policy)** — Opt-in `requiredHeaders: "enforce"` (programmatic `policy`, config `llm.policy`, repeatable `--policy` on both CLIs, or the `X-AIMock-Policy` request header) rejects Anthropic HTTP requests (`/v1/messages`, `/v1/messages/count_tokens`, Anthropic-flavored `/v1/models`) that omit `anthropic-version` with a native 400. Off by default, and independent of strict mode
- **[Native model catalogs and token counts](https://aimock.copilotkit.dev/claude-messages#model-catalog)** — Anthropic `GET /v1/models` (selected by `anthropic-version`, or `x-api-key` without `Authorization`) and Gemini `GET /v1beta/models` list and detail routes with provider paging, built from fixture model IDs with synthetic metadata; `POST /v1/messages/count_tokens` and Gemini `:countTokens` return a deterministic estimate of about 1 token per 4 characters of messages (plus tool definitions), minimum 1. These routes never proxy or record
- **Context-Based Fixture Routing** — `X-AIMock-Context` header scopes fixtures per integration; fixtures with `match.context` only match requests carrying that context, fixtures without it remain shared
- **[Drift Detection](https://aimock.copilotkit.dev/drift-detection)** — Daily CI validation against real APIs
- **[Streaming Physics](https://aimock.copilotkit.dev/streaming-physics)** — Configurable `ttft`, `tps`, and `jitter`
Expand Down
14 changes: 11 additions & 3 deletions docs/claude-messages/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -153,8 +153,9 @@ <h2 id="token-counts">Approximate Token Counts</h2>
<p>
<code>POST /v1/messages/count_tokens</code> accepts a Messages-style body and returns
<code>{"input_tokens": N}</code>. It converts messages, system text, and tools through the
existing request translator, then estimates tokens with <code>ceil(chars / 4)</code>. This
deterministic estimate is not a tokenizer or an upstream count.
existing request translator, then estimates about 1 token per 4 characters of messages
(plus tool definitions), minimum 1. This deterministic estimate is not a tokenizer or an
upstream count.
</p>
<div class="code-block">
<div class="code-block-header">Count a prompt <span class="lang-tag">shell</span></div>
Expand Down Expand Up @@ -340,8 +341,15 @@ <h2>SSE Event Sequence</h2>
<li><code>message_delta</code> &mdash; stop_reason, usage</li>
<li><code>message_stop</code></li>
</ol>
<p>
If the client disconnects mid-stream, aimock stops sending events and skips any remaining
per-chunk latency. The journal entry records <code>response.interrupted: true</code> and
<code>response.interruptReason: "client aborted"</code>. Fixture interruptions
(<code>truncateAfterChunks</code>, <code>disconnectAfterMs</code>) keep their own reasons.
See <a href="/error-injection">Error Injection</a>.
</p>

<h2>Cache Usage</h2>
<h2 id="cache-usage">Cache Usage</h2>
<p>
Set <code>response.usage.cache_read_input_tokens</code> and
<code>cache_creation_input_tokens</code> in a fixture to emit cache counts on Messages and
Expand Down
4 changes: 4 additions & 0 deletions docs/error-injection/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,10 @@ <h2>Interruption Behavior</h2>
Interrupted requests are recorded in the journal with
<code>response.interrupted: true</code> and <code>response.interruptReason</code>
</li>
<li>
A client that disconnects from a Claude Messages stream is recorded the same way, with
<code>response.interruptReason: "client aborted"</code>
</li>
</ul>

<div class="info-box">
Expand Down
7 changes: 5 additions & 2 deletions docs/fixtures/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -935,8 +935,11 @@ <h2>Response Override Fields</h2>
Gemini field names (<code>promptTokenCount</code>,
<code>candidatesTokenCount</code>, <code>totalTokenCount</code>). OpenAI Chat
Completions includes usage in the response body; the Responses API uses a separate
<code>response.usage</code> object. When omitted, token counts are auto-computed
from content length
<code>response.usage</code> object. Claude Messages and Bedrock Claude invoke also
accept <code>cache_read_input_tokens</code>,
<code>cache_creation_input_tokens</code> and a <code>cache_creation</code> object
(see <a href="/claude-messages#cache-usage">Cache Usage</a>). When omitted, token
counts are auto-computed from content length
</td>
</tr>
<tr>
Expand Down
9 changes: 5 additions & 4 deletions docs/gemini/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -149,10 +149,11 @@ <h2 id="token-counts">Approximate Token Counts</h2>
including system text and tools.
</p>
<p>
aimock converts the request through its existing translator, then estimates tokens with
<code>ceil(chars / 4)</code>. The estimate includes converted messages, system text, and
serialized converted tools. This deterministic estimate is not a tokenizer or an upstream
count. Optional cache counts and token-detail arrays are omitted.
aimock converts the request through its existing translator, then estimates about 1 token
per 4 characters of messages (plus tool definitions), minimum 1. The estimate includes
converted messages, system text, and serialized converted tools. This deterministic
estimate is not a tokenizer or an upstream count. Optional cache counts and token-detail
arrays are omitted.
</p>
<div class="code-block">
<div class="code-block-header">Count a prompt <span class="lang-tag">shell</span></div>
Expand Down
20 changes: 20 additions & 0 deletions docs/streaming-physics/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,21 @@ <h2>JSON Fixture File</h2>
}</code></pre>
</div>

<h2 id="non-streaming">Non-Streaming Responses</h2>
<p>
A fixture with <code>streamingProfile</code> or <code>recordedTimings</code> also delays
<code>stream: false</code> OpenAI Chat Completions responses: text, tool calls, combined
content and tool calls, and <code>blocks</code> replies. aimock waits for the sum of the
per-chunk delays that a streamed reply at the same <code>chunkSize</code> would get
(<code>ttft</code>, then a <code>tps</code> gap for each further content chunk), then
writes the JSON body. A streamed reply also pays for its role and finish chunks, so the
two totals are close but not identical.
</p>
<p>
This applies only to OpenAI Chat Completions. Non-streaming responses from other providers
are not delayed.
</p>

<h2>Interaction with <code>latency</code></h2>
<ul>
<li>
Expand All @@ -172,6 +187,11 @@ <h2>Interaction with <code>latency</code></h2>
If <code>streamingProfile</code> is set but has neither <code>ttft</code> nor
<code>tps</code>, it falls back to <code>latency</code>.
</li>
<li>
<code>latency</code> and <code>--latency</code> are gaps between SSE chunks only. They
never delay a non-streaming response, including the fallback case above. Use
<code>streamingProfile</code> to delay <code>stream: false</code> replies.
</li>
</ul>

<h2>Realistic Profiles</h2>
Expand Down
Loading
Loading