Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,13 @@ We will support the Databricks AI Gateway as a first-class backend, delivered as
- **Attribution and correlation.** Required opaque session/turn/model-call/attempt/purpose tags use the documented `Databricks-Ai-Gateway-Request-Tags` JSON header. Optional bounded User-Agent attribution and real `traceparent` remain secondary signals. Keep header request id, completion body id and provider invocation id distinct; exact row/tag/header joins require profile-specific live qualification (CONTRACT-005). Reuse the inspected downstream capture/read pattern with async I/O and stronger multi-invocation/conflict handling.
- **Identity.** A request's actor credential is used when present (ADR-011). Otherwise the application's own identity comes from the Databricks SDK's unified authentication (a service principal, or a developer's command-line profile), which this module alone imports and only for host discovery and authentication. A `require_user` mode refuses a call that has no user credential instead of falling back to the application's identity, as the first downstream platform's implementation does.
- **Reference launcher.** `--databricks-profile NAME` derives the workspace host from the explicitly selected installed CLI profile; no URL is required. The CLI bridge owns bounded model-service discovery and OAuth token refresh, while the existing compatible adapter owns inference. Discovery initially covers `system.ai` services advertising `mlflow/v1/chat/completions`, with a picker before session creation and an explicit `--model` bypass. Catalog metadata is not invocation permission or context-capacity evidence. The bridge pins the discovered host and refuses later changes; it never persists credentials. This launcher qualifies existing CLI OAuth user profiles only; SDK application identity and full gateway attribution/history qualification remain separate profiles.
- **Declared wire capabilities.** The reference profile omits the unsupported `parallel_tool_calls` option in native inference and offline estimation, uses the gateway request-tag header with string-valued opaque tags, and permits validated partial usage on intermediate SSE chunks while requiring complete terminal input/output counts. These choices do not add an inference client or relax tool approval. Broader cache accounting and history/row joins require separate qualification.
- **Declared wire capabilities.** The reference profile omits the unsupported `parallel_tool_calls` option in native inference and offline estimation, uses the gateway request-tag header with string-valued opaque tags, and permits validated partial usage on intermediate SSE chunks while requiring complete terminal input/output counts. These choices do not add an inference client or relax tool approval. History/row joins require separate qualification.
- **Gateway stream dialect.** Gateway backends stream frames outside the strict `openai-chat-text-tools-v1` chunk shape: a prompt-filter preamble (`object: ""`, empty `choices`, `prompt_filter_results`), content-filter results and filter errors on choices, null `logprobs` and `refusal`, `obfuscation`, `latency_checkpoint` and `routing` metadata, and reasoning-summary content parts before answer text. We will not widen the strict profile. A separate, explicitly selected stream profile, `databricks-ai-gateway-stream-v1`, accepts exactly those fields against closed allow-lists and forwards only normalized frames that the strict validator then re-checks; the SDK never sees a gateway field. The reference launcher selects it with its Databricks profile; `OpenAICompatModel` and `CompatibleModelSettings` default to the strict profile.
- *Reasoning-summary parts are validated and dropped.* A list-valued delta `content` must hold only `{"type": "reasoning", "summary": [{"type": "summary_text", "text", "signature"?}]}` parts. They never become answer text, deltas, tool input or telemetry. Text parts in list form, other part types and other keys refuse. Preserving reasoning as typed non-text output needs a model-port change and its own decision.
- *Filter metadata is shape-checked, not kept.* Category maps allow only `filtered`/`detected` booleans and a `severity` string; a filter error allows only a string `code` and `message`. None of it enters `ModelResponse`, receipts or audit. A prompt category reporting `filtered: true` refuses as `prompt_filtered`; an output category reporting `filtered: true` requires a `content_filter` terminal, which normalizes to `filtered`. Tether is not the content-safety layer; the gateway's terminal reason governs.
- *Unknown fields still refuse* under both profiles. The raw stream bytes remain the attempt's wire observation, so dropped fields stay in the original exchange record under existing payload authorization.
- Nonstreaming responses still use the strict body validator; this dialect is qualified for streaming only.
- Nonstreaming bodies, including the stream fallback when a server ignores `stream: true`, still use the strict body validator; this dialect is qualified for streaming only. Structure-only checks show gateway JSON bodies carry the same filter, annotation and reasoning extras, which the strict validator refuses.
- *Gateway cache counts map to the port's cache subcounts.* Two backends report Anthropic-style `cache_read_input_tokens` and `cache_creation_input_tokens` at the top level of `usage`, sometimes with a `cache_creation` breakdown by retention; the content-filtered backend reports OpenAI-style `prompt_tokens_details.cached_tokens` and `cache_write_tokens`. Live checks showed both styles are subsets of `prompt_tokens`: an identical 10,420-token prompt reported `prompt_tokens` 10,420 with 10,403 tokens written, then 10,403 read, then neither, and `total_tokens` stayed prompt plus completion. That matches `TokenUsage`, whose cache counts are subcounts never added to totals again. In streamed frames the profile therefore moves the top-level counts into `prompt_tokens_details` before validation and forwarding, so they become `cache_read_tokens` and `cache_write_tokens`; the `cache_creation` breakdown must use only the two known retention keys and sum to the creation count, and is then dropped. A nested count that conflicts with its top-level twin, a negative count or a cache sum above `prompt_tokens` refuses as `invalid_response`. The strict profile, and the strict nonstreaming fallback under either profile, leave top-level cache counts missing rather than guessing their meaning. Tether's request mapping sends no cache markers, so these counts are nonzero only when a deployment caches on its own.
- **Configuration is explicit.** The adapter reads no environment variable. The application builds its settings object and passes the tier map and the identity mode in.
- **Never the only backend.** Every feature of the core runs against the scripted model and against the default adapter on a plain OpenAI-compatible server. The adapter conformance matrix includes at least one backend that is not the Databricks AI Gateway. No contract field, port method, or default exists only to serve the Databricks backend.
- **Gateway authoritative history.** CONTRACT-005 makes inference-table model exchanges a first-class selected HistoryProvider source, composed with a writable supplement. The execution journal holds facts and content refs; a gateway request is the sent model projection, not the full original conversation. Reader discovery/query authentication uses the explicit calling actor, bounded SQL and configured locators; no schema mutation or denied-user fallback. The profile may use the platform SDK for these adapter-owned reads as well as authentication. Delayed/missing/partial/ambiguous observations are explicit.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ async def checked_stream(port: ModelPort, request: ModelRequest, *, max_text_byt
| Concurrency | An adapter MUST be safe to call from many tasks at once. Caller mode uses only that request's credential; application mode uses only the explicitly configured application identity; anonymous mode sends no authorization. No mode inherits identity from another task. |
| Credential | Caller mode reads credential.authorization() once per physical attempt, never caches it across attempts, and refuses missing/empty/expired credentials before egress without application fallback. Application/anonymous modes are explicit request selections; passing a caller credential in either is identity_mode_conflict rather than silent credential substitution. Adapter construction validates the selected application credential mechanism. No credential is logged, persisted or included in an error reason. |
| Retries | An adapter MUST NOT retry in more than one layer; the layer that retries owns the policy. |
| Stream profile | The compatible adapter's stream profile is an explicit selection; the default `openai-chat-text-tools-v1` refuses any field outside its chunk shape and forwards original bytes. A dialect profile (`databricks-ai-gateway-stream-v1`, ADR-013) MUST check each extra field against a closed allow-list, MUST forward only normalized frames that the strict validator re-checks, and MUST NOT turn dropped non-text parts into text, deltas or usage. Unknown fields refuse under every profile. The receipt's `schema_profile` names the selected profile. A stream-validation failure carries the observed header request id and receipt. |
| Stream profile | The compatible adapter's stream profile is an explicit selection; the default `openai-chat-text-tools-v1` refuses any field outside its chunk shape and forwards original bytes. A dialect profile (`databricks-ai-gateway-stream-v1`, ADR-013) MUST check each extra field against a closed allow-list, MUST forward only normalized frames that the strict validator re-checks, and MUST NOT turn dropped non-text parts into text, deltas or usage. It MAY map a provider's cache counts to `cache_read_tokens`/`cache_write_tokens` only after their semantics are qualified as subcounts of the input total, and MUST refuse counts that exceed it or conflict. Unknown fields refuse under every profile. The receipt's `schema_profile` names the selected profile. A stream-validation failure carries the observed header request id and receipt. |
| Terminal outcome | The adapter normalizes only documented wire terminal reasons to `finish_reason`; missing/unrecognized reasons are `unknown`. A stream without a terminal response is `ModelUnavailable("incomplete_stream")`. Only `stop` without calls or `tool_calls` with complete valid calls permits continuation/final success. Truncated, filtered, unknown or structurally inconsistent outcomes MUST NOT dispatch a call or be labelled final success. A structurally valid bounded `length`, `filtered` or `unknown` response remains an observed terminal for CONTRACT-002 incomplete-text persistence; it is not a transport outage. Structural inconsistency, invalid arguments and byte-limit failures remain explicit failures. A native SDK default cannot promote a missing/unrecognized wire reason from `unknown` to `stop`; exact SDK text/call projection parity is required when SDK normalization succeeds. If the public SDK rejects an unrecognized reason, a complete bounded response that passed Tether wire validation and successful recorder observation MAY be returned unchanged only as non-executable `unknown` evidence, after every local transport/recorder error has propagated. This validated unknown-terminal fallback makes no SDK projection-parity claim and MUST NOT rewrite captured bytes, retry, dispatch calls or expose SDK diagnostics. |
| Wire integrity | Invalid JSON arguments, duplicate/ambiguous call ids in one reply, invalid names or incomplete calls MUST be rejected, never repaired into executable arguments. Provider-specific history formatting may add clearly synthetic non-executable separators; it MUST preserve the original log, invocation identity and effect status. |
| Bounds | `max_output_tokens` is mapped to the configured endpoint output-limit field. A conflicting reserved value in `extra` is `ModelUnavailable("parameter_conflict")`. Oversized/unmeasurable input is rejected by CONTRACT-004 before transport; the adapter caps response bytes and tool argument bytes by explicit settings before materialization. |
Expand Down
Loading
Loading