Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -328,5 +328,5 @@ a successful start. Snapshot request/identity scalars independently of mutable p
Runtime uses explicit text byte caps with cancellation-responsive incremental UTF-8 accounting.
Scripted explicit fragments emit once; replay regenerates physical identities and refuses inconsistent
successful cassette text at load without changing cassette-v1 matching. The selected native adapter
remains nonstreaming and binds its whole-text delta to its explicit successful transport attempt.
only streams and binds each text delta to its explicit successful transport attempt.
Do not infer current-policy live disclosure or broker support from frame conformance.
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ We will support the Databricks AI Gateway as a first-class backend, delivered as
- *Reasoning-summary parts are validated and dropped.* A list-valued delta `content` must hold only `{"type": "reasoning", "summary": [{"type": "summary_text", "text", "signature"?}]}` parts. They never become answer text, deltas, tool input or telemetry. Text parts in list form, other part types and other keys refuse. Preserving reasoning as typed non-text output needs a model-port change and its own decision.
- *Filter metadata is shape-checked, not kept.* Category maps allow only `filtered`/`detected` booleans and a `severity` string; a filter error allows only a string `code` and `message`. None of it enters `ModelResponse`, receipts or audit. A prompt category reporting `filtered: true` refuses as `prompt_filtered`; an output category reporting `filtered: true` requires a `content_filter` terminal, which normalizes to `filtered`. Tether is not the content-safety layer; the gateway's terminal reason governs.
- *Unknown fields still refuse* under both profiles. The raw stream bytes remain the attempt's wire observation, so dropped fields stay in the original exchange record under existing payload authorization.
- Nonstreaming bodies, including the stream fallback when a server ignores `stream: true`, still use the strict body validator; this dialect is qualified for streaming only. Structure-only checks show gateway JSON bodies carry the same filter, annotation and reasoning extras, which the strict validator refuses.
- The adapter only streams. A gateway that answers a streaming request with one JSON body still gets the strict body validator; no gateway tier has been observed doing so. Structure-only checks show gateway JSON bodies carry the same filter, annotation and reasoning extras, which the strict validator refuses.
- *Gateway cache counts map to the port's cache subcounts.* Two backends report Anthropic-style `cache_read_input_tokens` and `cache_creation_input_tokens` at the top level of `usage`, sometimes with a `cache_creation` breakdown by retention; the content-filtered backend reports OpenAI-style `prompt_tokens_details.cached_tokens` and `cache_write_tokens`. Live checks showed both styles are subsets of `prompt_tokens`: an identical 10,420-token prompt reported `prompt_tokens` 10,420 with 10,403 tokens written, then 10,403 read, then neither, and `total_tokens` stayed prompt plus completion. That matches `TokenUsage`, whose cache counts are subcounts never added to totals again. In streamed frames the profile therefore moves the top-level counts into `prompt_tokens_details` before validation and forwarding, so they become `cache_read_tokens` and `cache_write_tokens`; the `cache_creation` breakdown must use only the two known retention keys and sum to the creation count, and is then dropped. A nested count that conflicts with its top-level twin, a negative count or a cache sum above `prompt_tokens` refuses as `invalid_response`. The strict profile, and the strict nonstreaming fallback under either profile, leave top-level cache counts missing rather than guessing their meaning. Tether's request mapping sends no cache markers, so these counts are nonzero only when a deployment caches on its own.
- **Configuration is explicit.** The adapter reads no environment variable. The application builds its settings object and passes the tier map and the identity mode in.
- **Never the only backend.** Every feature of the core runs against the scripted model and against the default adapter on a plain OpenAI-compatible server. The adapter conformance matrix includes at least one backend that is not the Databricks AI Gateway. No contract field, port method, or default exists only to serve the Databricks backend.
Expand Down
4 changes: 2 additions & 2 deletions docs/helix/02-design/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ ddx:
activity: design
status: draft
owner: Erik LaBianca
updated: 2026-10-29
updated: 2026-11-08
authoring:
home: repo
links:
Expand Down Expand Up @@ -442,6 +442,6 @@ SessionService concern; this checkpoint introduces no second runner or live brok
Model-port version 3 checks strict frame copies, scoped claimed identity, consistent effective
attempt state and exact delta/terminal text using bounded incremental UTF-8 work. Request scope and
identity state are immutable scalar snapshots. Scripted and replay doubles preserve explicit text
boundaries once; the native nonstreaming adapter carries the successful transport identity. No
boundaries once; the native streaming adapter carries the successful transport identity. No
frame identity is itself proof of accepted start or permission to disclose live text. The following
Engine/SessionService integration must own that verification, current access and bounded delivery.
Original file line number Diff line number Diff line change
Expand Up @@ -233,8 +233,9 @@ Model-port version 3 adds optional claimed physical identities to text frames an
nonempty-delta/terminal equality rule. Cassette layout and matching remain version 1; physical
identities are regenerated for replay, not recorded as matching authority. Existing inconsistent
experimental fixtures must be regenerated rather than silently repaired. The selected native
adapter remains nonstreaming and emits one whole-text delta bound to the successful transport
attempt, including after a retry before any text. Live disclosure additionally requires Engine-owned
adapter only streams. It binds every text delta to the successful transport attempt, including
after a retry before any text. A server that answers a streaming request with one JSON body gets
strict body validation and becomes one whole-text delta; there is no separate nonstreaming call. Live disclosure additionally requires Engine-owned
proof of successful start and current session access; these frame claims alone establish neither.


Expand Down
86 changes: 2 additions & 84 deletions src/tether/adapters/pydantic_ai/compatible.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Explicit nonstreaming compatible profile using the native Pydantic AI model."""
"""Explicit streaming compatible profile using the native Pydantic AI model."""

from __future__ import annotations

Expand All @@ -13,24 +13,21 @@

import httpx2 as httpx
from openai import APIConnectionError
from pydantic_ai.exceptions import UnexpectedModelBehavior
from pydantic_ai.messages import PartDeltaEvent, PartStartEvent, TextPart, TextPartDelta

from tether.models.canonical import copy_request
from tether.runtime.pydantic_ai.messages import from_sdk_response
from tether.types import (
AttemptIdentity,
Credential,
ModelExecutionLimits,
ModelRequest,
ModelResponse,
ModelUnavailable,
StreamItem,
TextDelta,
)

from .estimator import OpenAIWireEstimator
from .native import request_native, request_native_stream, validate_payload
from .native import request_native_stream, validate_payload
from .tokenizer import TokenizerProfile
from .transport import AttemptTransport
from .wire import STREAM_PROFILES, STRICT_STREAM_PROFILE
Expand Down Expand Up @@ -128,51 +125,6 @@ def estimator(self, *, profile: TokenizerProfile, vocabulary: bytes) -> OpenAIWi
vocabulary=vocabulary,
)

async def _once(self, request: ModelRequest, boundary: AttemptTransport) -> ModelResponse:
error: Exception | None = None
normalized: ModelResponse | None = None
try:
response = await request_native(
request,
base_url=self._base_url,
model_name=self._tiers[request.model],
output_limit_field=self._output_limit_field,
supports_parallel_tool_calls=self._supports_parallel_tool_calls,
transport=boundary,
)
normalized = from_sdk_response(
response, boundary.response.usage if boundary.response else None
)
except Exception as failure:
error = failure
if boundary.local_error is not None:
raise boundary.local_error
if isinstance(error, ModelUnavailable):
raise error
if (
isinstance(error, UnexpectedModelBehavior)
and boundary.response is not None
and boundary.response.finish_reason == "unknown"
):
# A validated observed unknown is incomplete evidence, never SDK execution authority.
return boundary.response
if error is not None:
reason = (
"transport_error" if isinstance(error, APIConnectionError) else "invalid_response"
)
raise ModelUnavailable(reason, receipt=boundary.receipt) from None
observed = boundary.response
if observed is None or normalized is None:
raise ModelUnavailable("invalid_response")
# A missing wire reason remains unknown even if the SDK defaults it to stop.
finish_matches = observed.finish_reason in {"unknown", normalized.finish_reason}
if (observed.content, observed.tool_calls) != (
normalized.content,
normalized.tool_calls,
) or not finish_matches:
raise ModelUnavailable("sdk_projection_changed")
return observed

async def stream(self, request: ModelRequest) -> AsyncIterator[StreamItem]:
request = copy_request(request)
self._validate_request(request)
Expand Down Expand Up @@ -299,37 +251,3 @@ async def _stream_attempt(
) != (response.content, response.tool_calls, response.finish_reason):
raise ModelUnavailable("sdk_projection_changed")
yield response

async def _retry(self, request: ModelRequest) -> tuple[ModelResponse, AttemptIdentity]:
for attempt in range(self._limits.max_attempts):
boundary = AttemptTransport(
self._transport,
request,
self._limits,
self._application_credential,
request_tags_header=self._request_tags_header,
stream_profile=self._stream_profile,
)
try:
response = await self._once(request, boundary)
if boundary.identity is None:
raise ModelUnavailable("invalid_response")
return response, boundary.identity
except ModelUnavailable as error:
if (
error.reason
not in {
"transport_error",
"http 429",
"http 500",
"http 502",
"http 503",
"http 504",
}
or attempt + 1 == self._limits.max_attempts
):
raise
await asyncio.sleep(
retry_delay(boundary.retry_after, self._limits.max_retry_delay_seconds, attempt)
)
raise ModelUnavailable("attempts_exhausted")
62 changes: 0 additions & 62 deletions src/tether/adapters/pydantic_ai/native.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@

import httpx2 as httpx
from openai import AsyncOpenAI
from pydantic_ai.messages import ModelResponse
from pydantic_ai.models import ModelRequestParameters, StreamedResponse
from pydantic_ai.models.openai import OpenAIChatModel, OpenAIChatModelSettings
from pydantic_ai.profiles.openai import OpenAIModelProfile
Expand Down Expand Up @@ -48,67 +47,6 @@ def validate_payload(request: ModelRequest, tiers: dict[str, str]) -> None:
raise ModelUnavailable("unsupported_effort")


async def request_native(
request: ModelRequest,
*,
base_url: str,
model_name: str,
output_limit_field: str,
transport: httpx.AsyncBaseTransport,
supports_parallel_tool_calls: bool = True,
) -> ModelResponse:
async with (
httpx.AsyncClient(
transport=transport,
trust_env=False,
follow_redirects=False,
headers={"Accept-Encoding": "identity"},
) as http,
AsyncOpenAI(
api_key="-".join(("tether", "transport", "placeholder")),
organization="",
project="",
admin_api_key="",
webhook_secret="",
base_url=base_url,
max_retries=0,
http_client=http,
) as client,
):
model = OpenAIChatModel(
model_name,
provider=OpenAIProvider(openai_client=client),
profile=OpenAIModelProfile(
openai_chat_supports_max_completion_tokens=output_limit_field
== "max_completion_tokens",
openai_supports_strict_tool_definition=False,
openai_unsupported_model_settings=[]
if supports_parallel_tool_calls
else ["parallel_tool_calls"],
),
)
parameters = ModelRequestParameters(
function_tools=[
ToolDefinition(
name=t.name,
description=t.description,
parameters_json_schema=t.input_schema,
strict=False,
)
for t in request.tools
]
)
settings = OpenAIChatModelSettings(
max_tokens=request.max_output_tokens,
extra_body=request.extra,
)
if supports_parallel_tool_calls:
settings["parallel_tool_calls"] = False
return await model.request(
to_sdk_messages(request.messages, request.system), settings, parameters
)


@asynccontextmanager
async def request_native_stream(
request: ModelRequest,
Expand Down
38 changes: 3 additions & 35 deletions tests/test_compatible_retry_paths.py
Original file line number Diff line number Diff line change
Expand Up @@ -108,25 +108,7 @@ def test_retry_delay_honors_numeric_and_http_date_headers() -> None:
assert 0 <= retry_delay(None, 0.01, 0) <= 0.01


async def test_retry_path_recovers_after_transient_failure(
model_request: ModelRequest, limits: ModelExecutionLimits
) -> None:
statuses = iter([503, 200])

def handler(_: httpx.Request) -> httpx.Response:
status = next(statuses)
return httpx.Response(status, json=_wire() if status == 200 else {"e": 1})

model = _model(limits, handler)
try:
response, identity = await model._retry(model_request)
finally:
await model.aclose()
assert response.content == "Fabricated answer."
assert identity is not None


async def test_retry_path_refuses_permanent_and_exhausted_failures(
async def test_stream_does_not_retry_a_permanent_status(
model_request: ModelRequest, limits: ModelExecutionLimits
) -> None:
calls: list[int] = []
Expand All @@ -138,24 +120,10 @@ def handler(_: httpx.Request) -> httpx.Response:
model = _model(limits, handler)
try:
with pytest.raises(ModelUnavailable, match="http 400"):
await model._retry(model_request)
assert len(calls) == 1
finally:
await model.aclose()

calls.clear()

def always_unavailable(_: httpx.Request) -> httpx.Response:
calls.append(1)
return httpx.Response(503, json={"e": 1})

model = _model(limits, always_unavailable)
try:
with pytest.raises(ModelUnavailable, match="http 503"):
await model._retry(model_request)
await complete(model, model_request)
finally:
await model.aclose()
assert len(calls) == limits.max_attempts
assert len(calls) == 1


async def test_unknown_finish_reason_is_returned_as_incomplete_evidence(
Expand Down
Loading