diff --git a/AGENTS.md b/AGENTS.md index e949402..3828e0a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -328,5 +328,5 @@ a successful start. Snapshot request/identity scalars independently of mutable p Runtime uses explicit text byte caps with cancellation-responsive incremental UTF-8 accounting. Scripted explicit fragments emit once; replay regenerates physical identities and refuses inconsistent successful cassette text at load without changing cassette-v1 matching. The selected native adapter -remains nonstreaming and binds its whole-text delta to its explicit successful transport attempt. +only streams and binds each text delta to its explicit successful transport attempt. Do not infer current-policy live disclosure or broker support from frame conformance. diff --git a/docs/helix/02-design/adr/ADR-013-the-databricks-ai-gateway-is-a-first-class-backend-delivered-as-a-profile-and-never-the-only-backend.md b/docs/helix/02-design/adr/ADR-013-the-databricks-ai-gateway-is-a-first-class-backend-delivered-as-a-profile-and-never-the-only-backend.md index 76dda2c..425ae08 100644 --- a/docs/helix/02-design/adr/ADR-013-the-databricks-ai-gateway-is-a-first-class-backend-delivered-as-a-profile-and-never-the-only-backend.md +++ b/docs/helix/02-design/adr/ADR-013-the-databricks-ai-gateway-is-a-first-class-backend-delivered-as-a-profile-and-never-the-only-backend.md @@ -40,7 +40,7 @@ We will support the Databricks AI Gateway as a first-class backend, delivered as - *Reasoning-summary parts are validated and dropped.* A list-valued delta `content` must hold only `{"type": "reasoning", "summary": [{"type": "summary_text", "text", "signature"?}]}` parts. They never become answer text, deltas, tool input or telemetry. Text parts in list form, other part types and other keys refuse. Preserving reasoning as typed non-text output needs a model-port change and its own decision. - *Filter metadata is shape-checked, not kept.* Category maps allow only `filtered`/`detected` booleans and a `severity` string; a filter error allows only a string `code` and `message`. None of it enters `ModelResponse`, receipts or audit. A prompt category reporting `filtered: true` refuses as `prompt_filtered`; an output category reporting `filtered: true` requires a `content_filter` terminal, which normalizes to `filtered`. Tether is not the content-safety layer; the gateway's terminal reason governs. - *Unknown fields still refuse* under both profiles. The raw stream bytes remain the attempt's wire observation, so dropped fields stay in the original exchange record under existing payload authorization. - - Nonstreaming bodies, including the stream fallback when a server ignores `stream: true`, still use the strict body validator; this dialect is qualified for streaming only. Structure-only checks show gateway JSON bodies carry the same filter, annotation and reasoning extras, which the strict validator refuses. + - The adapter only streams. A gateway that answers a streaming request with one JSON body still gets the strict body validator; no gateway tier has been observed doing so. Structure-only checks show gateway JSON bodies carry the same filter, annotation and reasoning extras, which the strict validator refuses. - *Gateway cache counts map to the port's cache subcounts.* Two backends report Anthropic-style `cache_read_input_tokens` and `cache_creation_input_tokens` at the top level of `usage`, sometimes with a `cache_creation` breakdown by retention; the content-filtered backend reports OpenAI-style `prompt_tokens_details.cached_tokens` and `cache_write_tokens`. Live checks showed both styles are subsets of `prompt_tokens`: an identical 10,420-token prompt reported `prompt_tokens` 10,420 with 10,403 tokens written, then 10,403 read, then neither, and `total_tokens` stayed prompt plus completion. That matches `TokenUsage`, whose cache counts are subcounts never added to totals again. In streamed frames the profile therefore moves the top-level counts into `prompt_tokens_details` before validation and forwarding, so they become `cache_read_tokens` and `cache_write_tokens`; the `cache_creation` breakdown must use only the two known retention keys and sum to the creation count, and is then dropped. A nested count that conflicts with its top-level twin, a negative count or a cache sum above `prompt_tokens` refuses as `invalid_response`. The strict profile, and the strict nonstreaming fallback under either profile, leave top-level cache counts missing rather than guessing their meaning. Tether's request mapping sends no cache markers, so these counts are nonzero only when a deployment caches on its own. - **Configuration is explicit.** The adapter reads no environment variable. The application builds its settings object and passes the tier map and the identity mode in. - **Never the only backend.** Every feature of the core runs against the scripted model and against the default adapter on a plain OpenAI-compatible server. The adapter conformance matrix includes at least one backend that is not the Databricks AI Gateway. No contract field, port method, or default exists only to serve the Databricks backend. diff --git a/docs/helix/02-design/architecture.md b/docs/helix/02-design/architecture.md index 4f816b3..1137f89 100644 --- a/docs/helix/02-design/architecture.md +++ b/docs/helix/02-design/architecture.md @@ -5,7 +5,7 @@ ddx: activity: design status: draft owner: Erik LaBianca - updated: 2026-10-29 + updated: 2026-11-08 authoring: home: repo links: @@ -442,6 +442,6 @@ SessionService concern; this checkpoint introduces no second runner or live brok Model-port version 3 checks strict frame copies, scoped claimed identity, consistent effective attempt state and exact delta/terminal text using bounded incremental UTF-8 work. Request scope and identity state are immutable scalar snapshots. Scripted and replay doubles preserve explicit text -boundaries once; the native nonstreaming adapter carries the successful transport identity. No +boundaries once; the native streaming adapter carries the successful transport identity. No frame identity is itself proof of accepted start or permission to disclose live text. The following Engine/SessionService integration must own that verification, current access and bounded delivery. diff --git a/docs/helix/02-design/contracts/CONTRACT-001-model-port-and-test-doubles.md b/docs/helix/02-design/contracts/CONTRACT-001-model-port-and-test-doubles.md index d833956..e323dfc 100644 --- a/docs/helix/02-design/contracts/CONTRACT-001-model-port-and-test-doubles.md +++ b/docs/helix/02-design/contracts/CONTRACT-001-model-port-and-test-doubles.md @@ -233,8 +233,9 @@ Model-port version 3 adds optional claimed physical identities to text frames an nonempty-delta/terminal equality rule. Cassette layout and matching remain version 1; physical identities are regenerated for replay, not recorded as matching authority. Existing inconsistent experimental fixtures must be regenerated rather than silently repaired. The selected native -adapter remains nonstreaming and emits one whole-text delta bound to the successful transport -attempt, including after a retry before any text. Live disclosure additionally requires Engine-owned +adapter only streams. It binds every text delta to the successful transport attempt, including +after a retry before any text. A server that answers a streaming request with one JSON body gets +strict body validation and becomes one whole-text delta; there is no separate nonstreaming call. Live disclosure additionally requires Engine-owned proof of successful start and current session access; these frame claims alone establish neither. diff --git a/src/tether/adapters/pydantic_ai/compatible.py b/src/tether/adapters/pydantic_ai/compatible.py index fcc8c94..7403ce6 100644 --- a/src/tether/adapters/pydantic_ai/compatible.py +++ b/src/tether/adapters/pydantic_ai/compatible.py @@ -1,4 +1,4 @@ -"""Explicit nonstreaming compatible profile using the native Pydantic AI model.""" +"""Explicit streaming compatible profile using the native Pydantic AI model.""" from __future__ import annotations @@ -13,24 +13,21 @@ import httpx2 as httpx from openai import APIConnectionError -from pydantic_ai.exceptions import UnexpectedModelBehavior from pydantic_ai.messages import PartDeltaEvent, PartStartEvent, TextPart, TextPartDelta from tether.models.canonical import copy_request from tether.runtime.pydantic_ai.messages import from_sdk_response from tether.types import ( - AttemptIdentity, Credential, ModelExecutionLimits, ModelRequest, - ModelResponse, ModelUnavailable, StreamItem, TextDelta, ) from .estimator import OpenAIWireEstimator -from .native import request_native, request_native_stream, validate_payload +from .native import request_native_stream, validate_payload from .tokenizer import TokenizerProfile from .transport import AttemptTransport from .wire import STREAM_PROFILES, STRICT_STREAM_PROFILE @@ -128,51 +125,6 @@ def estimator(self, *, profile: TokenizerProfile, vocabulary: bytes) -> OpenAIWi vocabulary=vocabulary, ) - async def _once(self, request: ModelRequest, boundary: AttemptTransport) -> ModelResponse: - error: Exception | None = None - normalized: ModelResponse | None = None - try: - response = await request_native( - request, - base_url=self._base_url, - model_name=self._tiers[request.model], - output_limit_field=self._output_limit_field, - supports_parallel_tool_calls=self._supports_parallel_tool_calls, - transport=boundary, - ) - normalized = from_sdk_response( - response, boundary.response.usage if boundary.response else None - ) - except Exception as failure: - error = failure - if boundary.local_error is not None: - raise boundary.local_error - if isinstance(error, ModelUnavailable): - raise error - if ( - isinstance(error, UnexpectedModelBehavior) - and boundary.response is not None - and boundary.response.finish_reason == "unknown" - ): - # A validated observed unknown is incomplete evidence, never SDK execution authority. - return boundary.response - if error is not None: - reason = ( - "transport_error" if isinstance(error, APIConnectionError) else "invalid_response" - ) - raise ModelUnavailable(reason, receipt=boundary.receipt) from None - observed = boundary.response - if observed is None or normalized is None: - raise ModelUnavailable("invalid_response") - # A missing wire reason remains unknown even if the SDK defaults it to stop. - finish_matches = observed.finish_reason in {"unknown", normalized.finish_reason} - if (observed.content, observed.tool_calls) != ( - normalized.content, - normalized.tool_calls, - ) or not finish_matches: - raise ModelUnavailable("sdk_projection_changed") - return observed - async def stream(self, request: ModelRequest) -> AsyncIterator[StreamItem]: request = copy_request(request) self._validate_request(request) @@ -299,37 +251,3 @@ async def _stream_attempt( ) != (response.content, response.tool_calls, response.finish_reason): raise ModelUnavailable("sdk_projection_changed") yield response - - async def _retry(self, request: ModelRequest) -> tuple[ModelResponse, AttemptIdentity]: - for attempt in range(self._limits.max_attempts): - boundary = AttemptTransport( - self._transport, - request, - self._limits, - self._application_credential, - request_tags_header=self._request_tags_header, - stream_profile=self._stream_profile, - ) - try: - response = await self._once(request, boundary) - if boundary.identity is None: - raise ModelUnavailable("invalid_response") - return response, boundary.identity - except ModelUnavailable as error: - if ( - error.reason - not in { - "transport_error", - "http 429", - "http 500", - "http 502", - "http 503", - "http 504", - } - or attempt + 1 == self._limits.max_attempts - ): - raise - await asyncio.sleep( - retry_delay(boundary.retry_after, self._limits.max_retry_delay_seconds, attempt) - ) - raise ModelUnavailable("attempts_exhausted") diff --git a/src/tether/adapters/pydantic_ai/native.py b/src/tether/adapters/pydantic_ai/native.py index a522055..0f830fa 100644 --- a/src/tether/adapters/pydantic_ai/native.py +++ b/src/tether/adapters/pydantic_ai/native.py @@ -5,7 +5,6 @@ import httpx2 as httpx from openai import AsyncOpenAI -from pydantic_ai.messages import ModelResponse from pydantic_ai.models import ModelRequestParameters, StreamedResponse from pydantic_ai.models.openai import OpenAIChatModel, OpenAIChatModelSettings from pydantic_ai.profiles.openai import OpenAIModelProfile @@ -48,67 +47,6 @@ def validate_payload(request: ModelRequest, tiers: dict[str, str]) -> None: raise ModelUnavailable("unsupported_effort") -async def request_native( - request: ModelRequest, - *, - base_url: str, - model_name: str, - output_limit_field: str, - transport: httpx.AsyncBaseTransport, - supports_parallel_tool_calls: bool = True, -) -> ModelResponse: - async with ( - httpx.AsyncClient( - transport=transport, - trust_env=False, - follow_redirects=False, - headers={"Accept-Encoding": "identity"}, - ) as http, - AsyncOpenAI( - api_key="-".join(("tether", "transport", "placeholder")), - organization="", - project="", - admin_api_key="", - webhook_secret="", - base_url=base_url, - max_retries=0, - http_client=http, - ) as client, - ): - model = OpenAIChatModel( - model_name, - provider=OpenAIProvider(openai_client=client), - profile=OpenAIModelProfile( - openai_chat_supports_max_completion_tokens=output_limit_field - == "max_completion_tokens", - openai_supports_strict_tool_definition=False, - openai_unsupported_model_settings=[] - if supports_parallel_tool_calls - else ["parallel_tool_calls"], - ), - ) - parameters = ModelRequestParameters( - function_tools=[ - ToolDefinition( - name=t.name, - description=t.description, - parameters_json_schema=t.input_schema, - strict=False, - ) - for t in request.tools - ] - ) - settings = OpenAIChatModelSettings( - max_tokens=request.max_output_tokens, - extra_body=request.extra, - ) - if supports_parallel_tool_calls: - settings["parallel_tool_calls"] = False - return await model.request( - to_sdk_messages(request.messages, request.system), settings, parameters - ) - - @asynccontextmanager async def request_native_stream( request: ModelRequest, diff --git a/tests/test_compatible_retry_paths.py b/tests/test_compatible_retry_paths.py index bac6a34..902bb33 100644 --- a/tests/test_compatible_retry_paths.py +++ b/tests/test_compatible_retry_paths.py @@ -108,25 +108,7 @@ def test_retry_delay_honors_numeric_and_http_date_headers() -> None: assert 0 <= retry_delay(None, 0.01, 0) <= 0.01 -async def test_retry_path_recovers_after_transient_failure( - model_request: ModelRequest, limits: ModelExecutionLimits -) -> None: - statuses = iter([503, 200]) - - def handler(_: httpx.Request) -> httpx.Response: - status = next(statuses) - return httpx.Response(status, json=_wire() if status == 200 else {"e": 1}) - - model = _model(limits, handler) - try: - response, identity = await model._retry(model_request) - finally: - await model.aclose() - assert response.content == "Fabricated answer." - assert identity is not None - - -async def test_retry_path_refuses_permanent_and_exhausted_failures( +async def test_stream_does_not_retry_a_permanent_status( model_request: ModelRequest, limits: ModelExecutionLimits ) -> None: calls: list[int] = [] @@ -138,24 +120,10 @@ def handler(_: httpx.Request) -> httpx.Response: model = _model(limits, handler) try: with pytest.raises(ModelUnavailable, match="http 400"): - await model._retry(model_request) - assert len(calls) == 1 - finally: - await model.aclose() - - calls.clear() - - def always_unavailable(_: httpx.Request) -> httpx.Response: - calls.append(1) - return httpx.Response(503, json={"e": 1}) - - model = _model(limits, always_unavailable) - try: - with pytest.raises(ModelUnavailable, match="http 503"): - await model._retry(model_request) + await complete(model, model_request) finally: await model.aclose() - assert len(calls) == limits.max_attempts + assert len(calls) == 1 async def test_unknown_finish_reason_is_returned_as_incomplete_evidence(