feat(outputschema): allow format, pattern and type lists, add opt-in outputSchemaStrict by danielle-mustillo · Pull Request #3284 · kagent-dev/kagent

danielle-mustillo · 2026-10-19T17:38:41Z

Why

An AgentTemplate outputSchema cannot use format, pattern, or a type list such as ["string","null"]. So a schema cannot ask for an ISO date, a ticket ID like OPS-123, or an optional string. OpenAI follows format and pattern only in strict mode, so this PR also adds outputSchemaStrict.

What changed

The compiler accepts RE2 pattern, type lists, and format with the 9 formats that OpenAI and Claude both document.
kagent validates each final answer. A failure fails the Task with the schema path, such as at /properties/due (format: date-time).
outputSchemaStrict sends strict to OpenAI, Azure OpenAI, and Foundry. This boolean will trigger the provider to send output in the targeted format/pattern. Defaults to false and is provider dependent (no op on Claude for example)
The Agent shows Compatible=False for a schema that would fail at run time, such as a strict schema that breaks OpenAI's rules or a Gemini schema that Go ADK rejects.

Blast radius

OutputSchemaStrict is a new boolean configuration on AgentTemplate which passes the strict flag to upstream providers that support it (OpenAI, Azure OpenAI, and Foundry requests)
By default OutputSchemaStrict: false. The Responses API (OpenAI models) used to default to strict, so an agent that relied on that needs that behaviour again need to set outputSchemaStrict: true.
A Gemini agent whose schema Go ADK cannot check now shows Compatible=False instead of failing each Task.
Gemini and Mistral now receive pattern and the formats.

Lines changed by type

From git diff --numstat origin/main...HEAD. Generated is *.pb.go, *_pb2.py*, and the CRD YAML.

Type	Files	Added	Removed
Code	13	619	61
API (proto)	1	7	0
Test	15	1296	8
UI	3	29	0
Docs	1	29	5
Generated	5	156	101
Total	38	2136	175

Verification

Kind, on OpenAI, Azure OpenAI, Foundry (Claude), Anthropic, Bedrock (Claude) direct and through agentgateway. Strict schemas returned valid answers. Strict on and off.
Kind, Gemini & Mistral , called directly.
Unit tests, lint, the Go E2E test TestSessionStructuredOutput/kagent, and a Playwright test of the checkbox pass.

Accept `format` and `pattern` on string schemas in `outputSchema` and `outputSchemaFrom`. `format` accepts the formats both OpenAI and Claude structured outputs document: date-time, time, date, duration, email, hostname, ipv4, ipv6, and uuid. Other formats, and either keyword on a non-string schema, fail compilation. Patterns must compile as RE2. Raw-schema providers (OpenAI, Azure OpenAI, Claude, Bedrock, Foundry) receive both keywords unchanged. The Gemini projection keeps `pattern` and only the date and date-time formats Gemini documents. The final output validator now asserts `format` as well as `pattern`. jsonschema-go treats `format` as an annotation, so each format is rewritten to an anchored RE2 pattern in the validator's copy of the schema, which keeps anyOf branch selection inside the validator. Signed-off-by: Danielle Mustillo <[email protected]>

Claude documents \b and \B as unsupported in structured output patterns. On Bedrock, a pattern with \b was accepted, but the model returned the escape text followed by degenerate repetition until max_tokens. Reject both assertions at compile time. Signed-off-by: Danielle Mustillo <[email protected]>

The ConfigMap path shares the compile gate with the inline field. Test that a referenced schema with format and pattern compiles unchanged into the runtime config, and that an unsupported format and a word-boundary pattern are rejected on this path too. Signed-off-by: Danielle Mustillo <[email protected]>

Add AgentTemplate outputSchemaStrict. When set, the Go ADK sends strict: true with the output schema on the Chat Completions and Responses APIs. The kagent Harness compiler rejects a strict schema that OpenAI would refuse: an object without additionalProperties false, a property missing from required, or an anyOf root. Signed-off-by: Danielle Mustillo <[email protected]>

…form Choosing "no output schema" in the AgentTemplate form removed outputSchema and outputSchemaFrom but kept outputSchemaStrict, so the CRD rule rejected the save. The outputSchemaStrict docs now say that Foundry's OpenAI-compatible surface also sends strict, and that the strict rules apply to every provider's schema. Comments that restated code are removed, and the printf meta-schema constant is named as a template. Signed-off-by: Danielle Mustillo <[email protected]>

The Responses API treats an omitted strict as true, while Chat Completions treats it as false. kagent omitted strict when outputSchemaStrict was false, so false could not turn strict mode off on the Responses API: a schema with an optional property compiled, then every request failed with 400. outputSchemaStrict is now optional: - unset omits strict, so the provider default applies (same request as main) - false sends strict: false - true sends strict: true and keeps the compile-time strict check The proto field becomes optional bool so the runtime can tell unset from false. Signed-off-by: Danielle Mustillo <[email protected]>

- Make outputSchemaStrict a plain boolean that defaults to false. False, or an omitted field, now sends strict: false with the schema to OpenAI, Azure OpenAI and Foundry in the openai format. Before, an omitted field omitted strict, and the Responses API, which is strict when strict is omitted, returned 400 for a schema that breaks the strict rules. True sends strict: true, still requires a schema, and still runs the strict check at compile time. Support for strict: true depends on the vendor, and the docs say so. - Carry outputSchemaStrict in the provider model config. The compiler writes it to strict_output_schema, a plain bool, on the OpenAI, Azure OpenAI and Foundry (openai format only) model configs, next to api_format, and CreateLLM copies it into the model's config. CreateLLM keeps its signature, and ADKOutputConfig.strict is removed. The old type assertion on *models.OpenAIModel depended on running before fileextract.WithFileText wrapped the model, and no test covered it. Mistral, which reuses OpenAIModel, still sends no strict. - Bound the size of output schema patterns. CheckPatterns, which replaces CheckPortablePatterns, adds up the program size of every pattern and format, estimated from the parse tree, and rejects a schema above 100,000 instructions. It runs before Resolve. Five 11 KB patterns had made the controller allocate 1.4 GB, and 1,586 format: ipv6 properties made the runtime validator allocate 517 MB. Walk properties and $defs in sorted order, so the same schema gives the same Compatible message on every compile. - Reject February 29 outside leap years, and email local parts with a leading, trailing or doubled dot. The date pattern matches time.Parse on every February 28, 29 and 30 from 0000 to 9999. - Reject keywords next to $ref in a strict schema. OpenAI and Azure OpenAI return 400 "$ref cannot have keywords" for it. - Strict errors no longer repeat their prefix. A schema-valued additionalProperties now says to list the keys or set outputSchemaStrict to false. The Gemini error names untyped nodes. - Docs and comments: the example sets additionalProperties: false, which Claude requires. anyOf, type null and an untyped node reject null too. The word boundary error names Claude and no longer advises ^ and $, which Cohere does not support. Document Claude's regex and parameter limits, what each provider receives, and the format checks' leniency. - Tests: CEL rows write outputSchemaStrict true and false explicitly, with and without a schema. Strict reaches each of the three providers through a recording endpoint with strict false and true, and Mistral sends none. The runtime offers set_model_response exactly when UsesSetModelResponse says so. Formats and word boundaries are checked under every child schema kind. The structured-output E2E sets outputSchemaStrict: true and checks that every request carrying the schema sends strict: true. Signed-off-by: Danielle Mustillo <[email protected]>

Assert each string format as a pattern in its own allOf entry and record that entry's schema path, so a format failure reports the node and the format instead of the keyword pattern. A node's own pattern still reports (pattern). Signed-off-by: Danielle Mustillo <[email protected]>

Signed-off-by: Danielle Mustillo <[email protected]>

Delete comments that repeat a name or describe our own design, and name the format map by its key (formatByAllOfPath) instead of explaining it. Rename the CEL test helper to unstructuredStrictTemplate for the same reason. Signed-off-by: Danielle Mustillo <[email protected]>

Signed-off-by: Danielle Mustillo <[email protected]>

EItanya

🤖 AI-generated review. I looked through the findings and they both seem worth looking at.

Reviewed 33c36f5. One correctness finding and one optional cleanup:

Format validation disagrees with JSON Schema semantics — format.go:49. The validator rejects valid email addresses such as "joe bloggs"@example.com, but accepts invalid values such as time: "12:00:00" without an offset and oversized hostnames. Consequently, valid answers can fail Tasks, while invalid answers can be published successfully. The validation probe reproduced 50 mismatches across 440 official JSON Schema string cases from the format conformance suite. Can we correct the format assertions and add the relevant conformance cases as regression coverage?
Optional: keep validation error interpretation inside outputschema — executor.go:184. The A2A executor now understands jsonschema-go's error wrapping and the validator's format-to-pattern transformation. Can we move this interpretation into outputschema.Validator and expose a sanitized location? That keeps schema transformation and its diagnostics under one owner.

Focused Go tests passed across the affected ADK, translator, and API packages. Live provider, E2E, and UI checks were not rerun for this review.

…rong Most cases come from the JSON Schema Test Suite draft2020-12 optional/format files at 89e83a74. A time needs an offset, a duration cannot skip a unit in the middle, an email can have a quoted local part or an IPv4 or IPv6 address literal, and a hostname is at most 253 characters. The hostname cases test that limit at its edge. The cases that the current patterns get wrong fail until the next commit. Signed-off-by: Danielle Mustillo <[email protected]>

…ations The format patterns now follow the JSON Schema Test Suite: - A time needs an offset. - A duration cannot skip a unit in the middle, such as P1Y2D or PT1H2S. - An email can have a quoted local part or an IPv4 or IPv6 address literal. - A hostname is at most 253 characters. RE2 has no lookahead, so a format can assert more than one pattern. Each pattern counts toward the CheckPatterns size limit. The address literals and the hostname length pattern raise the cost of email and hostname, so a schema now fits 76 email or 131 hostname properties, down from 380 and 392. Punycode and leap second placement stay loose, and docs/architecture/structured-output.md says so. Validator.Validate returns a *ViolationError. Its Location names the schema node and keyword, such as "/properties/due (format: date-time)", and omits the value and the generated pattern. outputschema adds the hidden format patterns, so it also parses jsonschema-go's error text and maps a pattern back to its format. The A2A executor no longer does either, and its error message is unchanged. Signed-off-by: Danielle Mustillo <[email protected]>

danielle-mustillo · 2026-10-22T17:47:09Z

Done in 53bf136.

Fixed time, duration, email and hostname. Two checks stay loose and are documented: non-English domain names (xn--...) aren't decoded, and a leap second (:60) is accepted at any time. A regex can't check either.
Done.

… answers Drop the regular expressions that asserted each string format on the final answer, and the violation location and multiple-value messages that came with them. The executor is back to the main branch's validation. format stays limited to the formats OpenAI and Claude both document, and is sent to the provider unchanged. JSON Schema 2020-12 treats format as an annotation, so jsonschema-go does not check it. Signed-off-by: Danielle Mustillo <[email protected]>

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 19, 2026

danielle-mustillo mentioned this pull request Oct 19, 2026

feat(outputschema): let an agent ask OpenAI for strict structured output #3286

Closed

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 20, 2026

danielle-mustillo added 4 commits October 20, 2026 14:50

danielle-mustillo changed the title ~~feat(outputschema): allow string format and pattern~~ feat(outputschema): allow string format and pattern, with opt-in OpenAI strict mode Oct 20, 2026

danielle-mustillo force-pushed the danielle-mustillo/output-schema-format-pattern branch from ee5b36e to c569e25 Compare October 20, 2026 18:54

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 20, 2026

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 21, 2026

github-actions Bot added the enhancement New feature or request label Oct 21, 2026

danielle-mustillo force-pushed the danielle-mustillo/output-schema-format-pattern branch from b0d3cfc to c32e31a Compare October 22, 2026 04:08

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 22, 2026

danielle-mustillo added 2 commits October 22, 2026 00:46

fix(outputschema): send every supported format to Gemini

da99fef

Signed-off-by: Danielle Mustillo <[email protected]>

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Oct 22, 2026

danielle-mustillo added 6 commits October 22, 2026 01:46

refactor(outputschema): map type names with a switch again

3db714c

Signed-off-by: Danielle Mustillo <[email protected]>

docs(outputschema): trim ToGenAISchema comment

aa01421

Signed-off-by: Danielle Mustillo <[email protected]>

docs(api): shorten the outputSchemaStrict description

f40c6f6

Signed-off-by: Danielle Mustillo <[email protected]>

docs(outputschema): keep the structured output doc high level

f355b10

Signed-off-by: Danielle Mustillo <[email protected]>

docs(outputschema): tighten strict and structured output docs

0acef5a

Signed-off-by: Danielle Mustillo <[email protected]>

docs(outputschema): keep the code pointer with the schema profile

33c36f5

Signed-off-by: Danielle Mustillo <[email protected]>

EItanya reviewed Oct 22, 2026

View reviewed changes

danielle-mustillo added 2 commits October 22, 2026 13:26

EItanya approved these changes Oct 22, 2026

View reviewed changes

Navigation Menu

Repository navigation

feat(outputschema): allow format, pattern and type lists, add opt-in outputSchemaStrict - #3284

feat(outputschema): allow format, pattern and type lists, add opt-in outputSchemaStrict#3284
EItanya merged 29 commits into
kagent-dev:mainfrom
danielle-mustillo:danielle-mustillo/output-schema-format-pattern

danielle-mustillo commented Oct 19, 2026 •

edited

Loading

Uh oh!

EItanya left a comment

Uh oh!

danielle-mustillo commented Oct 22, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

danielle-mustillo commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changed

Blast radius

Lines changed by type

Verification

Uh oh!

EItanya left a comment

Choose a reason for hiding this comment

Uh oh!

danielle-mustillo commented Oct 22, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

danielle-mustillo commented Oct 19, 2026 •

edited

Loading