Skip to content

Make jsonformat strict schemas OpenAI-valid by requiring all properties - #689

Open
PratikDhanave wants to merge 2 commits into
microsoft:mainfrom
PratikDhanaveFork:jsonformat-strict-require-all-properties
Open

Make jsonformat strict schemas OpenAI-valid by requiring all properties#689
PratikDhanave wants to merge 2 commits into
microsoft:mainfrom
PratikDhanaveFork:jsonformat-strict-require-all-properties

Conversation

@PratikDhanave

Copy link
Copy Markdown
Contributor

What

jsonformat marks every response format as Strict: true, and the OpenAI providers forward that schema verbatim with strict: true. But jsonschema-go inference only appends a field to required when it is not omitempty/omitzero, while still adding it to properties and setting additionalProperties: false.

So a struct like:

type Struct struct {
    Name  string `json:"name"`
    Age   int    `json:"age"`
    Email string `json:"email,omitempty"`
}

produces properties: {name, age, email} but required: [name, age]. OpenAI strict structured outputs require every key in properties to also appear in required, so this schema is rejected with HTTP 400.

This change rewrites the stored schema so every object requires all of its properties, expressing optionality via nullable types instead of omission.

Why (parity)

This mirrors .NET's AIJsonSchemaCreateOptions.RequireAllProperties = true, which the .NET/Python SDKs apply when emitting strict schemas: all properties become required and optional ones are made nullable rather than omitted. Doing the transform once in jsonformat (rather than in each provider) keeps every provider that reads ResponseFormat.Schema consistent.

How it stays correct locally

The strict form is only appropriate for what is sent to the model. Locally, Format.Marshal/Unmarshal/Normalize validate Go values (including tool arguments and tool outputs) that legitimately omit omitempty fields. To avoid breaking that, resolvedSchema validates against a relaxed deep-clone that drops the nullable (optional) properties back out of required; the strict schema itself is left untouched for transmission. Model responses under strict mode contain every key, so they still validate.

Tests

  • Added TestStrictRequiresAllProperties in the canonical jsonformat_test.go: builds a ResponseFormat for a struct with omitempty fields (including a nested object and an array element) and asserts each object's required covers every key in properties. It fails before this change (required is missing the optional keys) and passes after.
  • The existing package tests (TestNormalizeAppliesDefaults, encoding round-trips) and the repo-wide tool-argument/structured-output tests continue to pass, confirming local validation still honors optionality.

Copilot AI review requested due to automatic review settings July 24, 2026 01:39
@PratikDhanave
PratikDhanave requested a review from a team as a code owner July 24, 2026 01:39
@PratikDhanave
PratikDhanave force-pushed the jsonformat-strict-require-all-properties branch from 56c5742 to 390f45b Compare July 24, 2026 01:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the jsonformat response-format schema generation so that when Strict: true schemas are sent to OpenAI, every object lists all keys in properties within its required array (matching OpenAI strict structured-output requirements). It also preserves local Go-side validation semantics by resolving/validating against a relaxed clone of the schema.

Changes:

  • Rewrite inferred JSON Schemas in jsonformat so every object requires all properties; properties that were previously optional are made nullable.
  • Relax strict-only requirements for local validation by cloning the schema and dropping nullable properties from required before resolving.
  • Add a test asserting required covers all properties for nested objects and array elements under strict mode.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
agent/format/jsonformat/jsonformat.go Adds strict-schema rewrite (require-all-properties + nullable optional props) and relaxed schema resolution for local validation.
agent/format/jsonformat/jsonformat_test.go Adds coverage to ensure OpenAI strict schemas have required covering all properties for nested/object/array-element schemas.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +53 to +55
// requiredKeys returns the "required" and "properties" key sets of the object
// schema at path (a sequence of property names to descend through).
func objectSchema(t *testing.T, format agent.ResponseFormat, path ...string) *jsonschema.Schema {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in b0a8e9a: reworded the comment to describe what objectSchema actually does (returns the object schema reached by descending through path, stepping into array item schemas), dropping the stale requiredKeys/key-set wording.

Comment thread agent/format/jsonformat/jsonformat.go Outdated
Comment on lines +176 to +177
// makeNullable marks s as accepting the JSON null value in addition to its
// existing type(s), so an optional property can be omitted by returning null.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in b0a8e9a: reworded to clarify that in a strict schema every property stays required, and an originally optional property signals "no value" by being null rather than by being omitted.

@github-actions github-actions Bot added the parity-approved Go API consistency review found no parity issues label Jul 24, 2026
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Parity Review — No Issues Found

This PR makes jsonformat strict schemas OpenAI-valid by requiring all properties and expressing optionality via nullable types rather than omission.

Scope: agent/format/jsonformat/jsonformat.go and jsonformat_test.go. No exported symbols were added, removed, or renamed — all new logic (makeStrict, relaxStrict, walkSchema, cloneSchema, makeNullable, isNullable) is unexported.

Cross-repo parity: The makeStrict transform directly mirrors .NET's AIJsonSchemaCreateOptions.RequireAllProperties = true from Microsoft.Extensions.AI, which the .NET Agent Framework SDK applies when emitting strict structured-output schemas. The Python implementation delegates to microsoft-extensions-ai for the same transform. Centralising the rewrite in jsonformat (rather than each provider) is consistent with how the upstream SDKs place this concern in their JSON-utilities layer.

Local validation: The PR preserves local Marshal/Unmarshal/Normalize semantics by resolving a relaxStrict deep-clone (nullable-but-optional properties dropped from required) for validation, while leaving the strict schema untouched for transmission — matching the existing principle of separating model-wire schemas from local validation.

public-api-change label: Not applicable — no exported API surface changed.

Parity verdict: The change aligns Go behaviour with .NET and Python. parity-approved is correct.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by Go API Consistency Review Agent · 89.8 AIC · ⌖ 5.66 AIC · ⊞ 5.9K ·

OpenAI strict structured outputs require every key in an object's
"properties" to also appear in "required". The jsonschema-go inference
omits omitempty/omitzero fields from "required", so an inferred schema
such as {properties:{name,age}, required:[name], additionalProperties:false}
is rejected by OpenAI with HTTP 400.

Rewrite the schema stored in the response format so every object requires
all of its properties, expressing optionality via nullable types rather
than omission. This mirrors .NET's
AIJsonSchemaCreateOptions.RequireAllProperties = true.

The strict form is only correct for transmission; local validation of Go
values still omits omitempty fields, so resolvedSchema validates against a
relaxed clone that drops nullable (optional) properties from "required".
This keeps tool-argument and structured-output validation behavior intact.
@PratikDhanave
PratikDhanave force-pushed the jsonformat-strict-require-all-properties branch from b0a8e9a to dae1ad1 Compare July 24, 2026 09:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parity-approved Go API consistency review found no parity issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants