Skip to content

Core: Added {} and null support for default values within SchemaParser#14052

Closed
Verest wants to merge 1 commit into
apache:mainfrom
Verest:defaultValues
Closed

Core: Added {} and null support for default values within SchemaParser#14052
Verest wants to merge 1 commit into
apache:mainfrom
Verest:defaultValues

Conversation

@Verest
Copy link
Copy Markdown

@Verest Verest commented Sep 11, 2025

Overview

SchemaParser does not support parsing schema's default values containing {} (structs) or explicit nulls (any supported data type). From the spec:

  • When an optional field is added, the defaults may be null and should be explicitly set
  • The default value of a struct can be either null or {}

Struct Change

This PR adds a new Literal for specifically EmptyStructLike to represent {} values. I reused the existing EmptyStructLike, but made it public which will expose to the class to consumers of iceberg-api and by extension iceberg-core.

I did not tether the Literal to the generic StructLike interface as:

  • A non-empty StructLike Literal has no functional use at the moment.
  • A non-empty StructLike Literal seemed riskier for potential side effects, given Literal is used outside of default values.
  • Determining if a StructLike is empty appeared to require a recursive search for non-null values, which is unnecessary complexity.

I did not add conversions between String / EmptyStructLike Literals as that conversion is unrelated to this PR and probably undesired.

Null Change

This PR is leaving existing behavior around nulls default values for Schema -> JSON parsing. A small refactor was done to support null default values in metadata.json files when parsing JSON -> Schema.

Additional changes (if needed) around Schema -> JSON parsing can be done in another PR.

Asides

  • Assumed boolean supportsDefaultValues() method of DataTestBase is N/A for TestSchemaParser as the underlying implementation defers to a writeAndValidate method. My changes are unrelated to writing the actual default data.
  • Assuming the goal of {} is for a writer engine to be able to determine if a default struct value is null or Empty. I.e. using EmptyStructType as a flag for this scenario, so writers can react appropriately.
  • Related thread on slack

Comment thread api/src/test/java/org/apache/iceberg/TestSchema.java
@Verest
Copy link
Copy Markdown
Author

Verest commented Sep 11, 2025

@rdblue Here is a draft PR adding in null / {} supports for default values. Pinging you as requested.

@Verest Verest marked this pull request as ready for review September 25, 2025 18:00
@github-actions
Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Oct 26, 2025
@Verest
Copy link
Copy Markdown
Author

Verest commented Oct 27, 2025

Needs reviewed.

@github-actions github-actions Bot removed the stale label Oct 28, 2025
@Verest Verest force-pushed the defaultValues branch 2 times, most recently from b8871d1 to 3ff6d7c Compare November 4, 2025 19:11
@Verest Verest marked this pull request as draft December 4, 2025 18:35
@Verest Verest marked this pull request as ready for review December 4, 2025 22:50
@Verest Verest changed the title Draft: Added null and {} support for default values within SchemaParser Draft: Added {} and null support for default values within SchemaParser Dec 4, 2025
@Verest Verest changed the title Draft: Added {} and null support for default values within SchemaParser Core: Added {} and null support for default values within SchemaParser Dec 30, 2025
@github-actions
Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Jan 30, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 7, 2026

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

@github-actions github-actions Bot closed this Feb 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant