Core: Added {} and null support for default values within SchemaParser#14052
Core: Added {} and null support for default values within SchemaParser#14052Verest wants to merge 1 commit into
Conversation
|
@rdblue Here is a draft PR adding in |
c47e84e to
fbee5b3
Compare
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
|
Needs reviewed. |
b8871d1 to
3ff6d7c
Compare
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
|
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
Overview
SchemaParserdoes not support parsing schema's default values containing{}(structs) or explicitnulls (any supported data type). From the spec:nullor{}Struct Change
This PR adds a new
Literalfor specificallyEmptyStructLiketo represent{}values. I reused the existingEmptyStructLike, but made it public which will expose to the class to consumers oficeberg-apiand by extensioniceberg-core.I did not tether the Literal to the generic
StructLikeinterface as:I did not add conversions between String / EmptyStructLike
Literals as that conversion is unrelated to this PR and probably undesired.Null Change
This PR is leaving existing behavior around nulls default values for Schema -> JSON parsing. A small refactor was done to support null default values in metadata.json files when parsing JSON -> Schema.
Additional changes (if needed) around Schema -> JSON parsing can be done in another PR.
Asides
boolean supportsDefaultValues()method ofDataTestBaseis N/A forTestSchemaParseras the underlying implementation defers to awriteAndValidatemethod. My changes are unrelated to writing the actual default data.{}is for a writer engine to be able to determine if a default struct value is null or Empty. I.e. usingEmptyStructTypeas a flag for this scenario, so writers can react appropriately.