refactor: Move SchemaAdapter from parquet module to data source #10680
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
HawaiianSpork
force-pushed
the
schema-adapter-top-level
branch
from
15618af to
631ff7d
Compare
This is not a change in behavior except moving the public location of SchemaAdapter. SchemaAdapter was exposed in apache#10515 to allow callers to define their own implementation. This PR then changes the location so that it could be used in other data sources.
HawaiianSpork
force-pushed
the
schema-adapter-top-level
branch
from
631ff7d to
a6fa3d9
Compare
alamb
approved these changes
May 27, 2024
Contributor
alamb
left a comment
alamb
left a comment
There was a problem hiding this comment.
Thank you @HawaiianSpork -- this is very nicely done 👏
| } | ||
| } | ||
|
|
||
| #[cfg(test)] |
Contributor
There was a problem hiding this comment.
Moving the tests to be along side the schema mapper makes a lot of sense
9 tasks
findepi
pushed a commit
to findepi/datafusion
that referenced
this pull request
Jul 16, 2024
…he#10680) * refactor: Move SchemaAdapter from parquet module to data source This is not a change in behavior except moving the public location of SchemaAdapter. SchemaAdapter was exposed in apache#10515 to allow callers to define their own implementation. This PR then changes the location so that it could be used in other data sources. * fix comments surrounding tests to be accurate.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is not a change in behavior except moving the public location of SchemaAdapter. SchemaAdapter was exposed in #10515 to allow callers to define their own implementation.
Let me know if you want me to create a corresponding issue.
Rationale for this change
This PR then changes the location so that SchemaAdapter could be used in other data sources since it isn't parquet specific.
What changes are included in this PR?
Moves the public location of
SchemaAdapterfrom inside theparquetmodule to thedatasourcemodule.Are these changes tested?
Reusing existing tests
Are there any user-facing changes?
This is an API change but the previous exposed API had not been released.