support generics in from row and to row conversions#37347
support generics in from row and to row conversions#37347tilgalas wants to merge 1 commit intoapache:masterfrom
Conversation
Summary of ChangesHello @tilgalas, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly advances the Apache Beam schema system by implementing robust support for generic types in Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
|
Run Java PreCommit |
|
Run Java_IOs_Direct PreCommit |
|
Assigning reviewers: R: @ahmedabu98 for label java. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
|
assign to next reviewer |
|
Assigning reviewers: R: @Abacn for label java. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
|
Reminder, please take a look at this pr: @Abacn |
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @chamikaramj for label java. Available commands:
|
|
Reminder, please take a look at this pr: @chamikaramj |
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @ahmedabu98 for label java. Available commands:
|
|
Sorry for the slowness on review - it is a large change so it takes a lot to page in. |
|
Reminder, please take a look at this pr: @ahmedabu98 @kennknowles |
No worries, I appreciate the update! |
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @chamikaramj for label java. Available commands:
|
|
Reminder, please take a look at this pr: @chamikaramj @kennknowles |
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @ahmedabu98 for label java. Available commands:
|
|
Reminder, please take a look at this pr: @ahmedabu98 @kennknowles |
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @Abacn for label java. Available commands:
|
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request enhances the Apache Beam schema system to improve support for generic types in AutoValue, JavaBean, and POJO schemas. Key changes include making FieldValueTypeInformation builder methods public, refining FromRowUsingCreator to use getGetterTargetType(), and extensively refactoring GetterBasedSchemaProvider to propagate TypeDescriptor information for nested types (rows, arrays, iterables, maps) to preserve generic type details. AutoValueUtils is updated for more accurate constructor parameter matching with generics, and Row and RowWithGetters now store and utilize getterTargetType. New test cases validate these generic type handling improvements across various nested structures. A review comment highlights that the fieldValueTypeInformationFor implementation in AwsTypes.java may not fully handle generic types for collections and maps within AWS SDK objects, potentially losing generic type parameters and suggesting a more robust approach to resolve component types.
| static FieldValueTypeInformation fieldValueTypeInformationFor(SdkField<?> sdkField) { | ||
| TypeDescriptor<?> type = TypeDescriptor.of(sdkField.marshallingType().getTargetClass()); | ||
| return FieldValueTypeInformation.builder() | ||
| .setName(normalizedNameOf(sdkField)) | ||
| .setType(type) | ||
| .setRawType(sdkField.marshallingType().getClass()) | ||
| .setElementType(FieldValueTypeInformation.getIterableComponentType(type)) | ||
| .setMapKeyType(FieldValueTypeInformation.getMapKeyType(type)) | ||
| .setMapValueType(FieldValueTypeInformation.getMapValueType(type)) | ||
| .setOneOfTypes(Collections.emptyMap()) | ||
| .setNullable(true) | ||
| .build(); | ||
| } |
There was a problem hiding this comment.
The implementation of fieldValueTypeInformationFor appears to not fully handle generic types for collections and maps within AWS SDK objects.
Using TypeDescriptor.of(sdkField.marshallingType().getTargetClass()) will lose generic type parameters. For example, for a List<String>, getTargetClass() returns List.class, and TypeDescriptor.of(List.class) does not retain the String element type. This means FieldValueTypeInformation.getIterableComponentType(type) will likely resolve to Object.
To correctly support collections and maps, you could inspect sdkField.marshallingType(). If it's a list or map, you can recursively determine the component types, similar to how the fieldType method in this class does it using MarshallingInfo.MEMBER_MARSHALLING_TYPE.
For instance, for lists, you could do something like:
if (sdkField.marshallingType() == MarshallingType.LIST) {
MarshallingType<?> memberMarshallingType = sdkField.getMemberProperty(MarshallingInfo.MEMBER_MARSHALLING_TYPE);
// ... use memberMarshallingType to get element type information
}This would provide more accurate type information, aligning better with the overall goal of this pull request to improve generics support.
4 participants
This is a 5th PR in the series of PRs (based on the now closed #31648) whose ultimate goal is to add support for generic classes to schema providers.
This PR builds on top of the previous PRs by updating "FromRow" and "ToRow" functions so that they understand the generic type information and can correctly perform the necessary conversions. With this PR we should be achieving full support for generics in schema providers.
See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.