AVRO-2825: [csharp] Resolve: C# Logical Types throw exception on unknown logical type#2741
Closed
TomBruns wants to merge 4 commits intoapache:mainfrom
TomBruns:main
Closed
AVRO-2825: [csharp] Resolve: C# Logical Types throw exception on unknown logical type#2741TomBruns wants to merge 4 commits intoapache:mainfrom TomBruns:main
TomBruns wants to merge 4 commits intoapache:mainfrom
TomBruns:main
Conversation
added 3 commits
February 16, 2024 07:42
Language implementations must ignore unknown logical types when reading, and should use the underlying Avro type.
also resolved exception thrown if namespace is missing from schema
… type" This reverts commit c9c1bed.
… the schema file does not contain a namespace This reverts commit 3f75a9a.
KalleOlaviNiemitalo
suggested changes
Feb 16, 2024
Comment on lines
+195
to
+202
| try | ||
| { | ||
| if (null != jo["logicalType"]) // logical type based on a primitive | ||
| return LogicalSchema.NewInstance(jtok, props, names, encspace); | ||
| } | ||
| // swallow exception from unknown logicalType | ||
| catch { } | ||
|
|
Contributor
There was a problem hiding this comment.
Can LogicalSchema.NewInstance instead be made to work with unrecognized logical types? For these purposes:
- Avoid swallowing any exceptions thrown for other reasons.
- Allow applications to parse a schema and read the name of the logical type from LogicalSchema.LogicalTypeName (or even LogicalType.Name) regardless of whether the library recognizes it.
- Allow applications to parse a schema and serialize it back to JSON without losing the unrecognized logical types here:
avro/lang/csharp/src/apache/main/Schema/Property.cs
Lines 45 to 46 in 49e619b
LogicalSchema.LogicalType could then be null, or perhaps an instance of a new NotSupportedLogicalType class that would pass everything through without conversion.
Author
There was a problem hiding this comment.
I will work on that, thanks for the feedback
Comment on lines
-896
to
+903
| if (string.IsNullOrEmpty(nspace)) | ||
| { | ||
| throw new CodeGenException("Namespace required for record schema " + recordSchema.Name); | ||
| } | ||
|
|
||
| CodeNamespace codens = AddNamespace(nspace); | ||
|
|
||
| //if (string.IsNullOrEmpty(nspace)) | ||
| //{ | ||
| // throw new CodeGenException("Namespace required for record schema " + recordSchema.Name); | ||
| //} | ||
|
|
||
| // AVRO spec DOES NOT require a Namespace but this code does. | ||
| // Workaround is to inject a fixed string that will be obvious if required | ||
| CodeNamespace codens = (!string.IsNullOrEmpty(nspace)) ? AddNamespace(nspace) : AddNamespace(@"SchemaHadNoNamespace"); |
Contributor
There was a problem hiding this comment.
This is not related to logical types. Please move to a separate pull request.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| return LogicalSchema.NewInstance(jtok, props, names, encspace); | ||
| } | ||
| // swallow exception from unknown logicalType | ||
| catch { } |
Check notice
Code scanning / CodeQL
Generic catch clause
| return LogicalSchema.NewInstance(jtok, props, names, encspace); | ||
| } | ||
| // swallow exception from unknown logicalType | ||
| catch { } |
Check notice
Code scanning / CodeQL
Poor error handling: empty catch block
| var secondField = ((RecordSchema)schema).Fields.FirstOrDefault(f => f.Name == @"secondField"); | ||
| Assert.IsNotNull(secondField); | ||
|
|
||
| var secondFieldSchema = ((Field)secondField).Schema; |
Check warning
Code scanning / CodeQL
Cast to same type
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AVRO-2825
What is the purpose of the change
NOTE: This PR WILL change the behavior of the current nuget package. It corrects it to align with the AVRO spec.
Verifying this change
This change is already covered by existing tests:
test > AvroGen > AvroGenSchemaTests.cs > NotSupportedSchema
test > Schema > SchemaTests.cs > TestUnknownLogical
This change added tests and can be verified as follows:
Documentation