Skip to content

[FLINK-34228] Add long UTF serializer/deserializer#24191

Merged
gyfora merged 1 commit into
apache:masterfrom
pvary:long_utf
Mar 19, 2024
Merged

[FLINK-34228] Add long UTF serializer/deserializer#24191
gyfora merged 1 commit into
apache:masterfrom
pvary:long_utf

Conversation

@pvary
Copy link
Copy Markdown
Contributor

@pvary pvary commented Jan 24, 2024

What is the purpose of the change

Add a way to serialize UTF8 strings where the resulting byte array is longer than 64k

Brief change log

  • Added DataOutputSerializer.writeLongUTF
  • Added DataInputDeserializer.readLongUTF

Verifying this change

  • Added DataInputOutputSerializerTest.testLongUTFWriteRead unit test

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? no

@flinkbot
Copy link
Copy Markdown
Collaborator

flinkbot commented Jan 24, 2024

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@pvary
Copy link
Copy Markdown
Contributor Author

pvary commented Jan 24, 2024

@flinkbot run azure

2 similar comments
@pvary
Copy link
Copy Markdown
Contributor Author

pvary commented Jan 24, 2024

@flinkbot run azure

@pvary
Copy link
Copy Markdown
Contributor Author

pvary commented Jan 25, 2024

@flinkbot run azure

@pvary pvary force-pushed the long_utf branch 2 times, most recently from 3a0bbe1 to 5afff51 Compare January 25, 2024 13:43
@pvary
Copy link
Copy Markdown
Contributor Author

pvary commented Jan 26, 2024

@flinkbot run azure

@pvary
Copy link
Copy Markdown
Contributor Author

pvary commented Mar 1, 2024

@flinkbot run azure

To check the patch in 1.20

@gyfora gyfora merged commit f759352 into apache:master Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants