Skip to content

PIP-85 Add Schema Information to Message in Java Client API - #10476

Merged
codelipenghui merged 23 commits into
apache:masterfrom
eolivelli:impl/message-getschema
May 12, 2021
Merged

PIP-85 Add Schema Information to Message in Java Client API#10476
codelipenghui merged 23 commits into
apache:masterfrom
eolivelli:impl/message-getschema

Conversation

@eolivelli

@eolivelli eolivelli commented May 4, 2021

Copy link
Copy Markdown
Contributor

This is the implementation for PIP-85
https://docs.google.com/document/d/1VWi5LHP44V31nP4bCui9d5RXwH6xc_phrUes6tvNguk/

  • introduce Optional<Schema<?>> Message.getReaderSchema() - new Public API
  • introduce Optional<Object> SchemaReader.getNativeSchema()` - new Public API
  • introduce Schema<?> atSchemaVersion(schemaVersion) to AbstractSchema (private API)
  • make KeyValueSchema extend AbstractSchema (private API)
  • rename MessageImpl and TopicMessageImpl getSchema to getSchemaInternal

The schema returned by Message.getReaderSchema has these properties:

  • getSchemaInfo returns the actual schema used to decode the Message
  • getNativeSchema returns the original native (ie. Avro) Schema for the correct version
  • it can be used for "decoding" payloads
  • it cannot be used for "encoding" payloads

@eolivelli
eolivelli marked this pull request as draft May 4, 2021 06:37
@eolivelli

Copy link
Copy Markdown
Contributor Author

This PR is a "draft", it is a reference for the discussion on the GDoc, please read the GDoc before commenting here

@eolivelli

Copy link
Copy Markdown
Contributor Author

Latest commit broke the tests.
Btw this is still a draft, until we finish the discussion on gdoc

@eolivelli
eolivelli force-pushed the impl/message-getschema branch from 148c6ea to b585bb2 Compare May 10, 2021 06:52
@eolivelli
eolivelli marked this pull request as ready for review May 11, 2021 09:29
@eolivelli

eolivelli commented May 11, 2021

Copy link
Copy Markdown
Contributor Author

The patch is ready for review.

@eolivelli
eolivelli force-pushed the impl/message-getschema branch from d635506 to 49ddab9 Compare May 11, 2021 11:11

@congbobo184 congbobo184 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! left some comment.

if (schema == null) {
return Optional.empty();
}
if (schema instanceof AutoConsumeSchema) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why AutoConsumeSchema cant extend AbstractSchema?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@congbobo184 I am afraid that this change may have side effects, AutoConsumeSchema is very special as well as AbstractSchema

If you want I can try to apply this suggestion in a follow up patch

* @throws SchemaSerializationException in case of unknown schema version
* @throws NullPointerException in case of null schemaVersion
*/
public Schema<?> atSchemaVersion(byte[] schemaVersion) throws SchemaSerializationException {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why named atSchemaVersion? may getSchemaByVersion is easy to understand?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sounds to me pretty good, let's see this example:

Schema<?> schema = .....
Schema<?> schemaVersion2 = schema.atSchemaVersion(2);

In this example is like having immutable objects, and you call a method to create a new object with different content

With getSchemaByVersion it looks to that a Schema is a "container" (or a registry) for Schemas

Schema<?> schema = .....
Schema<?> schemaVersion2 = schema.getSchemaByVersion(2);

@codelipenghui codelipenghui added this to the 2.8.0 milestone May 12, 2021
@codelipenghui
codelipenghui merged commit 4224110 into apache:master May 12, 2021
eolivelli added a commit to datastax/pulsar that referenced this pull request May 12, 2021
…0476)

This is the implementation for PIP-85
https://docs.google.com/document/d/1VWi5LHP44V31nP4bCui9d5RXwH6xc_phrUes6tvNguk/

- introduce `Optional<Schema<?>> Message.getReaderSchema()` - new Public API
- introduce  `Optional<Object>` SchemaReader.getNativeSchema()` - new Public API
- introduce `Schema<?> atSchemaVersion(schemaVersion)` to `AbstractSchema` (private API)
- make `KeyValueSchema` extend AbstractSchema (private API)
- rename MessageImpl and TopicMessageImpl `getSchema` to `getSchemaInternal`

The schema returned by Message.getReaderSchema has these properties:
- getSchemaInfo returns the actual schema used to decode the Message
- getNativeSchema returns the original native (ie. Avro) Schema for the correct version
- it can be used for "decoding" payloads
- it cannot be used for "encoding" payloads

(cherry picked from commit 4224110)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants