[Pulsar SQL] Fix Pulsar SQL query bytes schema data error - #9631
Conversation
|
/pulsarbot run-failure-checks |
1 similar comment
|
/pulsarbot run-failure-checks |
codelipenghui
added
area/sql
| metricsTracker.start_RECORD_DESERIALIZE_TIME(); | ||
|
|
||
| SchemaInfo schemaInfo; | ||
| SchemaInfo schemaInfo = getBytesSchemaInfo(pulsarSplit.getSchemaType(), pulsarSplit.getSchemaName()); |
There was a problem hiding this comment.
I noticed the current getSchemaVersion in the RawMessageImpl return null
@Override
public byte[] getSchemaVersion() {
if (msgMetadata != null && msgMetadata.getMetadata().hasSchemaVersion()) {
return msgMetadata.getMetadata().getSchemaVersion();
} else {
return null;
}
}
It should not throw java.nio.BufferUnderflowException. Could you please confirm here?
There was a problem hiding this comment.
This method may cause the exception java.nio.BufferUnderflowException.
private SchemaInfo loadSchema(BytesSchemaVersion bytesSchemaVersion) throws PulsarAdminException {
ClassLoader originalContextLoader = Thread.currentThread().getContextClassLoader();
try {
Thread.currentThread().setContextClassLoader(InjectionManagerFactory.class.getClassLoader());
return pulsarAdmin.schemas()
.getSchemaInfo(topicName.toString(), ByteBuffer.wrap(bytesSchemaVersion.get()).getLong());
} finally {
Thread.currentThread().setContextClassLoader(originalContextLoader);
}
}
The schema version of Schema.BYTES is an empty bytes array. So ByteBuffer.wrap(bytesSchemaVersion.get()).getLong() will cause the exception java.nio.BufferUnderflowException.
| if (!schemaType.equals(SchemaType.BYTES) && !schemaType.equals(SchemaType.NONE)) { | ||
| return null; | ||
| } | ||
| if (schemaName.equals(Schema.BYTES.getSchemaInfo().getName())) { |
There was a problem hiding this comment.
Can you compare SchemaType instead of comparing schemaName?
There was a problem hiding this comment.
The schema types of the Schema.BYTES and Schema.BYTEBUFFER all are SchemaType.BYTES, so I use the schema name to make the comparison.
e73e177 to
4633855
Compare
|
/pulsarbot run-failure-checks |
|
/pulsarbot run-failure-checks |
|
/pulsarbot run-failure-checks |
|
This issue is introduced by #8422. So it not a bug fix for the 2.7.0. Removed the |
4 participants
Motivation
Currently, the Pulsar SQL query bytes schema data will cause an error.
Reproduce
Error log
Modifications
Add check for bytes schema, if the schema is bytes schema use the schema info of the bytes schema directly.
Verifying this change
Add a new integration test for different schemas.
Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changes