[fix][sql][branch-3.0] Fix long decimal compatibility in Trino 368.

[shibd]

Motivation

After upgrading to trino368 #20016, the long decimal type is represented by a block type.

However, Trino expects to get an Int128 type, which causes an error.

java.lang.IllegalArgumentException: Expected field 6 to be type interface io.trino.spi.block.Block but is class io.trino.spi.type.Int128
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:463)
	at org.apache.pulsar.sql.presto.PulsarRecordCursor.checkFieldType(PulsarRecordCursor.java:761)
	at org.apache.pulsar.sql.presto.PulsarRecordCursor.getFieldValueProvider(PulsarRecordCursor.java:710)
	at org.apache.pulsar.sql.presto.PulsarRecordCursor.getObject(PulsarRecordCursor.java:716)
	at io.trino.spi.connector.RecordPageSource.getNextPage(RecordPageSource.java:116)
	at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:311)
	at io.trino.operator.Driver.processInternal(Driver.java:388)
	at io.trino.operator.Driver.lambda$processFor$9(Driver.java:292)
	at io.trino.operator.Driver.tryWithLock(Driver.java:685)

Modifications

1. Remove file type check in PulsarRecordCursor.getObject(): We can't simply assume that a Block is an object in trino368;
2. If block is Int128ArrayBlock, convert to Int32: This is a simple workaround for the current version. In the new version of Trino, FieldValueProvider has provided getObject to replace getBlock, keeping it consistent with the RecordCursor interface. Since it's only contributed back to branch-3.0, a simple fix is provided here.

Verifying this change

• Verify JSON and AVRO schema with LongDecimalType.

trino> SELECT * FROM pulsar."public/default"."my-topic";
 booleanfield | datefield  | decimalfield | doublefield | floatfield | intfield |        longdecimalfield         | longfield | stringfield |  timefield   |     timestampfield      |              uuidfield               | __partition_>
--------------+------------+--------------+-------------+------------+----------+---------------------------------+-----------+-------------+--------------+-------------------------+--------------------------------------+------------->
 true         | 2024-10-08 |        22.33 |        22.2 |        2.2 |       22 | 1234567891234567891234567891.23 |       222 | message_1   | 20:59:50.000 | 2024-10-08 12:59:50.672 | 2bedf524-f17b-410e-b499-05f0d16a3ce7 |            ->
 true         | 2024-10-08 |        22.33 |        22.2 |        2.2 |       22 | 1234567891234567891234567891.23 |       222 | message_1   | 20:59:50.000 | 2024-10-08 12:59:50.685 | 8c6507b8-7211-45e0-a4d6-71bea3a8f58b |            ->
 true         | 2024-10-08 |        22.33 |        22.2 |        2.2 |       22 | 1234567891234567891234567891.23 |       222 | message_1   | 20:59:50.000 | 2024-10-08 12:59:50.706 | ade887cf-32b9-4846-ad7f-7294ce8638b6 |            ->
 true         | 2024-10-08 |        22.33 |        22.2 |        2.2 |       22 | 1234567891234567891234567891.23 |       222 | message_1   | 20:59:50.000 | 2024-10-08 12:59:50.717 | 73c32a72-3bcd-4d2d-adec-668074a85049 |            ->
 true         | 2024-10-08 |        22.33 |        22.2 |        2.2 |       22 | 1234567891234567891234567891.23 |       222 | message_1   | 20:59:50.000 | 2024-10-08 12:59:50.487 | 284237e5-0972-4eb2-92cf-4b56be12e3b6 |            ->
 true         | 2024-10-08 |        22.33 |        22.2 |        2.2 |       22 | 1234567891234567891234567891.23 |       222 | message_1   | 20:59:50.000 | 2024-10-08 12:59:50.612 | d00c5314-0ae7-41d3-8730-a462fe2fa09f |            ->
 true         | 2024-10-08 |        22.33 |        22.2 |        2.2 |       22 | 1234567891234567891234567891.23 |       222 | message_1   | 20:59:50.000 | 2024-10-08 12:59:50.627 | a185064e-d5eb-421d-abcb-f20467c04274 |            ->
 true         | 2024-10-08 |        22.33 |        22.2 |        2.2 |       22 | 1234567891234567891234567891.23 |       222 | message_1   | 20:59:50.000 | 2024-10-08 12:59:50.642 | 839a70d0-48b7-4f87-97e8-67df65381c49 |            ->
 true         | 2024-10-08 |        22.33 |        22.2 |        2.2 |       22 | 1234567891234567891234567891.23 |       222 | message_1   | 20:59:50.000 | 2024-10-08 12:59:50.654 | 7c19b818-482f-4322-a5fe-1a721f4d9c06 |            ->
(9 rows)

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository:

[lhotari]

LGTM

[lhotari]

shibd closed this 1 hour ago
shibd reopened this 1 hour ago

@shibd Do you use closing and reopening to trigger CI? Please use /pulsarbot rerun-failure-checks to retry Pulsar CI instead. Closing and reopening is wasteful for Pulsar CI resources and should only be performed if there's a change that is needed from the target branch (for example a CI fix) or when the CI hasn't been run for over 3 days.

We have quite a few flaky tests in branch-3.0 (as well as in other branches) so getting a fully green CI pass is hard. One potential improvement for Pulsar CI would be to run only failed tests in retries.