[SPARK-56892][SQL] Bulk read optimization for Parquet DELTA_BINARY_PACKED decoding

[iemejia]

What changes were proposed in this pull request?

Replace per-element lambda dispatch in readIntegers/readLongs with bulk paths that compute prefix sums in-place over the unpacked delta buffer and write via putInts/putLongs (backed by System.arraycopy on-heap).

Three optimizations in this PR:

1. Bulk read for INT32/INT64: readBulkIntegers and readBulkLongs replace the generic readValues() lambda-per-value path. A single loadMiniBlockBulk method handles block/mini-block loading, prefix-sum computation, and delegates the type-specific write to a BulkWriter callback (called once per mini-block, not per value).

2. Zero-allocation unsigned long encoding: Replace new BigInteger(Long.toUnsignedString(v)).toByteArray() (3 allocations per value: String + BigInteger + byte[]) with a loop-based byte[] encoder using Long.numberOfLeadingZeros to compute minimal BigInteger-compatible encoding directly. The shared utility encodeUnsignedLongBigEndian is extracted into VectorizedReaderBase and applied to all call sites (VectorizedDeltaBinaryPackedReader, UnsignedLongUpdater, ParquetDictionary).

3. Widening overrides (readIntegersAsLongs, readIntegersAsDoubles): Since the delta decoder already works on long[] internally, these overrides skip the int narrowing step and write longs/doubles directly from the prefix-sum buffer. Benefits DateToTimestampNTZUpdater, IntegerToLongUpdater, and IntegerToDoubleUpdater via the two-pass updater pattern.

4. Benchmark fix: Add unsignedLongVec.reset() before readUnsignedLongs to prevent unbounded arrayData() growth across benchmark iterations (OOM).

Why are the changes needed?

The DELTA_BINARY_PACKED decoder was 2-5x slower than PLAIN encoding for INT32/INT64 reads due to per-element lambda dispatch and lack of bulk vector writes. The readUnsignedLongs path allocated 3 objects per value (12,288 allocations per 4096-row batch) due to BigInteger(Long.toUnsignedString(v)).

Benchmark Results (AMD EPYC 7763, JDK 17/21/25)

Results from GHA benchmark workflow runs committed to this branch. Both baseline (upstream master) and this PR ran on AMD EPYC 7763 64-Core Processor.

DELTA_BINARY_PACKED INT64 (primary optimization target):

Case	JDK 17 Baseline	JDK 17 PR	Speedup	JDK 21 Baseline	JDK 21 PR	Speedup	JDK 25 Baseline	JDK 25 PR	Speedup
readLongs, constant	191.0	535.8	2.8x	195.2	477.9	2.4x	163.0	508.3	3.1x
readLongs, monotonic	144.9	534.1	3.7x	158.8	478.2	3.0x	139.1	520.9	3.7x
readLongs, small-delta random	134.8	368.4	2.7x	139.4	334.4	2.4x	122.7	356.6	2.9x
readLongs, wide random	97.8	187.7	1.9x	102.9	182.3	1.8x	94.4	189.4	2.0x
skipLongs, constant	170.1	612.6	3.6x	175.4	520.5	3.0x	152.1	603.1	4.0x
skipLongs, monotonic	170.1	611.2	3.6x	175.6	526.5	3.0x	152.0	603.7	4.0x
skipLongs, small-delta random	152.8	427.1	2.8x	152.5	356.2	2.3x	133.9	397.2	3.0x

DELTA_BINARY_PACKED INT32:

Case	JDK 17 Baseline	JDK 17 PR	Speedup	JDK 21 Baseline	JDK 21 PR	Speedup	JDK 25 Baseline	JDK 25 PR	Speedup
readIntegers, constant	482.3	375.1	0.78x	451.8	482.9	1.07x	487.1	476.4	0.98x
readIntegers, monotonic	314.9	375.1	1.19x	369.7	478.9	1.30x	303.1	476.4	1.57x
readIntegers, small-delta random	257.5	297.3	1.15x	308.6	333.9	1.08x	241.3	333.6	1.38x
readIntegers, wide random	209.3	240.8	1.15x	249.2	270.9	1.09x	201.4	267.5	1.33x
skipIntegers, constant	335.4	608.9	1.82x	415.1	519.8	1.25x	364.8	606.4	1.66x
skipIntegers, monotonic	335.1	612.2	1.83x	415.6	529.0	1.27x	365.3	603.8	1.65x

DELTA_BYTE_ARRAY (indirect benefit from INT64 bulk path):

Case	JDK 17 Baseline	JDK 17 PR	Speedup	JDK 21 Baseline	JDK 21 PR	Speedup	JDK 25 Baseline	JDK 25 PR	Speedup
readBinary, no overlap, len=16	29.2	38.5	1.32x	29.6	38.7	1.31x	28.2	38.5	1.37x
readBinary, half overlap, len=16	25.0	31.0	1.24x	25.4	31.2	1.23x	24.3	31.7	1.30x
readBinary, full overlap, len=16	24.8	30.2	1.22x	25.1	30.9	1.23x	24.0	31.3	1.30x

DELTA_LENGTH_BYTE_ARRAY:

Case	JDK 17 Baseline	JDK 17 PR	Speedup	JDK 21 Baseline	JDK 21 PR	Speedup	JDK 25 Baseline	JDK 25 PR	Speedup
readBinary, payloadLen=8	48.1	63.3	1.32x	51.7	65.7	1.27x	48.7	63.8	1.31x
skipBinary, payloadLen=8	95.8	174.7	1.82x	106.3	182.0	1.71x	97.0	183.2	1.89x

Variant reads (unsigned long encoding + widening overrides):

Case	JDK 17 Baseline	JDK 17 PR	Speedup	JDK 21 Baseline	JDK 21 PR	Speedup	JDK 25 Baseline	JDK 25 PR	Speedup
readIntegersAsLongs (INT32 -> Long)	121.4	291.1	2.4x	117.6	286.8	2.4x	105.8	283.9	2.7x
readIntegersAsDoubles (INT32 -> Double)	121.4	258.8	2.1x	117.6	258.1	2.2x	105.8	253.0	2.4x
readUnsignedLongs (INT64 -> Decimal(20,0))	4.6	37.6	8.2x	4.5	36.9	8.2x	5.0	36.8	7.4x

Note: Baseline for readIntegersAsLongs/readIntegersAsDoubles is readUnsignedIntegers which uses the per-row default readInteger() virtual dispatch path (same cost as the default widening methods without overrides).

Note: These results are from independent GHA runs (not back-to-back on the same runner). Some cross-run variance is present in unchanged paths. The INT64 bulk-read improvement (1.8x-3.7x), widening overrides (2.1x-2.7x), and the unsigned long encoding fix (7-9x) are the primary contributions of this PR.

Full committed results: JDK 17, JDK 21, JDK 25

Does this PR introduce any user-facing change?

No. This is a performance improvement to internal Parquet decoding. No API or behavior changes.

How was this patch tested?

• Existing unit tests: ParquetDeltaEncodingInteger (13 tests), ParquetDeltaEncodingLong (13 tests), ParquetDeltaByteArrayEncodingSuite, ParquetDeltaLengthByteArrayEncodingSuite, ParquetVectorizedSuite (25 tests), ParquetIOSuite (unsigned Parquet logical types test) -- all pass.
• Benchmark: VectorizedDeltaReaderBenchmark run via GHA workflow on JDK 17, 21, and 25.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: OpenCode (Claude claude-opus-4.6)

[LuciferYang]

 SPARK-34817: Read UINT_64 as Decimal from parquet: org.apache.spark.sql.execution.datasources.parquet.ParquetIOSuite
org.scalatest.exceptions.TestFailedException: 
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=311,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env: 

== Parsed Logical Plan ==
UnresolvedDataSource format: parquet, isStreaming: false, paths: 1 provided

== Analyzed Logical Plan ==
a: decimal(20,0)
Relation [a#1073655] parquet

== Optimized Logical Plan ==
Relation [a#1073655] parquet

== Physical Plan ==
*(1) ColumnarToRow
+- FileScan parquet [a#1073655] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/home/runner/work/spark/spark/target/tmp/spark-3d5f4b5f-6812-4791..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<a:decimal(20,0)>

== Results ==

== Results ==
!== Correct Answer - 1006 ==   == Spark Answer - 1006 ==
!struct<>                      struct<a:decimal(20,0)>

The failed tests are likely related to this PR. Could you fix them first? @iemejia

[LuciferYang]

We should address this as a separate PR as a priority.

[LuciferYang]

When v == 0L, the method returns early with start = 0 before buf.putLong(1, v) is called. The caller writes 9 - start bytes:

int start = encodeUnsignedLongBigEndian(v, unsignedLongBuf);
c.putByteArray(rowId, unsignedLongBuf.array(), start, 9 - start); // 9 bytes when start=0
``` But `scratch[1..8]` still contains data from the *previous* non-zero call to this method (since `unsignedLongBuf` is reused across rows). This produces a corrupt encoding: the downstream `BigInteger` constructor would interpret the stale bytes as a non-zero value.

**Concrete example:** Given a column with values `[42L, 0L]`, the first call encodes 42 into `scratch[1..8]`. The second call (`v = 0L`) sets `scratch[0] = 0` and returns `start = 0` without overwriting `scratch[1..8]`. The caller writes 9 bytes `[0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x2A]`, which `BigInteger` interprets as 42, not 0.

**Suggested fix:** Move `buf.putLong(1, v)` before the zero-check so the buffer is always current:
```java
buf.putLong(1, v);
if (v == 0L) {
  scratch[0] = 0;
  return 0;
}

[iemejia]

Good catch. Fixed — moved buf.putLong(1, v) before the zero-check so the buffer is always current when reused across rows.

[LuciferYang]

readBulkLoop always returns remaining = 0 (the while-loop exhausts it). So valuesRead = total - 0 = total, overwriting the accumulated count from prior calls. The bounds check if (valuesRead + total > totalValueCount) under-counts the cumulative values read, so the guard fires too late and may allow reads past the page boundary on multi-batch pages.

This is a pre-existing bug in the original readValues method (same file, line 243 in master), carried forward unchanged into the new bulk paths.

Suggested fix (applies to readBulkIntegers, readBulkLongs, and the original readValues):

valuesRead += total;

[iemejia]

Fixed in readBulkIntegers, readBulkLongs, and also in the original readValues (the pre-existing bug you mentioned). All three now use valuesRead += total.

[LuciferYang]

The baseline result files were captured on AMD EPYC 7763 (Zen 3); the new results are from AMD EPYC 9V45 (Zen 5). This is a generational CPU upgrade with significantly different IPC, cache hierarchy, and memory subsystem. The reported speedup numbers (e.g. 3.8x for readLongs) include both the code optimization AND the hardware improvement, making them unreliable for isolating the PR's contribution.

Also, please update the results for Java 21 and Java 25 as well.

[iemejia]

You're right. I've reverted the results file to the upstream baseline. I expect the CI workflow to regenerate it on consistent hardware with Java 17/21/25.

[LuciferYang]

You can navigate to the Benchmark workflow page in your repository. Select the current branch, specify the benchmark class name, pick the desired JDK version, and check Commit the benchmark results to the current branch. Note that only one JDK version can be selected per run, so you will need to launch three separate workflows for the different JDK versions.

[iemejia]

Thanks for the instructions! I had to rebase onto upstream master to pick up the updated benchmark workflow with JDK 25 support and the create-commit option. Triggered all three runs (JDK 17, 21, 25) -- they should commit the results directly to the branch once complete.

[LuciferYang]

The loop runs until remaining == 0, so the return value is always 0. This makes the return value redundant and obscures the valuesRead bug in the caller (see item #2). Consider changing the return type to void to make the exhaustive-loop contract explicit, and fix the callers to use valuesRead += total directly.

[iemejia]

Done. Changed readBulkLoop to void and removed the dead return remaining since the loop always exhausts it.

[LuciferYang]

The scratch buffer is lazily allocated on first readIntegers call. Since miniBlockSizeInValues is known at initFromPage time and the buffer is small (typically 128 ints = 512 bytes), it could be allocated eagerly alongside unpackedValuesBuffer for simpler code and deterministic allocation behavior.

[iemejia]

Done. Moved intScratchBuffer allocation to initFromPage alongside unpackedValuesBuffer and removed the lazy null-check from bulkWriteInts.

[LuciferYang]

Return start = 8 instead of start = 0 so the caller emits the single 0x00 byte at scratch[8] (already written by putLong):

buf.putLong(1, v);
if (v == 0L) {
    return 8;  // scratch[8] is already 0x00; caller writes 9 - 8 = 1 byte: [0x00]
}

[iemejia]

Good catch. Changed to return 8 so the caller writes the single 0x00 byte at scratch[8] (already zeroed by putLong), consistent with the minimal encoding contract.

[LuciferYang]

Could you update the PR description by referencing the GHA test results?
also cc @dongjoon-hyun @yaooqinn @viirya @sunchao

[LuciferYang]

It’s a bit odd why the data for skipBytes / skipShorts has dropped. Could this be just a coincidence?

[iemejia]

It does not make much sense. Notice also that the Github Runner was assigned to a different machine too. I am not sure how we can guarantee consistency, but at least most of the improvements look nice. Thanks for helping me with this one @LuciferYang I hope we will go faster with the other ones now that I know how it works.

[iemejia]

I am going to do a second run maybe to see if I get better numbers, we'll see.

[iemejia]

@LuciferYang I got the PR to land in the right worker for Java 21 and now the results are more consistent and better

[sunchao]

Summary

I reviewed the current head 456e0022ce5283ac0d66a41354a84c00d723ea32 through independent correctness, performance, and test/compatibility passes.

The production implementation looks correct. The bulk DELTA_BINARY_PACKED decoder preserves reader state across partial miniblocks and block boundaries, the UINT64 encoding helper handles zero and unsigned extremes correctly, and the earlier correctness concerns in the review history are fixed on this head.

One P2 remains before merge: the benchmark claims in the PR description do not describe the final checked-in benchmark results and still compare runs from different CPU generations as if the difference were primarily caused by this patch.

Prior State and Problem

The INT32 and INT64 DELTA_BINARY_PACKED paths reconstructed and wrote values through generic per-element callback loops. That added dispatch overhead and prevented the vectorized reader from using the destination column vector's bulk write operations.

The UINT64-to-decimal path converted each value through Long.toUnsignedString, BigInteger, and toByteArray, creating several temporary objects per decoded value. The benchmark also did not reset the variable-width UINT64 result vector between iterations, allowing its backing data to accumulate until the benchmark could run out of memory.

Design Approach

The PR adds specialized INT32 and INT64 bulk paths. A shared miniblock loop loads packed deltas, reconstructs absolute values through in-place prefix sums, and delegates each decoded segment to a type-specific bulk writer.

INT32 values are narrowed into reusable scratch storage before putInts, while INT64 values can be written through putLongs. The PR also extracts a shared allocation-free UINT64 encoder into VectorizedReaderBase, using a reusable eight-byte big-endian buffer at all three call sites.

Correctness / Compatibility Analysis

The new loop correctly handles partial miniblocks, transitions between miniblocks and blocks, repeated bulk calls, and mixtures of scalar and bulk reads. The cumulative valuesRead accounting advances by the complete decoded count.

The UINT64 helper overwrites the reusable buffer before handling zero and returns the correct minimal unsigned byte range. This fixes the stale-buffer zero-value failure from an earlier revision.

The existing reader API and Parquet file semantics are unchanged. Both on-heap and off-heap vectors support the bulk operations used here, and no public API or data-format compatibility change is introduced.

Key Design Decisions

• Prefix sums are computed directly in the unpacked delta buffer, avoiding an additional long-array copy.
• INT32 uses reusable scratch storage because the packed decoder reconstructs longs while the destination vector expects ints.
• Singleton runs still use the scalar updater path, avoiding bulk-copy setup for common nullable single-value runs.
• UINT64 encoding is centralized so delta, dictionary, and normal vector-updater paths use identical unsigned conversion semantics.
• Resetting the UINT64 benchmark vector is in scope because it is necessary to measure the changed variable-width path without accumulated state.

Implementation Sketch

1. Load a DELTA_BINARY_PACKED miniblock when the current miniblock is exhausted.
2. Add the block minimum delta and reconstruct absolute values with in-place prefix sums.
3. Write each available range through putInts or putLongs.
4. Preserve miniblock, block, and cumulative-value state for the next read call.
5. Encode UINT64 values into reusable big-endian storage and expose only the minimal unsigned byte range to decimal conversion.

Behavioral Changes Worth Calling Out

There should be no user-visible result change. The intended effects are lower CPU overhead for vectorized INT32/INT64 reads and elimination of per-value temporary allocations in UINT64 decimal reads.

The current benchmark primarily covers large, contiguous on-heap reads. Short nullable runs and off-heap vectors are not directly characterized by it. I do not have evidence of a regression there, but these cases would make the performance evidence more representative of production reads.

Suggested Improvements

[P2] Refresh the benchmark results and avoid unsupported causal speedup claims

The PR description still reports the intermediate AMD EPYC 9V45 JDK 17 run: roughly 2.0x-3.2x for INT32, 3.0x-6.6x for INT64, and 14.6x for UINT64. The final checked-in JDK 17 results now use AMD EPYC 7763 and, against the checked-in upstream baseline, show approximately:

• INT32 reads: 0.77x, 1.19x, 1.15x, and 1.14x
• INT64 reads: 2.73x, 3.60x, 2.66x, and 1.89x
• UINT64 reads: 8.78x

Several algorithmically unchanged skip and single-value paths also move substantially between the baseline and PR benchmark files. For example, unchanged INT64 skip paths appear roughly 1.76x-3.16x faster. That is strong evidence of meaningful cross-run variance even when the final files report the same CPU model.

Please replace the PR-body tables, hardware note, and workflow links with the final results. More importantly, either run base and head back-to-back on the same runner or describe the checked-in numbers as observational results without attributing the full difference to this patch. The current statement that the magnitude “far exceeds” the hardware/environment difference is not supported by the unchanged control paths.

As non-blocking follow-ups, consider adding benchmark cases for nullable run lengths around 2-32 and both on-heap and off-heap vectors. A dedicated functional test combining DELTA_BINARY_PACKED with UINT64 would also close the only notable coverage gap, although the two components are already exercised independently.

Validation

Focused validation covered the integer and long delta-reader suites, partial and repeated reads, V2 pages, on/off-heap vectors, UINT64 zero and extreme values, and dictionary/plain UINT64 paths. Independent validation also compared one million random UINT64 encodings against the existing BigInteger representation. git diff --check passed.

[sunchao]

The production implementation looks correct. The remaining benchmark-description feedback is non-blocking and should not have been submitted as a changes-requested review.

Submitted with the wrong review state; the benchmark-description feedback is non-blocking.

[iemejia]

@sunchao @LuciferYang Updated the PR description with the final committed benchmark numbers (JDK 21, both sides on AMD EPYC 7763). Removed the old cross-hardware comparison and added a note about cross-run variance in unchanged paths. The benchmark result files for JDK 17, 21, and 25 are already committed to the branch and up to date.

[iemejia]

Apologies for the force-push — I accidentally rebased onto latest master while updating my other branches. No code changes, just a base update. Sorry for the noise.

[LuciferYang]

I will double-check this set of changes later today.

[LuciferYang]

int len = (64 - Long.numberOfLeadingZeros(v)) / 8 + 1;
    byte[] out = new byte[len];
    long x = v;
    for (int i = len - 1; i >= 0 && x != 0; i--) {
      out[i] = (byte) x;
      x >>>= 8;
    }
    return out;

[LuciferYang]

may be slightly better than the previous

[iemejia]

Done. Replaced with this approach — allocates only the minimal-length array directly without going through a 9-byte scratch buffer.

[LuciferYang]

  /**
   * Encodes an unsigned long as a minimal big-endian two's-complement byte array
   * compatible with BigInteger encoding, written into {@code scratch[start .. 8]}
   * (length = 9 - start). {@code scratch} must have length >= 9.
   */
  static int encodeUnsignedLongBigEndian(long v, byte[] scratch) {
    // Minimal BigInteger-compatible length is bitLength/8 + 1, so the
    // encoding occupies scratch[start .. 8]; the loop runs once past the
    // significant bytes, which writes the 0x00 sign byte when one is needed.
    int start = 8 - (64 - Long.numberOfLeadingZeros(v)) / 8;
    long x = v;
    for (int i = 8; i >= start; i--) {
      scratch[i] = (byte) x;
      x >>>= 8;
    }
    return start;
  }

[iemejia]

Done. Adopted this implementation — eliminates the ByteBuffer entirely and uses Long.numberOfLeadingZeros to compute the start offset directly. Also updated all call sites to pass byte[] instead of ByteBuffer.

[LuciferYang]

Was this benchmark result overwritten? I recall the previous run was already executed on an AMD EPYC 7763 (64-core) machine.

[iemejia]

Oh that's a mistake, let me retrigger the benchmark again to see if we match the AMD EPYC 7763 CPU for the diff.

[LuciferYang]

private final byte[] unsignedLongScratch = new byte[9];

[iemejia]

Done.

[iemejia]

@LuciferYang All three benchmark runs are now on AMD EPYC 7763 (JDK 17, 21, 25) and the results are pretty promising:

INT64 reads: 1.8x-3.7x across all JDKs and data patterns
INT64 skip: 2.3x-4.0x
Unsigned long encoding (with the new byte[] loop): 7.3x-8.6x
INT32 reads: 1.1x-1.6x (narrowing overhead limits gains)
DELTA_BYTE_ARRAY / DELTA_LENGTH_BYTE_ARRAY: 1.2x-1.9x indirect improvement

Updated the PR description with full JDK 17/21/25 comparison tables and the new workflow run links.

Thank you for all your help and the thorough review suggestions -- the byte[] loop approach is cleaner and avoids the ByteBuffer abstraction entirely, and moving the scratch buffer allocation to initFromPage makes the code more straightforward. Really appreciate the guidance on getting the benchmark workflow right too.

I believe this is ready to go now -- would you be able to merge it when you get a chance?

[iemejia]

@LuciferYang Sorry for the extra churn -- I added one more commit with readIntegersAsLongs and readIntegersAsDoubles overrides for the DELTA_BINARY_PACKED reader. It seemed worth including since the delta decoder already works on long[] internally, so these overrides skip the int narrowing step entirely and write longs/doubles directly from the prefix-sum buffer.

Local benchmark shows 2.1x for readIntegersAsLongs and 2.0x for readIntegersAsDoubles vs the per-row default path.

This benefits DateToTimestampNTZUpdater, IntegerToLongUpdater, and IntegerToDoubleUpdater when reading Parquet V2 DELTA_BINARY_PACKED encoded INT32 columns -- the improvement carries over automatically via the two-pass updater pattern in PR #55923.

The review delta is small (25 lines of new code in the reader + 8 lines of benchmark cases) if you want to focus just on the new commit.

[LuciferYang]

+1, LGTM

[LuciferYang]

Merged into master/branch-4.x. Thanks @iemejia and @sunchao

[iemejia]

Thank you @LuciferYang and @sunchao for all the guidance. As an easy follow up PTAL at #56479

[LuciferYang]

@iemejia The test failures seems related to the current PR. I will revert this change and reopen the PR first. We can fix the issues and merge it again later:

• https://github.com/apache/spark/actions/runs/27590688058/job/81572596324

 parquet widening conversion IntegerType -> LongType: org.apache.spark.sql.execution.datasources.parquet.ParquetTypeWideningSuite
org.scalatest.exceptions.TestFailedException: with dictionary encoding 'false' with timestamp rebase mode 'CORRECTED'' Vectorized reader 
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=311,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env: 

== Parsed Logical Plan ==
UnresolvedDataSource format: parquet, isStreaming: false, paths: 1 provided

== Analyzed Logical Plan ==
a: bigint
Relation [a#1708954L] parquet

== Optimized Logical Plan ==
Relation [a#1708954L] parquet

== Physical Plan ==
*(1) ColumnarToRow
+- FileScan parquet [a#1708954L] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/home/runner/work/spark/spark/target/tmp/spark-1407ec22-ddd8-4a7c..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<a:bigint>

== Results ==

== Results ==
!== Correct Answer - 3 ==   == Spark Answer - 3 ==
 struct<a:bigint>           struct<a:bigint>
![-2147483648]              [1]
![1]                        [2147483648]
 [2]                        [2]
    
       
sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException: with dictionary encoding 'false' with timestamp rebase mode 'CORRECTED'' Vectorized reader 
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=311,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:


== Parsed Logical Plan ==
UnresolvedDataSource format: parquet, isStreaming: false, paths: 1 provided


== Analyzed Logical Plan ==
a: bigint
Relation [a#1708954L] parquet


== Optimized Logical Plan ==
Relation [a#1708954L] parquet


== Physical Plan ==
*(1) ColumnarToRow
+- FileScan parquet [a#1708954L] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/home/runner/work/spark/spark/target/tmp/spark-1407ec22-ddd8-4a7c..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<a:bigint>


== Results ==


== Results ==
!== Correct Answer - 3 ==   == Spark Answer - 3 ==
struct<a:bigint>           struct<a:bigint>
![-2147483648]              [1]
![1]                        [2147483648]
[2]                        [2]

[LuciferYang]

reopen

[LuciferYang]

https://github.com/iemejia/spark/runs/81053637489

The same failure occurs here. It appears that automatically committing benchmark results masked the test failures.

@iemejia Feel free to ping me again once the issue is fixed.

[iemejia]

Argh it seems the PR did not reopen because I have already deleted the remote branch. Can you reopen it @LuciferYang or we move the review to the other site? (In the meantime I opened it in the side)

Ask me if you prefer me to squash all the commits of the previous part and let just the fix commit on top of it.