Fuzzing Crash: array_ops - BtrBlocksCompressor loses list data with empty strings in struct fields

[github-actions]

Fuzzing Crash Report

Analysis

Crash Location: fuzz/fuzz_targets/array_ops.rs:37:71 (unwrap on assertion failure in assert_array_eq)

Error Message:

called `Result::unwrap()` on an `Err` value: {==(: decimal128(1956805334888731279340369011, precision=28, scale=-36), : ["", "", "", "", "", ""]} != {==(: decimal128(1956805334888731279340369011, precision=28, scale=-36), : []} at index 11, lhs is root: vortex.struct({==(=decimal(28,-36)?, =list(utf8?)?}?, len=27) nbytes=1.13 kB

Stack Trace:

   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: unwrap<(), vortex_fuzz::error::VortexFuzzError>
   4: __libfuzzer_sys_run at ./fuzz/fuzz_targets/array_ops.rs:37:71
   5: rust_fuzzer_test_input
   6: {closure#0}
   7: do_call<libfuzzer_sys::test_input_wrap::{closure_env#0}, i32>

Root Cause:

The fuzzer detected data loss during compression (step 2) using the Default compressor (BtrBlocksCompressor). The test performed:

1. Filter operation (AllTrue mask) - succeeded
2. Filter operation (partial mask, 27/28 elements) - succeeded
3. Compress with BtrBlocksCompressor - FAILED

After compression, the array values at index 11 differ:

• Expected (lhs): Struct with list field containing ["", "", "", "", "", ""] (6 empty strings)
• Actual (rhs): Struct with list field containing [] (empty list)

The input structure:

• ChunkedArray of Structs (len: 28, 3 chunks)
• Each struct has 2 fields:
  • Field "==(\u{4}": Decimal(precision=28, scale=-36, nullable)
  • Field "" (empty name): List<Utf8(nullable), nullable>
• The List field uses ListViewArray encoding with empty VarBinViewArray elements
• At index 11, the ListViewArray has validity but all list sizes are 0 (6 empty lists)

The compressor appears to incorrectly deduplicate or compress empty list structures within struct fields, causing list data to be lost when converting from ListViewArray with 6 valid-but-empty sublists to an empty list.

Debug Output

FuzzArrayAction {
    array: ChunkedArray {
        dtype: Struct(
            StructFields {
                names: FieldNames([
                    FieldName("==(\u{4}"),
                    FieldName(""),
                ]),
                dtypes: [
                    FieldDType { inner: Owned(Decimal(DecimalDType { precision: 28, scale: -36 }, Nullable)) },
                    FieldDType { inner: Owned(List(Utf8(Nullable), Nullable)) },
                ],
            },
            Nullable,
        ),
        len: 28,
        chunk_offsets: [0, 6, 25, 28],
        chunks: [
            StructArray { len: 6, fields: [Decimal chunks, ListViewArray chunks with empty elements] },
            StructArray { len: 19, fields: [Decimal, ListViewArray with 15 VarBinViewArray elements] },
            StructArray { len: 3, fields: [Decimal, ListViewArray with empty elements] },
        ],
    },
    actions: [
        (Filter(AllTrue(28)), ...), // Step 0: success
        (Filter(Values(27 true)), ...), // Step 1: success
        (Compress(Default), ...), // Step 2: FAILURE - array value mismatch at index 11
    ],
}

The ListViewArray at index 11 (in the second chunk):

• Contains 6 list entries with all sizes = 0 (empty strings)
• Uses validity array with some entries valid
• After compression: becomes empty list instead of list with 6 empty elements

Summary

• Target: array_ops
• Crash File: crash-2a20d373d9e96fb70cc7336d457d211385d447e7
• Branch: develop
• Commit: d404eea
• Crash Artifact: https://github.com/vortex-data/vortex/actions/runs/19485211139/artifacts/4608287609

Reproduction

1. Download the crash artifact:

   • Direct download: https://github.com/vortex-data/vortex/actions/runs/19485211139/artifacts/4608287609
   • Or find operations-fuzzing-crash-artifacts at: https://github.com/vortex-data/vortex/actions/runs/19485211139
   • Extract the zip file

2. Reproduce locally:

# The artifact contains array_ops/crash-2a20d373d9e96fb70cc7336d457d211385d447e7
cargo +nightly fuzz run --sanitizer=none array_ops array_ops/crash-2a20d373d9e96fb70cc7336d457d211385d447e7

3. Get full backtrace:

RUST_BACKTRACE=full cargo +nightly fuzz run --sanitizer=none array_ops array_ops/crash-2a20d373d9e96fb70cc7336d457d211385d447e7

---

Auto-created by fuzzing workflow with Claude analysis

[github-actions]

Related Crash Detected

A similar crash was detected in the array_ops target.

Crash Details

Crash Location: fuzz/fuzz_targets/array_ops.rs:37:71

Error Message:

called `Result::unwrap()` on an `Err` value: [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}] != [] at index 2

Stack Trace:

   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: unwrap<(), vortex_fuzz::error::VortexFuzzError>
   4: __libfuzzer_sys_run at ./fuzz/fuzz_targets/array_ops.rs:37:71
   5: rust_fuzzer_test_input

Similarities to Original Issue:

• Same crash location: array_ops.rs:37:71
• Same operation sequence: Filter → Compress(Default) using BtrBlocksCompressor
• Same pattern: List with N empty/zero-size elements becomes empty list after compression
• Original issue: List with 6 empty strings → empty list
• This crash: List with 20 empty structs → empty list

Differences:

• Original issue: List field within a Struct (List<Utf8, nullable>)
• This crash: Top-level ListViewArray of empty Structs
• Different index where mismatch occurs (index 2 vs index 11)
• Note: These differences likely indicate the same root cause manifesting with different data structures

Key Data Structure

The crash involves a ListViewArray containing 20 empty struct elements. After compression, the list data is lost:

• Before compression: ListViewArray with elements pointing to 20 empty structs
• After compression: Empty list with 0 elements

The ListViewArray has:

• elements: StructArray (len=20) with no fields (empty struct)
• offsets: All zeros
• sizes: All zeros (each list entry has size 0)
• Operation: Filter (4/6 elements) → Compress(Default)

Occurrence Details

• Crash File: crash-d43755f1f95c5825dffcc2a4ebf0534bafa45791
• Branch: develop
• Commit: e25c0a2
• Crash Artifact: Not available (local fuzzing run)

Reproduction

cargo +nightly fuzz run --sanitizer=none array_ops array_ops/crash-d43755f1f95c5825dffcc2a4ebf0534bafa45791

---

Auto-detected by fuzzing workflow with Claude analysis

[github-actions]

Related Crash Detected

A similar crash was detected in the array_ops target.

Crash Details

Crash Location: fuzz/fuzz_targets/array_ops.rs:37:71

Error Message:

called `Result::unwrap()` on an `Err` value: [{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}] != [] at index 2, lhs is root: vortex.listview(list({}?)?, len=4) nbytes=65 B (100.00%)

Stack Trace:

   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: unwrap<(), vortex_fuzz::error::VortexFuzzError>
   4: __libfuzzer_sys_run at ./fuzz/fuzz_targets/array_ops.rs:37:71
   5: rust_fuzzer_test_input

Similarities to Original Issue:

• Same crash location: fuzz/fuzz_targets/array_ops.rs:37:71
• Same error pattern: ListViewArray with N empty elements incorrectly becomes an empty list (0 elements)
• Same operation type: Array transformation operation (Filter vs Compress)
• Both involve ListViewArray with empty child elements

Differences:

• Original issue: Compress operation loses 6 empty strings → becomes []
• This crash: Filter operation loses 20 empty structs → becomes []
• Original: List field in struct, this crash: standalone ListViewArray
• Note: These differences may indicate the same root cause manifesting in different operations

Debug Output

FuzzArrayAction {
    array: ListViewArray {
        dtype: List(
            Struct(
                StructFields {
                    names: FieldNames(
                        [],
                    ),
                    dtypes: [],
                },
                Nullable,
            ),
            Nullable,
        ),
        elements: StructArray {
            len: 20,
            dtype: Struct(
                StructFields {
                    names: FieldNames(
                        [],
                    ),
                    dtypes: [],
                },
                Nullable,
            ),
            fields: [],
            validity: AllValid,
            stats_set: ArrayStats { ... },
        },
        offsets: PrimitiveArray {
            dtype: Primitive(
                I32,
                NonNullable,
            ),
            buffer: Buffer<u8> {
                length: 24,
                alignment: Alignment(4),
                as_slice: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
            },
            validity: NonNullable,
        },
        sizes: PrimitiveArray {
            dtype: Primitive(
                I32,
                NonNullable,
            ),
            buffer: Buffer<u8> {
                length: 24,
                alignment: Alignment(4),
                as_slice: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
            },
            validity: NonNullable,
        },
        is_zero_copy_to_list: true,
        validity: Array(
            BoolArray {
                dtype: Bool(NonNullable),
                buffer: BitBuffer {
                    buffer: Buffer<u8> {
                        length: 1,
                        alignment: Alignment(1),
                        as_slice: [16],
                    },
                    offset: 0,
                    len: 6,
                },
                validity: NonNullable,
            },
        ),
    },
    actions: [
        (
            Filter(
                Values(
                    MaskValues {
                        buffer: BitBuffer {
                            buffer: Buffer<u8> {
                                length: 1,
                                alignment: Alignment(1),
                                as_slice: [53],
                            },
                            offset: 0,
                            len: 6,
                        },
                        true_count: 4,
                        density: 0.6666666666666666,
                    },
                ),
            ),
            Array(
                ListViewArray {
                    dtype: List(
                        Struct(
                            StructFields {
                                names: FieldNames([]),
                                dtypes: [],
                            },
                            Nullable,
                        ),
                        Nullable,
                    ),
                    elements: StructArray {
                        len: 20,
                        dtype: Struct(
                            StructFields {
                                names: FieldNames([]),
                                dtypes: [],
                            },
                            Nullable,
                        ),
                        fields: [],
                        validity: AllValid,
                    },
                    offsets: PrimitiveArray {
                        dtype: Primitive(U64, NonNullable),
                        buffer: Buffer<u8> {
                            length: 32,
                            alignment: Alignment(8),
                            as_slice: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
                        },
                        validity: NonNullable,
                    },
                    sizes: PrimitiveArray {
                        dtype: Primitive(U64, NonNullable),
                        buffer: Buffer<u8> {
                            length: 32,
                            alignment: Alignment(8),
                            as_slice: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
                        },
                        validity: NonNullable,
                    },
                    is_zero_copy_to_list: true,
                    validity: Array(
                        BoolArray {
                            dtype: Bool(NonNullable),
                            buffer: BitBuffer {
                                buffer: Buffer<u8> {
                                    length: 1,
                                    alignment: Alignment(1),
                                    as_slice: [4],
                                },
                                offset: 0,
                                len: 4,
                            },
                            validity: NonNullable,
                        },
                    ),
                },
            ),
        ),
        (
            Compress(Default),
            Array(
                ListViewArray {
                    dtype: List(
                        Struct(
                            StructFields {
                                names: FieldNames([]),
                                dtypes: [],
                            },
                            Nullable,
                        ),
                        Nullable,
                    ),
                    elements: StructArray {
                        len: 20,
                        dtype: Struct(
                            StructFields {
                                names: FieldNames([]),
                                dtypes: [],
                            },
                            Nullable,
                        ),
                        fields: [],
                        validity: AllValid,
                    },
                    offsets: PrimitiveArray {
                        dtype: Primitive(U64, NonNullable),
                        buffer: Buffer<u8> {
                            length: 32,
                            alignment: Alignment(8),
                            as_slice: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
                        },
                        validity: NonNullable,
                    },
                    sizes: PrimitiveArray {
                        dtype: Primitive(U64, NonNullable),
                        buffer: Buffer<u8> {
                            length: 32,
                            alignment: Alignment(8),
                            as_slice: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
                        },
                        validity: NonNullable,
                    },
                    is_zero_copy_to_list: true,
                    validity: Array(
                        BoolArray {
                            dtype: Bool(NonNullable),
                            buffer: BitBuffer {
                                buffer: Buffer<u8> {
                                    length: 1,
                                    alignment: Alignment(1),
                                    as_slice: [4],
                                },
                                offset: 0,
                                len: 4,
                            },
                            validity: NonNullable,
                        },
                    ),
                },
            ),
        ),
    ],
}

The fuzzer:

1. Applied a Filter operation (selecting 4 out of 6 elements) - succeeded
2. Applied Compress operation (BtrBlocksCompressor) - succeeded
3. But at step 1 (Filter result validation), the comparison shows:
   • Expected: ListViewArray with 20 empty struct elements
   • Actual: List with 0 elements

All offsets and sizes in the ListViewArray are 0, meaning each list contains 0 elements. The issue is that during Filter or subsequent operations, the ListViewArray structure with 20 empty structs in the elements array is being converted to a simple empty list.

Occurrence Details

• Crash File: crash-d43755f1f95c5825dffcc2a4ebf0534bafa45791
• Branch: develop
• Commit: 50ddc86
• Crash Artifact: (URL not provided in environment - check workflow run)

Reproduction

cargo +nightly fuzz run --sanitizer=none array_ops array_ops/crash-d43755f1f95c5825dffcc2a4ebf0534bafa45791

---

Auto-detected by fuzzing workflow with Claude analysis

[connortsui20]

The first crash was fixed by #5423. Not really sure about the others as I cannot reproduce