Skip to content

fix(parquet): Handle uint32 to uint64 array widening in reverseTransformArray #721

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kodiakhq merged 1 commit into from
Mar 23, 2026
Merged

fix(parquet): Handle uint32 to uint64 array widening in reverseTransformArray #721

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions parquet/read.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ func reverseTransformArray(dt arrow.DataType, arr arrow.Array) arrow.Array {
return reverseTransformTime64(dt.(*arrow.Time64Type), arr)
case *array.Date32:
return reverseTransformFromDate32(dt, arr)
case *array.Uint32:
return reverseTransformFromUint32(dt, arr)
case *array.Struct:
dt := dt.(*arrow.StructType)
children := make([]arrow.ArrayData, arr.NumField())
Expand Down Expand Up @@ -100,6 +102,23 @@ func reverseTransformArray(dt arrow.DataType, arr arrow.Array) arrow.Array {
}
}

func reverseTransformFromUint32(dt arrow.DataType, arr *array.Uint32) arrow.Array {
switch dt {
case arrow.PrimitiveTypes.Uint64:
Comment on lines +106 to +107
Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reverseTransformFromUint32 matches dt using direct interface equality (case arrow.PrimitiveTypes.Uint64). This will fail if the incoming schema uses a different *arrow.Uint64Type instance (pointer-inequality), causing an unexpected panic even though the logical type is uint64. Prefer a type switch on dt.(type) (like reverseTransformFromDate32) or use arrow.TypeEqual(dt, arrow.PrimitiveTypes.Uint64) for the match.

Suggested change
switch dt {
case arrow.PrimitiveTypes.Uint64:
switch dt.(type) {
case *arrow.Uint64Type:

Copilot uses AI. Check for mistakes.
builder := array.NewUint64Builder(memory.DefaultAllocator)
for i := 0; i < arr.Len(); i++ {
if arr.IsNull(i) {
builder.AppendNull()
continue
}
builder.Append(uint64(arr.Value(i)))
}
return builder.NewArray()
Comment on lines +105 to +116
Copy link

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new uint32→uint64 widening path is not exercised by existing parquet read/write tests. Add a targeted test that writes a schema with a uint64 column but produces/reads a uint32 physical array (or directly unit-tests reverseTransformArray with dt=uint64 and arr=*array.Uint32, including nulls and sliced arrays) to prevent regressions.

Copilot uses AI. Check for mistakes.
default:
panic(fmt.Errorf("unsupported conversion from %s to %s", arr.DataType(), dt))
}
}

func reverseTransformFromString(dt arrow.DataType, arr arrow.Array) arrow.Array {
builder := array.NewBuilder(memory.DefaultAllocator, dt)
for i := 0; i < arr.Len(); i++ {
Expand Down
Loading