Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions arrow-array/src/builder/struct_builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,12 @@ impl StructBuilder {
self.null_buffer_builder.append(is_valid);
}

/// Appends `n` non-null entries into the builder.
#[inline]
pub fn append_non_nulls(&mut self, n: usize) {
self.null_buffer_builder.append_n_non_nulls(n);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

append_nulls below currently appends an n-element vec of false instead of calling its append_n_nulls. It seems like both these methods should follow the same approach -- whichever way is faster?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @scovich The n-element vec has been removed here: #9428 I thought splitting it up into smaller PRs would make the discussion easier. Allocating the vec is, as expected, way slower. The null_buffer is implemented using a Bitmap in the form of a MutableBuffer, which is way faster than the vec.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I somehow missed that PR. We have our answer then!

}

/// Appends a null element to the struct.
#[inline]
pub fn append_null(&mut self) {
Expand Down Expand Up @@ -727,4 +733,60 @@ mod tests {
assert!(a1.is_valid(0));
assert!(a1.is_null(1));
}

#[test]
fn test_append_non_nulls() {
let int_builder = Int32Builder::new();
let fields = vec![Field::new("f1", DataType::Int32, false)];
let field_builders = vec![Box::new(int_builder) as Box<dyn ArrayBuilder>];

let mut builder = StructBuilder::new(fields, field_builders);
builder
.field_builder::<Int32Builder>(0)
.unwrap()
.append_slice(&[1, 2, 3, 4, 5]);
builder.append_non_nulls(5);

let arr = builder.finish();
assert_eq!(arr.len(), 5);
assert_eq!(arr.null_count(), 0);
for i in 0..5 {
assert!(arr.is_valid(i));
}
}

#[test]
fn test_append_non_nulls_with_nulls() {
let mut builder = StructBuilder::new(Fields::empty(), vec![]);
builder.append_null();
builder.append_non_nulls(3);
builder.append_nulls(2);
builder.append_non_nulls(1);

let arr = builder.finish();
assert_eq!(arr.len(), 7);
assert_eq!(arr.null_count(), 3);
assert!(arr.is_null(0));
assert!(arr.is_valid(1));
assert!(arr.is_valid(2));
assert!(arr.is_valid(3));
assert!(arr.is_null(4));
assert!(arr.is_null(5));
assert!(arr.is_valid(6));
}

#[test]
fn test_append_non_nulls_zero() {
let mut builder = StructBuilder::new(Fields::empty(), vec![]);
builder.append_non_nulls(0);
assert_eq!(builder.len(), 0);

builder.append(true);
builder.append_non_nulls(0);
assert_eq!(builder.len(), 1);

let arr = builder.finish();
assert_eq!(arr.len(), 1);
assert_eq!(arr.null_count(), 0);
}
}
Loading