multi dictionary benchmarks#22859
Conversation
Rich-T-kid
left a comment
There was a problem hiding this comment.
going to take another review tomorrow before pinging a reviewer to take a look!
| Arc::new(DictionaryArray::<UInt64Type>::try_new(key_array, values).unwrap()) | ||
| } | ||
|
|
||
| fn make_batch( |
There was a problem hiding this comment.
I think we should add a value type as a parameter here. strings are currently covered, in the future it may make sense for other types to also be covered. I left that out for this PR but I can introduce it if needed.
There was a problem hiding this comment.
I think that types that need to be benchmarks include (utf8, List<utf8>,Binary). I don't think their variants. (utf8View,List,binaryView...ect>,binaryView,LargeBinary,FixedSizeBinary) add enough difference to the point that there be any meaningful performance differences.
is there any recommendation on how to do this without multiplying the number of benchmarks we need to run by 3x?
| /// each column's distinct count is sampled from [target*0.95, target*1.05]. | ||
| const CARDINALITY_RANGE: usize = 10; | ||
|
|
||
| fn schema_for_cols(n_cols: usize) -> SchemaRef { |
There was a problem hiding this comment.
if/when we introduce other value data types, this function needs to change
|
|
||
| const SIZES: [usize; 2] = [8 * 1024, 64 * 1024]; | ||
| const N_COLS: [usize; 2] = [4, 8]; | ||
| const CARDS: [usize; 4] = [20, 100, 500, 1_000]; |
There was a problem hiding this comment.
Should I also include a 1-1 cardinality ratio similar to #21765?
|
@kumarUjjawal could you take a look a this and #22888 when you get a chance? Thank you |
kumarUjjawal
left a comment
There was a problem hiding this comment.
Thank you @Rich-T-kid
Left couple of comments, let me know what you think.
| perm.shuffle(&mut rng); | ||
| perm | ||
| } else { | ||
| (0..size) |
There was a problem hiding this comment.
We should generate shared row group ids first, then map each group id to a multi-column tuple. That keeps total group combinations near target_distinct
There was a problem hiding this comment.
good catch! previously column_count * distinct was being generated. I've updated the code to generate the group id's first and then pass it into the creation of the dictionarys. this way no matter how many columns get produced they will produce the same number of group id's.
Ive also added benchmarks for the cross-product group-bys to try and better reflect the worse case scenario. ex
GROUP BY department, city,rank
as opposed to the current benchmarks that cover the case of group-bys where the row-space may be shared. ex
GROUP BY employee_id, department, manager_id
…-product aggregations
|
Thanks for the review and comments @kumarUjjawal. Ive updated the PR could you take another look? |
|
@Rich-T-kid I was running this locally and I think this benchmark in not registered anywhere, I added it to the cargo toml only then it ran. Could you verify locally? |
|
I forgot to include the cargo file in the commit. should be good to go now @kumarUjjawal |
1333642 to
eb46aec
Compare
|
I replaced |
Which issue does this PR close?
Works towards closing #21878
Rationale for this change
There is currently no benchmarks that cover
GroupValueRow()when dictionary arrays are passed in. #22004 adds benchmarks for the single single column case. This PR adds benchmarks for the multi column caseWhat changes are included in this PR?
Adds a new Criterion harness for GroupValues over multi-column Dictionary<UInt64, Utf8> GROUP BY workloads, covering 4 and 8 group-by columns across batch sizes of 8 KiB and 64 KiB rows. It tests four realistic cardinalities (20, 100, 500, 1 000 distinct values) with per-column variance introduced by sampling each column's distinct count from a ±5% window around the target, so columns within the same batch have slightly different cardinalities. Two benchmark variants are included — repeated intern+emit and a partial-emit pattern that spills half the accumulated groups after each intern — to stress both the steady-state and incremental-flush paths of the group values implementation.
Are these changes tested?
n/a
Are there any user-facing changes?
No