Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
510b16c
Tmp
mustafasrepo Aug 7, 2024
6ef4369
Minor changes
mustafasrepo Aug 7, 2024
c3efafc
Minor changes
mustafasrepo Aug 7, 2024
2bf220d
Minor changes
mustafasrepo Aug 7, 2024
eb83917
Implement top down recursion with delete check
mustafasrepo Aug 7, 2024
0b66b15
Minor changes
mustafasrepo Aug 7, 2024
c769f9f
Minor changes
mustafasrepo Aug 7, 2024
0ad7063
Address reviews
mustafasrepo Aug 7, 2024
3661f06
Update comments
mustafasrepo Aug 7, 2024
60967c1
Minor changes
mustafasrepo Aug 7, 2024
6b87c4c
Make test deterministic
mustafasrepo Aug 7, 2024
8dd7e0a
Add fetch info to the statistics
mustafasrepo Aug 8, 2024
15423ae
Enforce distribution use inexact count estimate also.
mustafasrepo Aug 8, 2024
94fb83d
Minor changes
mustafasrepo Aug 8, 2024
9053b9f
Minor changes
mustafasrepo Aug 8, 2024
1171584
Minor changes
mustafasrepo Aug 8, 2024
711038d
Do not add unnecessary hash partitioning
mustafasrepo Aug 9, 2024
7e598e5
Minor changes
mustafasrepo Aug 9, 2024
12ad2c2
Add config option to use inexact row number estimates during planning
mustafasrepo Aug 9, 2024
2e3cc5d
Update config
mustafasrepo Aug 9, 2024
34af8ba
Minor changes
mustafasrepo Aug 9, 2024
98760bc
Minor changes
mustafasrepo Aug 9, 2024
1e4dada
Final review
ozankabak Aug 9, 2024
9fc4f3d
Address reviews
mustafasrepo Aug 9, 2024
1116058
Add handling for sort removal with fetch
mustafasrepo Aug 9, 2024
44dc292
Fix linter errors
mustafasrepo Aug 9, 2024
c6d2de6
Minor changes
mustafasrepo Aug 9, 2024
c7c85f4
Update config
mustafasrepo Aug 9, 2024
7c8967d
Cleanup stats under fetch
ozankabak Aug 9, 2024
ed35660
Update SLT comment
ozankabak Aug 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update config
  • Loading branch information
mustafasrepo committed Aug 9, 2024
commit 2e3cc5d45a514f0d2f84899dadacbddf30a74313
1 change: 1 addition & 0 deletions docs/source/user-guide/configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ Environment variables are read during `SessionConfig` initialisation so they mus
| datafusion.execution.keep_partition_by_columns | false | Should DataFusion keep the columns used for partition_by in the output RecordBatches |
| datafusion.execution.skip_partial_aggregation_probe_ratio_threshold | 0.8 | Aggregation ratio (number of distinct groups / number of input rows) threshold for skipping partial aggregation. If the value is greater then partial aggregation will skip aggregation for further input |
| datafusion.execution.skip_partial_aggregation_probe_rows_threshold | 100000 | Number of input rows partial aggregation partition should process, before aggregation ratio check and trying to switch to skipping aggregation mode |
| datafusion.execution.use_row_number_estimate_to_optimize_partitioning | false | Should DataFusion use row number estimate at the input to decide whether increasing parallelism is beneficial or not. By default, only exact row number (not estimates) are used for decision. Setting this flag to `true` will more likely produce better plans. |
| datafusion.optimizer.enable_distinct_aggregation_soft_limit | true | When set to true, the optimizer will push a limit operation into grouped aggregations which have no aggregate expressions, as a soft limit, emitting groups once the limit is reached, before all rows in the group are read. |
| datafusion.optimizer.enable_round_robin_repartition | true | When set to true, the physical plan optimizer will try to add round robin repartitioning to increase parallelism to leverage more CPU cores |
| datafusion.optimizer.enable_topk_aggregation | true | When set to true, the optimizer will attempt to perform limit operations during aggregations, if possible |
Expand Down