Flink: refactor sink shuffling statistics collection #10331

Merged

stevenzwu merged 10 commits into

apache:mainfrom

stevenzwu:refactor-stats

Jun 5, 2024

Commits on Jun 3, 2024

Flink: refactor sink shuffling statistics collection to support sketch statistics and auto migration from Map stats to reservoir sampling sketch if cardinality is detected high
stevenzwu
committed
add operator stats migration upon receiving global stats. also added unit test to cover the 2 scenarios of operator stats migrations
stevenzwu
committed
make MapDataStatistics and SketchDataStatistics non-public
stevenzwu
committed
fix copy-and-paste error in Javadoc as Peter pointed out
stevenzwu
committed
add more tests to TestSketchUtil
stevenzwu
committed
removed some code from SketchUtil
stevenzwu
committed
address Gang's comments
stevenzwu
committed
remove unneeded Javadoc
stevenzwu
committed

Commits on Jun 4, 2024

Address Peter's concern on dropping partial results by handling concurrent checkpoints properly.
stevenzwu
committed

Commits on Jun 5, 2024

remove volatile
stevenzwu
committed