Skip to content

[FLINK-37979] Remove obsolete MySQL CDC snapshot split assigner code#4048

Merged
lvyanquan merged 1 commit into
apache:masterfrom
morozov:FLINK-37979-obsolete-mysql-snapshot-split-assigner-code
Sep 16, 2025
Merged

[FLINK-37979] Remove obsolete MySQL CDC snapshot split assigner code#4048
lvyanquan merged 1 commit into
apache:masterfrom
morozov:FLINK-37979-obsolete-mysql-snapshot-split-assigner-code

Conversation

@morozov

@morozov morozov commented Jun 21, 2025

Copy link
Copy Markdown
Contributor

There is a redundant condition and a misleading comment in MySqlSnapshotSplitAssigner#captureNewlyAddedTables() that complicate understanding of the code:

if (AssignerStatus.isAssigningFinished(assignerStatus)) {
// start the newly added tables process under binlog reading phase
LOG.info(
"Found newly added tables, start capture newly added tables process under binlog reading phase");
this.startAssignNewlyAddedTables();
}

By the time when the above condition is being evaluated, an identical condition has been already evaluated earlier in the code:

if (sourceConfig.isScanNewlyAddedTableEnabled()
&& !sourceConfig.getStartupOptions().isSnapshotOnly()
&& AssignerStatus.isAssigningFinished(assignerStatus)) {

Therefore, the second evaluation will always result in true and thus is redundant.

Code Change History

Prior to #3519, table discovery would happen in any assigner status, so the condition in the end of the method made sense. Together with adding this condition in the beginning of the method, #3519 should have removed it from the end.

What Exactly is Obsolete

  1. The comment:
    Since [FLINK-35859][cdc-connector] Fix: The assigner is not ready to offer finished split information, this should not be called #3519, if the job is still in the snapshot reading phase, this code won't be executed. This comment should be removed as misleading.
  2. The second if (AssignerStatus.isAssigningFinished(assignerStatus)) statement. While the captureNewlyAddedTables() method is running, the assigner status cannot be modified by a different thread because this code is only executed as part of MySqlSourceEnumerator#open(), i.e. the source in its initialization phase.

@morozov morozov marked this pull request as ready for review June 21, 2025 16:50
@morozov morozov changed the title FLINK-37979: Remove obsolete MySQL CDC snapshot split assigner code [FLINK-37979] Remove obsolete MySQL CDC snapshot split assigner code Jun 23, 2025
@morozov morozov force-pushed the FLINK-37979-obsolete-mysql-snapshot-split-assigner-code branch from d29ee2f to c4f6d26 Compare June 23, 2025 04:16
@leonardBang leonardBang requested a review from ruanhang1993 July 9, 2025 04:06

@lvyanquan lvyanquan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Thanks for your detailed analysis.

@lvyanquan lvyanquan merged commit 2712b49 into apache:master Sep 16, 2025
20 of 21 checks passed
@morozov morozov deleted the FLINK-37979-obsolete-mysql-snapshot-split-assigner-code branch January 9, 2026 01:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants