Skip to content

Support read from specific MySQL Binlog position#39

Closed
shizhengchao wants to merge 1 commit into
apache:masterfrom
shizhengchao:issues21
Closed

Support read from specific MySQL Binlog position#39
shizhengchao wants to merge 1 commit into
apache:masterfrom
shizhengchao:issues21

Conversation

@shizhengchao

@shizhengchao shizhengchao commented Sep 19, 2020

Copy link
Copy Markdown
Contributor

Fix #21
Support read from specific MySQL Binlog position

Add optional flag source-offset-file and source-offset-pos to specify the source offsets in mysql-cdc.
source-offset-file means the binlog-file name, source-offset-pos means the position of the binlog.
two steps to get the binlog-file and position:

  1. open the mysql command-cli, :
show binary logs; // list the binlog-file
  1. get the position:
show binlog events in 'mysql-bin.xxxxxx'; // get the position

usage in flink sql:

CREATE TABLE flink_cdc_test (
  order_id INT,
  order_date TIMESTAMP(0),
  customer_name STRING,
  price DECIMAL(10, 5),
  product_id INT,
  order_status BOOLEAN
) WITH (
  'connector' = 'mysql-cdc',
  'hostname' = 'localhost',
  'port' = '3306',
  'username' = 'root',
  'password' = 'root123',
  'database-name' = 'flink',
  'table-name' = 'flink_cdc_test',
  'source-offset-file' = 'mysql-bin.000021',
  'source-offset-pos' = '30292'
);

@wuchong

wuchong commented Feb 26, 2021

Copy link
Copy Markdown
Member

Thanks for the contribution. I have merged this PR in befb4ad.

@wuchong wuchong closed this Feb 26, 2021
morozov pushed a commit to morozov/flink-cdc that referenced this pull request Dec 11, 2024
[FLINK-36326][cdc-connector][mysql] Send BinlogSplitUpdateRequestEvent only once to fix auto scan newly-added table failure (apache#3613)

Signed-off-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
Co-authored-by: yuxiqian <34335406+yuxiqian@users.noreply.github.com>
Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Read from specific MySQL Binlog position

2 participants