hbase->hbase的数据同步增加transformsql配置之后报错

[wuyiccc]

Search before asking

• I had searched in the issues and found no similar question.

• I had googled my question but i didn't get any help.

• I had read the documentation: ChunJun doc but it didn't help me.

Description

1. 情况描述

1. 版本: chunjun-1.12.5版本 本地测试
2. 配置hbase->hbase的数据同步正常,但是中间增加transformsql配置之后报错

2. JSON 配置如下

{
  "job": {
    "content": [
      {
        "reader": {
          "table": {
            "tableName": "sourcetable"
          },
          "name": "hbasereader",
          "parameter": {
            "hbaseConfig": {
              "hbase.zookeeper.property.clientPort": "*",
              "hbase.rootdir": "*",
              "hbase.cluster.distributed": "true",
              "hbase.zookeeper.quorum": "*",
              "zookeeper.znode.parent": "*",
              "hbase.security.authentication": "*",
              "hbase.security.authorization": true,
              "hbase.master.kerberos.principal": "*",
              "hbase.master.keytab.file": "*",
              "hbase.regionserver.keytab.file": "*",
              "hbase.regionserver.kerberos.principal": "*",
              "java.security.krb5.conf": "*",
              "principalFile": "*",
              "useLocalFile": true
            },
            "table": "di_flinkx_test:s_auction",
            "encoding": "utf-8",
            "column": [
              {
                "name": "rowkey",
                "type": "int"
              },
              {
                "name": "f1:title",
                "type": "varchar"
              },
              {
                "name": "f1:gmt_modified",
                "type": "varchar"
              }
            ],
            "startRowkey": "",
            "endRowkey": "",
            "isBinaryRowkey": true
          }
        },
        "transformer" : {
            "transformSql": "select * from sourcetable"
        },
        "writer": {
          "name": "hbasewriter",
          "parameter": {
            "hbaseConfig": {
              "hbase.zookeeper.property.clientPort": "*",
              "hbase.rootdir": "*",
              "hbase.cluster.distributed": "*",
              "hbase.zookeeper.quorum": "*",
              "zookeeper.znode.parent": "*",
              "hbase.security.authentication": "*",
              "hbase.security.authorization": true,
              "hbase.master.kerberos.principal": "*",
              "hbase.master.keytab.file": "*",
              "hbase.regionserver.keytab.file": "*",
              "hbase.regionserver.kerberos.principal": "*",
              "java.security.krb5.conf": "*",
              "principalFile": "***",
              "useLocalFile": true
            },
            "table": "di_flinkx_test:s_auction_target",
            "walFlag": true,
            "writeBufferSize": 300,
            "rowkeyExpress": "$(f1:id)",
            "column": [
              {
                "name": "f1:id",
                "type": "int"
              },
              {
                "name": "f1:c1",
                "type": "varchar"
              },
              {
                "name": "f1:c2",
                "type": "varchar"
              }
            ]
          }
        }
      }
    ],
    "setting": {
      "speed": {
        "channel": 1,
        "bytes": 0
      }
    }
  }
}

3. 报错情况

Exception in thread "main" java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.intellij.rt.execution.CommandLineWrapper.main(CommandLineWrapper.java:64)
Caused by: java.lang.IllegalArgumentException: Row key can't be set multiple times.
	at com.dtstack.chunjun.connector.hbase.HBaseTableSchema.setRowKey(HBaseTableSchema.java:130)
	at com.dtstack.chunjun.connector.hbase.HBaseTableSchema.fromTableSchema(HBaseTableSchema.java:387)
	at com.dtstack.chunjun.connector.hbase14.source.HBase14SourceFactory.createSource(HBase14SourceFactory.java:95)
	at com.dtstack.chunjun.Main.exeSyncJob(Main.java:186)
	at com.dtstack.chunjun.Main.main(Main.java:122)
	at com.dtstack.chunjun.local.test.LocalTest.main(LocalTest.java:178)
	... 5 more

Code of Conduct

• I agree to follow this project's Code of Conduct

[FlechazoW]

I think this is. a bug, and i will fix it.

[wuyiccc]

I think this is. a bug, and i will fix it.

ok, thanks

[kyo-tom]

Fixed via: #1149