Skip to content

[doc-#1415] update doc & starrocks batchsize set #1417

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FlechazoW merged 1 commit into from
Dec 1, 2022
Merged

[doc-#1415] update doc & starrocks batchsize set #1417

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...ocks/src/main/java/com/dtstack/chunjun/connector/starrocks/sink/StarRocksSinkFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ public StarRocksSinkFactory(SyncConf syncConf) {
JsonUtil.toObject(
JsonUtil.toJson(syncConf.getWriter().getParameter()), StarRocksConf.class);

int batchSize = syncConf.getWriter().getIntVal("batchSize", 10240);
int batchSize = syncConf.getWriter().getIntVal("batchSize", 1024);
starRocksConf.setBatchSize(batchSize);
super.initCommonConf(starRocksConf);
}
Expand Down
18 changes: 18 additions & 0 deletions docs_zh/ChunJun连接器/hbase/hbase-sink.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,17 @@ hbase1.4
- 默认值：无


- **column**

- 描述：要写入的hbase字段。
- name：指定写入的hbase列，必须为 列族:列名 的格式；
- type：指定源数据的类型，format指定日期类型的格式
- 必选：是
- 字段类型：List
- 默认值：无
- 注意：为了支持数据中某一列当做row_key这种情况，设置字符串rowkey为关键字，如果某一列name为rowkey，那么该列将不会写入


- **encoding**
- 描述：编码
- 必选：否
Expand Down Expand Up @@ -58,6 +69,13 @@ hbase1.4
<br />


- **batchSize**

- 描述：一次性批量提交的记录数大小，该值可以极大减少ChunJun与数据库的网络交互次数，并提升整体吞吐量。但是该值设置过大可能会造成ChunJun运行进程OOM情况
- 必选：否
- 参数类型：int
- 默认值：1


- **rowkeyExpress**
- 描述： 用于构造rowkey的描述信息，采用字符串格式，形式如下
Expand Down