Search before asking
What happened
When the running memory size of jm and tm is increased through the configuration in confProp, jobmanager.memory.process.size and taskmanager.memory.process.size only support the input of integer parameters, and the unit is bytes. When the transferred value exceeds Integer.MAX_ VALUE (2147483647) will throw the exception;
java.lang.NumberFormatException: For input string: "4294967296"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:583)
at java.lang.Integer.parseInt(Integer.java:615)
What you expected to happen
It is expected that the parameter can have units, such as g, m, k, etc
How to reproduce
flinkx -jobName "test_hdfs2doris_with_parquet"
-mode yarn-per-job
-jobType sync
-partitionKey dt
-partitionValue $[yyyy-MM-dd-1]
-job $pathDir"/HIVE2DORIS_20221212165533.json"
-flinkxDistDir /data/service/flinkx/flinkx-dist
-flinkConfDir /data/service/flink/conf
-flinkLibDir /data/service/flink/lib
-confProp "{"jobmanager.memory.process.size":"2048m","taskmanager.memory.process.size":"2048m"}" \
or
flinkx -jobName "test_hdfs2doris_with_parquet"
-mode yarn-per-job
-jobType sync
-partitionKey dt
-partitionValue $[yyyy-MM-dd-1]
-job $pathDir"/HIVE2DORIS_20221212165533.json"
-flinkxDistDir /data/service/flinkx/flinkx-dist
-flinkConfDir /data/service/flink/conf
-flinkLibDir /data/service/flink/lib
-confProp "{"jobmanager.memory.process.size":4294967296,"taskmanager.memory.process.size":4294967296}" \
Anything else
No response
Version
1.12_release
Are you willing to submit PR?
Code of Conduct
Search before asking
What happened
When the running memory size of jm and tm is increased through the configuration in confProp, jobmanager.memory.process.size and taskmanager.memory.process.size only support the input of integer parameters, and the unit is bytes. When the transferred value exceeds Integer.MAX_ VALUE (2147483647) will throw the exception;
java.lang.NumberFormatException: For input string: "4294967296"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:583)
at java.lang.Integer.parseInt(Integer.java:615)
What you expected to happen
It is expected that the parameter can have units, such as g, m, k, etc
How to reproduce
flinkx -jobName "test_hdfs2doris_with_parquet"
-mode yarn-per-job
-jobType sync
-partitionKey dt
-partitionValue $[yyyy-MM-dd-1]
-job $pathDir"/HIVE2DORIS_20221212165533.json"
-flinkxDistDir /data/service/flinkx/flinkx-dist
-flinkConfDir /data/service/flink/conf
-flinkLibDir /data/service/flink/lib
-confProp "{"jobmanager.memory.process.size":"2048m","taskmanager.memory.process.size":"2048m"}" \
or
flinkx -jobName "test_hdfs2doris_with_parquet"
-mode yarn-per-job
-jobType sync
-partitionKey dt
-partitionValue $[yyyy-MM-dd-1]
-job $pathDir"/HIVE2DORIS_20221212165533.json"
-flinkxDistDir /data/service/flinkx/flinkx-dist
-flinkConfDir /data/service/flink/conf
-flinkLibDir /data/service/flink/lib
-confProp "{"jobmanager.memory.process.size":4294967296,"taskmanager.memory.process.size":4294967296}" \
Anything else
No response
Version
1.12_release
Are you willing to submit PR?
Code of Conduct