Error when running start-workload.sh

[sunchao]

In the README it says:

./bin/start-workload.sh
    -Dauditreplay.input-path hdfs:///dyno/audit_logs/
    -Dauditreplay.num-threads 50
    -nn_uri hdfs://namenode_address:port/
    -start_time_offset 5m
    -mapper_class_name AuditReplayMapper

However, it seems both Dauditreplay.input-path and -Dauditreplay.num-threads are not valid options. Only valid options are: nn_uri, start_time_offset, start_timestamp_ms, and mapper_class_name?

[xkrogen]

Ah.. The -D flag is used to specify configurations, not input arguments -- the Client is run using ToolRunner so the standard Hadoop argument parsing is done for -D. However, it needs to have an = instead of a space separating the config key and value. Can you try the following instead?

./bin/start-workload.sh
    -Dauditreplay.input-path=hdfs:///dyno/audit_logs/
    -Dauditreplay.num-threads=50
    -nn_uri hdfs://namenode_address:port/
    -start_time_offset 5m
    -mapper_class_name AuditReplayMapper

[sunchao]

Thanks @xkrogen ! and sorry for the late reply. I've opened a PR to fix a few small issues I encountered while running start-workload.sh.

[hexiangheng]

when i running start-workload.sh but failed
The running command:
./bin/start-workload.sh -Dauditreplay.input-path=hdfs:///dyno/hdfs-audit.log -Dauditreplay.num-threads=50 -nn_uri hdfs://EC130:9000/ -start_time_offset 5m -mapper_class_name AuditReplayMapper

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hxh/hadoop/hadoop-3.3.0-SNAPSHOT/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hxh/hadoop/dynamometer-fat-0.1.0-SNAPSHOT/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread "main" java.lang.ClassNotFoundException: Class com.linkedin.dynamometer.workloadgenerator.AuditReplayMapper not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2529)
at com.linkedin.dynamometer.workloadgenerator.WorkloadDriver.getMapperClass(WorkloadDriver.java:146)
at com.linkedin.dynamometer.workloadgenerator.WorkloadDriver.run(WorkloadDriver.java:99)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at com.linkedin.dynamometer.workloadgenerator.WorkloadDriver.main(WorkloadDriver.java:139)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)

[xkrogen]

Hm, yes, this issue is what PR #62 is trying to fix. There's a small issue there.. Let me put together another patch for that

[xkrogen]

Hey @sunchao and @hexiangheng , I submitted my own PR for this #84, can you take a look there and see if it looks good?

[xkrogen]

Closed by PR #84