llama-bench : log benchmark progress

[akx]

Simply adds logging for the parameter set index and number of parameter sets before each test in llama-bench, to make it easier to gauge how long it'll still take.

---

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[slaren]

This will break every use case except when redirecting stdout.

[slaren]

Are you aware that you can use -oe to choose a printer to use with stderr?

I could see this implemented as type of printer that only prints a detailed progress report so that you can use eg. llama-bench -o json -oe progress > out.json. You can add more methods to the printer class if necessary to be able to print a more detailed progress progress. Do not use the LOG macro.

[akx]

This will break every use case except when redirecting stdout.

Every use case? 😱 😄

Fair, I added a commit to only print them when -verbose (i.e. when you'd get more stuff on stderr anyway). Would that work?

Are you aware that you can use -oe to choose a printer to use with stderr?

Yep, I am (via #9288 :) )

[akx]

@slaren Mind taking another look? Thanks :)

[slaren]

I do not really see the point of this change. Even with verbose, this would add output that I would rather not see in verbose mode. It seems that it is trying to solve the same problem for which -oe was added, and for me that works well enough.

[akx]

My UC is simply being able to see the progress of a benchmark run more granularly.

For instance, it's not quite clear that there's a warmup thing – I was initially mightily confused seeing 6 llama_print_timingses for what was supposed to be a 5-benchmark run, etc.

I think progress indication is also useful to figure out if e.g. a remote benchmark run may have crashed or timed out.

[slaren]

I wouldn't want to see this even in verbose mode, but that wouldn't be a problem if it was optional and gated behind a command line parameter. Also please change the LOG to fprintf to stderr directly, the logging library is not used in llama-bench.

[akx]

but that wouldn't be a problem if it was optional and gated behind a command line parameter.

Sure.

Also please change the LOG to fprintf to stderr directly

Will do.

the logging library is not used in llama-bench.

It is (since #8672 last week), which is why I was using it:

https://github.com/ggerganov/llama.cpp/blob/8ebe8ddebd68526757c631cd019de009697c63c2/examples/llama-bench/llama-bench.cpp#L1559

https://github.com/ggerganov/llama.cpp/blob/8ebe8ddebd68526757c631cd019de009697c63c2/examples/llama-bench/llama-bench.cpp#L1568

Should these get changed too?

[slaren]

Should these get changed too?

Yes, thanks for pointing that. It can be done in a different PR.
Edit: pushed a fix together with other fix.