Skip to content

llama-bench : add JSONL (NDJSON) output mode - #9288

Merged
slaren merged 3 commits into
ggml-org:masterfrom
akx:benchmark-jsonl
Sep 3, 2024
Merged

llama-bench : add JSONL (NDJSON) output mode#9288
slaren merged 3 commits into
ggml-org:masterfrom
akx:benchmark-jsonl

Conversation

@akx

@akx akx commented Sep 3, 2024

Copy link
Copy Markdown
Contributor

This PR adds a JSONL/NDJSON output mode to llama-bench. JSONL/NDJSON can be processed line by line even if the generating process fails to e.g. print out the final ] for the regular JSON output.

$ ./llama-bench -n 16,16,16,16 -ngl 99 -o jsonl > foo.jsonl
$ duckdb -c 'select n_gen, mean(avg_ts) from "out.jsonl" group by n_gen;'
┌───────┬──────────────┐
│ n_gen │ mean(avg_ts) │
│ int64 │    double    │
├───────┼──────────────┤
│     0 │   746.241384 │
│    16 │     22.08429 │
└───────┴──────────────┘

@akx
akx marked this pull request as draft September 3, 2024 07:10
@akx
akx marked this pull request as ready for review September 3, 2024 07:23
Comment thread examples/llama-bench/llama-bench.cpp Outdated
Comment thread examples/llama-bench/llama-bench.cpp
Co-authored-by: slaren <slarengh@gmail.com>
@akx
akx requested a review from slaren September 3, 2024 17:33
@akx akx mentioned this pull request Sep 3, 2024
4 tasks
@slaren
slaren merged commit 8962422 into ggml-org:master Sep 3, 2024
@akx akx mentioned this pull request Sep 4, 2024
4 tasks
dsx1986 pushed a commit to dsx1986/llama.cpp that referenced this pull request Oct 29, 2024
* llama-bench : add JSONL (NDJSON) output mode

* llama-bench : update usage docs
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 15, 2024
* llama-bench : add JSONL (NDJSON) output mode

* llama-bench : update usage docs
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 18, 2024
* llama-bench : add JSONL (NDJSON) output mode

* llama-bench : update usage docs
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
* llama-bench : add JSONL (NDJSON) output mode

* llama-bench : update usage docs
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
* llama-bench : add JSONL (NDJSON) output mode

* llama-bench : update usage docs
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
* llama-bench : add JSONL (NDJSON) output mode

* llama-bench : update usage docs
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
* llama-bench : add JSONL (NDJSON) output mode

* llama-bench : update usage docs
phibya pushed a commit to ziee-ai/llama.cpp that referenced this pull request May 29, 2026
* llama-bench : add JSONL (NDJSON) output mode

* llama-bench : update usage docs
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
* llama-bench : add JSONL (NDJSON) output mode

* llama-bench : update usage docs
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
* llama-bench : add JSONL (NDJSON) output mode

* llama-bench : update usage docs
fukuro-kun pushed a commit to fukuro-kun/fukuro-llama-cpp-turboquant that referenced this pull request Jul 5, 2026
* llama-bench : add JSONL (NDJSON) output mode

* llama-bench : update usage docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants