Skip to content
Open
Changes from 3 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
50f4c34
update new model list with new reuploaded model and ipex option in mo…
szeyu Aug 6, 2024
ec2f421
fix the typo of mistral repo id
szeyu Aug 6, 2024
dbdefa0
edit to the latest version of models available
szeyu Aug 6, 2024
0965d51
change the context length of 128k to 131072
szeyu Aug 6, 2024
b905aa9
Merge branch 'main' into szeyu-autoloader-1
szeyu Aug 12, 2024
5dbf495
onnx auto download model if repo id is provided as model path
szeyu Aug 13, 2024
fb2c63e
formated with black
szeyu Aug 13, 2024
f8c8f27
fixed with flake8
szeyu Aug 14, 2024
d54b4d8
add openvino description and the device gpu
szeyu Aug 14, 2024
1c3b393
update openvino in modelui list
szeyu Aug 14, 2024
75eff7c
first commit of benchmark code
szeyu Aug 15, 2024
608670c
update for the markdown to teach about benchmark code usage
szeyu Aug 15, 2024
21d95aa
Rename benchmark.md to README.md
szeyu Aug 15, 2024
a038376
Update README.md
szeyu Aug 15, 2024
ca93ba9
fixed the bias for encode and output_token_length for openvino
szeyu Aug 15, 2024
79af320
Merge branch 'szeyu-benchmark-2' of https://github.com/EmbeddedLLM/em…
szeyu Aug 15, 2024
632d651
Update loop_ellm_benchmark.py
szeyu Aug 15, 2024
e51527d
add prompt bias to fix the token encode margin error for directml
szeyu Aug 15, 2024
013adc4
Update ellm_benchmark.py
szeyu Aug 16, 2024
62e0b2c
Update loop_ellm_benchmark.py
szeyu Aug 16, 2024
03cc6b7
Update README.md
szeyu Aug 16, 2024
4998e2c
update the benchmark loop to loop without having the model load again…
szeyu Sep 2, 2024
769e558
Update loop_ellm_benchmark.py
szeyu Sep 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion benchmark/benchmark.md → benchmark/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Benchmark
Allow users to test on themselves to get the benchmark of model(s) on different backend. It will analyse the Token In / Out throughput for you in a statistical manner

## Benchmark a Model
To benchmark a model, run this
Expand Down Expand Up @@ -78,4 +79,4 @@ model_names = [
```
```shell
python loop_analyse_detailed_benchmark.py
```
```