Skip to content

Ttft analysis #138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
tttennnnn wants to merge 23 commits into
base: main
Choose a base branch
from
Draft

Ttft analysis #138

Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
a30adc2
Trace 40min; Round 1; Real start
Mar 2, 2026
43c8cb1
Trace 40min; Round 2
Mar 3, 2026
d709fd8
Account for master changes
Mar 3, 2026
7e1b724
Trace 20min; Round 1
Mar 6, 2026
f9f01cf
Trace 20min; Round 2
Mar 6, 2026
75acf26
Merge pull request #140 from hyscale-lab/main
tttennnnn Mar 6, 2026
fc9532f
Merge pull request #141 from hyscale-lab/main
tttennnnn Mar 6, 2026
f9621c9
Account for master changes
Mar 6, 2026
af45930
Merge pull request #142 from hyscale-lab/main
tttennnnn Mar 6, 2026
31469ec
Trace 10min; Round 1
Mar 6, 2026
26d80d8
Trace 10min; Round 2
Mar 7, 2026
7a6bc6c
Trace 20min; rerun TogetherAI
Mar 8, 2026
f0f4d5c
Trace 40min; rerun TogetherAI
Mar 8, 2026
44a2c25
Merge pull request #143 from hyscale-lab/main
tttennnnn Mar 8, 2026
acf865f
Merge main into ttft-analysis, keep perform_multiturn and workflow fr…
tttennnnn Mar 10, 2026
b516c36
Update ttft_analysis yml: split trace/multiturn into A/B/C, update ac…
tttennnnn Mar 10, 2026
eef4166
Add usage logging to perform_multiturn in ttft-analysis
tttennnnn Mar 10, 2026
2e32e4c
Merge branch 'main' into ttft-analysis
tttennnnn Mar 11, 2026
f83fb35
Fix run-name format placeholder count
tttennnnn Mar 11, 2026
f8982c5
Merge branch 'main' into ttft-analysis
tttennnnn Mar 11, 2026
69db7e5
Set VQA max_output to 1 for cleaner TTFT
tttennnnn Mar 11, 2026
59aa8d0
Merge branch 'main' into ttft-analysis
tttennnnn Mar 11, 2026
cffb7fa
Add trace duration choice (10/20/40min) to workflow
tttennnnn Mar 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Trace 20min; Round 2
  • Loading branch information
#MAISON SAPOL# committed Mar 6, 2026
commit f9f01cf53e2b2d3062e8d42da39b281610b2692c
10 changes: 6 additions & 4 deletions benchmarking/experiments/compare_providers_streaming_trace.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
{
"providers": [
"Google",
"Hyperbolic",
"OpenAI",
"PerplexityAI"
"Anthropic",
"AWSBedrock",
"Azure",
"Cloudflare",
"Groq",
"TogetherAI"
],
"models": [
"common-model"
Expand Down
Loading