pyroDash

Language / 语言: English | 中文

    

---

🔥 Updates

• 2026-07-23: Paper on arXiv:2607.20327.
• 2026-07-22: Release project page, math evaluation code, EasyHard-24K, and models on Hugging Face · pyromind (SFT, GRPO λ=0.05, GRPO λ=0.6). Milestone 1 math eval loop largely done; Collaborate Engine & one-click reproduce still in progress.

---

We propose PyroDash, a token-level dynamic reasoning paradigm for collaborative inference between small and large language models. PyroDash enables the small model to autonomously emit the control token <|llm_offload|> during autoregressive streaming decoding; the collaboration engine then dynamically offloads the local reasoning chain to a large model based on this control signal. This approach requires neither an additional router model nor retraining of the large model, and is naturally compatible with closed-source LLM services.

During training, PyroDash follows a three-stage progressive optimization pipeline: (1) train the control-token embedding layer so the small model acquires basic offloading expressiveness; (2) cold-start the offload capability to establish a collaboration pattern between the small and large models; and (3) apply GRPO reinforcement learning that jointly optimizes the dynamic offloading policy with a task-accuracy reward and a large-model call-cost penalty, achieving an adaptive balance between reasoning quality and compute cost. For more details, please refer to our paper (citation below).

---

🚀 Quick Start

1. Setup

git clone https://github.com/PyroMind-Dynamics/pyroDash.git
cd pyroDash
pip install -r requirements.txt

2. Run evaluation (evaluation/math_eval.sh)

Edit placeholders in evaluation/math_eval.sh, then:

bash evaluation/math_eval.sh

The script (1) starts a local vLLM server for the small model on port 8001, (2) runs math_eval.py, and (3) stops vLLM on exit.

Parameters

Variable / flag	Meaning	Example
MODEL	Local merged model path (vLLM serve + tokenizer)	/path/to/your/merged_model
--glm-base-url	OpenAI-compatible API for the large/relay model	http://your-glm-host:8000/v1
--glm-api-key	API key for that endpoint	your-glm-api-key
--glm-model	Served model name on the GLM side	your-glm-model
--output-dir	Per-dataset JSON output directory	./results_500
--datasets	Benchmarks (space-separated)	gsm8k minerva olympiad aime2024 aime2025

Tokenizer must include the special token <|llm_offload|>.

---

📊 Results

---

📋 TODO

Milestone 1 — Math eval loop

• Baselines: GLM-5.2 upper bound / Qwen3.5-4B lower bound + token cost stats
• vLLM + GLM relay (<|llm_offload|>) + per-dataset JSON + cost aggregation
• Comparisons: PyroDash / Query Router / Token Router + Pareto curve
• λ sweep & ablations
• One-click reproduce on PyroMind Console (end-to-end eval)
• Collaborate Engine

Milestone 2 — Coding + Agentic

• SWE-Bench (Verified / Lite) harness
• Terminal-Bench v2 harness
• Sandbox / scoring + offload trajectory & token stats
• Qwen3.5-4B / GLM-5.2 / PyroDash comparison + cost tables
• Unified Math + SWE + Terminal results & end-to-end scripts

Milestone 3 — Coding Plan release

• Product definition & collaborative inference integration
• Coding-scenario optimizations (completion / refactor / debug)
• Release & promotion

---

🔗 Resources

Resource	Link
Project website	PyroMind-Dynamics.github.io/pyroDash
Paper	arXiv:2607.20327
Dataset (EasyHard-24K)	huggingface.co/datasets/pyromind/easyhard-24k
Hugging Face org	huggingface.co/pyromind

---

📖 Citation

If you find PyroDash useful, please cite:

@misc{lyu2026pyrodash,
  title        = {PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference},
  author       = {Niqi Lyu and Pengtao Shi and Wei Qiu and Jianlin Zhong and Sicong Xia and Jianyao Ma and Yicheng Ding},
  year         = {2026},
  eprint       = {2607.20327},
  archivePrefix= {arXiv},
  primaryClass = {cs.CL},
  url          = {https://arxiv.org/abs/2607.20327}
}

Dataset:

@misc{pyromind2026easyhard24k,
  title        = {{EasyHard-24K} v0.02},
  author       = {{PyroMind Dynamics}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/datasets/pyromind/easyhard-24k}}
}