Commits

Branch selector

 main

User selector

All users

All time

Commit History

Commits on Apr 19, 2024

• [Bugfix][Core] Restore logging of stats in the async engine (#4150)

  ronensc

  authored
  7be4f56

  Copy full SHA for 7be4f56

• [Misc] fix docstrings (#4191)

  Show description for 8f20fc0
  

  UranusSeven

  and

  Zhong Wang

  authored
  8f20fc0

  Copy full SHA for 8f20fc0

• Bump version of 0.4.1 (#4177)

  simon-mo

  authored
  221d93e

  Copy full SHA for 221d93e

• [Bugfix] Fix LoRA loading check (#4138)

  Show description for d17c847
  

  jeejeelee

  and

  simon-mo

  authored
  d17c847

  Copy full SHA for d17c847

• Support eos_token_id from generation_config.json (#4182)

  simon-mo

  authored
  a134ef6

  Copy full SHA for a134ef6

Commits on Apr 18, 2024

• [Core] add an option to log every function call to for debugging hang/crash in distributed inference (#4079)

  Show description for 8a7a3e4
  

  youkaichao

  and

  simon-mo

  authored
  8a7a3e4

  Copy full SHA for 8a7a3e4

• [Bugfix] Fix CustomAllreduce nvlink topology detection (#3974)

  Show description for 8f9c28f
  agt

  authored
  8f9c28f

  Copy full SHA for 8f9c28f

• [CI/CD] add neuron docker and ci test scripts (#3571)

  liangfu

  authored
  cd2f63f

  Copy full SHA for cd2f63f

• [Misc] Bump transformers to latest version (#4176)

  njhill

  authored
  87fa80c

  Copy full SHA for 87fa80c

• [Bugfix] Support logprobs when using guided_json and other constrained decoding fields (#4149)

  jamestwhedbee

  authored
  e1bb2fd

  Copy full SHA for e1bb2fd

• [Docs] document that Meta Llama 3 is supported (#4175)

  simon-mo

  authored
  705578a

  Copy full SHA for 705578a

• [Bugfix][Kernel] allow non-power-of-two head sizes in prefix prefill (#4128)

  mmoskal

  authored
  e8cc796

  Copy full SHA for e8cc796

• [Bugfix] Get available quantization methods from quantization registry (#4098)

  mgoin

  authored
  53b018e

  Copy full SHA for 53b018e

• Allow model to be served under multiple names (#2894)

  Show description for 66ded03
  

  hmellor

  and

  payoto

  authored
  66ded03

  Copy full SHA for 66ded03

• [Core] nccl integrity check and test (#4155)

  Show description for 6dc1fc9
  youkaichao

  authored
  6dc1fc9

  Copy full SHA for 6dc1fc9

• [Typing] Mypy typing part 2 (#4043)

  Show description for 533d2a1
  

  rkooo567

  and

  SangBin Cho

  authored
  533d2a1

  Copy full SHA for 533d2a1

Commits on Apr 17, 2024

• [Kernel] Add punica dimension for Swallow-MS-7B LoRA (#4134)

  ucciicci

  authored
  a532225

  Copy full SHA for a532225

• [Bugfix] fix output parsing error for trtllm backend (#4137)

  Show description for fe3b5bb
  

  elinx

  and

  ywang96

  authored
  fe3b5bb

  Copy full SHA for fe3b5bb

• [Core] RayWorkerVllm --> WorkerWrapper to reduce duplication (#4024)

  Show description for 8438e05
  youkaichao

  authored
  8438e05

  Copy full SHA for 8438e05

• [CI] Move CPU/AMD tests to after wait (#4123)

  cadedaniel

  authored
  11d652b

  Copy full SHA for 11d652b

• [Misc] [CI] Fix CI failure caught after merge (#4126)

  cadedaniel

  authored
  d150e4f

  Copy full SHA for d150e4f

Commits on Apr 16, 2024

• [Speculative decoding 6/9] Integrate speculative decoding with LLMEngine (#3894)

  cadedaniel

  authored
  e95cd87

  Copy full SHA for e95cd87

• [Core] Refactor model loading code (#4097)

  Yard1

  authored
  69e1d2f

  Copy full SHA for 69e1d2f

• LM Format Enforcer Guided Decoding Support (#3868)

  Show description for 0543476
  

  noamgat

  and

  simon-mo

  authored
  0543476

  Copy full SHA for 0543476

• [Core] Fix engine-use-ray broken (#4105)

  rkooo567

  authored
  4e7ee66

  Copy full SHA for 4e7ee66

Commits on Apr 15, 2024

• [Typing] Fix Sequence type GenericAlias only available after Python 3.9. (#4092)

  rkooo567

  authored
  37e84a4

  Copy full SHA for 37e84a4

• [Bugfix] Fix ray workers profiling with nsight (#4095)

  rickyyx

  authored
  4695397

  Copy full SHA for 4695397

• [Doc] Add better clarity for tensorizer usage (#4090)

  Show description for d619ae2
  

  sangstar

  and

  ywang96

  authored
  d619ae2

  Copy full SHA for d619ae2

• [Core] Simplifications to executor classes (#4071)

  njhill

  authored
  eb46fbf

  Copy full SHA for eb46fbf

• [Misc][Minor] Fix CPU block num log in CPUExecutor. (#4088)

  bigPYJ1151

  authored
  0003e91

  Copy full SHA for 0003e91

• [Bugfix] Fix filelock version requirement (#4075)

  zhuohan123

  authored
  e11e200

  Copy full SHA for e11e200

• [Misc] Upgrade triton to 2.2.0 (#4061)

  esmeetu

  authored
  8db1bf3

  Copy full SHA for 8db1bf3

Commits on Apr 14, 2024

• [Docs] document that mixtral 8x22b is supported (#4073)

  simon-mo

  authored
  aceb17c

  Copy full SHA for aceb17c

• [BugFix] Fix tensorizer extra in setup.py (#4072)

  njhill

  authored
  563c54f

  Copy full SHA for 563c54f

• [Core] avoid too many cuda context by caching p2p test (#4021)

  youkaichao

  authored
  2cd6b4f

  Copy full SHA for 2cd6b4f

Pagination

Previous

Next