Skip to content

Latest commit

 

History

History
50 lines (42 loc) · 1.75 KB

README.md

File metadata and controls

50 lines (42 loc) · 1.75 KB

SGLang Kernels NPU

Introduction

SGLang Kernels for Ascend NPU

Software and hardware

Supported Hardware Models: Atlas A3 Series Products Platform: aarch64/x86 Supporting Software

  • Driver Ascend HDK 25.0.RC1.1, CANN 8.3.RC1 or later versions (refer to the "CANN Software Installation Guide" to install the CANN development kit package, as well as the supporting firmware and drivers)
  • Before installing CANN software, you need to install the relevant dependency list
  • Python >= 3.7
  • pybind11 (install via pip install pybind11)

Quick Start

Compile and Run

  1. Prepare the CANN environment variables (modify according to the installation path)
source /usr/local/Ascend/ascend-toolkit/set_env.sh
  1. Build the project Executing the engineering build script build.sh
# Building Project
bash build.sh

Installation

  1. Pip install the .whl file into your Python environment
pip install output/sgl_kernel_npu*.whl

# (Optional) Confirm whether the import can be successfully
python -c "import sgl_kernel_npu; print(sgl_kernel_npu.__path__)"
  1. Execute the environment variables for CANN (modify according to the installation path)
source /usr/local/Ascend/ascend-toolkit/set_env.sh
  1. In the Python project, import sgl_kernel_npu.

Test

Execute sgl_kernel_npu test scripts, for example

python3 tests/python/sgl_kernel_npu/test_hello_world.py