Skip to content

Add PPU backend support #3807

Merged
lvhan028 merged 13 commits intoInternLM:mainfrom
guozixu2001:gzx/add_ppu_support
Aug 12, 2025
Merged

Add PPU backend support #3807
lvhan028 merged 13 commits intoInternLM:mainfrom
guozixu2001:gzx/add_ppu_support

Conversation

@guozixu2001
Copy link
Copy Markdown
Contributor

Motivation

This pull request introduces backend support for PPU devices in lmdeploy.

Modification

This PR implements the necessary components to integrate PPU as a new device backend. The key changes include:

  1. Added PpuOpsBackend: A new backend implementation (lmdeploy/pytorch/backends/dlinfer/ppu/op_backend.py) is introduced to handle PPU-specific operations.
  2. System-wide Integration:
    • Added 'ppu' as a valid device option in the command-line interface (cli/utils.py), engine configuration (messages.py), and utility functions (utils.py).
    • Updated the backend selector (selector.py) and kernel dispatcher (dispatcher.py) to recognize and load the PpuOpsBackend when the device is set to 'ppu'.

BC-breaking (Optional)

No.

Checklist

  1. Pre-commit or other linting tools are used to fix the potential lint issues.
  2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
  3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
  4. The documentation has been modified accordingly, like docstring or example tutorials.

@jinminxi104 jinminxi104 self-requested a review August 1, 2025 07:10
@jinminxi104 jinminxi104 marked this pull request as ready for review August 6, 2025 06:06
@jinminxi104 jinminxi104 self-requested a review August 6, 2025 06:12
@jinminxi104 jinminxi104 marked this pull request as draft August 6, 2025 06:14
@jinminxi104 jinminxi104 marked this pull request as ready for review August 8, 2025 06:01
@lvhan028
Copy link
Copy Markdown
Collaborator

May resolve the conflict.
Could you kindly provide the user guide as well?

@lvhan028 lvhan028 added the enhancement New feature or request label Aug 12, 2025
@lvhan028 lvhan028 requested a review from grimoire August 12, 2025 03:51
Copy link
Copy Markdown
Collaborator

@grimoire grimoire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lvhan028 lvhan028 merged commit 6012442 into InternLM:main Aug 12, 2025
5 checks passed
yao-fengchen added a commit to DeepLink-org/lmdeploy that referenced this pull request Aug 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants