Skip to content

Add DLPack and numpy array protocols to OrtValue Python wrapper - #27836

Closed
Rishi-Dave wants to merge 1 commit into
microsoft:mainfrom
Rishi-Dave:rishidave/feat/ortvalue-dlpack-array-protocols
Closed

Add DLPack and numpy array protocols to OrtValue Python wrapper#27836
Rishi-Dave wants to merge 1 commit into
microsoft:mainfrom
Rishi-Dave:rishidave/feat/ortvalue-dlpack-array-protocols

Conversation

@Rishi-Dave

@Rishi-Dave Rishi-Dave commented Mar 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add __dlpack__(), __dlpack_device__(), and from_dlpack() to the Python OrtValue class, enabling zero-copy tensor sharing with PyTorch, JAX, NumPy, and other DLPack-compatible frameworks.
  • Add __array__() protocol so np.array(ort_value) and np.asarray(ort_value) work directly.
  • The C binding (C.OrtValue) already supported these methods — this PR exposes them through the public Python wrapper class.

Motivation

Fixes #24071

The Python OrtValue wrapper class did not expose the DLPack or numpy array protocols, even though the underlying C binding had full support (added in #23110). Users had to access the internal _ortvalue attribute to use __dlpack__ or from_dlpack, which is fragile and undocumented.

With this change, standard Python interop patterns work out of the box:

# DLPack protocol — zero-copy to/from PyTorch, JAX, etc.
torch_tensor = torch.from_dlpack(ort_value)
np_array = np.from_dlpack(ort_value)
ort_value = OrtValue.from_dlpack(torch_tensor)

# Numpy array protocol
np_array = np.array(ort_value)
np_array = np.asarray(ort_value)

Changes

  • onnxruntime/python/onnxruntime_inference_collection.py: Added 4 methods to OrtValue:
    • __dlpack__(*, stream=None) — returns a DLPack capsule (delegates to C.OrtValue.__dlpack__)
    • __dlpack_device__() — returns (device_type, device_id) tuple
    • from_dlpack(data) — classmethod accepting any __dlpack__-compatible object or raw capsule, with automatic bool tensor detection via is_dlpack_uint8_tensor
    • __array__(dtype=None, copy=None) — numpy array protocol, compatible with NumPy 1.x and 2.x
  • onnxruntime/test/python/onnxruntime_test_python.py: Added 9 test cases covering:
    • __dlpack__ and __dlpack_device__ on the Python wrapper
    • from_dlpack with OrtValue→OrtValue, numpy→OrtValue, and raw capsule→OrtValue
    • np.from_dlpack(ort_value) interop
    • np.array(), np.asarray(), and dtype conversion via __array__
    • Full round-trip across multiple dtypes (float32, float64, int32, int64)

Test Plan

  • All 9 new tests pass locally against pip-installed onnxruntime with DLPack enabled
  • Existing test_ort_value_dlpack and test_ort_value_dlpack_zero_size tests unaffected
  • ruff check and ruff format --check pass on both modified files
  • CI validation

@Rishi-Dave
Rishi-Dave force-pushed the rishidave/feat/ortvalue-dlpack-array-protocols branch from 6afdcc8 to 8311f75 Compare March 26, 2026 12:22
@Rishi-Dave Rishi-Dave changed the title feat: expose DLPack and numpy __array__ protocols on OrtValue Add DLPack and numpy __array__ protocols to Python OrtValue class Mar 26, 2026
…alue

Expose DLPack and numpy array protocols on the Python OrtValue wrapper
class, enabling zero-copy interop with frameworks like PyTorch, JAX, and
numpy via standard Python protocols:

- __dlpack__(stream) and __dlpack_device__() delegate to the existing
  C binding methods, allowing torch.from_dlpack(ort_value) and
  np.from_dlpack(ort_value) to work directly.
- from_dlpack(data) classmethod accepts any __dlpack__-compatible object
  (PyTorch tensors, numpy arrays, JAX arrays, other OrtValues) or raw
  DLPack capsules, with automatic bool tensor detection.
- __array__(dtype, copy) implements the numpy array protocol so that
  np.array(ort_value) and np.asarray(ort_value) work seamlessly.

Previously users had to access the internal _ortvalue attribute to use
these protocols.

Fixes microsoft#24071
@Rishi-Dave
Rishi-Dave force-pushed the rishidave/feat/ortvalue-dlpack-array-protocols branch from 8311f75 to acc297b Compare April 6, 2026 12:30
@Rishi-Dave Rishi-Dave changed the title Add DLPack and numpy __array__ protocols to Python OrtValue class Add DLPack and numpy array protocols to OrtValue Python wrapper Apr 6, 2026
@tianleiwu tianleiwu closed this Apr 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Make OrtValue compatible with numpy __array__ and dlpack protocols

2 participants