Native macOS driver for Kinect v2 — built for Apple Silicon, powered by Metal
Kinect4Metal is a ground-up reimagining of Kinect v2 support for macOS. While derived from libfreenect2, this project goes far deeper into the Apple ecosystem, leveraging Metal compute shaders, VideoToolbox hardware decoding, IOKit for low-level USB control, and modern Swift/SwiftUI integration.
The original libfreenect2 was a multi-platform project targeting Linux, Windows, and macOS. Kinect4Metal takes a different approach: macOS-only, Apple-native, no compromises.
- Metal-First Architecture: Depth processing runs entirely on GPU via Metal compute shaders — no OpenCL translation layer, no CPU fallback overhead
- VideoToolbox Integration: Hardware-accelerated JPEG decoding through Apple's native video pipeline
- Apple Silicon Optimized: Native arm64, unified memory architecture awareness, M1/M2/M3/M4 Pro/Max/Ultra support
- Modern APIs: IOKit USB access, Accelerate framework SIMD, Core Image integration
- Swift-Ready: Clean C++ API with Swift bridging headers for SwiftUI/AppKit apps
- Metal compute shaders for real-time depth processing
- VideoToolbox JPEG decoder (zero-copy to Metal textures)
- Universal binary (arm64 + x86_64)
- macOS 11+ (Big Sur and newer)
- CMake build with Xcode generator support
- Framework target for easy app integration
- IOKit USB Rewrite: Direct USB isochronous transfers without libusb
- AVFoundation Source: Expose Kinect as virtual camera device
- Core ML Integration: On-device skeletal tracking and gesture recognition
- RealityKit Bridge: Depth data → ARKit point clouds
- visionOS: Spatial computing depth input (research phase)
| Component | Requirement |
|---|---|
| macOS | 11.0 (Big Sur) or newer |
| Xcode | 13.0+ (Metal compiler) |
| Hardware | USB 3.0 port, Kinect v2 sensor |
| Optional | Homebrew for dependencies |
brew install libusb glfw jpeg-turbo cmake pkg-configgit clone https://github.com/RESMP-DEV/kinect4metal.git
cd kinect4metal
mkdir build && cd build
cmake .. -G Xcode # Or use: cmake ..
cmake --build . --config Release -j$(sysctl -n hw.ncpu)./bin/Protonect metal| Option | Default | Description |
|---|---|---|
ENABLE_METAL |
ON | Metal compute depth processing |
ENABLE_VIDEOTOOLBOX |
ON | Hardware JPEG decoding |
ENABLE_OPENGL |
ON | Legacy OpenGL viewer |
BUILD_FRAMEWORK |
ON | Build as Kinect4Metal.framework |
BUILD_EXAMPLES |
ON | Build Protonect and SwiftUI examples |
cmake .. -DBUILD_FRAMEWORK=ON -G Xcode
xcodebuild -scheme kinect4metal -configuration Release
# Output: build/lib/Kinect4Metal.framework┌─────────────────────────────────────────────────────────────┐
│ Your Application │
├─────────────────────────────────────────────────────────────┤
│ Swift Bindings │ C++ API │ SwiftUI Views │
├─────────────────────────────────────────────────────────────┤
│ Kinect4Metal.framework │
├──────────────┬──────────────┬───────────────┬───────────────┤
│ Metal Depth │ VideoToolbox │ Registration │ Frame Sync │
│ Processing │ RGB Decode │ & Calibration │ & Threading │
├──────────────┴──────────────┴───────────────┴───────────────┤
│ IOKit USB / libusb │
├─────────────────────────────────────────────────────────────┤
│ Kinect v2 Hardware │
└─────────────────────────────────────────────────────────────┘
macOS requires explicit permission for USB device access:
# Check USB devices
system_profiler SPUSBDataType | grep -A5 "Xbox NUI Sensor"
# If not visible, check System Settings → Privacy & Security → USBFor signed apps, add to entitlements:
<key>com.apple.security.device.usb</key>
<true/>Metal pipeline is required on Apple Silicon for optimal performance:
| Pipeline | M1 | M1 Pro | M4 Max |
|---|---|---|---|
| Metal | 30fps | 30fps | 30fps |
| OpenCL (legacy) | 15fps | 20fps | N/A |
| CPU | 8fps | 12fps | 15fps |
Each Kinect v2 requires ~3Gbps USB bandwidth. Apple Silicon Macs have favorable USB topology:
- Thunderbolt Architecture: Most M-series chips have independent controllers per port (not shared hubs)
- Thunderbolt 5: Up to 80Gbps bidirectional — theoretical capacity for 20+ sensors per port
- Unified Memory: Zero-copy potential between USB DMA and Metal textures
Sensor count is limited by:
- Software efficiency (current bottleneck — Kinect4Metal aims to improve this)
- CPU/GPU processing capacity
- Physical USB bandwidth (rarely the limit on modern Macs)
See CONTRIBUTING.md for development setup and guidelines.
Kinect4Metal is derived from libfreenect2 by the OpenKinect project. See NOTICE for attribution details.
MIT License — see LICENSE for details.