Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism

GNNear: Accelerating Full-Batch Training of Graph NeuralNetworks with Near-Memory Processing

The Artifact of NeoMem: Hardware/Software Co-Design for CXL-Native Memory Tiering

LLM Inference analyzer for different hardware platforms

Compare different hardware platforms via the Roofline Model for LLM inference tasks.

Efficient operation implementation based on the Cambricon Machine Learning Unit (MLU) .

LLM inference in C/C++

ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale

Latency and Memory Analysis of Transformer Models for Training and Inference

This is the FreePDK45 V1.4 Process Development Kit for the 45 nm technology

Serving multiple LoRA finetuned LLM as one

A benchmark suite for xillybus

An integrated power, area, and timing modeling framework for multicore and manycore architectures

Python-based research interface for blackbox and hyperparameter optimization, based on the internal Google Vizier Service.

Hardware utilities with Spinal HDL

Provide Python access to the NVML library for GPU diagnostics

Cavs: An Efficient Runtime System for Dynamic Neural Networks

Yinghan's Code Sample

RISC-V Instruction Set Manual

Deep learning toolkit-enabled VLSI placement

Bridging polyhedral analysis tools to the MLIR framework

Polyhedral High-Level Synthesis in MLIR

Neural network graphs and training metrics for PyTorch, Tensorflow, and Keras.

Research and development for optimizing transformers

Reproduce Fast ConvNets @CVPR 2020