XNNPACK is a highly optimized solution for neural network inference on ARM, x86, WebAssembly, and RISC-V platforms. XNNPACK is not intended for direct use by deep learning practitioners and researchers; instead it provides low-level performance primitives for accelerating high-level machine learning frameworks, such as TensorFlow Lite, TensorFlow.js, PyTorch, ONNX Runtime, ExecuTorch, and MediaPipe.
- ARM64 on Android, iOS, macOS, Linux, and Windows
- ARMv7 (with NEON) on Android
- ARMv6 (with VFPv2) on Linux
- x86 and x86-64 (up to AVX512) on Windows, Linux, macOS, Android, and iOS simulator
- WebAssembly MVP
- WebAssembly SIMD
- WebAssembly Relaxed SIMD (experimental)
- RISC-V (RV32GC and RV64GC)
- Hexagon (with HVX)
XNNPACK implements the following neural network operators:
- 2D Convolution (including grouped and depthwise)
- 2D Deconvolution (AKA Transposed Convolution)
- 2D Average Pooling
- 2D Max Pooling
- 2D ArgMax Pooling (Max Pooling + indices)
- 2D Unpooling
- 2D Bilinear Resize
- 2D Depth-to-Space (AKA Pixel Shuffle)
- Add (including broadcasting, two inputs only)
- Subtract (including broadcasting)
- Divide (including broadcasting)
- Maximum (including broadcasting)
- Minimum (including broadcasting)
- Multiply (including broadcasting)
- Squared Difference (including broadcasting)
- Global Average Pooling
- Channel Shuffle
- Fully Connected
- Abs (absolute value)
- Bankers' Rounding (rounding to nearest, ties to even)
- Ceiling (rounding to integer above)
- Clamp (includes ReLU and ReLU6)
- Convert (includes fixed-point and half-precision quantization and dequantization)
- Copy
- ELU
- Floor (rounding to integer below)
- HardSwish
- Leaky ReLU
- Negate
- Sigmoid
- Softmax
- Square
- Tanh
- Transpose
- Truncation (rounding to integer towards zero)
- PReLU
All operators in XNNPACK support NHWC layout, but additionally allow custom stride along the Channel dimension. Thus, operators can consume a subset of channels in the input tensor, and produce a subset of channels in the output tensor, providing a zero-cost Channel Split and Channel Concatenation operations.