- Hexagon backend adds support for the LIGHTNING_INDEXER operation, including initial support for the op.
- Hexagon LIGHTNING_INDEXER adds an HMX path for prefill.
- Hexagon LIGHTNING_INDEXER adds a q8_0 decode gather transpose and two-block decode compute.
- Hexagon LIGHTNING_INDEXER adds a pipelined HMX path for both decode and prefill.
- Hexagon LIGHTNING_INDEXER adds support for K in f32 format.
- Hexagon LIGHTNING_INDEXER adds support for q4_0 quantized K.
- Hexagon LIGHTNING_INDEXER adds dimension collapse support.
- Hexagon backend adds a GGML_HEXAGON_LI_SELECT environment option to force the HMX or HVX paths.
- Hexagon LIGHTNING_INDEXER removes scalar VTCM reads in the HMX q4_0 K preparation and removes vector spills.
- Hexagon LIGHTNING_INDEXER host and kernel kparams alignment is updated and software devices are removed from the kernel.
- Hexagon LIGHTNING_INDEXER DMA pipeline uses canonical double buffering in all paths.
- Hexagon LIGHTNING_INDEXER kparams are made consistent with other ops.
From llama.cpp
hexagon: add LIGHTNING_INDEXER support (#31289)
-
hexagon: initial support for LIGHTNING_INDEXER op
-
hex-lght-idxr: hmx for prefill
-
hex-lght-idxr: q8_0 decode gather transpose, two-block decode compute
-
hex-lght-idxr: pipelined hmx path for decode and prefill
-
hex-lid: add support for K (f32)
-
hex-lid: add support for K (f32)
-
hex-lid: add support for q4_0
-
hex-lid: remove scalar VTCM reads in the HMX q4_0 K prep and remove vector spills
-
hex-lid: host/kernel kparams alighnment and remove sw devices in kernel
-
hex-lit: update dma pipeline to use canonical dbl buffering in all paths
-
hex-lit: add GGML_HEXAGON_LI_SELECT to force hmx/hvx paths
-
hex-lit: add support for dims collapse
-
hex-lit: make kparams constistent with other ops
Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.com
Website:
Attestations:
macOS/iOS:
- macOS Apple Silicon (arm64)
- macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED
- macOS Intel (x64)
- iOS XCFramework
Linux:
- Ubuntu x64 (CPU)
- Ubuntu arm64 (CPU)
- Ubuntu s390x (CPU)
- Ubuntu 26.04 s390x (CPU)
- Ubuntu 26.04 s390x (zDNN)
- Ubuntu x64 (Vulkan)
- Ubuntu arm64 (Vulkan)
- Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries
- Ubuntu x64 (CUDA 13) - CUDA 13.4 libraries
- Ubuntu arm64 (CUDA 13) - CUDA 13.4 libraries
- Ubuntu x64 (ROCm 10.0)
- Ubuntu x64 (OpenVINO)
- Ubuntu x64 (SYCL FP32)
- Ubuntu x64 (SYCL FP16)
- Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide
Android:
Windows:
- Windows x64 (CPU)
- Windows arm64 (CPU)
- Windows arm64 (OpenCL Adreno)
- Windows x64 (CUDA 12) - CUDA 12.4 DLLs
- Windows x64 (CUDA 13) - CUDA 13.4 DLLs
- Windows arm64 (CUDA 13) - CUDA 13.4 DLLs
- Windows x64 (Vulkan)
- Windows arm64 (Vulkan)
- Windows x64 (OpenVINO)
- Windows x64 (SYCL)
- Windows x64 (ROCm 10.0)
openEuler:
- DISABLED
- openEuler x86 (310p)
- openEuler x86 (910b, ACL Graph)
- openEuler aarch64 (310p)
- openEuler aarch64 (910b, ACL Graph)
UI: