English | 简体中文
✒️ TechnicalReport | 📖 Documentation | 🤗 Hugging Face | 🤖 ModelScope
- [26/09/01] We've released the hy4 preview MIX-STQ1_0 version! Compressed from 1.5TB to 214 GiB with only a 0.7% performance drop on SWE-bench Pro. Using Prima.cpp, we ran this 214 GB model on a Laptop 4090 (32GB RAM, 16GB VRAM) + a Server with 4x A4000 (32GB RAM, 16GB VRAM each), hitting 1.02 tokens/s. See the blog and the deployment guideline! 🔥🔥🔥
- [26/07/29] We have open-sourced AngelSpec, a torch-native, disaggregated speculative-decoding training framework with support for a variety of draft methods, led by DFly and MTP + TTT, and released the MTP and DFly drafter weights for Hy3-A21B — DFly delivers up to 2.40× average end-to-end speedup over AR, and D-cut adds up to +15.7% throughput at high concurrency. [Paper] | [GitHub] | [Docs] | [Hugging Face] 🔥🔥🔥
- [26/07/20] We now support scale-only quantization-aware distillation on Megatron-Core for Qwen3-MoE and Hy3, with TP/EP/CP/SP distributed training. [Docs]
- [26/07/06] We now support FP8-Static quantization , SmoothQuant for Hy3 (MoE A21B).[Docs]
- [26/06/04] We have released Stem, a sparse attention algorithm that accelerates the Prefill stage of long-context LLMs by dynamically selecting top-k key blocks for block-sparse attention, significantly reducing latency while preserving generation quality. [Docs]
- [26/06/01] We have released DFlare, a block-diffusion speculative decoding framework with layer-wise fusion that achieves up to 5.52× end-to-end speedup. [Docs]
- [26/05/27] We have released D-Cut, an adaptive verification depth pruning technique for speculative decoding. [Docs]
- [26/05/20] We support Distillation for full-precision HuggingFace models and quantized QAT-style models, as detailed in the distillation documentation.
- [26/05/08] We have released STQ1_0 kernel for 1.25-bit model and given a PR to llama.cpp