Chủ đề #quantization trên GitHub — repository trong kho Beeknoee có gắn topic này.
Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
Faster Whisper transcription with CTranslate2
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
Your Cheat Sheet for AI Engineering Interview – Questions and Answers.
Phonon: open speech recognition models (Phonon-2, Phonon-1) — CLI, CPU and CUDA images
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
Kyojin: open-source engine that runs the largest open models on one mini PC or one graphics card, about 2× faster than llama.cpp. AMD (ROCm) and NVIDIA (CUDA), built on ExLlamaV3.
State-of-the-art LLM compression, built for production inference with vLLM
Các chủ đề phổ biến khác trong kho Beeknoee
