Glyd keeps a bf16 model's weights and KV cache in fewer bits in GPU memory and rebuilds the exact values as the model runs. It is the same model in about a third less memory, so it fits a smaller GPU or serves more requests on the same one.
curl -LsSf https://getglyd.com/install.sh | sh
glyd run Qwen/Qwen3-8B # settings for your GPU, then a chat in the terminalWith vLLM, or in Python:
pip install "glyd[vllm]"
vllm serve Qwen/Qwen3-8B --quantization glydimport glyd
model = glyd.from_pretrained("Qwen/Qwen3-8B")Linux (x86_64 or aarch64) with an NVIDIA GPU: Ampere, Ada or Hopper, driver 580 or newer.
| Measured | bf16 | Glyd |
|---|---|---|
| Qwen3-8B under vLLM on an L4: weights | 15.27 GiB | 10.38 GiB (−32.0%) |
| Qwen3-32B: 48 GB GPUs it needs | 2 | 1 |
| Qwen2.5-72B on two H100s: requests a second, at full load | 0.88 | 3.59 (4.1x) |
| Qwen3-30B-A3B on an H100: requests a second, at full load | 12.32 | 17.10 (1.39x) |
The freed memory becomes KV cache, which is where the extra requests come from. Logs: vllm-v028-l4-2026-10-04, lambda-gpu_4x_a6000-20260926-084757, h100x2-qwen2.5-72b-2026-10-03, h100-moe-v028-2026-10-03. Every result, and where Glyd is still slower than bf16: getglyd.com/benchmarks.
What lossless means here: every weight, and every key and value in the KV cache, reads back as the
exact bf16 value. Outputs can still differ from bf16's in the last bits, because Glyd's kernels add the
same products in a different order, as any two GPU kernels do. exact=True gives bf16's logits exactly.
Underneath the GPU work, Glyd is a lossless compression library and CLI in Rust with a C ABI: a drop-in
alternative to LZ4, Snappy and zstd that also turns logs and table dumps into typed columns (-r) and
compresses a new version of a file against the old one (--base).
brew install surya-koritala/glyd/glyd # or a release's tarball
glyd --max events.json -o events.glyd # fewer bytes than zstd -3, faster reads