Lesson 5: Beyond GGUF: The Format Zoo
Lesson 5: Beyond GGUF: The Format Zoo
GGUF is the llama.cpp universe's format, but it's not the only one. Each runtime reads a different container, and each quant method has different strengths. Here's the full zoo, all of it real and all of it findable on Hugging Face for Qwen3.8-27B:
| Format | Bits | Runtimes | Strengths |
|---|---|---|---|
GPTQ -GPTQ-Int4 | int4 (W4A16) | AutoGPTQ, vLLM, TGI, ExLlama | Oldest calibrated method; error-correction pass |
AWQ -AWQ, AWQ-INT4 | int4 (W4A16) | vLLM, LM Studio, TGI | Activation-aware — protects outlier channels; usually beats GPTQ |
EXL2 / EXL3 exl2, exl3, 4.0bpw | any bpw | ExLlamaV2/V3 | Exact-size quants; fastest single-GPU llama-class loader; self-calibrated (SC) variants |
MLX -4bit, -8bit | 4 / 8 | Apple Silicon (mlx_lm) | The right choice on Macs — native Metal, fast |
bitsandbytes nf4, fp4 | 4 | transformers load_in_4bit | On-the-fly loading; the standard for QLoRA fine-tuning |
FP8 -FP8, FP8-Dynamic | 8 (E4M3) | vLLM, TGI, TensorRT | Datacenter-grade; ~half of bf16; near-lossless |
NVFP4 -NVFP4 | 4 (FP, E1M2) | vLLM on NVIDIA Blackwell (RTX 50xx) | 4-bit float with Blackwell tensor-core speed; ~13.7 GB for 27B |
QAT -QAT | 4 / 8 | vLLM, TensorRT, transformers | Quantization-aware trained — the best 4-bit quality |
AQLM 2bit 1x16 | 2 | transformers, vLLM (research) | Additive quantization; the 2-bit quality leader |
HQQ -HQQ | 2-8 | transformers | Quantizes in seconds on any hardware |
EETQ -EETQ | 8 | transformers (Linux x86) | Fast, dependency-light 8-bit |
Decoding the names you asked about
int4 / int8: generic terms for integer quantization. "4-bit model" and "int4 model" usually mean the same thing. Precision matters: int4 on NVIDIA cards historically means "weights rounded to 16 levels with scales" — exactly what AWQ/GPTQ do better via calibration.
AWQ (Activation-aware Weight Quantization): instead of treating all weights equally, AWQ measures activations and finds the ~1% of channels that dominate outputs, then protects them by scaling before rounding. Result: better quality than GPTQ at the same int4 size, and it's the default in LM Studio and vLLM for good reason. Variants you'll see: AWQ-INT4, AWQ-5.0bpw, AWQ-Marlin.
Marlin: not a quant — a kernel. Marlin kernels run int4 GPTQ/AWQ weights extremely fast on NVIDIA GPUs. A -Marlin suffix means "the same quant, with fast kernels"; pick it when your runtime supports it (vLLM does).
nvfp4: NVIDIA's FP4 format — a 4-bit floating point (1 sign, 1 exponent, 2 mantissa bits) designed for Blackwell tensor cores (RTX 50-series). It's the current frontier of "small AND fast": a Qwen3.8-27B NVFP4 file is ~13.7 GB with big speedups on 50-series cards (unsloth's NVFP4 uses their Dynamic 3.0 calibration). On older GPUs FP4 falls back to slow paths — check your card before downloading. The gittensor variant NVFP4-RTX5090 in the search results says exactly what it needs.
flash: on the format side, flash usually means FlashAttention kernels (faster attention, longer contexts, less memory). The famous Qwen2.5-7B-Instruct-1M-flash was a FlashAttention build that fit 1M-token context into a GPU. There are also fine-tunes and model families named "Flash" — check whether the word describes attention kernels or a tuned variant.
MLX vs GGUF: the Mac question
If you're on Apple Silicon this is the fork in the road, so it deserves its own showdown. MLX is Apple's machine-learning framework — native Metal kernels, built around the Mac's unified memory — while GGUF is the universal llama.cpp container that also runs on Macs (llama.cpp has a mature Metal backend). Both will run the same model on the same Mac; the differences are practical:
| GGUF (llama.cpp) | MLX | |
|---|---|---|
| Runs on | CPU, NVIDIA, AMD, Intel, Apple | Apple Silicon only |
| File shape | One self-contained .gguf file | A folder of safetensors shards + config |
| Quant ladder | Q4_K_M, Q6_K, IQ, UD, imatrix — huge choice | 4-bit and 8-bit (plus some 3/6-bit community work) |
| Speed on Mac | Fast (Metal, mature) | Usually faster — native Metal kernels, no conversion layer |
| Tooling | Ollama, LM Studio, llama.cpp, koboldcpp; MTP support | mlx_lm (CLI + OpenAI-compatible server), built-in LoRA/QLoRA fine-tuning |
| Pick when | You use Ollama/LM Studio, want a specific quant (UD/imatrix/IQ), or also run the model on non-Mac hardware | You're all-in on Apple Silicon and want maximum native speed or on-Mac fine-tuning |
Quality at the same bit level is comparable — MLX 4-bit is roughly Q4-class, MLX 8-bit is near-lossless. The honest guidance: on a Mac, MLX is the fastest native path; GGUF is the most compatible path. You can't go wrong either way, and you can switch formats without switching models — both live on Hugging Face (official/unsloth repos for GGUF, mlx-community for MLX). One real-world tiebreaker: if you want Ollama's one-command workflow or a specific GGUF quant (like UD-Q4_K_M), go GGUF; if you want every last token per second out of your M-series chip, go MLX.
safetensors vs gguf
The official Qwen/Qwen3.8-27B repo ships safetensors — the raw, unquantized weights (fp8 in the -FP8 repo, bf16 in the main one) that transformers/vLLM load directly. GGUF, EXL3, MLX, AWQ etc. are downstream conversions. As a rule: safetensors = framework-native; GGUF = llama.cpp family; pick by your software, not by prestige.
🧠 Knowledge Check
1. AWQ improves over naive int4 by:
2. nvfp4 is:
3. Which format should a Mac (Apple Silicon) user prefer?
4. On a Mac, when should you pick GGUF over MLX?