Qwen3.8-27B
Five files, 6.19 to 14.25 GB, and a vision projector for images. How they score
Get the filesOpen-weight models in standard GGUF, made with our own post-training method: no training, no fine-tuning. The files load in upstream llama.cpp, with nothing to patch.
Open weights · Standard GGUF · Stock llama.cpp · 13,320 downloads
hf download KarLexAI/Qwen3.8-27B-GGUF Qwen3.8-27B-KLX-IQ1_S.gguf --local-dir .
llama-server -m Qwen3.8-27B-KLX-IQ1_S.gguf -ngl 99 -c 32768 --jinjaThen open http://localhost:8080 and chat; thinking is on by default, and Qwen's sampling settings are in the file. The two large models need more memory than the file size, GPU and system RAM together, and a llama.cpp build that supports them: Kimi K3.1 Pro from #31779 (merged 6 November 2026), DeepSeek V4.1 Flash as tested at commit bcb0152. Their commands: Kimi K3.1 Pro, DeepSeek V4.1 Flash.
Five files, 6.19 to 14.25 GB, and a vision projector for images. How they score
Get the filesThree files, 593 to 710 GB, and a vision projector for images. How they score
Get the filesOne 232.6 GB file of a model that is 510 GB as released, and a vision projector for images. How they score
Get the filesThe smallest GGUFs of a reasoning model usually stop reasoning: they loop, run out of tokens and miss easy problems.
| Size | AIME 2025+2026 | IFEval | MATH-500 | |
|---|---|---|---|---|
| KLX-IQ1_S | 6.19 GB | 53.8 | 50.5 | 87.8 |
| KLX-IQ2_XXS | 7.21 GB | 82.7 | 88.5 | 98.4 |
| KLX-IQ2_XS | 8.00 GB | 89.4 | 93.5 | 97.8 |
| KLX-IQ3_XXS | 10.93 GB | 94.2 | 94.3 | 99.0 |
| KLX-IQ4_XS | 14.25 GB | 95.2 | 94.3 | 98.8 |
| Full BF16 model | About 54 GB | 96.0 | 93.4 | 99.2 |
Scores in percent, from one harness on stock llama.cpp, in thinking mode with Qwen's sampler. AIME 2025+2026: 60 problems, 8 samples each. IFEval: 541 prompts, prompt-level strict. MATH-500: 500 problems. Up to 81,920 output tokens (32,768 for IFEval). The name gives each file's size class; each file mixes several standard tensor types. Every score and the setup
The same questions with the same settings, against each publisher's own file, unchanged.
| Against (Unsloth unless named) | AIME 2025+2026 | IFEval | MATH-500 | |
|---|---|---|---|---|
| 6.2 GB · IQ1_S | UD-IQ1_S, 6.19 GB | +52.5 ± 4.9 | +28.8 ± 2.6 | +55.2 ± 2.5 |
| 7.2 GB · IQ2_XXS | UD-IQ2_XXS, 7.27 GB | +8.75 ± 2.6 | +8.3 ± 1.9 | +1.8 ± 0.9 |
| 8 GB · IQ2_XS | ISTA GSQ-RCO IQ2_XS, 8.42 GB | +0.2 ± 1.5 | −1.1 ± 1.1 | +0.2 ± 0.8 |
| UD-IQ2_S, 8.37 GB | +1.0 ± 1.4 | +2.8 ± 1.3 | −0.8 ± 0.8 | |
| 10.9 GB · IQ3_XXS | UD-IQ3_XXS, 10.93 GB | +1.0 ± 1.2 | −0.4 ± 0.8 | +0.2 ± 0.4 |
| 14.3 GB · IQ4_XS | UD-IQ4_XS, 14.25 GB | −0.6 ± 0.9 | −1.1 ± 0.8 | +0.2 ± 0.5 |
KLX minus the other file, in points, ± one standard error. Highlighted: KLX ahead by more than two standard errors. Compared files: unsloth/Qwen3.8-27B-GGUF and ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF. At 10.93 and 14.25 GB both makers' files are close to full quality, and these benchmarks cannot separate them. Outside this table: PrismML's Bonsai 2 27B (7.21 GB), which runs on PrismML's own llama.cpp fork and is not a standard GGUF, scores 93.1 on AIME 2025+2026 against 82.7 for our 7.21 GB file.
Kimi K3.1 Pro has 2.8 trillion parameters, 104 billion active per token, and is 1.56 TB even as a lossless GGUF.
| Size | Mean KL, KLX | Mean KL, standard quant | KLX lower by | |
|---|---|---|---|---|
| KLX-IQ1_S | 592.7 GB | 0.106 | 0.107 | Level |
| KLX-IQ1_M | 647.9 GB | 0.084 | 0.089 | 6.2% |
| KLX-IQ2_XXS | 709.9 GB | 0.072 | 0.084 | 14.5% |
Mean KL divergence from the lossless GGUF on held-out text at 4,096 tokens of context; lower is closer to the full model. The standard quant is built with llama.cpp at the same size and tensor-type mix. Scores: one run of KLX-IQ1_S on stock llama.cpp, one sample per question, so about ± 7 points of sampling error on AIME's 30 problems; 91.0% on MATH-500 and 94.0% on IFEval, 100 questions each. The full model was not run on the same setup, so these are not a comparison. The two larger files keep part of the experts in system RAM on one node. Every number and the setup
DeepSeek V4.1 Flash is a mixture of experts with 763 billion parameters, 510 GB as released. KLX-IQ1_S is 232.6 GB: on four H100 80 GB GPUs each holds about 44 GiB, the rest of the file stays memory-mapped in system RAM, and eight conversations of 80K tokens run at once.
| Score, % | Tokens per answer | ||||
|---|---|---|---|---|---|
| KLX-IQ1_S | UD-IQ1_S | Difference | KLX-IQ1_S | UD-IQ1_S | |
| AIME 2025 | 100.0 | 96.7 | +3.3 ± 3.3 | 7,823 | 9,498 |
| AIME 2026 | 96.7 | 96.7 | 0 | 9,020 | 11,454 |
| MATH-500 | 91.0 | 89.6 | +1.4 ± 1.1 | 1,097 | 1,308 |
| IFEval | 95.5 | 95.7 | −0.2 ± 0.8 | 2,267 | 2,031 |
Both files side by side on one 8× H100 80 GB node with 1.9 TB of RAM, four GPUs each, on stock llama.cpp: the same questions and seeds, one sample per question, DeepSeek's sampling, thinking on, up to 81,920 output tokens (32,768 for IFEval). AIME 2025 and 2026: all 30 problems each. MATH-500: all 500. IFEval: 539 of 541 prompts, prompt-level strict; two requests failed on the server for both files. No score difference is statistically significant. Compared file: unsloth/DeepSeek-V4.1-Flash-1017-GGUF.
| Size | Mean KL | Same top token | Perplexity, relative | |
|---|---|---|---|---|
| KLX-IQ1_S | 232.62 GB | 0.0820 ± 0.0003 | 89.48% | ×1.044 |
| Unsloth UD-IQ1_S | 232.98 GB | 0.1075 ± 0.0004 | 87.84% | ×1.026 |
Each file against the lossless GGUF of the full model (FP8 weights and FP4 experts, as released) on 200 chunks of 4,096 tokens of held-out text. Lower KL is closer to the full model; perplexity is given as a multiple of the full model's, and it points the other way from KL. Highlighted: KLX ahead by more than two standard errors. Every number and the setup
The architecture and tokenizer are unchanged, and every file uses only standard llama.cpp tensor types. The Qwen3.8-27B files leave out the multi-token prediction head. All three models also read images: the vision encoder is in separate mmproj files in each repository, unquantized (F16 and BF16) and converted with upstream llama.cpp; add --mmproj to the command. For Kimi K3.1 Pro, image input needs a llama.cpp build with #31980 (merged 10 November 2026). The Qwen3.8-27B and Kimi K3.1 Pro files keep their model's chat template; the DeepSeek V4.1 Flash file carries Unsloth's llama.cpp port of DeepSeek's template (MIT), which renders the same prompts. KarLex AI is not affiliated with Qwen, Moonshot AI, DeepSeek, Unsloth, ISTA or PrismML.
Every KLX-GGUF file is free to download. If they save you memory, a like on the repository helps others find them.