Skip to content
Releases

Three models, every size in one place.

7 Nov 2026 · Apache-2.0 · 2,634 downloads

Qwen3.8-27B

Five files, 6.19 to 14.25 GB, and a vision projector for images. How they score

Get the files
8 Nov 2026 · Kimi K3.1-Pro License · 10,300 downloads

Kimi K3.1 Pro

Three files, 593 to 710 GB, and a vision projector for images. How they score

Get the files
10 Nov 2026 · MIT · 386 downloads

DeepSeek V4.1 Flash

One 232.6 GB file of a model that is 510 GB as released, and a vision projector for images. How they score

Get the files
Qwen3.8-27B

It still thinks at 6.2 GB.

The smallest GGUFs of a reasoning model usually stop reasoning: they loop, run out of tokens and miss easy problems.

  • 8% vs 95%of AIME answers that looped at 6.2 GB: the KLX file, then Unsloth's
  • 53.8%on AIME 2025+2026 at 6.19 GB, where Unsloth's file of the same size scores 1.3%KLX ahead
Scores in percent of every Qwen3.8-27B KLX-GGUF file and of the full BF16 model
SizeAIME 2025+2026IFEvalMATH-500
KLX-IQ1_S6.19 GB53.850.587.8
KLX-IQ2_XXS7.21 GB82.788.598.4
KLX-IQ2_XS8.00 GB89.493.597.8
KLX-IQ3_XXS10.93 GB94.294.399.0
KLX-IQ4_XS14.25 GB95.294.398.8
Full BF16 modelAbout 54 GB96.093.499.2

Scores in percent, from one harness on stock llama.cpp, in thinking mode with Qwen's sampler. AIME 2025+2026: 60 problems, 8 samples each. IFEval: 541 prompts, prompt-level strict. MATH-500: 500 problems. Up to 81,920 output tokens (32,768 for IFEval). The name gives each file's size class; each file mixes several standard tensor types. Every score and the setup

Head to head

Against GGUFs of the same size.

The same questions with the same settings, against each publisher's own file, unchanged.

Paired differences in points, KLX minus the other file, with one standard error
Against (Unsloth unless named)AIME 2025+2026IFEvalMATH-500
6.2 GB · IQ1_SUD-IQ1_S, 6.19 GB+52.5 ± 4.9+28.8 ± 2.6+55.2 ± 2.5
7.2 GB · IQ2_XXSUD-IQ2_XXS, 7.27 GB+8.75 ± 2.6+8.3 ± 1.9+1.8 ± 0.9
8 GB · IQ2_XSISTA GSQ-RCO IQ2_XS, 8.42 GB+0.2 ± 1.5−1.1 ± 1.1+0.2 ± 0.8
UD-IQ2_S, 8.37 GB+1.0 ± 1.4+2.8 ± 1.3−0.8 ± 0.8
10.9 GB · IQ3_XXSUD-IQ3_XXS, 10.93 GB+1.0 ± 1.2−0.4 ± 0.8+0.2 ± 0.4
14.3 GB · IQ4_XSUD-IQ4_XS, 14.25 GB−0.6 ± 0.9−1.1 ± 0.8+0.2 ± 0.5

KLX minus the other file, in points, ± one standard error. Highlighted: KLX ahead by more than two standard errors. Compared files: unsloth/Qwen3.8-27B-GGUF and ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF. At 10.93 and 14.25 GB both makers' files are close to full quality, and these benchmarks cannot separate them. Outside this table: PrismML's Bonsai 2 27B (7.21 GB), which runs on PrismML's own llama.cpp fork and is not a standard GGUF, scores 93.1 on AIME 2025+2026 against 82.7 for our 7.21 GB file.

Kimi K3.1 Pro

The whole model on one node.

Kimi K3.1 Pro has 2.8 trillion parameters, 104 billion active per token, and is 1.56 TB even as a lossless GGUF.

  • 593 GBKLX-IQ1_S, about 1.7 bits per weight: every layer and expert on the GPUs of one 8×H100 node, with 131K tokens of context
  • 80.0%on AIME 2025 for KLX-IQ1_S, all 30 problems
  • 19 tokens/sgenerating for one request, all on GPU, with stock llama.cpp
Mean KL divergence from the full model of each Kimi K3.1 Pro KLX-GGUF file and of a standard quant of the same size
SizeMean KL, KLXMean KL, standard quantKLX lower by
KLX-IQ1_S592.7 GB0.1060.107Level
KLX-IQ1_M647.9 GB0.0840.0896.2%
KLX-IQ2_XXS709.9 GB0.0720.08414.5%

Mean KL divergence from the lossless GGUF on held-out text at 4,096 tokens of context; lower is closer to the full model. The standard quant is built with llama.cpp at the same size and tensor-type mix. Scores: one run of KLX-IQ1_S on stock llama.cpp, one sample per question, so about ± 7 points of sampling error on AIME's 30 problems; 91.0% on MATH-500 and 94.0% on IFEval, 100 questions each. The full model was not run on the same setup, so these are not a comparison. The two larger files keep part of the experts in system RAM on one node. Every number and the setup

DeepSeek V4.1 Flash

763 billion parameters on four GPUs.

DeepSeek V4.1 Flash is a mixture of experts with 763 billion parameters, 510 GB as released. KLX-IQ1_S is 232.6 GB: on four H100 80 GB GPUs each holds about 44 GiB, the rest of the file stays memory-mapped in system RAM, and eight conversations of 80K tokens run at once.

  • Levelwith Unsloth's UD-IQ1_S on AIME, MATH-500 and IFEval, on the same 1,099 questions and seedsTie
  • 24% lowerKL divergence from the full model than UD-IQ1_S, from a file 358 MB smallerKLX ahead
Scores in percent of DeepSeek V4.1 Flash KLX-IQ1_S and Unsloth UD-IQ1_S on the same questions, the paired difference, and the mean tokens per answer
Score, %Tokens per answer
KLX-IQ1_SUD-IQ1_SDifferenceKLX-IQ1_SUD-IQ1_S
AIME 2025100.096.7+3.3 ± 3.37,8239,498
AIME 202696.796.709,02011,454
MATH-50091.089.6+1.4 ± 1.11,0971,308
IFEval95.595.7−0.2 ± 0.82,2672,031

Both files side by side on one 8× H100 80 GB node with 1.9 TB of RAM, four GPUs each, on stock llama.cpp: the same questions and seeds, one sample per question, DeepSeek's sampling, thinking on, up to 81,920 output tokens (32,768 for IFEval). AIME 2025 and 2026: all 30 problems each. MATH-500: all 500. IFEval: 539 of 541 prompts, prompt-level strict; two requests failed on the server for both files. No score difference is statistically significant. Compared file: unsloth/DeepSeek-V4.1-Flash-1017-GGUF.

Closeness of each DeepSeek V4.1 Flash file to the full model: mean KL divergence, how often the top token matches, and perplexity relative to the full model
SizeMean KLSame top tokenPerplexity, relative
KLX-IQ1_S232.62 GB0.0820 ± 0.000389.48%×1.044
Unsloth UD-IQ1_S232.98 GB0.1075 ± 0.000487.84%×1.026

Each file against the lossless GGUF of the full model (FP8 weights and FP4 experts, as released) on 200 chunks of 4,096 tokens of held-out text. Lower KL is closer to the full model; perplexity is given as a multiple of the full model's, and it points the other way from KL. Highlighted: KLX ahead by more than two standard errors. Every number and the setup

The architecture and tokenizer are unchanged, and every file uses only standard llama.cpp tensor types. The Qwen3.8-27B files leave out the multi-token prediction head. All three models also read images: the vision encoder is in separate mmproj files in each repository, unquantized (F16 and BF16) and converted with upstream llama.cpp; add --mmproj to the command. For Kimi K3.1 Pro, image input needs a llama.cpp build with #31980 (merged 10 November 2026). The Qwen3.8-27B and Kimi K3.1 Pro files keep their model's chat template; the DeepSeek V4.1 Flash file carries Unsloth's llama.cpp port of DeepSeek's template (MIT), which renders the same prompts. KarLex AI is not affiliated with Qwen, Moonshot AI, DeepSeek, Unsloth, ISTA or PrismML.

Open weights

Free on
Hugging Face.

Every KLX-GGUF file is free to download. If they save you memory, a like on the repository helps others find them.