CSAT-benchmark-smol

Small open-weight language models take the 2026 Korean college entrance exam, the 수능. Totals out of 450.

Scores

no thinkingout of 450

  1. Gemma 4 12B399.2no thinking 327.5
  2. Ternary Bonsai 2 27B385.2
  3. Ternary Bonsai 2 9B320.7
  4. Gemma 4 E4B316.8no thinking 283.5
  5. Nanbeige4.2 3B304.2
  6. LFM3 1.7B261.2
  7. Gemma 4 E2B259.3no thinking 237.7
  8. Qwen3.5 2B234.5
  9. Kanana-2 1.3B193
  10. LFM3-VL 2B177

Total by GB

The line is the best total at each size or under.
  • Gemma 4
  • Gemma 4, no thinking
  • Ternary Bonsai 2
  • Nanbeige4.2
  • Qwen3.5
  • Kanana-2
  • LFM3
  1. Gemma 4 12B, 7.16 GB, 399.2
  2. Ternary Bonsai 2 27B, 8.6 GB, 385.2
  3. Ternary Bonsai 2 9B, 3.46 GB, 320.7
  4. Gemma 4 E4B, 6.14 GB, 316.8
  5. Nanbeige4.2 3B, 5.19 GB, 304.2
  6. LFM3 1.7B, 1.84 GB, 261.2
  7. Gemma 4 E2B, 4.34 GB, 259.3
  8. Qwen3.5 2B, 1.94 GB, 234.5
  9. Kanana-2 1.3B, 2.58 GB, 193
  10. LFM3-VL 2B, 1.59 GB, 177
  11. Gemma 4 12B, no thinking, 7.16 GB, 327.5
  12. Gemma 4 E4B, no thinking, 6.14 GB, 283.5
  13. Gemma 4 E2B, no thinking, 4.34 GB, 237.7
  1. 1 GB
  2. 2 GBGalaxy Watch8
  3. 4 GB
  4. 6 GBiPhone 15
  5. 8 GBiPhone 16 ProRTX 5060
  6. 12 GBGalaxy S26RTX 5070
  7. 16 GBRTX 5080
  8. 24 GBRTX 4090
  9. 32 GBRTX 5090

Size of the weights, GB. Under a size: devices with that much memory.

All scores

ModelGBKorean out of 100Math out of 100English out of 100History out of 50Inquiry out of 100Total
  1. Gemma 4 12B QAT 4-bit7.1695.598.7973375Total 399.2
  2. Gemma 4 12B QAT 4-bit, no thinking7.167976923545.5Total 327.5
  3. Ternary Bonsai 2 27B 8.686.598.7942779Total 385.2
  4. Ternary Bonsai 2 9B 3.4665.590.7902054.5Total 320.7
  5. Gemma 4 E4B QAT 4-bit6.1457.581.3903850Total 316.8
  6. Gemma 4 E4B QAT 4-bit, no thinking6.145868902542.5Total 283.5
  7. Nanbeige4.2 3B FP85.195786.7931750.5Total 304.2
  8. LFM3 1.7B 8-bit1.844586.7761835.5Total 261.2
  9. Gemma 4 E2B QAT 4-bit4.344268.3902732Total 259.3
  10. Gemma 4 E2B QAT 4-bit, no thinking4.343966.7773025Total 237.7
  11. Qwen3.5 2B 4-bit1.9447.559781832Total 234.5
  12. Kanana-2 1.3B BF162.5828.547583029.5Total 193
  13. LFM3-VL 2B 4-bit1.5916.55975215.5Total 177

Korean, Math, English and Inquiry are out of 100, History out of 50. GB is the size of the weights. Updated .

Why

The defences proposed against AI in the exam hall work on the network: jamming, Wi-Fi shut off, radio detectors. A model of this size needs no network. Each one here fits in the memory of a laptop, and the smallest fit in a phone.

How

Each model answers every question in its own request. The answers are graded and totalled out of 450 the way CSAT-benchmark does it.