CSAT-benchmark-smol
Small open-weight language models take the 2026 Korean college entrance exam, the 수능. Totals out of 450.
Scores
no thinkingout of 450
Total by GB
The line is the best total at each size or under.- Gemma 4
- Gemma 4, no thinking
- Ternary Bonsai 2
- Nanbeige4.2
- Qwen3.5
- Kanana-2
- LFM3
- Gemma 4 12B, 7.16 GB, 399.2
- Ternary Bonsai 2 27B, 8.6 GB, 385.2
- Ternary Bonsai 2 9B, 3.46 GB, 320.7
- Gemma 4 E4B, 6.14 GB, 316.8
- Nanbeige4.2 3B, 5.19 GB, 304.2
- LFM3 1.7B, 1.84 GB, 261.2
- Gemma 4 E2B, 4.34 GB, 259.3
- Qwen3.5 2B, 1.94 GB, 234.5
- Kanana-2 1.3B, 2.58 GB, 193
- LFM3-VL 2B, 1.59 GB, 177
- Gemma 4 12B, no thinking, 7.16 GB, 327.5
- Gemma 4 E4B, no thinking, 6.14 GB, 283.5
- Gemma 4 E2B, no thinking, 4.34 GB, 237.7
- 1 GB
- 2 GBGalaxy Watch8
- 4 GB
- 6 GBiPhone 15
- 8 GBiPhone 16 ProRTX 5060
- 12 GBGalaxy S26RTX 5070
- 16 GBRTX 5080
- 24 GBRTX 4090
- 32 GBRTX 5090
Size of the weights, GB. Under a size: devices with that much memory.
All scores
ModelGBKorean out of 100Math out of 100English out of 100History out of 50Inquiry out of 100Total
- Gemma 4 12B QAT 4-bitGB 7.16Korean 95.5Math 98.7English 97History 33Inquiry 75Total 399.2
- Gemma 4 12B QAT 4-bit, no thinkingGB 7.16Korean 79Math 76English 92History 35Inquiry 45.5Total 327.5
- Ternary Bonsai 2 27B GB 8.6Korean 86.5Math 98.7English 94History 27Inquiry 79Total 385.2
- Ternary Bonsai 2 9B GB 3.46Korean 65.5Math 90.7English 90History 20Inquiry 54.5Total 320.7
- Gemma 4 E4B QAT 4-bitGB 6.14Korean 57.5Math 81.3English 90History 38Inquiry 50Total 316.8
- Gemma 4 E4B QAT 4-bit, no thinkingGB 6.14Korean 58Math 68English 90History 25Inquiry 42.5Total 283.5
- Nanbeige4.2 3B FP8GB 5.19Korean 57Math 86.7English 93History 17Inquiry 50.5Total 304.2
- LFM3 1.7B 8-bitGB 1.84Korean 45Math 86.7English 76History 18Inquiry 35.5Total 261.2
- Gemma 4 E2B QAT 4-bitGB 4.34Korean 42Math 68.3English 90History 27Inquiry 32Total 259.3
- Gemma 4 E2B QAT 4-bit, no thinkingGB 4.34Korean 39Math 66.7English 77History 30Inquiry 25Total 237.7
- Qwen3.5 2B 4-bitGB 1.94Korean 47.5Math 59English 78History 18Inquiry 32Total 234.5
- Kanana-2 1.3B BF16GB 2.58Korean 28.5Math 47English 58History 30Inquiry 29.5Total 193
- LFM3-VL 2B 4-bitGB 1.59Korean 16.5Math 59English 75History 21Inquiry 5.5Total 177
Korean, Math, English and Inquiry are out of 100, History out of 50. GB is the size of the weights. Updated .
Why
The defences proposed against AI in the exam hall work on the network: jamming, Wi-Fi shut off, radio detectors. A model of this size needs no network. Each one here fits in the memory of a laptop, and the smallest fit in a phone.
How
Each model answers every question in its own request. The answers are graded and totalled out of 450 the way CSAT-benchmark does it.