Benchmark Detail View

Vending Machine System

MEDIUM Challenge100 models testedTop Score: 78.9

Success Rate

65.3%

Quality Score

Tests Passed

Models Tested

100

Vending Machine Benchmark - Individual Model Results

Showing 100 of 100 models

	Family
1 GPT 5.2 OpenAI • 12/2025 12/2025	OpenAI	12/2025	78.9	77.4%	92.0%
2 GPT 5.3 Codex OpenAI • 02/2026 02/2026	OpenAI	02/2026	78.7	77.4%	90.0%
3 GPT 5.4 OpenAI • 03/2026 03/2026	OpenAI	03/2026	78.7	77.4%	90.0%
4 Gemini 2.5 Pro Google • 08/2025 08/2025	Google	08/2025	78.5	77.4%	88.0%
5 GPT 5.3 Chat OpenAI • 03/2026 03/2026	OpenAI	03/2026	78.3	77.4%	86.0%
6 GPT 5 nano OpenAI • 08/2025 08/2025	OpenAI	08/2025	78.3	77.4%	86.0%
7 OSS 120B OpenAI • 08/2025 08/2025	OpenAI	08/2025	78.3	77.4%	86.0%
8 Claude Opus 4.7 Anthropic • 04/2026 04/2026	Anthropic	04/2026	78.1	77.4%	84.0%
9 Gemini 3.1 Pro Preview Google • 02/2026 02/2026	Google	02/2026	78.1	77.4%	84.0%
10 GPT 4.1 nano OpenAI • 08/2025 08/2025	OpenAI	08/2025	78.1	77.4%	84.0%
11 GPT 5.1 Codex OpenAI • 11/2025 11/2025	OpenAI	11/2025	78.1	77.4%	84.0%
12 GPT 5.2 OpenAI • 12/2025 12/2025	OpenAI	12/2025	78.1	77.4%	84.0%
13 Claude 3.5 Haiku Anthropic • 08/2025 08/2025	Anthropic	08/2025	77.9	77.4%	82.0%
14 DeepSeek V3.2 Exp DeepSeek • 12/2025 12/2025	DeepSeek	12/2025	77.9	77.4%	82.0%
15 GLM 4.6 Z.AI • 10/2025 10/2025	Z.AI	10/2025	77.9	77.4%	82.0%
16 GPT 4 OpenAI • 08/2025 08/2025	OpenAI	08/2025	77.9	77.4%	82.0%
17 GPT 5.1 OpenAI • 11/2025 11/2025	OpenAI	11/2025	77.9	77.4%	82.0%
18 DeepSeek R1 DeepSeek • 08/2025 08/2025	DeepSeek	08/2025	77.9	77.4%	82.0%
19 Claude 4.5 Opus Anthropic • 11/2025 11/2025	Anthropic	11/2025	77.7	77.4%	80.0%
20 Claude 4.6 Opus Anthropic • 02/2026 02/2026	Anthropic	02/2026	77.7	77.4%	80.0%
21 DeepSeek V3.2 Exp DeepSeek • 10/2025 10/2025	DeepSeek	10/2025	77.7	77.4%	80.0%
22 Gemini 2.5 Flash Google • 08/2025 08/2025	Google	08/2025	77.7	77.4%	80.0%
23 Gemini 3 Flash Preview Google • 12/2025 12/2025	Google	12/2025	77.7	77.4%	80.0%
24 Grok 4.1 Fast xAI • 02/2026 02/2026	xAI	02/2026	77.7	77.4%	80.0%
25 Kimi K2 Thinking Moonshot AI • 12/2025 12/2025	Moonshot AI	12/2025	77.7	77.4%	80.0%
26 Llama 4 Maverick Meta • 08/2025 08/2025	Meta	08/2025	77.7	77.4%	80.0%
27 Llama 4 Scout Meta • 08/2025 08/2025	Meta	08/2025	77.7	77.4%	80.0%
28 MiniMax M2.5 Minimax • 02/2026 02/2026	Minimax	02/2026	77.7	77.4%	80.0%
29 GPT 5.2 Codex OpenAI • 01/2026 01/2026	OpenAI	01/2026	77.7	77.4%	80.0%
30 GPT 5 Codex OpenAI • 10/2025 10/2025	OpenAI	10/2025	77.7	77.4%	80.0%
31 GPT 5 OpenAI • 09/2025 09/2025	OpenAI	09/2025	77.7	77.4%	80.0%
32 o3 mini (High) OpenAI • 08/2025 08/2025	OpenAI	08/2025	77.7	77.4%	80.0%
33 Qwen3 Coder Plus Qwen • 10/2025 10/2025	Qwen	10/2025	77.7	77.4%	80.0%
34 Qwen3 Max Qwen • 10/2025 10/2025	Qwen	10/2025	77.7	77.4%	80.0%
35 Claude 3.7 Sonnet Anthropic • 08/2025 08/2025	Anthropic	08/2025	77.5	77.4%	78.0%
36 Claude 4.5 Sonnet Anthropic • 10/2025 10/2025	Anthropic	10/2025	77.5	77.4%	78.0%
37 DeepSeek V3.2 Speciale DeepSeek • 02/2026 02/2026	DeepSeek	02/2026	77.5	77.4%	78.0%
38 Kimi K2.5 Moonshot AI • 02/2026 02/2026	Moonshot AI	02/2026	77.5	77.4%	78.0%
39 GPT 4 Turbo OpenAI • 08/2025 08/2025	OpenAI	08/2025	77.5	77.4%	78.0%
40 GPT 5.1 OpenAI • 11/2025 11/2025	OpenAI	11/2025	77.5	77.4%	78.0%
41 GPT 5 OpenAI • 08/2025 08/2025	OpenAI	08/2025	77.5	77.4%	78.0%
42 o3 mini OpenAI • 08/2025 08/2025	OpenAI	08/2025	77.5	77.4%	78.0%
43 o4 mini (High) OpenAI • 08/2025 08/2025	OpenAI	08/2025	77.5	77.4%	78.0%
44 Claude 3.7 Sonnet (Thinking) Anthropic • 08/2025 08/2025	Anthropic	08/2025	77.3	77.4%	76.0%
45 Grok 3 xAI • 08/2025 08/2025	xAI	08/2025	77.3	77.4%	76.0%
46 Grok 4 Fast xAI • 10/2025 10/2025	xAI	10/2025	77.3	77.4%	76.0%
47 Horizon Beta Other • 08/2025 08/2025	Other	08/2025	77.3	77.4%	76.0%
48 Kimi K2 (0905) Moonshot AI • 10/2025 10/2025	Moonshot AI	10/2025	77.3	77.4%	76.0%
49 GPT 5 OpenAI • 08/2025 08/2025	OpenAI	08/2025	77.3	77.4%	76.0%
50 o4 mini OpenAI • 08/2025 08/2025	OpenAI	08/2025	77.3	77.4%	76.0%
51 Step 3.5 Flash StepFun • 02/2026 02/2026	StepFun	02/2026	77.3	77.4%	76.0%
52 Claude 3.5 Sonnet Anthropic • 08/2025 08/2025	Anthropic	08/2025	77.1	77.4%	74.0%
53 GPT 5.1 Codex Mini OpenAI • 11/2025 11/2025	OpenAI	11/2025	77.1	77.4%	74.0%
54 GPT 5 nano OpenAI • 09/2025 09/2025	OpenAI	09/2025	77.1	77.4%	74.0%
55 GPT 4o OpenAI • 08/2025 08/2025	OpenAI	08/2025	77.1	77.4%	74.0%
56 Claude 4.6 Sonnet Anthropic • 02/2026 02/2026	Anthropic	02/2026	76.9	77.4%	72.0%
57 GPT 4.1 OpenAI • 08/2025 08/2025	OpenAI	08/2025	76.9	77.4%	72.0%
58 Nova 2 Lite V1 Amazon • 02/2026 02/2026	Amazon	02/2026	76.7	77.4%	70.0%
59 GPT 4.1 mini OpenAI • 08/2025 08/2025	OpenAI	08/2025	76.7	77.4%	70.0%
60 o1 mini OpenAI • 08/2025 08/2025	OpenAI	08/2025	76.7	77.4%	70.0%
61 GPT 5 mini OpenAI • 09/2025 09/2025	OpenAI	09/2025	76.5	77.4%	68.0%
62 GPT 5 mini OpenAI • 08/2025 08/2025	OpenAI	08/2025	76.3	77.4%	66.0%
63 GLM 5 Z.AI • 02/2026 02/2026	Z.AI	02/2026	75.6	74.2%	88.0%
64 Codestral 25.08 Mistral • 08/2025 08/2025	Mistral	08/2025	75.4	74.2%	86.0%
65 Sonoma Sky Alpha Other • 09/2025 09/2025	Other	09/2025	75.4	74.2%	86.0%
66 Claude 4.1 Opus Anthropic • 08/2025 08/2025	Anthropic	08/2025	75.0	74.2%	82.0%
67 Claude 4 Opus Anthropic • 08/2025 08/2025	Anthropic	08/2025	75.0	74.2%	82.0%
68 Claude 4 Sonnet Anthropic • 08/2025 08/2025	Anthropic	08/2025	75.0	74.2%	82.0%
69 Gemini 2.5 Flash Lite Google • 08/2025 08/2025	Google	08/2025	74.8	74.2%	80.0%
70 MiniMax M2.1 Minimax • 12/2025 12/2025	Minimax	12/2025	46.5	41.9%	88.0%
71 Trinity Large Preview Arcee AI • 02/2026 02/2026	Arcee AI	02/2026	46.5	41.9%	88.0%
72 Devstral 25.12 Mistral • 12/2025 12/2025	Mistral	12/2025	46.4	41.9%	86.0%
73 Mistral Large 25.12 Mistral • 12/2025 12/2025	Mistral	12/2025	46.4	41.9%	86.0%
74 Gemini 3 Pro Preview Google • 11/2025 11/2025	Google	11/2025	46.0	41.9%	82.0%
75 GLM 4.7 Z.AI • 12/2025 12/2025	Z.AI	12/2025	46.0	41.9%	82.0%
76 Mistral Medium 3 Mistral • 08/2025 08/2025	Mistral	08/2025	46.0	41.9%	82.0%
77 Nova Pro V1 Amazon • 08/2025 08/2025	Amazon	08/2025	46.0	41.9%	82.0%
78 OSS 20B OpenAI • 08/2025 08/2025	OpenAI	08/2025	46.0	41.9%	82.0%
79 Qwen3 Coder Qwen • 08/2025 08/2025	Qwen	08/2025	46.0	41.9%	82.0%
80 Coder Large Other • 08/2025 08/2025	Other	08/2025	45.8	41.9%	80.0%
81 Grok Code Fast 1 xAI • 09/2025 09/2025	xAI	09/2025	45.8	41.9%	80.0%
82 Grok 4 xAI • 08/2025 08/2025	xAI	08/2025	45.5	41.9%	78.0%
83 Kimi K2 Moonshot AI • 08/2025 08/2025	Moonshot AI	08/2025	45.5	41.9%	78.0%
84 MIMO V2 Flash Minimax • 12/2025 12/2025	Minimax	12/2025	45.5	41.9%	78.0%
85 GLM 4.5 Z.AI • 08/2025 08/2025	Z.AI	08/2025	45.4	41.9%	76.0%
86 GPT 4o OpenAI • 08/2025 08/2025	OpenAI	08/2025	45.4	41.9%	76.0%
87 Qwen3 Coder Next Qwen • 02/2026 02/2026	Qwen	02/2026	45.4	41.9%	76.0%
88 Gemini 2.0 Flash 001 Google • 08/2025 08/2025	Google	08/2025	45.0	41.9%	72.0%
89 Grok 3 Mini xAI • 08/2025 08/2025	xAI	08/2025	45.0	41.9%	72.0%
90 Claude 3 Haiku Anthropic • 08/2025 08/2025	Anthropic	08/2025	43.6	38.7%	88.0%
91 Gemma 3 4B IT Google • 08/2025 08/2025	Google	08/2025	43.6	38.7%	88.0%
92 Magnum V4 72B NousResearch • 08/2025 08/2025	NousResearch	08/2025	43.6	38.7%	88.0%
93 GPT 3.5 Turbo OpenAI • 08/2025 08/2025	OpenAI	08/2025	43.6	38.7%	88.0%
94 GPT 4o mini OpenAI • 08/2025 08/2025	OpenAI	08/2025	43.6	38.7%	88.0%
95 Qwen3 14B Qwen • 08/2025 08/2025	Qwen	08/2025	43.4	38.7%	86.0%
96 DeepSeek V3 DeepSeek • 08/2025 08/2025	DeepSeek	08/2025	43.2	38.7%	84.0%
97 Claude 4.5 Haiku Anthropic • 10/2025 10/2025	Anthropic	10/2025	43.0	38.7%	82.0%
98 Nova Micro V1 Amazon • 08/2025 08/2025	Amazon	08/2025	37.4	32.3%	84.0%
99 Nova Lite V1 Amazon • 08/2025 08/2025	Amazon	08/2025	32.2	25.8%	90.0%
100 Command A Cohere • 08/2025 08/2025	Cohere	08/2025	11.7	3.2%	88.0%

Top Performers

Vending Machine Champions

OpenAI

78.9

GPT 5.2

Success Rate

77.4%

Tests Passed

Quality

Issues

31 total tests

OpenAI

78.7

GPT 5.3 Codex

Success Rate

77.4%

Tests Passed

Quality

Issues

31 total tests

OpenAI

78.7

GPT 5.4

Success Rate

77.4%

Tests Passed

Quality

Issues

31 total tests

Explore More Benchmarks

See how models perform across different programming challenges and complexity levels.