Benchmark Detail View
Vending Machine System
MEDIUM Challenge99 models testedTop Score: 78.9
Success Rate
65.2%
Quality Score
80
Tests Passed
20
Models Tested
99
Vending Machine Benchmark - Individual Model Results
Showing 99 of 99 models
1 GPT 5.2 OpenAI • 12/2025 12/2025 | 78.9 | 77.4% |
2 GPT 5.3 Codex OpenAI • 02/2026 02/2026 | 78.7 | 77.4% |
3 GPT 5.4 OpenAI • 03/2026 03/2026 | 78.7 | 77.4% |
4 Gemini 2.5 Pro Google • 08/2025 08/2025 | 78.5 | 77.4% |
5 GPT 5.3 Chat OpenAI • 03/2026 03/2026 | 78.3 | 77.4% |
6 GPT 5 nano OpenAI • 08/2025 08/2025 | 78.3 | 77.4% |
7 OSS 120B OpenAI • 08/2025 08/2025 | 78.3 | 77.4% |
8 Gemini 3.1 Pro Preview Google • 02/2026 02/2026 | 78.1 | 77.4% |
9 GPT 4.1 nano OpenAI • 08/2025 08/2025 | 78.1 | 77.4% |
10 GPT 5.1 Codex OpenAI • 11/2025 11/2025 | 78.1 | 77.4% |
11 GPT 5.2 OpenAI • 12/2025 12/2025 | 78.1 | 77.4% |
12 Claude 3.5 Haiku Anthropic • 08/2025 08/2025 | 77.9 | 77.4% |
13 DeepSeek V3.2 Exp DeepSeek • 12/2025 12/2025 | 77.9 | 77.4% |
14 GLM 4.6 Z.AI • 10/2025 10/2025 | 77.9 | 77.4% |
15 GPT 4 OpenAI • 08/2025 08/2025 | 77.9 | 77.4% |
16 GPT 5.1 OpenAI • 11/2025 11/2025 | 77.9 | 77.4% |
17 DeepSeek R1 DeepSeek • 08/2025 08/2025 | 77.9 | 77.4% |
18 Claude 4.5 Opus Anthropic • 11/2025 11/2025 | 77.7 | 77.4% |
19 Claude 4.6 Opus Anthropic • 02/2026 02/2026 | 77.7 | 77.4% |
20 DeepSeek V3.2 Exp DeepSeek • 10/2025 10/2025 | 77.7 | 77.4% |
21 Gemini 2.5 Flash Google • 08/2025 08/2025 | 77.7 | 77.4% |
22 Gemini 3 Flash Preview Google • 12/2025 12/2025 | 77.7 | 77.4% |
23 Grok 4.1 Fast xAI • 02/2026 02/2026 | 77.7 | 77.4% |
24 Kimi K2 Thinking Moonshot AI • 12/2025 12/2025 | 77.7 | 77.4% |
25 Llama 4 Maverick Meta • 08/2025 08/2025 | 77.7 | 77.4% |
26 Llama 4 Scout Meta • 08/2025 08/2025 | 77.7 | 77.4% |
27 MiniMax M2.5 Minimax • 02/2026 02/2026 | 77.7 | 77.4% |
28 GPT 5.2 Codex OpenAI • 01/2026 01/2026 | 77.7 | 77.4% |
29 GPT 5 Codex OpenAI • 10/2025 10/2025 | 77.7 | 77.4% |
30 GPT 5 OpenAI • 09/2025 09/2025 | 77.7 | 77.4% |
31 o3 mini (High) OpenAI • 08/2025 08/2025 | 77.7 | 77.4% |
32 Qwen3 Coder Plus Qwen • 10/2025 10/2025 | 77.7 | 77.4% |
33 Qwen3 Max Qwen • 10/2025 10/2025 | 77.7 | 77.4% |
34 Claude 3.7 Sonnet Anthropic • 08/2025 08/2025 | 77.5 | 77.4% |
35 Claude 4.5 Sonnet Anthropic • 10/2025 10/2025 | 77.5 | 77.4% |
36 DeepSeek V3.2 Speciale DeepSeek • 02/2026 02/2026 | 77.5 | 77.4% |
37 Kimi K2.5 Moonshot AI • 02/2026 02/2026 | 77.5 | 77.4% |
38 GPT 4 Turbo OpenAI • 08/2025 08/2025 | 77.5 | 77.4% |
39 GPT 5.1 OpenAI • 11/2025 11/2025 | 77.5 | 77.4% |
40 GPT 5 OpenAI • 08/2025 08/2025 | 77.5 | 77.4% |
41 o3 mini OpenAI • 08/2025 08/2025 | 77.5 | 77.4% |
42 o4 mini (High) OpenAI • 08/2025 08/2025 | 77.5 | 77.4% |
43 Claude 3.7 Sonnet (Thinking) Anthropic • 08/2025 08/2025 | 77.3 | 77.4% |
44 Grok 3 xAI • 08/2025 08/2025 | 77.3 | 77.4% |
45 Grok 4 Fast xAI • 10/2025 10/2025 | 77.3 | 77.4% |
46 Horizon Beta Other • 08/2025 08/2025 | 77.3 | 77.4% |
47 Kimi K2 (0905) Moonshot AI • 10/2025 10/2025 | 77.3 | 77.4% |
48 GPT 5 OpenAI • 08/2025 08/2025 | 77.3 | 77.4% |
49 o4 mini OpenAI • 08/2025 08/2025 | 77.3 | 77.4% |
50 Step 3.5 Flash StepFun • 02/2026 02/2026 | 77.3 | 77.4% |
51 Claude 3.5 Sonnet Anthropic • 08/2025 08/2025 | 77.1 | 77.4% |
52 GPT 5.1 Codex Mini OpenAI • 11/2025 11/2025 | 77.1 | 77.4% |
53 GPT 5 nano OpenAI • 09/2025 09/2025 | 77.1 | 77.4% |
54 GPT 4o OpenAI • 08/2025 08/2025 | 77.1 | 77.4% |
55 Claude 4.6 Sonnet Anthropic • 02/2026 02/2026 | 76.9 | 77.4% |
56 GPT 4.1 OpenAI • 08/2025 08/2025 | 76.9 | 77.4% |
57 Nova 2 Lite V1 Amazon • 02/2026 02/2026 | 76.7 | 77.4% |
58 GPT 4.1 mini OpenAI • 08/2025 08/2025 | 76.7 | 77.4% |
59 o1 mini OpenAI • 08/2025 08/2025 | 76.7 | 77.4% |
60 GPT 5 mini OpenAI • 09/2025 09/2025 | 76.5 | 77.4% |
61 GPT 5 mini OpenAI • 08/2025 08/2025 | 76.3 | 77.4% |
62 GLM 5 Z.AI • 02/2026 02/2026 | 75.6 | 74.2% |
63 Codestral 25.08 Mistral • 08/2025 08/2025 | 75.4 | 74.2% |
64 Sonoma Sky Alpha Other • 09/2025 09/2025 | 75.4 | 74.2% |
65 Claude 4.1 Opus Anthropic • 08/2025 08/2025 | 75.0 | 74.2% |
66 Claude 4 Opus Anthropic • 08/2025 08/2025 | 75.0 | 74.2% |
67 Claude 4 Sonnet Anthropic • 08/2025 08/2025 | 75.0 | 74.2% |
68 Gemini 2.5 Flash Lite Google • 08/2025 08/2025 | 74.8 | 74.2% |
69 MiniMax M2.1 Minimax • 12/2025 12/2025 | 46.5 | 41.9% |
70 Trinity Large Preview Arcee AI • 02/2026 02/2026 | 46.5 | 41.9% |
71 Devstral 25.12 Mistral • 12/2025 12/2025 | 46.4 | 41.9% |
72 Mistral Large 25.12 Mistral • 12/2025 12/2025 | 46.4 | 41.9% |
73 Gemini 3 Pro Preview Google • 11/2025 11/2025 | 46.0 | 41.9% |
74 GLM 4.7 Z.AI • 12/2025 12/2025 | 46.0 | 41.9% |
75 Mistral Medium 3 Mistral • 08/2025 08/2025 | 46.0 | 41.9% |
76 Nova Pro V1 Amazon • 08/2025 08/2025 | 46.0 | 41.9% |
77 OSS 20B OpenAI • 08/2025 08/2025 | 46.0 | 41.9% |
78 Qwen3 Coder Qwen • 08/2025 08/2025 | 46.0 | 41.9% |
79 Coder Large Other • 08/2025 08/2025 | 45.8 | 41.9% |
80 Grok Code Fast 1 xAI • 09/2025 09/2025 | 45.8 | 41.9% |
81 Grok 4 xAI • 08/2025 08/2025 | 45.5 | 41.9% |
82 Kimi K2 Moonshot AI • 08/2025 08/2025 | 45.5 | 41.9% |
83 MIMO V2 Flash Minimax • 12/2025 12/2025 | 45.5 | 41.9% |
84 GLM 4.5 Z.AI • 08/2025 08/2025 | 45.4 | 41.9% |
85 GPT 4o OpenAI • 08/2025 08/2025 | 45.4 | 41.9% |
86 Qwen3 Coder Next Qwen • 02/2026 02/2026 | 45.4 | 41.9% |
87 Gemini 2.0 Flash 001 Google • 08/2025 08/2025 | 45.0 | 41.9% |
88 Grok 3 Mini xAI • 08/2025 08/2025 | 45.0 | 41.9% |
89 Claude 3 Haiku Anthropic • 08/2025 08/2025 | 43.6 | 38.7% |
90 Gemma 3 4B IT Google • 08/2025 08/2025 | 43.6 | 38.7% |
91 Magnum V4 72B NousResearch • 08/2025 08/2025 | 43.6 | 38.7% |
92 GPT 3.5 Turbo OpenAI • 08/2025 08/2025 | 43.6 | 38.7% |
93 GPT 4o mini OpenAI • 08/2025 08/2025 | 43.6 | 38.7% |
94 Qwen3 14B Qwen • 08/2025 08/2025 | 43.4 | 38.7% |
95 DeepSeek V3 DeepSeek • 08/2025 08/2025 | 43.2 | 38.7% |
96 Claude 4.5 Haiku Anthropic • 10/2025 10/2025 | 43.0 | 38.7% |
97 Nova Micro V1 Amazon • 08/2025 08/2025 | 37.4 | 32.3% |
98 Nova Lite V1 Amazon • 08/2025 08/2025 | 32.2 | 25.8% |
99 Command A Cohere • 08/2025 08/2025 | 11.7 | 3.2% |
Top Performers
#1
OpenAI78.9
GPT 5.2
Success Rate
77.4%24
Tests Passed
Q
92
Quality
4
Issues
31 total tests
#2
OpenAI78.7
GPT 5.3 Codex
Success Rate
77.4%24
Tests Passed
Q
90
Quality
5
Issues
31 total tests
#3
OpenAI78.7
GPT 5.4
Success Rate
77.4%24
Tests Passed
Q
90
Quality
5
Issues
31 total tests
Explore More Benchmarks
See how models perform across different programming challenges and complexity levels.