Tested prompt · Math
Pens at 3 for $4: every AI model's reply, tested
We sent this everyday math prompt to all 20 models in llmwise, the same way the app sends a message, and checked every reply the same way. Here's each one as it came, with whether it passed, what it cost and how long it took.
Based on 20 of our test runs on , through OpenRouter with the app's own prompt and settings. Updated .
Short answer
19 of 20 models passed this math prompt's check (final answer); GPT-6 Luna didn't. The cheapest reply that passed was Claude Haiku 5.5's, at $0.00016; the fastest, DeepSeek V4.1 Flash's in 0.9 s. The dearest reply, Claude Fable 5.1's, cost 99 times as much ($0.0159).
The prompt, as sent, and its check
Checked by final answer, the same way for every model.
Pens at 3 for $4 (everyday)
A shop sells pens at 3 for $4, or $1.50 for a single pen. What's the cheapest way to pay for exactly 10 pens, and what does it cost? Show your working briefly, then end with a line that says "Final answer: " followed by the answer alone.
The final answer must be 13.5.
Exactly what this prompt's replies are checked against, with every other prompt of our test runs.
Every model's result
All 20 models on this prompt, in catalog order.
| Model | Result | Cost | Time | Reply |
|---|---|---|---|---|
| Claude Fable 5.1Anthropic | Passed: Final answer Buy three 3-for- 4 packs and one single pen, total 13.50: right. | $0.0159 | 5.4 s | 208 tokens |
| Claude Opus 5.5Anthropic | Passed: Final answer 3 packs of 3 plus 1 single pen, costing 13.50: right. | $0.0084 | 5.3 s | 189 tokens |
| Claude Sonnet 5.5Anthropic | Passed: Final answer Buy 3 packs of 3 and 1 single pen, for 13.50: right. | $0.0036 | 2.2 s | 250 tokens |
| Claude Sonnet 5Anthropic | Passed: Final answer Buy 3 packs of 3 pens ( 4 each) plus 1 single pen ( 1.50), total 13.50.: right. | $0.0061 | 6.3 s | 255 tokens |
| Claude Haiku 5.5Anthropic | Passed: Final answer Buy 3 packs of 3 and 1 single pen, costing 13.50: right. | $0.00016 | 1.7 s | 212 tokens |
| Claude Haiku 4.5Anthropic | Passed: Final answer Buy 3 packs of 3 pens and 1 single pen for 13.50: right. | $0.0014 | 2.2 s | 199 tokens |
| GPT-6 AstraOpenAI | Passed: Final answer Buy three groups of 3 and one single pen for 13.50.: right. | $0.0082 | 2.4 s | 83 tokens |
| GPT-6.1 SolOpenAI | Passed: Final answer Buy three 3-pen bundles and one single pen for 13.50.: right. | $0.0010 | 2.4 s | 89 tokens |
| GPT-6 SolOpenAI | Passed: Final answer 13.50: right. | $0.0024 | 3.3 s | 56 tokens |
| GPT-6 LunaOpenAI | Failed: Final answer 2 bundles and 4 singles for 12; expected 13.5. | $0.00014 | 2.9 s | 87 tokens |
| Gemini 4 ArgonGoogle | Passed: Final answer Buy three 3-packs and one single pen for 13.50: right. | $0.0015 | 4.4 s | 171 tokens |
| Gemini 3.1 Pro (preview)Google | Passed: Final answer 3 sets of 3 and 1 single pen for 13.50: right. | $0.0121 | 9.6 s | 160 tokens |
| Gemini 3.8 FlashGoogle | Passed: Final answer 3 sets of 3 pens and 1 single pen for 13.50: right. | $0.0017 | 4.7 s | 114 tokens |
| DeepSeek V4.1 FlashDeepSeek | Passed: Final answer Buy 3 packs of 3 (9 pens) plus 1 single pen, for 13.50.: right. | $0.00041 | 0.9 s | 146 tokens |
| DeepSeek V4 ProDeepSeek | Passed: Final answer 3 packs of 3 pens plus 1 single pen, costing 13.50: right. | $0.0010 | 2.7 s | 85 tokens |
| Grok 4.7xAI | Passed: Final answer 3 packs + 1 single ( 13.50): right. | $0.0111 | 18.5 s | 211 tokens |
| Kimi K3Moonshot | Passed: Final answer Buy three 3-for- 4 packs and one single pen; 13.50: right. | $0.0046 | 5.4 s | 75 tokens |
| GLM 5.3Z.ai | Passed: Final answer 13.50 (three 3-for- 4 packs plus one single pen): right. | $0.00018 | 1.5 s | 83 tokens |
| GLM 5.3 FlashZ.ai | Passed: Final answer Buy three packs of 3 plus one single pen, costing 13.50: right. | $0.00027 | 14.6 s | 161 tokens |
| Mistral Large 4Mistral | Passed: Final answer Buy three 3-packs plus one single pen, for 13.50 total.: right. | $0.0022 | 22.0 s | 301 tokens |
Cost: what OpenRouter charged us for the reply. Time: from sending to the whole reply. Reply: its length in tokens, thinking not counted. In llmwise you pay per message, not per token: each of these replies counted as one message on Pro.
Every reply
The reply that failed first, then the ones that passed, from the cheapest up.
GPT-6 Luna
Failed: Final answer 2 bundles and 4 singles for 12; expected 13.5.
For 10 pens, compare buying 0–3 bundles of 3: - 0 bundles: 10 singles = $15 - 1 bundle: 7 singles = $14.50 - 2 bundles: 4 singles = $12 - 3 bundles: 1 single = $13.50 Final answer: 2 bundles and 4 singles for $12
402 tokens in, 198 out (111 of them reasoning) · 2.9 s · $0.0001 · 1 message on Pro · answered by openai/gpt-6-luna via OpenAI ·
Claude Haiku 5.5
Passed: Final answer Buy 3 packs of 3 and 1 single pen, costing 13.50: right.
Options per pen: - Bundle of 3 costs $4 (about $1.33 per pen). - Single pen costs $1.50. For 10 pens, use as many bundles as possible: - 3 bundles = 9 pens = $12.00 - 1 single pen = $1.50 - Total = $13.50 Check alternatives: - 2 bundles + 4 singles = $8 + $6 = $14.00 - 0 bundles + 10 singles = $15.00 - 3 bundles + 1 single is the cheapest. Final answer: Buy 3 packs of 3 and 1 single pen, costing $13.50
548 tokens in, 212 out · 1.7 s · $0.0002 · 1 message on Pro · answered by anthropic/claude-haiku-5.5 via Anthropic ·
GLM 5.3
Passed: Final answer 13.50 (three 3-for- 4 packs plus one single pen): right.
Three 3-packs = 9 pens for $12, plus 1 single pen at $1.50 → $13.50. Alternative: two 3-packs (6 pens, $8) + 4 singles ($6) = $14. So the first option is cheaper. Final answer: $13.50 (three 3-for-$4 packs plus one single pen)
417 tokens in, 83 out · 1.5 s · $0.0002 · 1 message on Pro · answered by z-ai/glm-5.3 via Baidu ·
GLM 5.3 Flash
Passed: Final answer Buy three packs of 3 plus one single pen, costing 13.50: right.
**Working:** - 10 = 3a + b, where a = packs of 3 ($4 each), b = singles ($1.50 each) - a=3, b=1: 3×$4 + $1.50 = **$13.50** - a=2, b=4: $8 + $6.00 = $14.00 - a=1, b=7: $4 + $10.50 = $14.50 - a=0, b=10: $15.00 Buying 12 pens (4 packs) would cost $16 but exceeds 10, so it doesn't qualify. Final answer: Buy three packs of 3 plus one single pen, costing $13.50
418 tokens in, 393 out (232 of them reasoning) · 14.6 s · $0.0003 · 1 message on Pro · answered by z-ai/glm-5.3-flash via Wafer ·
DeepSeek V4.1 Flash
Passed: Final answer Buy 3 packs of 3 (9 pens) plus 1 single pen, for 13.50.: right.
3-pack rate: $4/3 ≈ $1.33 per pen, cheaper than $1.50 singles. Options: - 3 packs (9 pens) + 1 single = $12 + $1.50 = $13.50 - 2 packs (6) + 4 singles = $8 + $6 = $14.00 - 1 pack + 7 singles = $4 + $10.50 = $14.50 - 10 singles = $15.00 Best is 3 packs plus 1 single pen. Final answer: Buy 3 packs of 3 (9 pens) plus 1 single pen, for $13.50.
442 tokens in, 291 out (145 of them reasoning) · 0.9 s · $0.0004 · 1 message on Pro · answered by deepseek/deepseek-v4.1-flash via Together ·
GPT-6.1 Sol
Passed: Final answer Buy three 3-pen bundles and one single pen for 13.50.: right.
Three singles cost $4.50, so each 3-for-$4 bundle saves $0.50. Use as many bundles as possible: 3 bundles give 9 pens, then buy 1 single. Cost: \(3 \times \$4 + \$1.50 = \$13.50\). Final answer: Buy three 3-pen bundles and one single pen for $13.50.
404 tokens in, 127 out (38 of them reasoning) · 2.4 s · $0.0010 · 1 message on Pro · answered by openai/gpt-6.1-sol via OpenAI ·
DeepSeek V4 Pro
Passed: Final answer 3 packs of 3 pens plus 1 single pen, costing 13.50: right.
Buy packs of 3 where possible. - 3 packs = 9 pens: 3 × $4 = $12 - 1 single pen: $1.50 - Total: $12 + $1.50 = $13.50 Other combinations cost more, so this is cheapest. Final answer: 3 packs of 3 pens plus 1 single pen, costing $13.50
414 tokens in, 232 out (147 of them reasoning) · 2.7 s · $0.0010 · 1 message on Pro · answered by deepseek/deepseek-v4-pro-0813 via Wafer ·
Claude Haiku 4.5
Passed: Final answer Buy 3 packs of 3 pens and 1 single pen for 13.50: right.
# Pen Purchase Problem **Option 1: Buy 3-packs only** - 10 pens ÷ 3 = 3 packs with 1 pen left over - 3 packs × $4 = $12 - 1 single pen × $1.50 = $1.50 - Total: $13.50 **Option 2: Buy all single pens** - 10 pens × $1.50 = $15.00 **Option 3: Buy 3 packs + adjust** - Same as Option 1: $13.50 The cheapest way is to buy three 3-packs ($12) and one single pen ($1.50). Final answer: Buy 3 packs of 3 pens and 1 single pen for $13.50
434 tokens in, 199 out · 2.2 s · $0.0014 · 1 message on Pro · answered by anthropic/claude-haiku-4.5 via Amazon Bedrock ·
Gemini 4 Argon
Passed: Final answer Buy three 3-packs and one single pen for 13.50: right.
To get 10 pens as cheaply as possible, maximize the 3-pack deal ($1.33 per pen) and buy the remainder as singles ($1.50 per pen): - 3 packs of 3 pens (9 pens) = 3 × $4.00 = $12.00 - 1 single pen = $1.50 - Total for 10 pens = $12.00 + $1.50 = $13.50 (Buying 4 packs of 3 would give 12 pens for $16.00, which is more expensive and not *exactly* 10 pens.) Final answer: Buy three 3-packs and one single pen for $13.50
650 tokens in, 171 out · 4.4 s · $0.0015 · 1 message on Pro · answered by google/gemini-4-argon via Google AI Studio ·
Gemini 3.8 Flash
Passed: Final answer 3 sets of 3 pens and 1 single pen for 13.50: right.
To get exactly 10 pens at the lowest cost, buy as many 3-pen deals as possible: * 3 deals of 3 pens = 9 pens for $12.00 ($4.00 × 3) * 1 single pen = 1 pen for $1.50 Total cost: $12.00 + $1.50 = $13.50 Final answer: 3 sets of 3 pens and 1 single pen for $13.50
431 tokens in, 354 out (240 of them reasoning) · 4.7 s · $0.0017 · 1 message on Pro · answered by google/gemini-3.8-flash via Google ·
Mistral Large 4
Passed: Final answer Buy three 3-packs plus one single pen, for 13.50 total.: right.
- 3-pack deal: $4 for 3 pens ≈ $1.33/pen — cheaper than the $1.50 single. - For exactly 10 pens, use as many 3-packs as possible without going over: 3 packs = 9 pens for 3 × $4 = $12. - Remaining 1 pen bought singly: $1.50. - Total: $12 + $1.50 = $13.50. Final answer: Buy three 3-packs plus one single pen, for $13.50 total.
408 tokens in, 917 out (616 of them reasoning) · 22.0 s · $0.0022 · 1 message on Pro · answered by mistralai/mistral-large-4-0 via Mistral ·
GPT-6 Sol
Passed: Final answer 13.50: right.
Three groups of 3 pens cost \(3 \times \$4 = \$12\). One single pen costs \$1.50, bringing the total to \$13.50. Using fewer groups would cost more. Final answer: $13.50
402 tokens in, 164 out (108 of them reasoning) · 3.3 s · $0.0024 · 1 message on Pro · answered by openai/gpt-6-sol via OpenAI ·
Claude Sonnet 5.5
Passed: Final answer Buy 3 packs of 3 and 1 single pen, for 13.50: right.
Per-pen cost: a 3-pack costs $4 ÷ 3 ≈ $1.33 per pen, while a single costs $1.50. So the packs are cheaper and should be used as much as possible. Options for exactly 10 pens: - 3 packs (9 pens) + 1 single: 3 × $4 + $1.50 = $13.50 - 2 packs (6 pens) + 4 singles: $8 + $6 = $14.00 - 1 pack + 7 singles: $4 + $10.50 = $14.50 - 0 packs + 10 singles: $15.00 The cheapest is 3 packs plus 1 single pen. Final answer: Buy 3 packs of 3 and 1 single pen, for $13.50
549 tokens in, 250 out · 2.2 s · $0.0036 · 1 message on Pro · answered by anthropic/claude-sonnet-5.5 via Anthropic ·
Kimi K3
Passed: Final answer Buy three 3-for- 4 packs and one single pen; 13.50: right.
Three 3-for-$4 packs give 9 pens for $12; add 1 single for $1.50 to reach exactly 10. Total = $12 + $1.50 = $13.50. Final answer: Buy three 3-for-$4 packs and one single pen; $13.50
489 tokens in, 322 out (247 of them reasoning) · 5.4 s · $0.0046 · 1 message on Pro · answered by moonshotai/kimi-k3 via Phala ·
Claude Sonnet 5
Passed: Final answer Buy 3 packs of 3 pens ( 4 each) plus 1 single pen ( 1.50), total 13.50.: right.
Compare unit costs: 3-for-$4 = $1.33/pen, cheaper than the $1.50 single price. So maximize the number of 3-packs. 10 pens = 3 packs of 3 (9 pens) + 1 single pen Cost = 3×$4 + 1×$1.50 = $12 + $1.50 = $13.50 Check alternative (2 packs + 4 singles): 2×$4 + 4×$1.50 = $8 + $6 = $14 (worse) Check buying an extra pack for surplus: 4 packs = $16 for 12 pens (worse, and you don't need 12) Final answer: Buy 3 packs of 3 pens ($4 each) plus 1 single pen ($1.50), total $13.50.
545 tokens in, 505 out (250 of them reasoning) · 6.3 s · $0.0061 · 1 message on Pro · answered by anthropic/claude-sonnet-5 via Claude Platform on AWS ·
GPT-6 Astra
Passed: Final answer Buy three groups of 3 and one single pen for 13.50.: right.
A group of 3 costs $4, compared with $4.50 for 3 singles, so buy as many groups as possible. 10 pens = 3 groups of 3 + 1 single. Cost = 3 × $4 + $1.50 = $13.50. Final answer: Buy three groups of 3 and one single pen for $13.50.
402 tokens in, 83 out · 2.4 s · $0.0082 · 1 message on Pro · answered by openai/gpt-6-astra via OpenAI ·
Claude Opus 5.5
Passed: Final answer 3 packs of 3 plus 1 single pen, costing 13.50: right.
Each pen costs about $1.33 in a pack of 3 ($4 ÷ 3) and $1.50 on its own. So use as many packs as possible. For exactly 10 pens: - 3 packs + 1 single = $12 + $1.50 = **$13.50** - 2 packs + 4 singles = $8 + $6 = $14.00 - 1 pack + 7 singles = $4 + $10.50 = $14.50 - 10 singles = $15.00 Final answer: 3 packs of 3 plus 1 single pen, costing $13.50
547 tokens in, 311 out (122 of them reasoning) · 5.3 s · $0.0084 · 1 message on Pro · answered by anthropic/claude-opus-5.5 via Claude Platform on AWS ·
Grok 4.7
Passed: Final answer 3 packs + 1 single ( 13.50): right.
To buy exactly 10 pens, let \(x\) be the number of 3-packs (at \$4 each) and \(y\) the number of singles (at \$1.50 each), so \(3x + y = 10\) with \(x, y\) non-negative integers. The cost is \(4x + 1.5y\). Feasible options: - \(x=0\), \(y=10\): \$15.00 - \(x=1\), \(y=7\): \$14.50 - \(x=2\), \(y=4\): \$14.00 - \(x=3\), \(y=1\): \$13.50 Packs are cheaper per pen than singles, so maximizing packs (without exceeding 10) is optimal. Buying 4 packs yields 12 pens for \$16, which both exceeds 10 and costs more. Final answer: 3 packs + 1 single ($13.50)
1,637 tokens in, 1,596 out (1,385 of them reasoning) · 18.5 s · $0.0111 · 1 message on Pro · answered by x-ai/grok-4.7 via xAI ·
Gemini 3.1 Pro
Passed: Final answer 3 sets of 3 and 1 single pen for 13.50: right.
To get exactly 10 pens at the lowest price, you should buy as many of the bulk deals (3 for $4) as possible, since $1.33 per pen is cheaper than the $1.50 single price. - 10 pens = 3 sets of 3 pens + 1 single pen - Cost of 3 sets: 3 x $4.00 = $12.00 - Cost of 1 single: 1 x $1.50 = $1.50 - Total cost: $12.00 + $1.50 = $13.50 Final answer: 3 sets of 3 and 1 single pen for $13.50
431 tokens in, 933 out (773 of them reasoning) · 9.6 s · $0.0121 · 1 message on Pro · answered by google/gemini-3.1-pro-preview via Google ·
Claude Fable 5.1
Passed: Final answer Buy three 3-for- 4 packs and one single pen, total 13.50: right.
Options for 10 pens: - 3 packs of 3 (9 pens) + 1 single: 3 × $4 + $1.50 = $13.50 - 4 packs of 3 (12 pens, but that's more than exactly 10 — not allowed here): $16.00 - 2 packs + 4 singles: $8 + $6 = $14.00 - 1 pack + 7 singles: $4 + $10.50 = $14.50 - 10 singles: $15.00 Cheapest is three 3-packs plus one single pen. Final answer: Buy three 3-for-$4 packs and one single pen, total $13.50
547 tokens in, 208 out · 5.4 s · $0.0159 · 1 message on Pro · answered by anthropic/claude-fable-5.1 via Anthropic ·
More math prompts
The other math prompts, each with every model's reply, and the results across all five.
Questions
Which AI does best on “Pens at 3 for $4”?
19 of 20 models passed this math prompt's check (final answer); GPT-6 Luna didn't. The cheapest reply that passed was Claude Haiku 5.5's, at $0.00016; the fastest, DeepSeek V4.1 Flash's in 0.9 s. The dearest reply, Claude Fable 5.1's, cost 99 times as much ($0.0159).
What does a reply to “Pens at 3 for $4” cost?
Through the models' APIs, what OpenRouter charged us ran from $0.00014 (GPT-6 Luna) to $0.0159 (Claude Fable 5.1) for this prompt. The cheapest that passed was Claude Haiku 5.5's, $0.00016. In llmwise you don't pay by the token: a reply like these counts as one message on Pro, whichever model answers.
Claude, GPT, Gemini, DeepSeek, Grok, Kimi, GLM, and Mistral, in one chat.
See what a message costs before you send it. Free is 5 messages to try; sign in with an email link, no password or card.