Tested prompt · AI email writer
An apology for a broken order: every AI model's reply, tested
We ran this everyday AI email writer test input on all 19 models in llmwise, the way the tool runs it, and checked every reply the same way. Here's each one as it came, with whether it passed, what it cost and how long it took.
Based on 19 of our test runs on , through OpenRouter with the tool's own instructions and the app's settings. Updated .
Short answer
All 19 models passed this AI email writer test input's check (key facts). The cheapest reply that passed was GPT-6 Luna's, at $0.000039; the fastest, DeepSeek V4.1 Flash's in 0.7 s. The dearest reply, Claude Fable 5.1's, cost 416 times as much ($0.0161).
The prompt, as sent, and its check
Checked by key facts, the same way for every model.
An apology for a broken order (everyday)
Write this email for me, ready to send. Start with a line that starts "Subject:", and put anything you don't know in [brackets]. Make the tone apologetic: own the problem and say what's being done. --- Reply to a customer, Dana Whitfield, whose order #58213 arrived with a cracked mug. Say sorry, tell her a replacement ships tomorrow at no cost to her, and that she can keep or recycle the broken one.
Sent with the AI email writer's own instructions as the system prompt, as a free run of the tool sends them.
The answer must state “subject:”, “dana”, “58213”, “replacement”, “tomorrow”, “sorry”.
How the AI email writer's test inputs are scored, with every model's results on all five.
Every model's result
All 19 models on this test input, in catalog order.
| Model | Result | Cost | Time | Reply |
|---|---|---|---|---|
| Claude Fable 5.1Anthropic | Passed: Stated all 6 facts with the section. | $0.0161 | 5.5 s | 193 tokens |
| Claude Opus 5.5Anthropic | Passed: Stated all 6 facts with the section. | $0.0070 | 3.0 s | 222 tokens |
| Claude Sonnet 5.5Anthropic | Passed: Stated all 6 facts with the section. | $0.0032 | 2.4 s | 192 tokens |
| Claude Sonnet 5Anthropic | Passed: Stated all 6 facts with the section. | $0.0031 | 3.8 s | 211 tokens |
| Claude Haiku 5.5Anthropic | Passed: Stated all 6 facts with the section. | $0.00018 | 1.6 s | 232 tokens |
| Claude Haiku 4.5Anthropic | Passed: Stated all 6 facts with the section. | $0.0010 | 1.9 s | 128 tokens |
| GPT-6 AstraOpenAI | Passed: Stated all 6 facts with the section. | $0.0045 | 1.9 s | 105 tokens |
| GPT-6.1 SolOpenAI | Passed: Stated all 6 facts with the section. | $0.00092 | 1.9 s | 108 tokens |
| GPT-6 SolOpenAI | Passed: Stated all 6 facts with the section. | $0.00071 | 1.3 s | 66 tokens |
| GPT-6 LunaOpenAI | Passed: Stated all 6 facts with the section. | $0.000039 | 1.3 s | 79 tokens |
| Gemini 3.1 Pro (preview)Google | Passed: Stated all 6 facts with the section. | $0.0060 | 8.1 s | 147 tokens |
| Gemini 3.8 FlashGoogle | Passed: Stated all 6 facts with the section. | $0.00041 | 2.1 s | 140 tokens |
| DeepSeek V4.1 FlashDeepSeek | Passed: Stated all 6 facts with the section. | $0.00022 | 0.7 s | 115 tokens |
| DeepSeek V4 ProDeepSeek | Passed: Stated all 6 facts with the section. | $0.0018 | 4.8 s | 99 tokens |
| Grok 4.7xAI | Passed: Stated all 6 facts with the section. | $0.0058 | 9.3 s | 97 tokens |
| Kimi K3Moonshot | Passed: Stated all 6 facts with the section. | $0.0019 | 4.6 s | 143 tokens |
| GLM 5.3Z.ai | Passed: Stated all 6 facts with the section. | $0.00069 | 0.8 s | 132 tokens |
| GLM 5.3 FlashZ.ai | Passed: Stated all 6 facts with the section. | $0.00012 | 1.5 s | 126 tokens |
| Mistral Large 4Mistral | Passed: Stated all 6 facts with the section. | $0.0017 | 9.4 s | 86 tokens |
Cost: what OpenRouter charged us for the reply. Time: from sending to the whole reply. Reply: its length in tokens, thinking not counted. In llmwise you pay per message, not per token: each of these replies counted as one message on Pro.
Every reply
Every reply passed: here they are from the cheapest up.
GPT-6 Luna
Passed: Stated all 6 facts with the section.
Subject: Apologies for the cracked mug in order #58213 Hi Dana Whitfield, I’m sorry your order #58213 arrived with a cracked mug. We take responsibility for the problem, and a replacement will ship tomorrow at no cost to you. You can keep or recycle the broken mug—there’s no need to return it. Best, [Your name]
381 tokens in, 79 out · 1.3 s · $0.0000 · 1 message on Pro · answered by openai/gpt-6-luna via OpenAI ·
GLM 5.3 Flash
Passed: Stated all 6 facts with the section.
Subject: Our apologies — replacement for order #58213 Hi Dana, I'm so sorry your order #58213 arrived with a cracked mug. That's not the experience we want you to have, and I take full responsibility for the item reaching you in that condition. To make it right, a replacement mug is shipping out tomorrow at no cost to you. In the meantime, please feel free to keep or recycle the broken one — no need to return anything. Thank you for your patience, and again, our sincerest apologies for the inconvenience. Warm regards, [Your name] [Company name]
397 tokens in, 126 out · 1.5 s · $0.0001 · 1 message on Pro · answered by z-ai/glm-5.3-flash via Friendli ·
Claude Haiku 5.5
Passed: Stated all 6 facts with the section.
Subject: Apologies for your cracked mug, order #58213 Dear Dana Whitfield, I'm sorry that your order #58213 arrived with a cracked mug. That is not the experience we want you to have, and I take full responsibility for the problem. A replacement mug is being shipped tomorrow at no cost to you. You don't need to return the damaged one: you're welcome to keep it or recycle it, whichever you prefer. If you have any further concerns, or if the replacement doesn't arrive as expected, please reply to this email or call us at [phone number]. Thank you for your patience, and I'm sorry again for the trouble. Warm regards, [your name] [your title] [company name]
520 tokens in, 232 out · 1.6 s · $0.0002 · 1 message on Pro · answered by anthropic/claude-haiku-5.5 via Anthropic ·
DeepSeek V4.1 Flash
Passed: Stated all 6 facts with the section.
Subject: Sorry about your cracked mug — order #58213 Dear Dana, I'm sorry your order #58213 arrived with a cracked mug. That's our mistake, and I understand how disappointing it is to open a box and find damaged goods. We've already arranged a replacement, which ships tomorrow at no cost to you. You're welcome to keep the broken mug for [use] or simply recycle it — no need to return it. Thank you for your patience, and again, my apologies. Best regards, [your name] [company name]
414 tokens in, 146 out (31 of them reasoning) · 0.7 s · $0.0002 · 1 message on Pro · answered by deepseek/deepseek-v4.1-flash via Together ·
Gemini 3.8 Flash
Passed: Stated all 6 facts with the section.
Subject: Replacement for your damaged mug (Order #58213) Dear Dana Whitfield, I am so sorry to hear that your mug from order #58213 arrived cracked. We take full responsibility for this packaging mishap, and I am issuing a free replacement right away. Your new mug will ship tomorrow at no cost to you, and we will send your tracking information as soon as it is on the way. There is no need to return the damaged item; please feel free to safely dispose of or recycle it. We truly appreciate your patience while we make this right for you. Sincerely, [Your Name] [Your Company Name]
405 tokens in, 140 out · 2.1 s · $0.0004 · 1 message on Pro · answered by google/gemini-3.8-flash via Google AI Studio ·
GLM 5.3
Passed: Stated all 6 facts with the section.
Subject: We're sorry about your cracked mug — replacement on the way Dear Dana, I'm so sorry to hear that the mug in your order #58213 arrived cracked. That's not the experience we want for you, and we truly apologize for the inconvenience. A replacement is shipping out tomorrow at no cost to you. There's no need to return the broken mug — please feel free to keep it or recycle it, whichever you prefer. Thank you for your patience and understanding. If you have any other questions, just reply to this email and we'll be glad to help. Sincerely, [Your name] [Company name]
397 tokens in, 132 out · 0.8 s · $0.0007 · 1 message on Pro · answered by z-ai/glm-5.3 via Friendli ·
GPT-6 Sol
Passed: Stated all 6 facts with the section.
Subject: Replacement for order #58213 Hi Dana Whitfield, I’m sorry the mug in your order #58213 arrived cracked. We’ll ship a replacement tomorrow at no cost to you. You can keep or recycle the broken one. We’re sorry for the trouble. Best, [your name]
381 tokens in, 66 out · 1.3 s · $0.0007 · 1 message on Pro · answered by openai/gpt-6-sol via OpenAI ·
GPT-6.1 Sol
Passed: Stated all 6 facts with the section.
Subject: Replacement for your damaged mug — order #58213 Hi Dana Whitfield, I’m sorry your order #58213 arrived with a cracked mug. We’re sending a replacement tomorrow at no cost to you to make this right. We take responsibility for getting your order to you in good condition, and we’re sorry we let you down. You can keep or recycle the broken one—there’s no need to return it. Thank you for giving us the chance to fix this. Best, [your name]
381 tokens in, 108 out · 1.9 s · $0.0009 · 1 message on Pro · answered by openai/gpt-6.1-sol via OpenAI ·
Claude Haiku 4.5
Passed: Stated all 6 facts with the section.
Subject: We're Sorry About Your Cracked Mug – Replacement on the Way Dear Dana Whitfield, Thank you for letting us know about order #58213. We sincerely apologize that your mug arrived cracked – that's our mistake, and we take full responsibility. We're shipping a replacement to you at no cost, and it will leave our warehouse tomorrow. You're welcome to keep the broken mug or recycle it; there's no need to return it. We appreciate your patience and will do better. Best regards, [Your name]
403 tokens in, 128 out · 1.9 s · $0.0010 · 1 message on Pro · answered by anthropic/claude-haiku-4.5 via Anthropic ·
Mistral Large 4
Passed: Stated all 6 facts with the section.
Subject: Replacement for your cracked mug from order #58213 Dear Dana Whitfield, I am writing to apologize about the cracked mug that arrived in your order #58213. We take full responsibility for this problem, and I am sorry it reached you in that condition. A replacement mug will ship tomorrow at no cost to you. You are welcome to keep or recycle the broken one; there is no need to return it. Sincerely, [your name]
389 tokens in, 689 out (603 of them reasoning) · 9.4 s · $0.0017 · 1 message on Pro · answered by mistralai/mistral-large-4-0 via Mistral ·
DeepSeek V4 Pro
Passed: Stated all 6 facts with the section.
Subject: Apology and replacement for your cracked mug – Order #58213 Dear Dana Whitfield, I'm writing about your order #58213 and the cracked mug you received. I'm very sorry we let you down. We're sending a replacement mug at no cost to you. It will ship tomorrow. Please keep or recycle the broken one as you prefer. Thank you for your patience, and again, our apologies. Sincerely, [Your Name] [Company Name]
388 tokens in, 421 out (322 of them reasoning) · 4.8 s · $0.0018 · 1 message on Pro · answered by deepseek/deepseek-v4-pro-0813 via Wafer ·
Kimi K3
Passed: Stated all 6 facts with the section.
Subject: Our sincere apologies — replacement for order #58213 on its way Dear Dana, I'm so sorry your order #58213 arrived with a cracked mug. That's not the quality we promise, and this one is on us. Here's what we're doing to make it right: a replacement mug ships tomorrow at no cost to you. There's no need to return the damaged one — please feel free to keep it or recycle it, whichever is easiest. Thank you for your patience, and again, my apologies for the trouble. If there's anything else I can help with, just reply to this email. Sincerely, [your name] [company name]
469 tokens in, 160 out (17 of them reasoning) · 4.6 s · $0.0019 · 1 message on Pro · answered by moonshotai/kimi-k3 via Wafer ·
Claude Sonnet 5
Passed: Stated all 6 facts with the section.
Subject: Our Apologies for the Damaged Item in Order #58213 Dear Dana Whitfield, I'm writing to apologize for the cracked mug you received in order #58213. This is not the experience we want you to have, and I'm sorry for the disappointment it caused when you opened your package. We are taking care of this right away: a replacement mug will ship tomorrow at no cost to you. There's no need to return the damaged item — please feel free to keep or recycle it, whichever is easiest for you. Thank you for your patience, and again, I apologize for the inconvenience. Warm regards, [your name] [your position/company name]
518 tokens in, 211 out · 3.8 s · $0.0031 · 1 message on Pro · answered by anthropic/claude-sonnet-5 via Anthropic ·
Claude Sonnet 5.5
Passed: Stated all 6 facts with the section.
Subject: Our apologies for the cracked mug in order #58213 Dear Dana, I'm so sorry that your mug arrived cracked in order #58213. We should have packed it better, and I'm sorry we let you down. We are sending you a replacement, and it ships tomorrow at no cost to you. There's no need to return the damaged mug. You are welcome to keep it or recycle it. I'll send the tracking details as soon as the replacement is on its way. Thank you for your patience, and again, my apologies for the trouble. Sincerely, [your name] [your company]
520 tokens in, 192 out · 2.4 s · $0.0032 · 1 message on Pro · answered by anthropic/claude-sonnet-5.5 via Anthropic ·
GPT-6 Astra
Passed: Stated all 6 facts with the section.
Subject: Replacement for your cracked mug — order #58213 Hi Dana Whitfield, I’m sorry your order #58213 arrived with a cracked mug. We take responsibility for getting your order to you intact, and we’re sending a replacement that ships tomorrow at no cost to you. You can keep or recycle the broken mug—there’s no need to return it. I’m sorry for the inconvenience, and thank you for giving us the chance to make this right. Best, [your name]
381 tokens in, 105 out · 1.9 s · $0.0045 · 1 message on Pro · answered by openai/gpt-6-astra via OpenAI ·
Grok 4.7
Passed: Stated all 6 facts with the section.
Subject: Apology for the cracked mug in order #58213 Dear Dana Whitfield, I'm writing to apologize because the mug in your order #58213 arrived cracked. This email confirms the free replacement and what you can do with the broken one. We own this problem, and it should not have happened. A replacement ships tomorrow at no cost to you, and you can keep or recycle the broken one. We are sorry for the inconvenience. Sincerely, [your name]
1,617 tokens in, 716 out (619 of them reasoning) · 9.3 s · $0.0058 · 1 message on Pro · answered by x-ai/grok-4.7 via xAI ·
Gemini 3.1 Pro
Passed: Stated all 6 facts with the section.
Subject: Replacement for your order #58213 Dear Dana Whitfield, I am writing regarding your recent order #58213, which unfortunately arrived with a cracked mug. I want to sincerely apologize for this mistake on our end. We take full responsibility for the damage and want to make it right immediately. A replacement mug will ship out to you tomorrow at no cost to you. You do not need to return the broken mug; please feel free to keep or recycle it. We are working with our shipping team to ensure our items are packed more securely in the future. Thank you for your patience and understanding. Best regards, [Your name] [Your company]
405 tokens in, 939 out (792 of them reasoning) · 8.1 s · $0.0060 · 1 message on Pro · answered by google/gemini-3.1-pro-preview via Google AI Studio ·
Claude Opus 5.5
Passed: Stated all 6 facts with the section.
Subject: Your order #58213: replacement mug on its way Dear Dana Whitfield, I'm so sorry that your order #58213 arrived with a cracked mug. That's not the experience we want for you, and we take full responsibility for it. A replacement mug will ship to you tomorrow at no cost to you. There's no need to send the broken one back. Please feel free to keep it or recycle it, whichever is easier for you. We're also [describe what's being done to prevent this, e.g., reviewing our packaging] so this doesn't happen again. Thank you for your patience, and again, I apologize for the trouble. Warm regards, [your name] [company name]
520 tokens in, 222 out · 3.0 s · $0.0070 · 1 message on Pro · answered by anthropic/claude-opus-5.5 via Google ·
Claude Fable 5.1
Passed: Stated all 6 facts with the section.
Subject: Your Order #58213 – Replacement Mug Shipping Tomorrow Hi Dana, I'm so sorry your order #58213 arrived with a cracked mug. This is on us, and I want to make it right: a replacement mug ships tomorrow at no cost to you. There's no need to return the damaged one. Please feel free to keep it or recycle it, whichever is easier for you. I'm sorry again for the disappointment and for the trouble this has caused. Thank you for letting us know so we could fix it. Warm regards, [your name] [company name]
520 tokens in, 193 out · 5.5 s · $0.0161 · 1 message on Pro · answered by anthropic/claude-fable-5.1 via Google ·
More AI email writer test inputs
The other AI email writer test inputs, each with every model's reply, and the results across all five.
Questions
Which AI does best on “An apology for a broken order”?
All 19 models passed this AI email writer test input's check (key facts). The cheapest reply that passed was GPT-6 Luna's, at $0.000039; the fastest, DeepSeek V4.1 Flash's in 0.7 s. The dearest reply, Claude Fable 5.1's, cost 416 times as much ($0.0161).
What does a reply to “An apology for a broken order” cost?
Through the models' APIs, what OpenRouter charged us ran from $0.000039 (GPT-6 Luna) to $0.0161 (Claude Fable 5.1) for this test input. In llmwise you don't pay by the token: a reply like these counts as one message on Pro, whichever model answers.
Claude, GPT, Gemini, DeepSeek, Grok, Kimi, GLM, and Mistral, in one chat.
See what a message costs before you send it. Free is 5 messages to try; sign in with an email link, no password or card.