Tested prompt · Customer support
A late order: every AI model's reply, tested
We sent this everyday customer support prompt to all 20 models in llmwise, the same way the app sends a message, and checked every reply the same way. Here's each one as it came, with whether it passed, what it cost and how long it took.
Based on 20 of our test runs on , through OpenRouter with the app's own prompt and settings. Updated .
Short answer
15 of 20 models passed this customer support prompt's check (rubric); GPT-6 Astra, GPT-6.1 Sol, GPT-6 Sol, Gemini 4 Argon, and GLM 5.3 Flash didn't. The cheapest reply that passed was GPT-6 Luna's, at $0.000091; the fastest, DeepSeek V4.1 Flash's in 1.3 s. The dearest reply, Claude Fable 5.1's, cost 219 times as much ($0.0199).
The prompt, as sent, and its check
Checked by rubric (graded), the same way for every model.
A late order (everyday)
[6 lines every customer support prompt of ours shares, word for word: the whole prompt, on the methods page] Customer message: "I ordered a rain jacket 9 business days ago with standard delivery and it still hasn't arrived. Where is it?" Write the reply to the customer, following the policy. At most 150 words, signed "Sam, Northwind support".
Graded on:
- Next step: Apologizes and gives a clear next step (checking or tracking the order).
- Applies the policy: Applies the delay rule: free express shipping on the next order.
- Tone: Friendly and brief.
Exactly what this prompt's replies are checked against, with every other prompt of our test runs.
Every model's result
All 20 models on this prompt, in catalog order.
| Model | Result | Cost | Time | Reply |
|---|---|---|---|---|
| Claude Fable 5.1Anthropic | Passed: Graded 5.0 of 5 on average (lowest 5). | $0.0199 | 6.1 s | 223 tokens |
| Claude Opus 5.5Anthropic | Passed: Graded 4.3 of 5 on average (lowest 3). | $0.0114 | 7.8 s | 186 tokens |
| Claude Sonnet 5.5Anthropic | Passed: Graded 5.0 of 5 on average (lowest 5). | $0.0038 | 2.5 s | 207 tokens |
| Claude Sonnet 5Anthropic | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0036 | 3.5 s | 213 tokens |
| Claude Haiku 5.5Anthropic | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.00019 | 1.8 s | 203 tokens |
| Claude Haiku 4.5Anthropic | Passed: Graded 5.0 of 5 on average (lowest 5). | $0.0014 | 2.0 s | 160 tokens |
| GPT-6 AstraOpenAI | Failed: Graded 3.7 of 5 on average (lowest 1). | $0.0123 | 4.1 s | 74 tokens |
| GPT-6.1 SolOpenAI | Failed: Graded 3.7 of 5 on average (lowest 1). | $0.0011 | 3.2 s | 56 tokens |
| GPT-6 SolOpenAI | Failed: Graded 3.7 of 5 on average (lowest 1). | $0.0025 | 3.2 s | 66 tokens |
| GPT-6 LunaOpenAI | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.000091 | 1.6 s | 80 tokens |
| Gemini 4 ArgonGoogle | Failed: Graded 3.7 of 5 on average (lowest 2). | $0.0017 | 5.0 s | 185 tokens |
| Gemini 3.1 Pro (preview)Google | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0125 | 8.7 s | 131 tokens |
| Gemini 3.8 FlashGoogle | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.00092 | 3.1 s | 136 tokens |
| DeepSeek V4.1 FlashDeepSeek | Passed: Graded 4.3 of 5 on average (lowest 3). | $0.00049 | 1.3 s | 159 tokens |
| DeepSeek V4 ProDeepSeek | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0023 | 4.1 s | 102 tokens |
| Grok 4.7xAI | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.0084 | 17.5 s | 116 tokens |
| Kimi K3Moonshot | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0045 | 2.5 s | 170 tokens |
| GLM 5.3Z.ai | Passed: Graded 5.0 of 5 on average (lowest 5). | $0.00039 | 2.6 s | 142 tokens |
| GLM 5.3 FlashZ.ai | Failed: Graded 3.7 of 5 on average (lowest 3). | $0.00026 | 6.2 s | 169 tokens |
| Mistral Large 4Mistral | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.00089 | 3.9 s | 148 tokens |
Cost: what OpenRouter charged us for the reply. Time: from sending to the whole reply. Reply: its length in tokens, thinking not counted. In llmwise you pay per message, not per token: each of these replies counted as one message on Pro.
Every reply
The 5 replies that failed first, then the ones that passed, from the cheapest up.
GLM 5.3 Flash
Failed: Graded 3.7 of 5 on average (lowest 3).
Dear customer, Thank you for reaching out, and I'm sorry your rain jacket hasn't arrived yet. That's certainly not the experience we want for you. Standard delivery normally takes 3–5 business days, and since your order is now 9 business days overdue, it has clearly been delayed. I apologize for the inconvenience. Because your order is more than 7 business days late, I'd be happy to add free express shipping to your next order as a gesture of apology. Just reply to this message or mention it when you place your next order, and we'll apply it. Please also keep an eye out for the package — if it hasn't arrived within the next few business days, contact us and we'll investigate with the carrier or arrange a replacement or refund. Thank you for your patience. Sam, Northwind support
530 tokens in, 259 out (90 of them reasoning) · 6.2 s · $0.0003 · 1 message on Pro · answered by z-ai/glm-5.3-flash via Wafer ·
GPT-6.1 Sol
Failed: Graded 3.7 of 5 on average (lowest 1).
Hi, I’m sorry your rain jacket hasn’t arrived. Standard delivery takes 3–5 business days, so your order is overdue. Please send us your order number so we can check the tracking and investigate the delay. Sam, Northwind support
513 tokens in, 124 out (68 of them reasoning) · 3.2 s · $0.0011 · 1 message on Pro · answered by openai/gpt-6.1-sol via OpenAI ·
Gemini 4 Argon
Failed: Graded 3.7 of 5 on average (lowest 2).
Hi there, I’m sorry to hear your rain jacket hasn't arrived yet! Standard delivery usually takes 3 to 5 business days, so your order is currently about 4 business days past its expected window. Please reply with your order number so I can check the tracking details with our carrier and see exactly where your package is. In the meantime, to apologize for the inconvenience, here is a code for 15% off your next purchase: **SORRY15**. Also, per our policy, if your order ends up being more than 7 business days late (over 12 business days total from when you ordered), we will gladly give you free express shipping on your next order as well. As soon as you send over your order number, I'll get you an update right away. Best regards, Sam, Northwind support
770 tokens in, 185 out · 5.0 s · $0.0017 · 1 message on Pro · answered by google/gemini-4-argon via Google AI Studio ·
GPT-6 Sol
Failed: Graded 3.7 of 5 on average (lowest 1).
Hi, I’m sorry your rain jacket hasn’t arrived. Standard delivery usually takes 3–5 business days, so I understand your concern after 9 business days. Could you send me your order number? I’ll check the tracking and find out where the package is. Sam, Northwind support
511 tokens in, 144 out (78 of them reasoning) · 3.2 s · $0.0025 · 1 message on Pro · answered by openai/gpt-6-sol via OpenAI ·
GPT-6 Astra
Failed: Graded 3.7 of 5 on average (lowest 1).
I’m sorry your rain jacket hasn’t arrived yet. Standard delivery normally takes 3–5 business days, so this is longer than expected. Please send your order number so we can check the tracking and look into the delay. If you received a shipping confirmation, its tracking link may also show the latest update. Sam, Northwind support
511 tokens in, 143 out (69 of them reasoning) · 4.1 s · $0.0123 · 1 message on Pro · answered by openai/gpt-6-astra via OpenAI ·
GPT-6 Luna
Passed: Graded 4.7 of 5 on average (lowest 4).
Hi, I’m sorry your rain jacket hasn’t arrived yet. Standard delivery usually takes 3–5 business days, so your order is now more than 7 business days late. I can’t see live tracking details here, but please share your order number so we can check its status. You’ll also receive free express shipping on your next order. Sam, Northwind support
511 tokens in, 80 out · 1.6 s · $0.0001 · 1 message on Pro · answered by openai/gpt-6-luna via OpenAI ·
Claude Haiku 5.5
Passed: Graded 4.7 of 5 on average (lowest 4).
Subject: Your rain jacket order Hi there, I'm sorry your rain jacket hasn't arrived. Standard delivery normally takes 3 to 5 business days, and yours is now 9 business days out, so it's well past due. Because your order is more than 7 business days late, you're eligible for free express shipping on your next order. Just mention this order when you place it, and we'll apply the upgrade. I'm also sorry for the wait. I don't have tracking details in this message, so if you can share your order number, I'll look into where it is and what we can do next. Thanks for your patience. Sam, Northwind support
716 tokens in, 203 out · 1.8 s · $0.0002 · 1 message on Pro · answered by anthropic/claude-haiku-5.5 via Anthropic ·
GLM 5.3
Passed: Graded 5.0 of 5 on average (lowest 5).
Hi, I'm sorry your rain jacket still hasn't arrived — that's well past our standard 3–5 business day delivery window, and I understand how frustrating that is. I'd like to look into where your package is right away. Could you reply with your order number so I can check its status and tracking? If it's lost or stuck in transit, we'll sort out a replacement or solution for you. Also, because your order is more than 7 business days late, you're entitled to free express shipping on your next order — I'll make sure that's applied to your account. Thanks for your patience, and again, apologies for the delay. Sam, Northwind support
529 tokens in, 163 out (21 of them reasoning) · 2.6 s · $0.0004 · 1 message on Pro · answered by z-ai/glm-5.3 via Baidu ·
DeepSeek V4.1 Flash
Passed: Graded 4.3 of 5 on average (lowest 3).
Hi there, Thanks for reaching out, and I'm sorry your rain jacket hasn't arrived yet. Standard delivery normally takes 3–5 business days, so at 9 business days your order is running late — I understand the frustration. I don't have your order details in front of me, so could you reply with your order number? I'll pull up the tracking and find out exactly where your jacket is and when it's due. If your order turns out to be more than 7 business days past its expected delivery date, you'll qualify for free express shipping on your next order, and I'll set that up for you. Once you send the order number, I'll get back to you quickly with an update. Thanks for your patience, Sam, Northwind support
553 tokens in, 336 out (177 of them reasoning) · 1.3 s · $0.0005 · 1 message on Pro · answered by deepseek/deepseek-v4.1-flash via Together ·
Mistral Large 4
Passed: Graded 4.7 of 5 on average (lowest 4).
Hi there, I'm sorry your rain jacket hasn't arrived yet — that's frustrating, and I understand your concern. Standard delivery takes 3–5 business days, so at 9 business days your order is running well behind schedule. Let me get this sorted: I'll escalate your order to our logistics team right away to locate the jacket and arrange delivery as soon as possible. If you'd like, I can also provide tracking details once they're available. As an apology for the delay — since your order is more than 7 business days late — you're entitled to free express shipping on your next order. Just mention this with your next purchase and we'll apply it. I'll follow up with an update within 24 hours. Thanks for your patience. Sam, Northwind support
519 tokens in, 258 out (110 of them reasoning) · 3.9 s · $0.0009 · 1 message on Pro · answered by mistralai/mistral-large-4-0 via Mistral ·
Gemini 3.8 Flash
Passed: Graded 4.7 of 5 on average (lowest 4).
Hi there, Thank you for reaching out, and I’m very sorry for the delay with your rain jacket. Standard delivery typically takes 3 to 5 business days, so your order is certainly running later than expected. Could you please reply with your order number? I will locate your package immediately and get an update on its delivery status for you. Because your order is more than 7 business days late, we would also like to offer you free express shipping on your next order to make up for the inconvenience. Once you share your order details, I'll get right on this. Best regards, Sam, Northwind support
550 tokens in, 136 out · 3.1 s · $0.0009 · 1 message on Pro · answered by google/gemini-3.8-flash via Google ·
Claude Haiku 4.5
Passed: Graded 5.0 of 5 on average (lowest 5).
Hi there, Thank you for reaching out. I'm sorry your rain jacket hasn't arrived yet. Standard delivery typically takes 3–5 business days, so at 9 business days, your order is definitely running late. Here's the good news: since your order is more than 7 business days late, you qualify for free express shipping on your next order. I'll make sure that's applied to your account. In the meantime, I'd like to help track down your current order. Could you please reply with your order number? That way I can check the status with our warehouse and courier right away. Thanks for your patience, and again, I apologize for the delay. Best, Sam Northwind Support
560 tokens in, 160 out · 2.0 s · $0.0014 · 1 message on Pro · answered by anthropic/claude-haiku-4.5 via Amazon Bedrock ·
DeepSeek V4 Pro
Passed: Graded 4.7 of 5 on average (lowest 4).
I'm sorry your rain jacket hasn't arrived yet. Standard delivery normally takes 3–5 business days, so 9 business days is outside the usual window, and I understand the concern. Please reply with your order number so I can check the tracking status and look into what's causing the delay. Just so you know: if the order ends up being more than 7 business days late, you'll qualify for free express shipping on your next order under our delivery policy. Sam, Northwind support
525 tokens in, 517 out (415 of them reasoning) · 4.1 s · $0.0023 · 1 message on Pro · answered by deepseek/deepseek-v4-pro-0813 via Wafer ·
Claude Sonnet 5
Passed: Graded 4.7 of 5 on average (lowest 4).
Hi there, I'm really sorry your rain jacket hasn't arrived yet — 9 business days is well past our standard 3–5 business day delivery window, so I completely understand the frustration. I'd recommend reaching out with your order number so we can track down exactly where your package is and get it moving. In the meantime, since your order is more than 7 business days late, you're entitled to free express shipping on your next order — I'll make sure that's applied to your account. If it turns out the jacket is lost in transit, we'll get a replacement or refund sorted for you right away. Thanks for your patience, and sorry again for the delay! Sam, Northwind support
713 tokens in, 213 out · 3.5 s · $0.0036 · 1 message on Pro · answered by anthropic/claude-sonnet-5 via Claude Platform on AWS ·
Claude Sonnet 5.5
Passed: Graded 5.0 of 5 on average (lowest 5).
Hi there, I'm sorry your rain jacket hasn't arrived. Standard delivery takes 3 to 5 business days, so at 9 business days your order is well past that window. Since it's more than 7 business days late, you're eligible for free express shipping on your next order. I'm happy to note that on your account. I don't have tracking details in front of me, so please send me your order number and I'll look into where the package is and what's causing the delay. If it turns out to be lost, we'll work out the next steps for you, such as a replacement or a refund. Thanks for your patience, and sorry again for the trouble. Sam, Northwind support
717 tokens in, 207 out · 2.5 s · $0.0038 · 1 message on Pro · answered by anthropic/claude-sonnet-5.5 via Anthropic ·
Kimi K3
Passed: Graded 4.7 of 5 on average (lowest 4).
Hi, Thanks for reaching out, and I'm sorry your rain jacket hasn't arrived yet. Standard delivery takes 3–5 business days, so your order is now 9 business days out and officially late — that's on us. Here's what I'd suggest: I'll flag your order with our shipping team right away to locate it and get you a tracking update within 24 hours. If it turns out to be lost in transit, we'll send a replacement or issue a full refund to your original payment method — your choice. Since your order is more than 7 business days late, you're also entitled to free express shipping on your next order. Just mention this conversation when you check out and we'll apply it. Again, apologies for the delay, and thank you for your patience. Sam, Northwind support
600 tokens in, 201 out (31 of them reasoning) · 2.5 s · $0.0045 · 1 message on Pro · answered by moonshotai/kimi-k3 via Together ·
Grok 4.7
Passed: Graded 4.3 of 5 on average (lowest 4).
Hello, I’m sorry your rain jacket still hasn’t arrived. Standard delivery is 3 to 5 business days, so an order placed 9 business days ago is past that window. I don’t have a live tracking update in this message, so I can’t say exactly where the package is. Please reply with your order number and I’ll trace the shipment and update you. Free express shipping on your next order applies if an order is more than 7 business days late. You’re not there yet, but I’ll keep an eye on it and apply that if it qualifies. Sam, Northwind support
1,748 tokens in, 1,109 out (993 of them reasoning) · 17.5 s · $0.0084 · 1 message on Pro · answered by x-ai/grok-4.7 via xAI ·
Claude Opus 5.5
Passed: Graded 4.3 of 5 on average (lowest 3).
Hi there, I'm sorry your rain jacket hasn't arrived yet. Standard delivery usually takes 3 to 5 business days, so your order is running late, and I understand how frustrating that is. Could you reply with your order number? I'll check the tracking details with our carrier and let you know exactly where your package is and when to expect it. If your order ends up more than 7 business days past its expected delivery date, you'll get free express shipping on your next order. Thanks for your patience, and I'll get back to you as soon as I have an update. Sam, Northwind support
715 tokens in, 395 out (209 of them reasoning) · 7.8 s · $0.0114 · 1 message on Pro · answered by anthropic/claude-opus-5.5 via Claude Platform on AWS ·
Gemini 3.1 Pro
Passed: Graded 4.7 of 5 on average (lowest 4).
Hello, I apologize that your rain jacket hasn't arrived yet. Because standard delivery typically takes 3 to 5 business days, your order from 9 business days ago is currently delayed. I am checking with our shipping carrier right now to locate your package and will follow up with you as soon as I have an exact update on its whereabouts. Please rest assured that if your order ends up being more than 7 business days late past its expected delivery window, we will provide you with free express shipping on your next order. Thank you for your patience while we get this sorted out. Sam, Northwind support
550 tokens in, 948 out (817 of them reasoning) · 8.7 s · $0.0125 · 1 message on Pro · answered by google/gemini-3.1-pro-preview via Google ·
Claude Fable 5.1
Passed: Graded 5.0 of 5 on average (lowest 5).
Hi there, I'm sorry your rain jacket hasn't arrived yet. Standard delivery normally takes 3 to 5 business days, so at 9 business days your order is well past our expected window. I'm looking into where the parcel is right now and will update you with tracking details as soon as I have them from the carrier. If it can't be located, we'll send a replacement straight away. Because your order is more than 7 business days late, you're entitled to free express shipping on your next order with us. I've noted that on your account, so it will be applied automatically. Please reply with your order number if it's not already on file, and I'll prioritise the trace. Thank you for your patience. Sam, Northwind support
715 tokens in, 223 out · 6.1 s · $0.0199 · 1 message on Pro · answered by anthropic/claude-fable-5.1 via Anthropic ·
More customer support prompts
The other customer support prompts, each with every model's reply, and the results across all five.
Questions
Which AI does best on “A late order”?
15 of 20 models passed this customer support prompt's check (rubric); GPT-6 Astra, GPT-6.1 Sol, GPT-6 Sol, Gemini 4 Argon, and GLM 5.3 Flash didn't. The cheapest reply that passed was GPT-6 Luna's, at $0.000091; the fastest, DeepSeek V4.1 Flash's in 1.3 s. The dearest reply, Claude Fable 5.1's, cost 219 times as much ($0.0199).
What does a reply to “A late order” cost?
Through the models' APIs, what OpenRouter charged us ran from $0.000091 (GPT-6 Luna) to $0.0199 (Claude Fable 5.1) for this prompt. In llmwise you don't pay by the token: a reply like these counts as one message on Pro, whichever model answers.
Claude, GPT, Gemini, DeepSeek, Grok, Kimi, GLM, and Mistral, in one chat.
See what a message costs before you send it. Free is 5 messages to try; sign in with an email link, no password or card.