A late order: every AI model's reply, tested

[6 lines every customer support prompt of ours shares, word for word: the whole prompt, on the methods page]
Customer message: "I ordered a rain jacket 9 business days ago with standard delivery and it still hasn't arrived. Where is it?"

Write the reply to the customer, following the policy. At most 150 words, signed "Sam, Northwind support".

Exactly what this prompt's replies are checked against, with every other prompt of our test runs.

1. GLM 5.3 Flash

   Failed: Graded 3.7 of 5 on average (lowest 3).

   Dear customer,
   
   Thank you for reaching out, and I'm sorry your rain jacket hasn't arrived yet. That's certainly not the experience we want for you.
   
   Standard delivery normally takes 3–5 business days, and since your order is now 9 business days overdue, it has clearly been delayed. I apologize for the inconvenience.
   
   Because your order is more than 7 business days late, I'd be happy to add free express shipping to your next order as a gesture of apology. Just reply to this message or mention it when you place your next order, and we'll apply it.
   
   Please also keep an eye out for the package — if it hasn't arrived within the next few business days, contact us and we'll investigate with the carrier or arrange a replacement or refund.
   
   Thank you for your patience.
   
   Sam, Northwind support

   530 tokens in, 259 out (90 of them reasoning) · 6.2 s · $0.0003 · 1 message on Pro · answered by z-ai/glm-5.3-flash via Wafer · September 27, 2026

2. GPT-6 Sol

   Failed: Graded 3.7 of 5 on average (lowest 1).

   Hi, I’m sorry your rain jacket hasn’t arrived. Standard delivery usually takes 3–5 business days, so I understand your concern after 9 business days.
   
   Could you send me your order number? I’ll check the tracking and find out where the package is.
   
   Sam, Northwind support

   511 tokens in, 144 out (78 of them reasoning) · 3.2 s · $0.0025 · 1 message on Pro · answered by openai/gpt-6-sol via OpenAI · September 27, 2026

3. GPT-6 Astra

   Failed: Graded 3.7 of 5 on average (lowest 1).

   I’m sorry your rain jacket hasn’t arrived yet. Standard delivery normally takes 3–5 business days, so this is longer than expected.
   
   Please send your order number so we can check the tracking and look into the delay. If you received a shipping confirmation, its tracking link may also show the latest update.
   
   Sam, Northwind support

   511 tokens in, 143 out (69 of them reasoning) · 4.1 s · $0.0123 · 1 message on Pro · answered by openai/gpt-6-astra via OpenAI · September 27, 2026

4. GPT-6 Luna

   Passed: Graded 4.7 of 5 on average (lowest 4).

   Hi,
   
   I’m sorry your rain jacket hasn’t arrived yet. Standard delivery usually takes 3–5 business days, so your order is now more than 7 business days late. I can’t see live tracking details here, but please share your order number so we can check its status. You’ll also receive free express shipping on your next order.
   
   Sam, Northwind support

   511 tokens in, 80 out · 1.6 s · $0.0001 · 1 message on Pro · answered by openai/gpt-6-luna via OpenAI · September 27, 2026

5. DeepSeek V4.1 Flash

   Passed: Graded 4.0 of 5 on average (lowest 4).

   Hi there,
   
   Thanks for reaching out, and I'm sorry your rain jacket hasn't arrived yet.
   
   Standard delivery normally takes 3 to 5 business days, so your order is running later than expected. I've flagged it with our shipping team and will come back to you with an update as soon as I have one. If you'd like, I can also check whether a replacement can be sent.
   
   One note on our policy: free express shipping on your next order applies when a delivery runs more than 7 business days late. Your order is currently around 4 to 6 business days past the standard window, so it doesn't quite qualify yet — but if it remains undelivered past that mark, I'll add it to your account.
   
   Thanks for your patience, and sorry again for the wait.
   
   Sam, Northwind support

   553 tokens in, 316 out (147 of them reasoning) · 2.5 s · $0.0004 · 1 message on Pro · answered by deepseek/deepseek-v4.1-flash via Novita · September 27, 2026

6. GLM 5.3

   Passed: Graded 5.0 of 5 on average (lowest 5).

   Hi,
   
   I'm sorry your rain jacket still hasn't arrived — that's well past our standard 3–5 business day delivery window, and I understand how frustrating that is.
   
   I'd like to look into where your package is right away. Could you reply with your order number so I can check its status and tracking? If it's lost or stuck in transit, we'll sort out a replacement or solution for you.
   
   Also, because your order is more than 7 business days late, you're entitled to free express shipping on your next order — I'll make sure that's applied to your account.
   
   Thanks for your patience, and again, apologies for the delay.
   
   Sam, Northwind support

   529 tokens in, 163 out (21 of them reasoning) · 2.6 s · $0.0004 · 1 message on Pro · answered by z-ai/glm-5.3 via Baidu · September 27, 2026

7. Gemini 3.8 Flash

   Passed: Graded 4.7 of 5 on average (lowest 4).

   Hi there,
   
   Thank you for reaching out, and I’m very sorry for the delay with your rain jacket. 
   
   Standard delivery typically takes 3 to 5 business days, so your order is certainly running later than expected. Could you please reply with your order number? I will locate your package immediately and get an update on its delivery status for you. 
   
   Because your order is more than 7 business days late, we would also like to offer you free express shipping on your next order to make up for the inconvenience. 
   
   Once you share your order details, I'll get right on this.
   
   Best regards,
   
   Sam, Northwind support

   550 tokens in, 136 out · 3.1 s · $0.0009 · 1 message on Pro · answered by google/gemini-3.8-flash via Google · September 27, 2026

8. Claude Haiku 4.5

   Passed: Graded 5.0 of 5 on average (lowest 5).

   Hi there,
   
   Thank you for reaching out. I'm sorry your rain jacket hasn't arrived yet. Standard delivery typically takes 3–5 business days, so at 9 business days, your order is definitely running late.
   
   Here's the good news: since your order is more than 7 business days late, you qualify for free express shipping on your next order. I'll make sure that's applied to your account.
   
   In the meantime, I'd like to help track down your current order. Could you please reply with your order number? That way I can check the status with our warehouse and courier right away.
   
   Thanks for your patience, and again, I apologize for the delay.
   
   Best,
   Sam
   Northwind Support

   560 tokens in, 160 out · 2.0 s · $0.0014 · 1 message on Pro · answered by anthropic/claude-haiku-4.5 via Amazon Bedrock · September 27, 2026

9. Claude Sonnet 5

   Passed: Graded 4.7 of 5 on average (lowest 4).

   Hi there,
   
   I'm really sorry your rain jacket hasn't arrived yet — 9 business days is well past our standard 3–5 business day delivery window, so I completely understand the frustration.
   
   I'd recommend reaching out with your order number so we can track down exactly where your package is and get it moving. In the meantime, since your order is more than 7 business days late, you're entitled to free express shipping on your next order — I'll make sure that's applied to your account.
   
   If it turns out the jacket is lost in transit, we'll get a replacement or refund sorted for you right away.
   
   Thanks for your patience, and sorry again for the delay!
   
   Sam, Northwind support

   713 tokens in, 213 out · 3.5 s · $0.0036 · 1 message on Pro · answered by anthropic/claude-sonnet-5 via Claude Platform on AWS · September 27, 2026

10. Claude Sonnet 5.5

    Passed: Graded 5.0 of 5 on average (lowest 5).

    Hi there,
    
    I'm sorry your rain jacket hasn't arrived. Standard delivery takes 3 to 5 business days, so at 9 business days your order is well past that window.
    
    Since it's more than 7 business days late, you're eligible for free express shipping on your next order. I'm happy to note that on your account.
    
    I don't have tracking details in front of me, so please send me your order number and I'll look into where the package is and what's causing the delay. If it turns out to be lost, we'll work out the next steps for you, such as a replacement or a refund.
    
    Thanks for your patience, and sorry again for the trouble.
    
    Sam, Northwind support

    717 tokens in, 207 out · 2.5 s · $0.0038 · 1 message on Pro · answered by anthropic/claude-sonnet-5.5 via Anthropic · September 28, 2026

11. DeepSeek V4 Pro

    Passed: Graded 4.3 of 5 on average (lowest 3).

    I'm sorry your rain jacket hasn't arrived yet. Standard delivery normally takes 3–5 business days, so your order is running late. I'm checking with our courier now to locate it and will get back to you with an update as soon as I have tracking information.
    
    In the meantime, please let us know if it still hasn't arrived after 7 business days past the latest expected delivery date. Under our policy, you would then qualify for free express shipping on your next order.
    
    We'll do our best to get this sorted out for you quickly.  
    
    Sam, Northwind support

    525 tokens in, 1,123 out (1,002 of them reasoning) · 20.2 s · $0.0041 · 1 message on Pro · answered by deepseek/deepseek-v4-pro-0813 via GMICloud · September 27, 2026

12. Kimi K3

    Passed: Graded 4.7 of 5 on average (lowest 4).

    Hi,
    
    Thanks for reaching out, and I'm sorry your rain jacket hasn't arrived yet. Standard delivery takes 3–5 business days, so your order is now 9 business days out and officially late — that's on us.
    
    Here's what I'd suggest: I'll flag your order with our shipping team right away to locate it and get you a tracking update within 24 hours. If it turns out to be lost in transit, we'll send a replacement or issue a full refund to your original payment method — your choice.
    
    Since your order is more than 7 business days late, you're also entitled to free express shipping on your next order. Just mention this conversation when you check out and we'll apply it.
    
    Again, apologies for the delay, and thank you for your patience.
    
    Sam, Northwind support

    600 tokens in, 201 out (31 of them reasoning) · 2.5 s · $0.0045 · 1 message on Pro · answered by moonshotai/kimi-k3 via Together · September 27, 2026

13. Grok 4.7

    Passed: Graded 4.3 of 5 on average (lowest 4).

    I’m sorry your rain jacket still hasn’t arrived. Standard delivery is 3 to 5 business days, so an order placed 9 business days ago is past that window.
    
    I can’t see the shipment from your message alone. Please reply with your order number and the name or email on the order, and I’ll check its status and tell you where it is.
    
    It is not yet more than 7 business days past the expected delivery date, so free express shipping on your next order doesn’t apply yet. I’ll confirm the dates when I look up the order.
    
    Sam, Northwind support

    1,748 tokens in, 1,289 out (1,174 of them reasoning) · 17.2 s · $0.0076 · 1 message on Pro · answered by x-ai/grok-4.7 via xAI · September 27, 2026

14. Claude Opus 5.5

    Passed: Graded 4.3 of 5 on average (lowest 3).

    Hi there,
    
    I'm sorry your rain jacket hasn't arrived yet. Standard delivery usually takes 3 to 5 business days, so your order is running late, and I understand how frustrating that is.
    
    Could you reply with your order number? I'll check the tracking details with our carrier and let you know exactly where your package is and when to expect it.
    
    If your order ends up more than 7 business days past its expected delivery date, you'll get free express shipping on your next order.
    
    Thanks for your patience, and I'll get back to you as soon as I have an update.
    
    Sam, Northwind support

    715 tokens in, 395 out (209 of them reasoning) · 7.8 s · $0.0114 · 1 message on Pro · answered by anthropic/claude-opus-5.5 via Claude Platform on AWS · September 27, 2026

15. Gemini 3.1 Pro

    Passed: Graded 4.7 of 5 on average (lowest 4).

    Hello,
    
    I apologize that your rain jacket hasn't arrived yet. Because standard delivery typically takes 3 to 5 business days, your order from 9 business days ago is currently delayed. 
    
    I am checking with our shipping carrier right now to locate your package and will follow up with you as soon as I have an exact update on its whereabouts. 
    
    Please rest assured that if your order ends up being more than 7 business days late past its expected delivery window, we will provide you with free express shipping on your next order. 
    
    Thank you for your patience while we get this sorted out.
    
    Sam, Northwind support

    550 tokens in, 948 out (817 of them reasoning) · 8.7 s · $0.0125 · 1 message on Pro · answered by google/gemini-3.1-pro-preview via Google · September 27, 2026

16. Claude Fable 5.1

    Passed: Graded 5.0 of 5 on average (lowest 5).

    Hi there,
    
    I'm sorry your rain jacket hasn't arrived yet. Standard delivery normally takes 3 to 5 business days, so at 9 business days your order is well past our expected window.
    
    I'm looking into where the parcel is right now and will update you with tracking details as soon as I have them from the carrier. If it can't be located, we'll send a replacement straight away.
    
    Because your order is more than 7 business days late, you're entitled to free express shipping on your next order with us. I've noted that on your account, so it will be applied automatically.
    
    Please reply with your order number if it's not already on file, and I'll prioritise the trace. Thank you for your patience.
    
    Sam, Northwind support

    715 tokens in, 223 out · 6.1 s · $0.0199 · 1 message on Pro · answered by anthropic/claude-fable-5.1 via Anthropic · September 27, 2026

• A frustrated customer
• A refund request outside the window
• A message with a planted instruction
• A return inside the window
• The best AI for customer support