A refund request outside the window: every AI model's reply, tested

[6 lines every customer support prompt of ours shares, word for word: the whole prompt, on the methods page]
Customer message: "I bought a coat 45 days ago and never wore it. I'd like a refund to my card, please."

Write the reply to the customer, following the policy. At most 150 words, signed "Sam, Northwind support".

Exactly what this prompt's replies are checked against, with every other prompt of our test runs.

1. GLM 5.3 Flash

   Failed: Graded 4.0 of 5 on average (lowest 2).

   Hi,
   
   Thanks for reaching out! Since your coat was delivered 45 days ago, it falls into our 31–60 day window, where we're able to offer an exchange or a store gift card instead of a refund to your card. The good news is that as long as the coat is unworn with tags attached, it's fully eligible for either option.
   
   If you'd like to proceed, just let me know whether you'd prefer an exchange or a gift card, and I'll send you a prepaid return label. Once the coat reaches our warehouse, we'll process your gift card or exchange within 5 business days.
   
   Sorry we can't refund the card directly in this case, but I'm happy to get an exchange or gift card sorted for you right away.
   
   Best,
   Sam, Northwind support

   530 tokens in, 165 out · 2.4 s · $0.0002 · 1 message on Pro · answered by z-ai/glm-5.3-flash via Together · September 27, 2026

2. Kimi K3

   Failed: Graded 4.0 of 5 on average (lowest 2).

   Hi there,
   
   Thanks for reaching out, and I'm glad the coat is still unworn with tags.
   
   Unfortunately, since your purchase was 45 days ago, it's outside our 30-day window for refunds to the original payment method. However, you're still within our 60-day window, so I'd be happy to offer you either:
   
   - An exchange for a different size, color, or item, or
   - A store gift card for the full purchase amount
   
   Just let me know which option you'd prefer, and I'll get the return process started for you right away. I'll send you a prepaid return label so you can send the coat back at no cost.
   
   Sorry I can't offer the card refund this time, but I hope one of these options works well for you. Feel free to reply with any questions!
   
   Best,
   Sam, Northwind support

   599 tokens in, 203 out (23 of them reasoning) · 2.6 s · $0.0045 · 1 message on Pro · answered by moonshotai/kimi-k3 via Together · September 27, 2026

3. GPT-6 Luna

   Passed: Graded 4.7 of 5 on average (lowest 4).

   Hi,
   
   Since your coat was purchased 45 days ago, it’s outside our 30-day refund window. However, within 60 days, unworn items are eligible for an exchange or a store gift card. We’re sorry we can’t refund it to your card.
   
   Sam, Northwind support

   511 tokens in, 64 out · 1.3 s · $0.0001 · 1 message on Pro · answered by openai/gpt-6-luna via OpenAI · September 27, 2026

4. DeepSeek V4 Pro

   Passed: Graded 4.3 of 5 on average (lowest 4).

   Hi there,
   
   Thanks for reaching out. Since your coat was purchased 45 days ago, it falls outside our 30-day refund window. For items between 30 and 60 days, we're unable to issue a refund to the original payment method.
   
   However, I can offer you either an exchange or a store gift card for the full value of the coat, as long as it's unworn with tags attached.
   
   Just let me know which option you'd prefer and I'll get it sorted for you.
   
   Best,
   Sam, Northwind support

   525 tokens in, 170 out (60 of them reasoning) · 3.9 s · $0.0003 · 1 message on Pro · answered by deepseek/deepseek-v4-pro-0813 via StreamLake · September 27, 2026

5. DeepSeek V4.1 Flash

   Passed: Graded 4.7 of 5 on average (lowest 4).

   Hi there,
   
   Thanks for getting in touch, and I'm sorry the coat didn't work out.
   
   Because it's been 45 days since your order, it falls outside our 30-day refund window. Within 30 days of delivery we refund to the original payment method, but from 30 up to 60 days we can offer an exchange or a store gift card instead — no refund to your card, I'm afraid.
   
   Since the coat is unworn with tags still attached, you're all set for either option. Just let me know which you'd prefer — a different item or a gift card — and I'll get it moving right away. If you'd like to exchange, tell me the size or style you're after and I'll confirm availability before you ship anything back.
   
   Happy to help however you'd like to proceed.
   
   Sam, Northwind support

   553 tokens in, 244 out (70 of them reasoning) · 2.3 s · $0.0003 · 1 message on Pro · answered by deepseek/deepseek-v4.1-flash via Novita · September 27, 2026

6. GLM 5.3

   Passed: Graded 4.7 of 5 on average (lowest 4).

   Hi,
   
   Thanks for reaching out! Since your coat was delivered 45 days ago, it falls outside our 30-day refund window, so I'm unable to refund your card. However, because it's within 60 days and unworn with tags, you're eligible for either an exchange or a store gift card for the item's value.
   
   Just reply with which option you'd prefer and we'll get it sorted right away.
   
   Sorry for the inconvenience, and thanks for understanding!
   
   Sam, Northwind support

   553 tokens in, 115 out (13 of them reasoning) · 1.1 s · $0.0009 · 1 message on Pro · answered by z-ai/glm-5.3 via Wafer · September 27, 2026

7. Claude Haiku 4.5

   Passed: Graded 4.7 of 5 on average (lowest 4).

   Hi there,
   
   Thank you for reaching out! I appreciate you wanting to return your coat.
   
   Since your purchase was 45 days ago, it falls outside our 30-day full refund window. However, you're within our extended return period (up to 60 days), so I can offer you either an exchange for a different item or a store gift card for the full purchase amount.
   
   Unfortunately, I'm unable to process a refund to your original payment method at this point, but either of these options would let you get full value from your purchase.
   
   Which would work better for you — an exchange or a gift card?
   
   Best regards,
   Sam
   Northwind Support

   561 tokens in, 147 out · 2.7 s · $0.0013 · 1 message on Pro · answered by anthropic/claude-haiku-4.5 via Amazon Bedrock · September 27, 2026

8. GPT-6 Sol

   Passed: Graded 4.3 of 5 on average (lowest 4).

   Hi,
   
   Thanks for reaching out. Our return window is based on the delivery date, so could you let us know when the coat arrived and whether it still has its tags?
   
   If it was delivered within the last 30 days and is unworn with tags, we can refund your original payment method. If it was delivered 31–60 days ago, we can offer an exchange or a store gift card, but not a refund to your card.
   
   Sam, Northwind support

   511 tokens in, 143 out (42 of them reasoning) · 3.0 s · $0.0025 · 1 message on Pro · answered by openai/gpt-6-sol via OpenAI · September 27, 2026

9. Gemini 3.8 Flash

   Passed: Graded 4.3 of 5 on average (lowest 4).

   Hi there,
   
   Thank you for reaching out to Northwind Outfitters.
   
   Under our policy, refunds to the original payment method are only available within 30 days of delivery. Because your purchase was 45 days ago, we are unable to issue a refund to your card.
   
   However, since you are within our 60-day window, we would be happy to offer you an exchange or a store gift card, provided the coat is unworn with the tags still attached. 
   
   Please let us know whether you would prefer an exchange or a gift card, and we will send you the return instructions right away.
   
   Best regards,
   
   Sam, Northwind support

   551 tokens in, 652 out (515 of them reasoning) · 5.9 s · $0.0029 · 1 message on Pro · answered by google/gemini-3.8-flash via Google · September 27, 2026

10. Claude Sonnet 5

    Passed: Graded 4.7 of 5 on average (lowest 4).

    Hi there,
    
    Thanks for reaching out! Since your coat was purchased 45 days ago, it falls outside our 30-day refund window, so I'm unable to process a refund to your card.
    
    The good news is that it's still within our 60-day window, so I can offer you an exchange for a different size/color, or a store gift card for the full value of the coat. Just let me know which you'd prefer, and if it's an exchange, what you'd like instead.
    
    Thanks for understanding, and sorry for any inconvenience!
    
    Sam, Northwind support

    704 tokens in, 174 out · 3.2 s · $0.0031 · 1 message on Pro · answered by anthropic/claude-sonnet-5 via Claude Platform on AWS · September 27, 2026

11. Claude Sonnet 5.5

    Passed: Graded 4.7 of 5 on average (lowest 4).

    Hi there,
    
    Thank you for reaching out. I'm sorry, but a refund to your card isn't possible for this coat. Our refund window is 30 days from delivery, and at 45 days you're past it.
    
    Since you're still within 60 days, you have two options:
    
    1. **Exchange** the coat for a different size or item.
    2. **Store gift card** for the value of the coat.
    
    The coat needs to be unworn with tags attached. If you'd like to go ahead with either option, just reply with your order number and which one you prefer, and I'll send return instructions right away.
    
    Thanks for your understanding.
    
    Sam, Northwind support

    708 tokens in, 201 out · 2.0 s · $0.0038 · 1 message on Pro · answered by anthropic/claude-sonnet-5.5 via Anthropic · September 28, 2026

12. Grok 4.7

    Passed: Graded 4.3 of 5 on average (lowest 4).

    Hi,
    
    Thanks for reaching out about your coat. Because it’s been 45 days since your purchase, this falls outside our 30-day refund window. From day 31 through day 60, we can offer an exchange or a store gift card for unworn items with tags still attached. We aren’t able to refund the original payment method after 30 days.
    
    If you’d like to proceed, reply with whether you prefer an exchange or a gift card, and we’ll send return instructions. Once the coat reaches our warehouse in eligible condition, we’ll process that option.
    
    Sam, Northwind support

    1,748 tokens in, 629 out (512 of them reasoning) · 7.6 s · $0.0041 · 1 message on Pro · answered by x-ai/grok-4.7 via xAI · September 27, 2026

13. Gemini 3.1 Pro

    Passed: Graded 4.7 of 5 on average (lowest 4).

    Hi there,
    
    Thank you for reaching out about returning your coat. 
    
    Because your purchase was made 45 days ago, it falls outside our 30-day window for refunds to the original payment method. However, since your item is unworn and still within our 60-day return period, I would be happy to help you with an exchange or issue a Northwind Outfitters store gift card for the full amount. 
    
    Please let me know which of these two options you prefer, and I will send you the return instructions. 
    
    Best regards,
    
    Sam, Northwind support

    551 tokens in, 763 out (640 of them reasoning) · 7.4 s · $0.0103 · 1 message on Pro · answered by google/gemini-3.1-pro-preview via Google · September 27, 2026

14. Claude Opus 5.5

    Passed: Graded 5.0 of 5 on average (lowest 5).

    Hi there,
    
    Thank you for reaching out about your coat.
    
    Our policy offers full refunds to the original payment method for unworn items with tags returned within 30 days of delivery. Since your purchase was about 45 days ago, it's outside the refund window, so unfortunately I can't refund it to your card.
    
    Because it's still within 60 days, you can choose either:
    - **An exchange** for a different size, colour, or item, or
    - **A store gift card** for the full value.
    
    Please make sure the coat is unworn with its tags still attached. Just let me know which option you'd prefer, and I'll send you the return instructions.
    
    Sorry I can't offer a card refund this time, and thanks for your understanding.
    
    Best regards,
    Sam, Northwind support

    706 tokens in, 355 out (114 of them reasoning) · 6.1 s · $0.0106 · 1 message on Pro · answered by anthropic/claude-opus-5.5 via Claude Platform on AWS · September 27, 2026

15. GPT-6 Astra

    Passed: Graded 4.7 of 5 on average (lowest 4).

    Hi,
    
    Thanks for reaching out! Our return window starts on the delivery date, rather than the purchase date. Could you confirm when your coat arrived and whether the tags are still attached?
    
    If it was delivered within the last 30 days and is unworn with tags, we can refund your original payment method within 5 business days of receiving the return at our warehouse.
    
    If it’s been more than 30 days but no more than 60 days since delivery, we can offer an exchange or a store gift card, but not a refund to your card.
    
    Sam, Northwind support

    511 tokens in, 171 out (48 of them reasoning) · 5.0 s · $0.0137 · 1 message on Pro · answered by openai/gpt-6-astra via OpenAI · September 27, 2026

16. Claude Fable 5.1

    Passed: Graded 4.3 of 5 on average (lowest 3).

    Hi there,
    
    Thank you for reaching out, and I'm sorry the coat didn't work out.
    
    Since your purchase was 45 days ago, it falls outside our 30-day window for refunds to the original payment method. However, because you're still within 60 days and the coat is unworn with its tags, you're eligible for either:
    
    - An exchange for a different size, colour, or item, or
    - A store gift card for the full purchase amount.
    
    Just let me know which option you'd prefer, and I'll send you the return instructions right away. Once the coat reaches our warehouse, we'll process your exchange or gift card promptly.
    
    Apologies that I can't offer a card refund in this case, and thanks for your understanding.
    
    Sam, Northwind support

    706 tokens in, 235 out · 4.5 s · $0.0204 · 1 message on Pro · answered by anthropic/claude-fable-5.1 via Anthropic · September 27, 2026

• A frustrated customer
• A late order
• A message with a planted instruction
• A return inside the window
• The best AI for customer support