Tested prompt · Summarization
Decisions and action items from a meeting: every AI model's reply, tested
We sent this everyday summarization prompt to all 20 models in llmwise, the same way the app sends a message, and checked every reply the same way. Here's each one as it came, with whether it passed, what it cost and how long it took.
Based on 20 of our test runs on , through OpenRouter with the app's own prompt and settings. Updated .
Short answer
All 20 models passed this summarization prompt's check (rubric). The cheapest reply that passed was GLM 5.3 Flash's, at $0.000088; the fastest, Claude Haiku 5.5's in 1.3 s. The dearest reply, Claude Fable 5.1's, cost 208 times as much ($0.0182).
The prompt, as sent, and its check
Checked by rubric (graded), the same way for every model.
Decisions and action items from a meeting (everyday)
From this meeting transcript, list the decisions, then the action items, each with its owner and deadline.
Maya: Okay, venue. The Linden Hall quote came in under budget, so let's go with it. Everyone fine? Great, that's decided.
Jonas: I can book it today, but I need the final headcount first.
Maya: Aisha, can you get the headcount to Jonas by Wednesday?
Aisha: Yes, Wednesday works.
Maya: And the date stays 6 December. Sam, the revised budget, can you send it by Friday?
Sam: Friday is fine. I'll include the catering quotes.
Jonas: One more thing: someone should send the attendee survey after the event.
Maya: I'll do that the week after.
Graded on:
- Decisions: Both decisions: Linden Hall, and the date stays 6 December.
- Action items: Every action item with the right owner and deadline, including Maya's survey.
- Clear and accurate: Easy to scan; nothing invented.
Exactly what this prompt's replies are checked against, with every other prompt of our test runs.
Every model's result
All 20 models on this prompt, in catalog order.
| Model | Result | Cost | Time | Reply |
|---|---|---|---|---|
| Claude Fable 5.1Anthropic | Passed: Graded 4.0 of 5 on average (lowest 3). | $0.0182 | 5.6 s | 192 tokens |
| Claude Opus 5.5Anthropic | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.0118 | 5.8 s | 283 tokens |
| Claude Sonnet 5.5Anthropic | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0043 | 2.2 s | 255 tokens |
| Claude Sonnet 5Anthropic | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.0033 | 2.8 s | 183 tokens |
| Claude Haiku 5.5Anthropic | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.00018 | 1.3 s | 189 tokens |
| Claude Haiku 4.5Anthropic | Passed: Graded 5.0 of 5 on average (lowest 5). | $0.0011 | 1.7 s | 106 tokens |
| GPT-6 AstraOpenAI | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0123 | 3.3 s | 146 tokens |
| GPT-6.1 SolOpenAI | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0013 | 2.3 s | 153 tokens |
| GPT-6 SolOpenAI | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.0034 | 3.5 s | 118 tokens |
| GPT-6 LunaOpenAI | Passed: Graded 5.0 of 5 on average (lowest 5). | $0.00010 | 1.5 s | 106 tokens |
| Gemini 4 ArgonGoogle | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.0013 | 2.9 s | 116 tokens |
| Gemini 3.1 Pro (preview)Google | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.0081 | 7.6 s | 99 tokens |
| Gemini 3.8 FlashGoogle | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.00094 | 3.2 s | 146 tokens |
| DeepSeek V4.1 FlashDeepSeek | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.00021 | 2.4 s | 99 tokens |
| DeepSeek V4 ProDeepSeek | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.0011 | 5.4 s | 103 tokens |
| Grok 4.7xAI | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.0036 | 5.3 s | 94 tokens |
| Kimi K3Moonshot | Passed: Graded 4.0 of 5 on average (lowest 3). | $0.0044 | 3.4 s | 144 tokens |
| GLM 5.3Z.ai | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.00024 | 1.5 s | 101 tokens |
| GLM 5.3 FlashZ.ai | Passed: Graded 4.3 of 5 on average (lowest 4). | $0.000088 | 1.9 s | 103 tokens |
| Mistral Large 4Mistral | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0031 | 14.3 s | 0 tokens |
Cost: what OpenRouter charged us for the reply. Time: from sending to the whole reply. Reply: its length in tokens, thinking not counted. In llmwise you pay per message, not per token: each of these replies counted as one message on Pro.
Every reply
Every reply passed: here they are from the cheapest up.
GLM 5.3 Flash
Passed: Graded 4.3 of 5 on average (lowest 4).
**Decisions** 1. Venue: Linden Hall selected (quote came in under budget). 2. Event date stays 6 December. **Action Items** | Task | Owner | Deadline | |---|---|---| | Send final headcount to Jonas | Aisha | Wednesday | | Book Linden Hall | Jonas | Today (once headcount received) | | Send revised budget, including catering quotes | Sam | Friday | | Send attendee survey after the event | Maya | Week after the event |
516 tokens in, 103 out · 1.9 s · $0.0001 · 1 message on Pro · answered by z-ai/glm-5.3-flash via Relace ·
GPT-6 Luna
Passed: Graded 5.0 of 5 on average (lowest 5).
## Decisions - **Venue:** Linden Hall; its quote is under budget. - **Event date:** 6 December. ## Action items - **Aisha:** Send the final headcount to Jonas by **Wednesday**. - **Jonas:** Book Linden Hall **after receiving the final headcount**; no deadline specified. - **Sam:** Send the revised budget, including catering quotes, by **Friday**. - **Maya:** Send the attendee survey **the week after the event**.
497 tokens in, 106 out · 1.5 s · $0.0001 · 1 message on Pro · answered by openai/gpt-6-luna via OpenAI ·
Claude Haiku 5.5
Passed: Graded 4.3 of 5 on average (lowest 4).
**Decisions** 1. Venue: Linden Hall, since its quote came in under budget. 2. Date: 6 December stays. **Action items** 1. Send final headcount to Jonas. Owner: Aisha. Deadline: Wednesday. 2. Book Linden Hall. Owner: Jonas. Deadline: today (once the headcount is in). 3. Send revised budget, including catering quotes. Owner: Sam. Deadline: Friday. 4. Send attendee survey after the event. Owner: Maya. Deadline: the week after the event.
704 tokens in, 189 out · 1.3 s · $0.0002 · 1 message on Pro · answered by anthropic/claude-haiku-5.5 via Anthropic ·
DeepSeek V4.1 Flash
Passed: Graded 4.3 of 5 on average (lowest 4).
**Decisions** - Venue: Linden Hall (its quote came in under budget). - Event date: stays 6 December. **Action items** | Action | Owner | Deadline | |---|---|---| | Get final headcount to Jonas | Aisha | Wednesday | | Book the venue | Jonas | Today (once headcount received) | | Send revised budget (including catering quotes) | Sam | Friday | | Send attendee survey | Maya | The week after the event |
541 tokens in, 102 out (3 of them reasoning) · 2.4 s · $0.0002 · 1 message on Pro · answered by deepseek/deepseek-v4.1-flash via Together ·
GLM 5.3
Passed: Graded 4.3 of 5 on average (lowest 4).
**Decisions** 1. Book Linden Hall as the venue (came in under budget). 2. Event date remains 6 December. **Action items** | Action | Owner | Deadline | |---|---|---| | Book Linden Hall | Jonas | Today (once headcount received) | | Send final headcount to Jonas | Aisha | Wednesday | | Send revised budget, including catering quotes | Sam | Friday | | Send attendee survey | Maya | The week after the event |
515 tokens in, 101 out · 1.5 s · $0.0002 · 1 message on Pro · answered by z-ai/glm-5.3 via Baidu ·
Gemini 3.8 Flash
Passed: Graded 4.3 of 5 on average (lowest 4).
**Decisions** * **Venue:** Linden Hall was chosen as the venue. * **Date:** The event date remains December 6. --- **Action Items** * **Provide final headcount to Jonas** * **Owner:** Aisha * **Deadline:** Wednesday * **Book the venue (Linden Hall)** * **Owner:** Jonas * **Deadline:** Once final headcount is received (Wednesday/after headcount arrives) * **Send revised budget (including catering quotes)** * **Owner:** Sam * **Deadline:** Friday * **Send attendee survey** * **Owner:** Maya * **Deadline:** The week after the event
530 tokens in, 146 out · 3.2 s · $0.0009 · 1 message on Pro · answered by google/gemini-3.8-flash via Google ·
Claude Haiku 4.5
Passed: Graded 5.0 of 5 on average (lowest 5).
# Decisions - **Venue**: Linden Hall (under budget) - **Event date**: 6 December # Action Items | Action | Owner | Deadline | |--------|-------|----------| | Provide final headcount | Aisha | Wednesday | | Book venue | Jonas | (after headcount received) | | Send revised budget with catering quotes | Sam | Friday | | Send attendee survey | Maya | Week after event |
535 tokens in, 106 out · 1.7 s · $0.0011 · 1 message on Pro · answered by anthropic/claude-haiku-4.5 via Amazon Bedrock ·
DeepSeek V4 Pro
Passed: Graded 4.3 of 5 on average (lowest 4).
## Decisions 1. **Venue selected**: Linden Hall (came in under budget) 2. **Event date confirmed**: December 6 ## Action Items | Owner | Action | Deadline | |-------|--------|----------| | Jonas | Book Linden Hall (once headcount received) | Today | | Aisha | Send final headcount to Jonas | Wednesday | | Sam | Send revised budget, including catering quotes | Friday | | Maya | Send attendee survey | Week after the event |
513 tokens in, 235 out (132 of them reasoning) · 5.4 s · $0.0011 · 1 message on Pro · answered by deepseek/deepseek-v4-pro-0813 via Wafer ·
GPT-6.1 Sol
Passed: Graded 4.7 of 5 on average (lowest 4).
## Decisions | Decision | Owner | Deadline/date | |---|---|---| | Use Linden Hall as the venue. | Maya confirmed; Jonas to book | Booking depends on final headcount | | Keep the event date as 6 December. | Maya confirmed | 6 December | ## Action items | Action item | Owner | Deadline | |---|---|---| | Send final headcount to Jonas. | Aisha | Wednesday | | Book Linden Hall once the final headcount is received. | Jonas | Not confirmed; offered to book today, pending headcount | | Send the revised budget, including catering quotes. | Sam | Friday | | Send the attendee survey. | Maya | The week after the event |
499 tokens in, 153 out · 2.3 s · $0.0013 · 1 message on Pro · answered by openai/gpt-6.1-sol via OpenAI ·
Gemini 4 Argon
Passed: Graded 4.3 of 5 on average (lowest 4).
**Decisions** * **Venue:** Linden Hall * **Event Date:** 6 December **Action Items** * **Provide final headcount to Jonas** — *Owner:* Aisha | *Deadline:* Wednesday * **Book Linden Hall** — *Owner:* Jonas | *Deadline:* Once headcount is received (Wednesday) * **Send revised budget (including catering quotes)** — *Owner:* Sam | *Deadline:* Friday * **Send attendee survey** — *Owner:* Maya | *Deadline:* The week after 6 December
750 tokens in, 116 out · 2.9 s · $0.0013 · 1 message on Pro · answered by google/gemini-4-argon via Google AI Studio ·
Mistral Large 4
Passed: Graded 4.7 of 5 on average (lowest 4).
**Decisions** 1. **Venue:** Linden Hall (approved; quote came in under budget). 2. **Date:** 6 December (confirmed). **Action Items** - **Aisha:** Send final headcount to Jonas — **Deadline:** Wednesday. - **Sam:** Send revised budget (including catering quotes) — **Deadline:** Friday. - **Jonas:** Book Linden Hall venue — **Deadline:** Upon receiving the headcount (he indicated he can book immediately once he has it). - **Maya:** Send attendee survey — **Deadline:** The week after the event.
506 tokens in, 1,295 out (1,318 of them reasoning) · 14.3 s · $0.0031 · 1 message on Pro · answered by mistralai/mistral-large-4-0 via Mistral ·
Claude Sonnet 5
Passed: Graded 4.3 of 5 on average (lowest 4).
## Decisions 1. **Venue:** Linden Hall selected (came in under budget). 2. **Event date:** Confirmed as 6 December. ## Action Items | Action | Owner | Deadline | |---|---|---| | Provide final headcount | Aisha | Wednesday | | Book the venue (Linden Hall) | Jonas | Today (pending headcount from Aisha) | | Send revised budget (incl. catering quotes) | Sam | Friday | | Send attendee survey | Maya | Week after the event |
701 tokens in, 192 out (9 of them reasoning) · 2.8 s · $0.0033 · 1 message on Pro · answered by anthropic/claude-sonnet-5 via Claude Platform on AWS ·
GPT-6 Sol
Passed: Graded 4.3 of 5 on average (lowest 4).
**Decisions** - Use Linden Hall as the venue; its quote is under budget. - Keep the event date as 6 December. **Action items** | Action | Owner | Deadline | |---|---|---| | Send the final headcount to Jonas | Aisha | Wednesday | | Book Linden Hall, once the final headcount is available | Jonas | Today, if the headcount is received | | Send the revised budget, including catering quotes | Sam | Friday | | Send the attendee survey | Maya | The week after the event |
497 tokens in, 243 out (125 of them reasoning) · 3.5 s · $0.0034 · 1 message on Pro · answered by openai/gpt-6-sol via OpenAI ·
Grok 4.7
Passed: Graded 4.3 of 5 on average (lowest 4).
**Decisions** - Venue is Linden Hall (quote came in under budget). - Event date stays 6 December. **Action items** - Aisha: send final headcount to Jonas by Wednesday. - Jonas: book Linden Hall once the headcount is in (ready to book the same day). - Sam: send the revised budget, including catering quotes, by Friday. - Maya: send the attendee survey the week after the event.
1,727 tokens in, 383 out (289 of them reasoning) · 5.3 s · $0.0036 · 1 message on Pro · answered by x-ai/grok-4.7 via xAI ·
Claude Sonnet 5.5
Passed: Graded 4.7 of 5 on average (lowest 4).
**Decisions** 1. Venue: Linden Hall (quote came in under budget). 2. Event date stays 6 December. **Action items** | # | Action | Owner | Deadline | |---|--------|-------|----------| | 1 | Send final headcount to Jonas | Aisha | Wednesday | | 2 | Book Linden Hall (after receiving headcount) | Jonas | Not stated; he offered to book once he has the headcount, so effectively Wednesday or shortly after | | 3 | Send revised budget, including catering quotes | Sam | Friday | | 4 | Send attendee survey | Maya | The week after the event | Note: Jonas's booking deadline wasn't explicitly set, so you may want to confirm it.
705 tokens in, 255 out · 2.2 s · $0.0043 · 1 message on Pro · answered by anthropic/claude-sonnet-5.5 via Anthropic ·
Kimi K3
Passed: Graded 4.0 of 5 on average (lowest 3).
## Decisions 1. **Venue: Linden Hall** — booked (quote came in under budget). 2. **Event date confirmed: 6 December** (unchanged). ## Action Items | # | Action | Owner | Deadline | |---|--------|-------|----------| | 1 | Send final headcount to Jonas | Aisha | Wednesday | | 2 | Book Linden Hall (after receiving headcount) | Jonas | Today, pending headcount | | 3 | Send revised budget, including catering quotes | Sam | Friday | | 4 | Send attendee survey after the event | Maya | The week after the event |
586 tokens in, 147 out (3 of them reasoning) · 3.4 s · $0.0044 · 1 message on Pro · answered by moonshotai/kimi-k3 via Fireworks ·
Gemini 3.1 Pro
Passed: Graded 4.3 of 5 on average (lowest 4).
**Decisions** * The venue will be Linden Hall. * The event date remains 6 December. **Action Items** * **Provide final headcount to Jonas:** Aisha (Deadline: Wednesday) * **Book the venue:** Jonas (Deadline: Upon receiving headcount on Wednesday) * **Send revised budget including catering quotes:** Sam (Deadline: Friday) * **Send attendee survey:** Maya (Deadline: The week after the event)
530 tokens in, 589 out (490 of them reasoning) · 7.6 s · $0.0081 · 1 message on Pro · answered by google/gemini-3.1-pro-preview via Google ·
Claude Opus 5.5
Passed: Graded 4.3 of 5 on average (lowest 4).
**Decisions** 1. **Venue:** Linden Hall, since its quote came in under budget. 2. **Date:** The event stays on 6 December. **Action items** | # | Action | Owner | Deadline | |---|--------|-------|----------| | 1 | Send the final headcount to Jonas | Aisha | Wednesday | | 2 | Book Linden Hall | Jonas | Once he has the headcount (after Wednesday; no firm date set) | | 3 | Send the revised budget, including catering quotes | Sam | Friday | | 4 | Send the attendee survey | Maya | The week after the event (the week after 6 December) | **Note:** Jonas offered to book "today," but he needs the headcount first, so the booking depends on Aisha's Wednesday delivery. You may want to set a firm booking date.
703 tokens in, 416 out (133 of them reasoning) · 5.8 s · $0.0118 · 1 message on Pro · answered by anthropic/claude-opus-5.5 via Claude Platform on AWS ·
GPT-6 Astra
Passed: Graded 4.7 of 5 on average (lowest 4).
## Decisions | Decision | Owner | Deadline / date | |---|---|---| | Use Linden Hall as the venue; its quote is under budget. | Maya / group | Not specified | | Keep the event date unchanged. | Maya / group | 6 December | ## Action items | Action | Owner | Deadline | |---|---|---| | Send the final headcount to Jonas. | Aisha | Wednesday | | Book Linden Hall. | Jonas | After receiving the final headcount; offered to book today if available | | Send the revised budget, including catering quotes. | Sam | Friday | | Send the attendee survey. | Maya | The week after the event |
497 tokens in, 146 out · 3.3 s · $0.0123 · 1 message on Pro · answered by openai/gpt-6-astra via OpenAI ·
Claude Fable 5.1
Passed: Graded 4.0 of 5 on average (lowest 3).
**Decisions** 1. Venue: Linden Hall (quote came in under budget). 2. Event date stays 6 December. **Action items** | Action | Owner | Deadline | |---|---|---| | Send final headcount to Jonas | Aisha | Wednesday | | Book Linden Hall (once headcount received) | Jonas | After Wednesday (intended to book same day headcount arrives) | | Send revised budget, including catering quotes | Sam | Friday | | Send attendee survey | Maya | Week after the event (week of 8 December) |
703 tokens in, 192 out · 5.6 s · $0.0182 · 1 message on Pro · answered by anthropic/claude-fable-5.1 via Anthropic ·
More summarization prompts
The other summarization prompts, each with every model's reply, and the results across all five.
Questions
Which AI does best on “Decisions and action items from a meeting”?
All 20 models passed this summarization prompt's check (rubric). The cheapest reply that passed was GLM 5.3 Flash's, at $0.000088; the fastest, Claude Haiku 5.5's in 1.3 s. The dearest reply, Claude Fable 5.1's, cost 208 times as much ($0.0182).
What does a reply to “Decisions and action items from a meeting” cost?
Through the models' APIs, what OpenRouter charged us ran from $0.000088 (GLM 5.3 Flash) to $0.0182 (Claude Fable 5.1) for this prompt. In llmwise you don't pay by the token: a reply like these counts as one message on Pro, whichever model answers.
Claude, GPT, Gemini, DeepSeek, Grok, Kimi, GLM, and Mistral, in one chat.
See what a message costs before you send it. Free is 5 messages to try; sign in with an email link, no password or card.