Tested prompt · Agents and tool use

Book a meeting from a sentence: every AI model's reply, tested

We sent this everyday agents and tool use prompt to all 16 models in llmwise, the same way the app sends a message, and checked every reply the same way. Here's each one as it came, with whether it passed, what it cost and how long it took.

Start free See pricing

Based on 16 of our test runs on September 28, 2026, through OpenRouter with the app's own prompt and settings. Updated September 28, 2026.

Short answer

15 of 16 models passed this agents and tool use prompt's check (tool calls); GLM 5.3 Flash didn't. The cheapest reply that passed was GPT-6 Luna's, at $0.000089; the fastest, Claude Haiku 4.5's in 1.2 s. The dearest reply, Claude Fable 5.1's, cost 147 times as much ($0.0131).

The prompt, as sent, and its check

Checked by tool calls, the same way for every model.

Book a meeting from a sentence (everyday)

You can call these tools:
- create_event(title: string, start: string in YYYY-MM-DDTHH:MM, duration_minutes: number, attendees: array of email addresses): adds an event to the user's calendar and invites the attendees.
- find_contact(name: string): looks up a contact's email address.

Today is Monday, 5 October 2026. The user's contacts include Priya Shah <priya@northwind.test>.
User: Put 30 minutes with Priya on my calendar this Thursday at 3pm to go over the Q4 plan.

Reply with only a JSON object, {"calls": [{"tool": "<name>", "arguments": {...}}]}, listing the tool calls to make now. If no call is right yet, reply {"calls": []}.

It must make exactly these calls, and no others:

[
  {
    "arguments": {
      "attendees": [
        "priya@northwind.test"
      ],
      "duration_minutes": 30,
      "start": "2026-10-08T15:00",
      "title": {
        "contains": "Q4"
      }
    },
    "tool": "create_event"
  }
]

Exactly what this prompt's replies are checked against, with every other prompt of our test runs.

Every model's result

All 16 models on this prompt, in catalog order.

Every model's reply to “Book a meeting from a sentence”
Model	Result	Cost	Time	Reply
Claude Fable 5.1Anthropic	Passed: Made the 1 expected call.	$0.0131	2.5 s	87 tokens
Claude Opus 5.5Anthropic	Passed: Made the 1 expected call.	$0.0059	7.2 s	87 tokens
Claude Sonnet 5.5Anthropic	Passed: Made the 1 expected call.	$0.0026	1.5 s	89 tokens
Claude Sonnet 5Anthropic	Passed: Made the 1 expected call.	$0.0037	5.7 s	86 tokens
Claude Haiku 4.5Anthropic	Passed: Made the 1 expected call.	$0.0010	1.2 s	99 tokens
GPT-6 AstraOpenAI	Passed: Made the 1 expected call.	$0.0077	1.9 s	55 tokens
GPT-6 SolOpenAI	Passed: Made the 1 expected call.	$0.0021	1.9 s	57 tokens
GPT-6 LunaOpenAI	Passed: Made the 1 expected call.	$0.000089	1.2 s	53 tokens
Gemini 3.1 Pro (preview)Google	Passed: Made the 1 expected call.	$0.0068	7.1 s	110 tokens
Gemini 3.8 FlashGoogle	Passed: Made the 1 expected call.	$0.0012	6.3 s	70 tokens
DeepSeek V4.1 FlashDeepSeek	Passed: Made the 1 expected call.	$0.00023	1.4 s	65 tokens
DeepSeek V4 ProDeepSeek	Passed: Made the 1 expected call.	$0.00026	3.5 s	65 tokens
Grok 4.7xAI	Passed: Made the 1 expected call.	$0.0048	9.1 s	59 tokens
Kimi K3Moonshot	Passed: Made the 1 expected call.	$0.0031	1.6 s	75 tokens
GLM 5.3Z.ai	Passed: Made the 1 expected call.	$0.00056	1.7 s	63 tokens
GLM 5.3 FlashZ.ai	Failed: Expected create_event(attendees: ["priya@northwind.test"], duration_minutes: 30, start: "2026-10-08T15:00", title: {"contains":"Q4"}); got find_contact(name: "Priya Shah"); create_event(title: "Q4 plan review", start: "2026-10-08T15:00", duration_minutes: 30, attendees: ["priya@northwind.test"]).	$0.00016	6.8 s	79 tokens

Cost: what OpenRouter charged us for the reply. Time: from sending to the whole reply. Reply: its length in tokens, thinking not counted. In llmwise you pay per message, not per token: each of these replies counted as one message on Pro.