Tested prompt · AI poem generator
A haiku on a bus window: every AI model's reply, tested
We ran this everyday AI poem generator test input on all 19 models in llmwise, the way the tool runs it, and checked every reply the same way. Here's each one as it came, with whether it passed, what it cost and how long it took.
Based on 19 of our test runs on , through OpenRouter with the tool's own instructions and the app's settings. Updated .
Short answer
17 of 19 models passed this AI poem generator test input's check (rubric); Claude Sonnet 5.5 and Claude Haiku 5.5 didn't. The cheapest reply that passed was GLM 5.3 Flash's, at $0.000071; the fastest, DeepSeek V4.1 Flash's in 0.9 s. The dearest reply, Claude Opus 5.5's, cost 179 times as much ($0.0126).
The prompt, as sent, and its check
Checked by rubric (graded), the same way for every model.
A haiku on a bus window (everyday)
Write an original poem about this, with a title on the first line. Keep every name and detail in it. Write it as a haiku. --- Autumn leaves stuck to the window of a city bus.
Sent with the AI poem generator's own instructions as the system prompt, as a free run of the tool sends them.
Graded on:
- Form: Apart from the title, it is exactly three lines of 5, 7 and 5 syllables.
- Image: It gives a precise image of the leaves on the bus window, and a feeling from it, without explaining it.
- Nothing extra: No commentary, notes or extra lines after the poem.
How the AI poem generator's test inputs are scored, with every model's results on all five.
Every model's result
All 19 models on this test input, in catalog order.
| Model | Result | Cost | Time | Reply |
|---|---|---|---|---|
| Claude Fable 5.1Anthropic | Passed: Graded 5.0 of 5 on average (lowest 5). | $0.0066 | 1.8 s | 32 tokens |
| Claude Opus 5.5Anthropic | Passed: Graded 5.0 of 5 on average (lowest 5). | $0.0126 | 7.3 s | 37 tokens |
| Claude Sonnet 5.5Anthropic | Failed: Graded 2.7 of 5 on average (lowest 1); but 68 words, over the 35 allowed. | $0.0028 | 2.5 s | 182 tokens |
| Claude Sonnet 5Anthropic | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0015 | 1.4 s | 48 tokens |
| Claude Haiku 5.5Anthropic | Failed: Graded 2.0 of 5 on average (lowest 1). | $0.000089 | 1.6 s | 78 tokens |
| Claude Haiku 4.5Anthropic | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.00054 | 1.0 s | 30 tokens |
| GPT-6 AstraOpenAI | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0052 | 10.2 s | 32 tokens |
| GPT-6.1 SolOpenAI | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0016 | 5.2 s | 27 tokens |
| GPT-6 SolOpenAI | Passed: Graded 4.3 of 5 on average (lowest 3). | $0.0015 | 4.3 s | 27 tokens |
| GPT-6 LunaOpenAI | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.000095 | 4.4 s | 32 tokens |
| Gemini 3.1 Pro (preview)Google | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0071 | 9.1 s | 18 tokens |
| Gemini 3.8 FlashGoogle | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.00019 | 1.1 s | 24 tokens |
| DeepSeek V4.1 FlashDeepSeek | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.00027 | 0.9 s | 22 tokens |
| DeepSeek V4 ProDeepSeek | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0037 | 33.6 s | 23 tokens |
| Grok 4.7xAI | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0082 | 14.6 s | 20 tokens |
| Kimi K3Moonshot | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.0014 | 5.8 s | 34 tokens |
| GLM 5.3Z.ai | Passed: Graded 4.7 of 5 on average (lowest 4). | $0.00025 | 1.8 s | 27 tokens |
| GLM 5.3 FlashZ.ai | Passed: Graded 4.3 of 5 on average (lowest 3). | $0.000071 | 1.2 s | 26 tokens |
| Mistral Large 4Mistral | Passed: Graded 4.3 of 5 on average (lowest 3). | $0.0092 | 60.8 s | 1,022 tokens |
Cost: what OpenRouter charged us for the reply. Time: from sending to the whole reply. Reply: its length in tokens, thinking not counted. In llmwise you pay per message, not per token: each of these replies counted as one message on Pro.
Every reply
The 2 replies that failed first, then the ones that passed, from the cheapest up.
Claude Haiku 5.5
Failed: Graded 2.0 of 5 on average (lowest 1).
Leaves on the bus glass pressed flat against the city window, riding along Let me check the syllables: "Leaves on the bus glass" (5), "pressed flat against the city" (7), "window, riding along" (5). Good.
497 tokens in, 78 out · 1.6 s · $0.0001 · 1 message on Pro · answered by anthropic/claude-haiku-5.5 via Anthropic ·
Claude Sonnet 5.5
Failed: Graded 2.7 of 5 on average (lowest 1); but 68 words, over the 35 allowed.
Leaves on the Bus Window Wet leaves on the glass, the city bus hums onward, gold pressed to the rain Syllable check: "Wet leaves on the glass" = 5. "the city bus hums onward" = the(1) cit-y(2) bus(1) hums(1) on-ward(2) = 7. "gold pressed to the rain" = 5. Correction, plain poem only: Leaves on the Bus Window Autumn leaves cling tight, the city bus rolling on, gold stamped on the glass
497 tokens in, 182 out · 2.5 s · $0.0028 · 1 message on Pro · answered by anthropic/claude-sonnet-5.5 via Anthropic ·
GLM 5.3 Flash
Passed: Graded 4.3 of 5 on average (lowest 3).
Wet leaves on the glass of the slow city bus ride— autumn holds on tight
377 tokens in, 28 out (2 of them reasoning) · 1.2 s · $0.0001 · 1 message on Pro · answered by z-ai/glm-5.3-flash via Friendli ·
GPT-6 Luna
Passed: Graded 4.7 of 5 on average (lowest 4).
Autumn Leaves on a City Bus Autumn leaves stick fast On the city bus window Gold blurs past the shops
368 tokens in, 306 out (274 of them reasoning) · 4.4 s · $0.0001 · 1 message on Pro · answered by openai/gpt-6-luna via OpenAI ·
Gemini 3.8 Flash
Passed: Graded 4.7 of 5 on average (lowest 4).
Glass Passengers Autumn leaves pressed flat, Stuck to a city bus pane, Riding through the rain.
374 tokens in, 24 out · 1.1 s · $0.0002 · 1 message on Pro · answered by google/gemini-3.8-flash via Google AI Studio ·
GLM 5.3
Passed: Graded 4.7 of 5 on average (lowest 4).
Stuck to the Window Yellow leaves cling fast, maple pressed against the glass— the bus rolls past town.
377 tokens in, 27 out · 1.8 s · $0.0003 · 1 message on Pro · answered by z-ai/glm-5.3 via Friendli ·
DeepSeek V4.1 Flash
Passed: Graded 4.7 of 5 on average (lowest 4).
Bus Window, October Autumn leaves pressed flat against the city bus glass, riding to somewhere
386 tokens in, 189 out (167 of them reasoning) · 0.9 s · $0.0003 · 1 message on Pro · answered by deepseek/deepseek-v4.1-flash via Together ·
Claude Haiku 4.5
Passed: Graded 4.7 of 5 on average (lowest 4).
Autumn Leaves on Glass Crimson leaves cling tight City bus speeds through the grey Seasons trapped in glass
386 tokens in, 30 out · 1.0 s · $0.0005 · 1 message on Pro · answered by anthropic/claude-haiku-4.5 via Azure ·
Kimi K3
Passed: Graded 4.7 of 5 on average (lowest 4).
Riding with the City Autumn leaves stuck fast to the window of the bus— city gold rides on
456 tokens in, 138 out (104 of them reasoning) · 5.8 s · $0.0014 · 1 message on Pro · answered by moonshotai/kimi-k3 via Wafer ·
Claude Sonnet 5
Passed: Graded 4.7 of 5 on average (lowest 4).
Bus Window Leaves Wet gold leaves pressed flat— city bus hums on its way, glass keeps autumn's map.
495 tokens in, 48 out · 1.4 s · $0.0015 · 1 message on Pro · answered by anthropic/claude-sonnet-5 via Anthropic ·
GPT-6 Sol
Passed: Graded 4.3 of 5 on average (lowest 3).
Window Passengers Autumn leaves stick fast To the city bus window As the wheels turn slow
368 tokens in, 226 out (199 of them reasoning) · 4.3 s · $0.0015 · 1 message on Pro · answered by openai/gpt-6-sol via OpenAI ·
GPT-6.1 Sol
Passed: Graded 4.7 of 5 on average (lowest 4).
Passengers of Autumn Autumn leaves cling fast To the city bus window Streetlights stain them gold
368 tokens in, 250 out (223 of them reasoning) · 5.2 s · $0.0016 · 1 message on Pro · answered by openai/gpt-6.1-sol via OpenAI ·
DeepSeek V4 Pro
Passed: Graded 4.7 of 5 on average (lowest 4).
Autumn Window Autumn leaves stuck to the window of a city bus, trembling with speed.
360 tokens in, 897 out (874 of them reasoning) · 33.6 s · $0.0037 · 1 message on Pro · answered by deepseek/deepseek-v4-pro-0813 via Wafer ·
GPT-6 Astra
Passed: Graded 4.7 of 5 on average (lowest 4).
Autumn on the City Bus Autumn leaves cling fast To the city bus window; Streets blur into rain.
368 tokens in, 136 out (104 of them reasoning) · 10.2 s · $0.0052 · 1 message on Pro · answered by openai/gpt-6-astra via OpenAI ·
Claude Fable 5.1
Passed: Graded 5.0 of 5 on average (lowest 5).
Passengers Wet leaves press their palms to the city bus window, riding without fare.
497 tokens in, 32 out · 1.8 s · $0.0066 · 1 message on Pro · answered by anthropic/claude-fable-5.1 via Google ·
Gemini 3.1 Pro
Passed: Graded 4.7 of 5 on average (lowest 4).
Passengers Wet autumn leaves stuck To the city bus window Riding through the rain
374 tokens in, 1,118 out (1,100 of them reasoning) · 9.1 s · $0.0071 · 1 message on Pro · answered by google/gemini-3.1-pro-preview via Google AI Studio ·
Grok 4.7
Passed: Graded 4.7 of 5 on average (lowest 4).
Pressed to the Glass Autumn leaves stuck fast to the city bus window gold against the glass
1,601 tokens in, 1,123 out (1,103 of them reasoning) · 14.6 s · $0.0082 · 1 message on Pro · answered by x-ai/grok-4.7 via xAI ·
Mistral Large 4
Passed: Graded 4.3 of 5 on average (lowest 3).
Autumn Leaves on Glass Autumn leaves are stuck To the city bus window Amber on the glass
372 tokens in, 4,274 out (3,252 of them reasoning) · 60.8 s · $0.0092 · 1 message on Pro · answered by mistralai/mistral-large-4-0 via Mistral ·
Claude Opus 5.5
Passed: Graded 5.0 of 5 on average (lowest 5).
Stowaways Autumn leaves stuck fast to a city bus window, rust-red stowaways
497 tokens in, 531 out (494 of them reasoning) · 7.3 s · $0.0126 · 1 message on Pro · answered by anthropic/claude-opus-5.5 via Google ·
More AI poem generator test inputs
The other AI poem generator test inputs, each with every model's reply, and the results across all five.
Questions
Which AI does best on “A haiku on a bus window”?
17 of 19 models passed this AI poem generator test input's check (rubric); Claude Sonnet 5.5 and Claude Haiku 5.5 didn't. The cheapest reply that passed was GLM 5.3 Flash's, at $0.000071; the fastest, DeepSeek V4.1 Flash's in 0.9 s. The dearest reply, Claude Opus 5.5's, cost 179 times as much ($0.0126).
What does a reply to “A haiku on a bus window” cost?
Through the models' APIs, what OpenRouter charged us ran from $0.000071 (GLM 5.3 Flash) to $0.0126 (Claude Opus 5.5) for this test input. In llmwise you don't pay by the token: a reply like these counts as one message on Pro, whichever model answers.
Claude, GPT, Gemini, DeepSeek, Grok, Kimi, GLM, and Mistral, in one chat.
See what a message costs before you send it. Free is 5 messages to try; sign in with an email link, no password or card.