Tutorials

Replay Lab Tutorial

Use historical traces to evaluate routing and model policy changes before rollout.

11 minUpdated 2026-02-15
Summary

Use historical traces to evaluate routing and model policy changes before rollout.

5 deep-dive sections1 code samples
Quick Start
  1. Start from your current production prompt/request.
  2. Run the exact tutorial flow step-by-step once.
  3. Measure impact in Usage before rollout.
  4. Promote only when quality/cost/reliability metrics match target.

What Replay Lab does

Replay Lab simulates historical request traffic against your current policy to estimate impact before you change production behavior.

  • Cost deltas
  • Latency deltas
  • Reliability and success-rate deltas

Replay flow

Historical traces -> simulation -> decision
1
Collect traces
Recent request history from usage logs
2
Simulate
Run policy/model alternatives
3
Analyze
Projected deltas for cost, latency, quality
4
Decide
Apply or reject policy update

Endpoint

MethodPathParameters
POST/api/v1/optimization/replaydays (1..90), sample_size (20..300)

API call

curl -X POST "https://llmwise.ai/api/v1/optimization/replay?days=30&sample_size=120" \
  -H "Authorization: Bearer mm_sk_YOUR_KEY"

Operating cadence

  • Weekly for steady traffic
  • Daily when provider/model behavior shifts
  • Before every major routing policy change
Do not skip this

Do not ship large policy changes directly to production without Replay. Replay is your low-risk validation gate.

Regression testing tutorialRate limits and reliabilityAuto routing and optimization
Docs Assistant

ChatKit-style guided help

Product-scoped assistant for LLMWise docs and API usage. It does not answer unrelated topics.

Sign in to ask implementation questions and get runnable snippets.

Sign in to use assistant
Previous
Mesh Mode Tutorial (Failover Routing)
Next
Prompt Regression Testing Tutorial