ReplayOpen source coming soon
Replay your agent’s work
with a new model.
Test a model swap or upgrade against recorded workflows. Replay uses the original context and tool results to show where the candidate’s actions change.
A retry becomes a second refund attempt.
Synthetic example
A $25 partial refund hits a network timeout. Both models retry the request.
Recorded tool resultnetwork_timeout · outcome unknown
- Current model
Reuses the original key.
create_refundamount_cents: 2500idempotency_key:"refund-1842"- Candidate model
Generates a fresh key.
create_refundamount_cents: 2500idempotency_key:"refund-1842-retry"
HOLD · Changed writeA fresh key can create another $25 refund. ModelSpar flags the change and stops the replay at the unmatched call.
By the time you rerun the case, the refund may already be complete. Replaying the recorded timeout brings the retry back into the test.
How it works
1. Your recorded runs
You provide traces from your current agent and the model you want to test.
2. A replay with the new model
ModelSpar runs the candidate using recorded context and tool results. Production tools remain disconnected.
3. A comparison report
The report shows changed actions, missing steps, and gaps in the evidence.
Technical details
The pilot accepts OpenInference JSONL, OTLP/JSON, and Datadog traces. Candidate models use an OpenAI-compatible endpoint with function tools. Existing candidate traces can also be compared directly.
You review which tools change state and approve the context sent to the candidate endpoint. Results cover the recorded cases and configured actions. Answer quality and broader safety require additional evaluation.