What is sandbox testing vs. live testing?
Two environments for evaluating an agent: controlled sandbox and real production traffic.
Sandbox testing and live testing are two environments for evaluating an AI agent. Sandbox testing runs real customer questions against the agent in a controlled setting; live testing evaluates it on real conversations in production.
Both should use genuine customer queries, including complex, vague, edge-case, sensitive, and multilingual scenarios, to get an accurate read on business performance and conversation quality.
See it on SupportWire
Related terms
- Continuous improvement loop A structured cycle: train, test, deploy, analyze, repeat. Systems that learn by default.
- Lightweight governance A low-friction way to keep iterating on an AI agent without bureaucracy.
- Proof of concept A focused evaluation of one strongest-fit agent in a realistic setting, not a multi-vendor bake-off.
- Human involvement rate How often humans step in, and whether those moments are genuinely high value.