How we review an assistant
Five things we check in every assistant.
We look at usability the way ISO 9241-11 describes it: can people get the task done, with reasonable effort, and come away satisfied? Automated runs set a baseline. Sessions with real people show the effort, understanding and trust behind the result.
This is experience testing, not functional QA. An assistant can work exactly as specified and still leave a customer stuck.
- Discover
- Can people tell what the assistant can do, and when to use it?
- Ask
- Does it understand plain language and the context a customer has already given?
- Answer
- Is the answer right, clear and quick enough to act on?
- Trust
- Do people trust it the right amount? We compare how confident they feel with whether the answer was correct.
- Escape
- Can people reach a person, make a complaint or ask for hardship help when they need to?