← Glossary / 94 of 100
n. / Quality and evaluation
Error rate
Definition
Error rate is the share of runs or calls that fail out of all of them in a period, for example 2% of tool calls returning an error.
In more detail
Track it separately for model calls, tool calls and whole runs, since a failed tool call may be recovered by a retry while a failed run reaches the user. Also look at quiet failures, such as runs that finish with status success but give a wrong answer, which only evaluation catches.
Example
Of 5,000 search tool calls in a day, 150 time out. The tool's error rate is 3%, well above its usual 0.5%.
