← Glossary / 91 of 100
n. / Quality and evaluation
Human feedback
Definition
Human feedback is ratings, corrections or comments from people on an AI system's outputs, from end users or from internal reviewers.
In more detail
A thumbs up or down from users is cheap but noisy. Expert review is slower and more reliable. Both are most useful when attached to the trace of the run they refer to, so you can see why an answer was rated badly. Reviewed examples become evaluation data and show whether automatic judges agree with people.
Example
A user clicks thumbs down on an answer. The click is stored as a score on that run, and a reviewer later reads the trace and writes the corrected answer.
