AIAgent evaluationStory 03
AgentHorizon catches plausible-looking computer-use failures
What changedA computer-use task can end on a convincing screen and still be wrong. AgentHorizon uses 1,373 instruction-trajectory pairs from 166 hours of recordings to test whether automatic judges catch hidden constraint violations and side effects across long tasks.
The useful part
Why it matters
A computer-use agent can look finished while silently publishing the wrong content or violating a user constraint; evaluation must inspect the trajectory.
Keep in mind
Good to know
The benchmark cannot guarantee generalization to every application.
Evidence