AIAI agentsStory 05
NVIDIA shows how to turn agent failures into regression tests
What changedNVIDIA published a November 4 walkthrough for improving agents using recorded failures rather than generic model rankings. NeMo Compass examines traces, NeMo Eval Author helps create evaluations, and developers change the surrounding agent instructions or code before testing again.
The useful part
Why it matters
Takeaway: Teams building agents should convert real failures into observable tests and protect held-out cases from the repair loop.
Keep in mind
Good to know
This is a worked tutorial, not proof that one harness change generalizes to every production agent.
Evidence