AIModel reliabilityStory 03
Long context still does not guarantee reliable memory
What changedStateBench tests whether a model can keep track of what is currently true after a sequence of dependent updates. Ai2 reports that tested frontier models struggle as those chains grow.
The useful part
Why it matters
Treat conversation history as context; verify live permissions, inventory and workflow state at the source before acting.
Keep in mind
Good to know
StateBench measures specific state-tracking tasks, not every long-document use case.
Evidence