AIResearch resultStory 05
Chemistry tests now show where AI reasoning breaks
What changedChemArgus scores every reasoning step and compares image-plus-text questions with controlled text descriptions to separate visual integration, propagation, and local reasoning failures. A low final answer score becomes a diagnosis of the broken step, which is more useful for scientific assistants than one pass-or-fail number.
The useful part
Why it matters
A low final answer score becomes a diagnosis of the broken step, which is more useful for scientific assistants than one pass-or-fail number.
Keep in mind
Good to know
This is a benchmark study; it does not show that the evaluated models can safely perform laboratory work.
Evidence