Halberd Capital: One agent was retrieving a stale document set.

The problem

Two research agents from different vendors were producing conflicting summaries of the same filings.

What we did

Run records from both were normalised into one view so the same task could be compared across vendors.

The result

One agent was retrieving a stale document set. The disagreement was a data problem, not a model problem.

We were about to run an evaluation exercise on the wrong question entirely.