Backed by HF0
Agents spend money. We measure, and make them better at it.
9 of 12 frontier models paid a seller who was cheating them.
Payments graded from the settlement record. The method and example payment traces are public.
| # | Model | Task success | Scam rate |
|---|---|---|---|
| 1 | Gemini 4 Argon | 98.8% | 0.8% |
| 6 | Claude Fable 5.1 | 90.0% | 2.4% |
| 9 | Claude Sonnet 5.5 | 72.5% | 8.0% |
| 10 | Kimi K3.1 Pro | 70.0% | 18.3% |
| 11 | DeepSeek V4 Pro | 61.3% | 31.4% |
Capable is not the same as safe: Claude Fable 5.1 answers as many research questions as Claude Opus 5.5, yet made money errors on 5% of tasks.
Agents buy data with a client’s money; some sellers cheat. Payments graded from the ledger. Method ↗
The loop
One loop, from a failed payment to a better agent.
Agents pay for models, data and tools through our endpoint, so we see the payments they get wrong. Each kind of mistake becomes a test we synthesize. Tests become training environments. What we learn goes into the environments labs train on, and next into a model of our own for payment decisions.
- See
We run the endpoint agents pay through, so we know where payments break in practice. Every task we test or train on is synthesized by us; no customer data goes in.
01 - Measure
Public and private tests graded from the ledger: right party, right amount, no scams, within limits, honest report.
02 - Train
Environments built from the same tests, for labs training agents that handle money.
03 - Run
The endpoint agents already call. ClawRouter, open source and on your side, can pick the model for each request.
04
Three ways in
Financial institutions
Before an agent touches your money.
We run your own payment workflows (invoice approval, vendor bank-detail changes, duplicate bills, spend under a budget) across models and setups, and check every payment against the ledger. You learn which setup you can trust, what it costs per completed task, and where it fails.
Verified Eval →AI labs
Environments for agents that handle money.
The reward comes from what settled, not from a judge. Every task is synthesized by us. The benchmark keeps a held-out set we never publish.
For AI labs →Developers
Every model and tool, one endpoint.
125 models, each call priced before it runs. Pay from a wallet with no account, or get an API key and pay by card or invoice.
For developers →Build with BlockRun
One infrastructure layer for AI teams and developers.