Open vs Frontier
You can go further than Coinbase
Coinbase cut its AI bill nearly in half by moving routine work to open models. Specializing goes further. Below, the same support tickets run through Gemma (Google’s US open model, fine-tuned on Oumi) and Claude Sonnet (frontier API), side by side. Watch the cost tick up.
Head-to-head on 150 held-out tickets: fine-tuned Gemma 93.3% category / 72% priority, Claude 95.3% / 60%, at about a tenth of the cost. The run below replays real recorded calls on 24 of those tickets.
0 / 24 tickets
Our model
Fine-tuned Gemma · US open weights · on Oumi
- Category
- —
- Priority
- —
- Cost
- $0.000000
Claude Sonnet
Frontier model · external API
- Category
- —
- Priority
- —
- Cost
- $0.000000
Cost so far
How it was tested
150 tickets neither model had seen, the same instructions for both, exact-match grading. No AI judge, no partial credit.
The honest trade-off
Gemma reasons before it answers, so it’s slower here (3.9s vs 1.6s typical). For background routing that rarely matters.
Build your own
Describe the decision, and Oumi synthesizes data, fine-tunes a small model and scores it. Start building