Which LLM provider
wins on structured output?
We ran 125 real executions across 5 providers, 5 schema types, and 5 prompts each — using the Reliant reliability layer to measure success rate and latency for structured JSON output.
Speed leaderboard
Average latency across all 5 schema types. All providers achieved 100% validation success.
Latency heatmap
Average latency (ms) per provider per schema. Greener = faster.
| Schema | Anthropic | OpenAI | Gemini | Groq | Mistral |
|---|---|---|---|---|---|
| Contact Extraction | 526ms | 826ms | 1354ms | 340ms | 281ms |
| Sentiment Analysis | 554ms | 763ms | 2243ms | 291ms | 263ms |
| Ticket Classification | 405ms | 1158ms | 2419ms | 182ms | 237ms |
| Contract Extraction | 399ms | 764ms | 828ms | 188ms | 304ms |
| Product Extraction | 482ms | 869ms | 803ms | 274ms | 272ms |
What the data says
Patterns worth knowing before choosing your provider for structured output.
At 255ms average, Groq with Llama 3.3 70B is the fastest option by far. Ideal for latency-sensitive pipelines. Gemini 2.5 Flash averaged 1,529ms — over 6x slower.
271ms average latency, 100% success rate, and the lowest cost per token among all providers tested. For high-volume batch processing, Mistral Small is the smart choice.
With Reliant's validation and retry layer, every provider achieved 100% structured output success. Without it, raw success rates vary significantly — especially on complex schemas.
At 473ms, Claude Haiku is the fastest Anthropic model and delivers strong structured output quality. Best choice when you need Anthropic's reasoning with lower latency.
Gemini peaked at 2,419ms on Ticket Classification but was faster on simpler schemas like Product Extraction (803ms). Complex schemas with enum constraints slow it down more than others.
Configure Groq as fallback for Anthropic to get both quality and speed resilience. If your primary provider is slow or down, Reliant automatically routes to the next one.
How we ran this
Full transparency on the benchmark setup.
- →5 providers tested in parallel
- →5 schema types × 5 prompts = 25 per provider
- →125 total executions
- →Models: cheapest tier of each provider
- →Max 2 retries per execution
- →Concurrency: 3 parallel per batch
- →Contact Extraction (name, email, phone)
- →Sentiment Analysis (enum + confidence score)
- →Ticket Classification (priority + category enums)
- →Contract Extraction (numeric values + dates)
- →Product Info Extraction (mixed types + boolean)
Run your own benchmark
Use Reliant to test your schemas against every provider — with automatic validation, retry, and fallback.
Start free → reliant.ia.br