// tools/api-latency-comparison
Inference API Latency Comparison
Warm generation times across 6 providers — this is the latency your users see once the model is loaded, not the cold start. For cold start data, see the Cold Start Benchmarks.
Single image · 1024×1024 · warm requests only · 2026-06-01
4-step distilled Flux · 4 steps
⚡
Fastest for Flux Schnell: Runware — 800ms median · 1.8s P95
Fastest raw inference. Purpose-built high-throughput stack.
fastest
pricing →RunwareInference API
P50 — median request
800ms
P95 — worst 5% of requests
1.8s
Fastest raw inference. Purpose-built high-throughput stack.
Optimized CUDA kernels, lowest latency on market for Flux Schnell.
fal.aiInference API
P50 — median request
1.8s
P95 — worst 5% of requests
3.5s
Widest model coverage, consistent warm fleet.
Novita AIInference API
P50 — median request
2.0s
P95 — worst 5% of requests
4.0s
Competitive on Flux and SDXL pricing.
Together AIInference API
P50 — median request
2.5s
P95 — worst 5% of requests
5.0s
Competitive pricing, free tier for new devs.
Shared fleet — usually warm for popular models.
ReplicateInference API
P50 — median request
2.5s
P95 — worst 5% of requests
5.0s
Largest model catalog including community models.
RunflowComfyUI ManagedComfyUI overhead included
P50 — median request
3.5s
P95 — worst 5% of requests
7.0s
Enterprise SLA, custom ComfyUI workflows, LoRA hot-swapping, multi-step pipelines.
Single-model ComfyUI workflow. Complex pipelines add ~0s marginal overhead.
Runflow runs ComfyUI workflows — latency includes workflow orchestration overhead. For complex multi-step pipelines (LoRA + upscale + post-process), this overhead becomes proportionally negligible.
Flux Schnell — side-by-side comparison
| Provider | P50 | P95 | Type |
|---|---|---|---|
| ★Runware | 800ms | 1.8s | Inference API |
| fal.ai | 1.8s | 3.5s | Inference API |
| Novita AI | 2.0s | 4.0s | Inference API |
| Together AI | 2.5s | 5.0s | Inference API |
| Replicate | 2.5s | 5.0s | Inference API |
| Runflow | 3.5s | 7.0s | ComfyUI Managed |
Reading these numbers
- • P50 is the median — half of requests are faster, half are slower. This is your "typical" user experience.
- • P95 is what 95% of requests are under. Your slowest 5% users see this. Design SLAs around P95.
- • Runflow is a ComfyUI platform — latency includes workflow scheduling overhead. It excels for complex multi-step pipelines (LoRA + upscale + post-processing) where that overhead is proportionally negligible.
- • For latency-sensitive UX (under 3s): Runware or fal.ai for Flux Schnell. Both <2s median warm.
- • For background/batch jobs: latency is irrelevant — optimize for cost per image instead.
Estimates from provider documentation and community benchmarks — not independent lab measurements · Cold start benchmarks →