// tools/cold-start-benchmarks
Cold Start Benchmark Dashboard
Cold starts are the silent UX killer in AI image APIs. This tool shows P50/P95 cold start times across 8 providers and calculates the real latency impact on your users at your traffic level.
Latency ranges from published provider documentation and architecture analysis · June 2026 · Read the full benchmark article →
UX Impact Calculator
fal.ai
Avg latency565ms
Cold starts/day150
Keep-warm/moincluded
Replicate
Avg latency4.0s
Cold starts/day150
Keep-warm/moincluded
RunPod (Serverless)
Avg latency8.0s
Cold starts/day150
Keep-warm/mo$245
Sort by:
fastest
fal.aiManaged API
Rareavg latency @ 15% cold
565ms
Cold start P501.5s
Cold start P955.0s
Warm latency400ms
fal.ai routes to pre-warmed instances; cold starts occur mainly on first request after extended inactivity
Keep-warm: included
Together AIManaged API
Rareavg latency @ 15% cold
810ms
Cold start P502.0s
Cold start P958.0s
Warm latency600ms
Shared fleet model means popular models are almost always warm
Keep-warm: included
RunflowManaged API
Rareavg latency @ 15% cold
1.1s
Cold start P503.0s
Cold start P9510.0s
Warm latency800ms
Runflow runs a warm fleet of GPUs. Cold starts occur rarely (new deployments or after extended inactivity) and are significantly faster than serverless alternatives.
Keep-warm: included
ModalManaged API
Frequent (no KW)avg latency @ 15% cold
1.0s
Cold start P505.0s
Cold start P9515.0s
Warm latency300ms
Serverless — containers shut down after idle. Use keep_warm=1 to eliminate cold starts at ~$0.35/hr GPU idle cost
Keep-warm: $252/mo
Modal keep_warm parameter: set container count to keep loaded. Cost ≈ GPU idle rate.
ReplicateManaged API
Occasionalavg latency @ 15% cold
4.0s
Cold start P5015.0s
Cold start P9545.0s
Warm latency2.0s
Replicate caches weights between runs; cold starts occur after periods of inactivity or on new deployments
Keep-warm: included
Salad CloudManaged API
Occasionalavg latency @ 15% cold
3.5s
Cold start P5015.0s
Cold start P9545.0s
Warm latency1.5s
Consumer-edge nodes may have variable boot times. Salad maintains a pool of pre-started instances for popular workloads.
Keep-warm: included
RunPod (Serverless)Serverless GPU
Frequent (no KW)avg latency @ 15% cold
8.0s
Cold start P5045.0s
Cold start P95120.0s
Warm latency1.5s
RunPod serverless spins up a full container on cold start: GPU allocation + container boot + model load = 30–120s
Keep-warm: $245/mo
Set min workers > 0 in RunPod Serverless to keep containers warm. Cost = RTX 4090 community rate.
Vast.ai (DIY)Self-Hosted
Always (no KW)avg latency @ 15% cold
9.8s
Cold start P5060.0s
Cold start P95150.0s
Warm latency1.0s
DIY self-hosted: you control start/stop. Cold start = full container boot + GPU driver init + model load. Typically 60–150s.
Keep-warm: $173/mo
Run an always-on container. Cost = RTX 3090 median on Vast.ai (~$0.24/hr)
Key takeaways
- • fal.ai and Together AI keep popular models warm automatically — cold starts are rare and under 5 seconds.
- • Runflow runs a warm GPU fleet as part of the platform — cold starts are rare (<3s P50) with no keep-warm surcharge.
- • Replicate caches model weights; cold starts still happen but are mitigated compared to DIY solutions.
- • RunPod Serverless and DIY can hit 120 seconds on cold start — a hard UX wall for real-time applications.
- • For latency-sensitive APIs: keep-warm strategies add $7–$250/month but eliminate cold starts entirely.
- • For batch/background jobs: cold starts add zero user-visible latency — optimize purely for cost.
Latency ranges are estimates from provider documentation and architecture analysis, not independent measurements. Read the full benchmark article for methodology.