GpuLedger

Self-Host vs API Break-Even Calculator

Self-hosting looks cheap at 100% utilization. Nobody runs at 100% utilization.

Tokens per month (millions)

API cost per 1M tokens ($)

GPUs required

GPU cost per hour ($)

Throughput per GPU (tokens/sec)

Realistic utilization (%)

Ops / MLOps hours per month

Engineer hourly rate ($)

Cheaper option

API ($1,350/mo)

Utilization is the whole game. GPUs bill for wall-clock hours whether or not requests arrive, so a fleet sized for peak and idle most of the night can cost several times the API rate per token actually served.

Self-host total: $5,550
API total: $4,200
Effective self-host $/1M: $3.77
Fleet capacity: Sufficient (1472M served)

What you entered

Tokens per month (millions): 1400API cost per 1M tokens ($): 3GPUs required: 2GPU cost per hour ($): 2.5Throughput per GPU (tokens/sec): 800Realistic utilization (%): 35Ops / MLOps hours per month: 20Engineer hourly rate ($): 95

1
API cost at list price
1400M tokens × $3/M= $4,200
2
GPU cost (billed whether idle or busy)
2 × $3/hr × 730 hr= $3,650
3
Engineering / ops cost
20 hr × $95/hr= $1,900
4
Tokens actually served at 35% utilization
4205M capacity × 35%= 1472M tokens
5
Effective self-host cost per 1M tokens
$5,550 ÷ 1472M served= $4/M (API: $3/M)

Result

Cheaper option: API ($1,350/mo)

At 35% utilization self-hosting costs $5,550/month versus $4,200 on the API — the API is $1,350 cheaper. Your effective self-host rate is $3.77/1M against an API rate of $3.00/1M.

Self-hosted GPUs versus a token API by volume

GPUs bill by the hour whether busy or idle, so self-hosting only wins above a volume threshold. Rows use $5 per million API tokens, $2.50/GPU-hour, and 20 ops hours a month at $120.

Monthly tokens	GPUs	Utilisation	API cost	Self-host cost	Cheaper
100 M	2	40%	$500	$6,050	API
1,000 M	2	40%	$5,000	$6,050	API
1,000 M	2	80%	$5,000	$6,050	API
5,000 M	4	60%	$25,000	$9,700	Self-host
10,000 M	8	70%	$50,000	$17,000	Self-host

Self-host cost barely moves with volume while API cost scales linearly, which is the whole shape of the decision. The ops line is the one most estimates omit and the one that most often decides it.

A self-host versus API calculator compares running an open-weight model on your own GPUs against paying a provider per token, and it does so on the input that decides the answer: utilization. The case for self-hosting is almost always presented at full load — take a GPU's peak throughput, multiply by hours in a month, divide the rental cost by that token count, and the per-million rate looks dramatically better than any API. The flaw is that the calculation assumes the GPU is busy every second of every hour, and no production system is.

The model prices both sides honestly. API cost is simply monthly tokens at the published per-million rate — it scales perfectly with usage and costs nothing when idle. Self-hosting costs GPU hours multiplied by the hourly rate across all 730 hours in a month, because rented or owned accelerators bill for wall-clock time regardless of whether requests arrive, plus a monthly engineering allocation for the deployment, monitoring, upgrades, and incident response that a managed API absorbs on your behalf. The calculator then computes theoretical capacity from GPU count and per-GPU throughput, applies your realistic utilization percentage to get tokens actually served, and divides total self-host cost by that number to produce an effective cost per million tokens. Comparing that against the API rate is the honest apples-to-apples figure, and it is usually a shock: at 35% utilization your effective rate is nearly three times the theoretical one.

That gap is the whole point. Real traffic is bursty and diurnal — busy during business hours, near-idle overnight — so a fleet sized to handle peak load sits substantially idle most of the time. Reserved capacity, autoscaling GPU pools, and batching help but cannot eliminate it, because scaling GPUs up takes minutes and model weights must be loaded. The source research is explicit that this decision must model p50/p95 load, idle time, staffing, failures, and quantization rather than perfect utilization. Self-hosting genuinely wins in specific conditions: very high sustained volume, strict data-residency or privacy requirements that rule out third-party APIs, latency requirements that need local inference, or a smaller quantized model that serves your task acceptably at a fraction of the compute. Use this to find your true break-even volume and to test how sensitive the answer is to utilization before committing to an infrastructure programme. It prices compute and ops only, excluding model licensing, data-transfer, storage for weights, and the multi-week engineering effort to reach production quality.

Utilization turns a good deal into a bad one

Theoretical cost per token assumes the GPU never idles. At 35% utilization your effective cost per million tokens is nearly 3× the headline figure, because you pay for 730 hours and use the equivalent of about 255. Any self-hosting business case that doesn't state its utilization assumption is quoting a number that cannot be achieved.

When self-hosting genuinely wins

Sustained high volume that keeps GPUs busy; data residency or privacy rules that prohibit sending data to a third party; latency requirements that need inference close to the application; or a smaller quantized model that handles your task well at a fraction of the compute. Outside those conditions, per-token APIs usually win on total cost once idle time and engineering effort are counted.

Frequently asked questions

2 GPUs at $2.50/hr serving 1,400M tokens/month at 35% utilization — is that cheaper than a $3/M API?

GPUs cost 2 × $2.50 × 730 = $3,650, plus 20 ops hours at $95 = $1,900, totalling $5,550. The API would cost 1,400 × $3 = $4,200. The API wins by $1,350 — your effective self-host rate is $3.77/M against the API's $3.00/M, because idle GPUs still bill.

Why is my effective cost per token so much higher than the theoretical rate?

Because GPUs bill for wall-clock hours, not tokens. Two GPUs at 800 tokens/sec could theoretically produce 4,205M tokens a month, costing about $1.32/M. At 35% utilization you serve only 1,472M of that but still pay the full $5,550 — so your real rate is $3.77/M, nearly three times the full-load figure.

Can I just autoscale the GPUs down when idle?

Partly. GPU autoscaling is slower and coarser than CPU autoscaling because model weights must load into VRAM, which takes minutes, and cold requests during scale-up hit latency. You can recover some idle cost, but rarely all of it — model a realistic utilization rather than assuming scaling solves it.

How is this different from the GPU VRAM planner?

The VRAM planner answers whether a model physically fits on given hardware and what throughput to expect. This one takes that hardware as given and answers the financial question — whether running it yourself beats paying an API once idle time and engineering effort are counted.