NotCheap MAKE EVERY CREDIT COUNT
Analyze Your Costs

Baseten, Modal and RunPod: GPU Serving Beyond Active Time

Share this guide

If you've ever self-hosted an open-source model on serverless GPUs, you know the budget surprise rarely comes from the inference itself. It comes from everything wrapped around it: container initialization, weights loading, cold-start latency, and warm-idle keepalives. For our realistic A100 80GB burst deployment, 100,000 requests yield runtime estimates of $272.35 on Modal, $266.00 on RunPod and $666.70 on Baseten, before persistent volume storage, network egress, and developer engineering time. Change your request pattern or runtime duration, and the cheapest provider can flip on you.

Sources checked 25 October 2026: Modal pricing, RunPod pricing, RunPod billing rules and Baseten pricing. USD before tax. Hardware availability, data center regions, and exact CPU/RAM combinations are worth confirming for your target architecture.

Choose a model and a resource allocation

Our reference workload serves an 8B parameter language model on a single A100 80GB GPU, with an average 500-token prompt and a 150-token response. We budget an average generation time of two seconds per request, with single-threaded sequential execution per replica. Treat these as capacity-planning inputs, not vendor benchmarks or SLA promises.

Modal bills the A100 80GB GPU at $0.000694 per second. Add one physical CPU core at $0.0000131/second and 32GiB of system host RAM at $0.00000222/GiB-second, and you land at $0.00077814/allocated second. Host system RAM is a separate thing from GPU VRAM, which is an easy one to mix up.

RunPod's Serverless A100 80GB console shows $0.00076/second, plus an hourly rate rounded to $2.72/hour. We use the displayed per-second tariff, but it's worth checking the effective rate on your own billing dashboard before you scale. RunPod meters from worker startup until full termination, rounded up to the nearest integer second. Active worker volume discounts need an enterprise inquiry, so don't assume an automatic 30% discount off list price.

Baseten publishes dedicated A100 80GB compute at $0.06667 per minute. We use that exact minute rate rather than a rounded $4/hour estimate. What to keep in mind: each runtime environment packages its own hardware optimizations and Triton/vLLM kernels, so tokens-per-second can vary from one platform to the next.

Two explicit traffic patterns

Take 100,000 requests of spiky production traffic: 5,000 separate bursts of 20 requests each. Every replica lifecycle has 40 seconds of active generation, 10 seconds of cold startup, and 20 seconds of warm idle to absorb follow-ups, so 70 seconds per allocation. Metered strictly by the second, that's 350,000 total seconds. Under Baseten's per-allocation minute rounding, each 70-second burst rounds up to two billed minutes, totaling 10,000 billed minutes.

Now 1,000,000 requests in larger, consolidated batches: 1,000 bursts of 1,000 requests. Each allocation runs 2,000 seconds of inference plus 10 seconds startup and 20 seconds warm idle, so 2,030 seconds total (34 billed minutes under that rounding rule).

Runtime budgetModal, CPU/RAM includedRunPod, posted second rateBaseten, minute model
100K requests, short bursts$272.35$266.00$666.70
1M requests, longer bursts$1,579.62$1,542.80$2,266.78
One warm replica, 730 hours$2,044.95$1,997.28$2,920.15

The ten-second startup, the twenty-second warm buffer, and Baseten's per-allocation rounding are explicit modeling assumptions, so compare them against your real container metrics. These are infrastructure runtime models, not guaranteed invoices. They don't claim every container cold-starts in exactly ten seconds, or that warm pods never get hit by node rebalancing.

Storage, transfer and engineering belong in the budget

Compute is only one line on the bill. RunPod charges $0.07/GB-month for network volumes under 1TB and roughly $0.10/GB-month for local container disk. Modal bills volume storage at $0.09/GB-month and meters network egress after standard plan allowances. Baseten's managed deployments and model artifact caches come with their own separate resource allocations. Price your actual checkpoint storage (safetensors weights), log retention, and network egress, because those add-ons aren't free just because they're small.

For a fuller total cost of ownership (TCO), count your own time too: an initial five hours of setup plus two hours of monthly maintenance at $100/hour adds up to $700 of first-month engineering value. With that labor included, your first-month commitment becomes $972.35 on Modal, $966.00 on RunPod, or $1,366.70 on Baseten, before storage and transfer. That's internal development cost rather than a line item on your cloud bill, but it's still real money.

Check capacity before calling something break-even

At two seconds per serial request, 1M requests represent 555.56 active GPU-hours before startup and idle. If heavier context or longer generation stretches that to four seconds, you need 1,111.11 hours of GPU time. A standard billing month has only 730 hours, so a single dedicated instance can't finish that slower workload without batching, concurrency, or autoscaling to a second instance, which promptly doubles your baseline spend.

At four seconds per request, the 100K short-burst case grows to 110 seconds per allocation (550,000 seconds total). Modal rises to $427.98 and RunPod to $418.00. Baseten still rounds 110 seconds to two minutes ($666.70), which shows how minute-based billing cushions small latency shifts while second-based billing tracks duration closely.

My practical rule of thumb: for bursty, intermittent workloads that can live with brief cold starts, scale-to-zero serverless on RunPod or Modal gives you very good unit economics. If you're running business-critical, user-facing applications that need zero cold starts, enterprise SLA guarantees, and turnkey model observability, Baseten's managed platform can be worth the premium. If those things are only nice-to-have, save your budget. Either way, test your real traffic distribution and payload duration before you sign an annual infrastructure contract.

PASS IT ON

Useful for your team?

Share this guide with a colleague.