NotCheap MAKE EVERY CREDIT COUNT
Analyze Your Costs

LLM observability costs: traces, spans, processed data and evaluations

One million application requests are not one million interchangeable observability billing units. A single request can fan out into several spans, carry large payloads, and trigger paid evaluation calls along the way. The friendliest way to budget is to start with what the platform actually ingests and retains, then layer on users and model-based scoring.

Translate application requests into real telemetry volume

The most common mistake I see when teams set up LLM observability is assuming that one million application requests means one million billable telemetry events. It's an easy assumption to make, but in modern agentic pipelines a single user prompt rarely triggers a single model call. An agent might plan, run three vector database searches, invoke two distinct tools, reflect on the intermediate responses, and then fire off a final synthesis prompt. Each of those sub-steps creates its own trace span. If each request averages eight spans, that one million user requests generate eight million spans. And if each span packages prompt payloads, system instructions, function arguments, and tool outputs averaging one kilobyte of stored telemetry data, you are ingesting roughly eight gigabytes of raw data before database indexing, compressed backups, and audit logging even enter the picture.

CandidatePublished meter or plan distinctionQuestion for the quote
LangSmithSeats, included base traces and usage services; current pricing also uses LSUWhich services consume the quoted units?
LangfuseUnits, data access period and cloud/self-hosted routeHow many observations and scores produce a unit?
Arize AXSpans, ingestion GB and retentionWhich limit does this workload reach first?
BraintrustProcessed data, scores, model credits and retentionAre evaluation models and retained data included?
W&B pricing destinationThe current URL directs to CoreWeave ForgeDoes the selected application-tracing plan match your workflow?

Here's the practical catch: every platform measures this data flow differently. LangSmith charges by seat licenses, included base traces, and LangChain Service Units (LSUs) that scale across tracing, online evaluations, and prompt playgrounds. Langfuse uses an observation unit meter tied to data retention windows, with dedicated cloud and self-hosted options. Arize AX bills across spans, raw ingestion gigabytes, and time-based retention windows. Braintrust prices processed gigabytes alongside explicit evaluation scoring calls. Even Weights & Biases has restructured its tracing offerings, and its current pricing links point directly to CoreWeave Forge. So before you compare headline prices, map your own span-to-request ratio against each vendor's primary metric. It takes a few minutes and saves a lot of squinting at invoices later.

A processed-data and scoring breakdown in practice

Let's walk through how these multi-dimensional meters interact on a real invoice, using Braintrust Starter as an illustrative baseline. The Starter tier includes one gigabyte of processed data, and bills subsequent ingestion at $4 per gigabyte. It also includes 10,000 evaluation scores, with additional scores billed at $2.50 per 1,000, backed by a 14-day retention window. Say your production pipeline generates eight billed gigabytes of processed telemetry and runs 100,000 evaluation scores in a month. The overage math splits into two lines: seven excess gigabytes at $4 each cost (8 − 1) × $4 = $28, while 90,000 excess scores cost (100,000 − 10,000) ÷ 1,000 × $2.50 = $225. Add them together and you get an overage charge of $253, before judge-model token costs or paid add-ons.

Notice where the money actually goes: the evaluation scoring line ($225) outweighs the raw data ingestion charge ($28) by more than eight to one. And that $225 covers only the platform's scoring meter. If your evaluation suite uses an external LLM as a judge, for example calling GPT-4o to score groundedness and hallucinations, you pay the model provider separately for every input and output token consumed during evaluation. It's tempting to assume an evaluation score allowance includes free model tokens, but that assumption is a fast track to an unpleasant billing surprise.

Identify which platform constraint binds first

Every observability tier comes with several limits running at once, and your workload will almost always hit one of them long before the others. Take the Arize AX Free tier, which offers 25,000 spans, one gigabyte of ingestion, and 15 days of data retention. If your application runs complex multi-agent workflows with twenty spans per request, you'll use up the 25,000 span allowance after just 1,250 user interactions, even though your total data volume might barely reach 50 megabytes. The span ceiling binds while the storage allowance sits almost untouched.

On the other side, LangSmith's Developer tier provides a single user seat with 5,000 included base traces, whereas the Plus plan introduces monthly seat fees alongside higher baseline trace allowances and flexible usage caps. Langfuse's Hobby plan provides free cloud units and 30-day historical data access, but scaling into production means choosing between managed cloud tiers and self-hosting. Figuring out whether your app is constrained by developer seats, trace counts, span depth, or storage gigabytes is the quickest way to see which vendor gives you the most sustainable unit economics.

Sampling strategies protect both budgets and diagnostic quality

Ingesting 100% of telemetry from a high-throughput production application is rarely necessary and often cost-prohibitive. Smart trace sampling can slash your observability bill by 80% to 90%, but how you sample matters a lot. A naive uniform random sample of 10% will catch 10% of your successful calls and 10% of your failures, which means 90% of your edge-case crashes, timeout errors, and user-facing bugs vanish without a diagnostic trace. That's usually the opposite of what you wanted.

A friendlier approach is tail-based or rule-based sampling: capture 100% of failed requests, 100% of high-latency outliers (such as queries in the 95th percentile and above), and a modest 2% to 5% sample of baseline successful queries for aggregate latency and token tracking. Before sending payloads to the observability backend, strip out redundant conversational history, base64-encoded image strings, and sensitive personally identifiable information (PII). Filtering repetitive payloads at the SDK layer keeps your diagnostic traces clean while meaningfully lowering billable ingestion volume.

Self-hosting is an operational commitment, not free software

Open-source platforms like Langfuse offer self-hosted deployment options, which can look very appealing if you want to avoid SaaS markups or meet strict data sovereignty requirements. Just keep in mind that self-hosting is an infrastructure decision, not a zero-cost shortcut. Running an observability stack in your own cloud account means provisioning and maintaining PostgreSQL databases, ClickHouse analytical stores, Redis caching layers, blob storage buckets, and redundant compute nodes.

You'll want to budget for ongoing database storage costs, automated backup routines, version upgrades, and the engineering hours needed to keep the cluster healthy during high-concurrency ingestion spikes. Before you commit, benchmark a realistic traffic load against your self-hosted cloud infrastructure bill and compare it honestly against the managed SaaS tier. And to see how these observability expenses fit into your broader product margins, plug your numbers into our micro-SaaS cost model alongside your raw inference bills.

Recalculate data and evaluation-score charges

Starts with the selected Braintrust Starter excess-rate example. Use provider-measured processed GB. Score input is the number beyond the included 10,000, initially 90,000. Add separately billed model usage and other services to fixed payments.

Evaluation-score overage

8 processed GB needed; 7 beyond included capacity.

$253.00 for this scenario · $31.625 per processed GB, allocating score spend

Data charge = max(0, processed GB − included GB) × excess GB rate. Add excess scores × price per score, then separately billed model/retention costs.

Put observability into the operating budget →

Sources

PASS IT ON

Useful for your team?

Share this guide with a colleague.