Token Bills vs. GPU Bills
If you already use token allocation, the main question is why a GPU bill needs a different method. The short video below walks through it, and the rest of this section covers the same ideas.Same Logs, Two Matches
Your request logs don’t change. Model API costs keep matching on tokens, exactly as they do today. For a GPU bill, Vantage uses what the logs and the bill have in common: the day, the model, and the provider. Every request that ran on a model that day is grouped together and matched to that model’s GPU cost for the day.Token Weights Decide the Split
Once a day’s requests are grouped, Vantage splits the day’s cost across the tags in the group. Each tag gets credit for its requests, plus its tokens, weighted by token type. You choose the token weights. Output tokens usually take more compute to serve than input tokens, so they usually get a higher weight. The shares always add back to the full bill, idle time included. See Work Units and Token Weights for the formula and a worked example. For example, if two teams sent requests togemma-3-27b on the same day, that day’s $480 GPU bill is split by each team’s share of the work units:
Supported Providers and Log Sources
Inference allocation supports these cost providers:Allocation Types by Provider
Each cost integration can use token allocation, inference allocation, or both, depending on how the provider bills. The provider selection table disables the checkbox for any allocation type a provider doesn’t support.inference tag to tell Vantage which requests belong to which. See Use Token and Inference Allocation Together.
Log Sources
Send request logs through any of these sources:- Custom LLM Enrichment
- LiteLLM Enrichment
- Cloudflare AI Gateway Enrichment
- Bifrost Enrichment
- For AWS only, AWS Bedrock LLM Enrichment, using the Bedrock model invocation logs you’ve already connected
provider value that names the cost provider, such as modal, baseten, or together. For AWS, Azure, and Google Cloud, use the values listed in the Custom LLM Enrichment provider reference. Vantage only matches a request to a provider’s costs when the request’s provider names that provider. If your gateway reports Modal traffic under a different provider, for example as a generic OpenAI-compatible endpoint, those requests can’t match your Modal costs. In that case, send them through Custom LLM Enrichment and set provider to modal.
How It Works
Inference allocation runs during each provider’s normal cost import. For each time-billed cost row, Vantage finds the requests that model served that day and splits the row across them:- Index: Vantage indexes your request logs by UTC day, provider, and model, the same way it does for token allocation.
- Route: If a cost integration has both token and inference allocation selected, requests tagged
inference=truego to inference allocation and all other requests go to token allocation. Each request is used once, so usage is never counted twice. - Convert to work units: Vantage converts each tag combination’s requests and tokens for that day into work units, using your token weights.
- Match and split: Vantage matches each time-billed cost row to the requests for the same provider, model, and day, then splits the row’s cost by each tag combination’s share of the work units.
- Tag: Each allocated row carries the tags from its requests, plus Managed AI Tags such as
vntg:ai:model.
How Requests Match Hourly Costs
Your provider bills compute by the hour, and Vantage ingests that spend as daily cost rows per App, deployment, endpoint, or model. Vantage matches cost rows to requests by UTC day and model, not by individual hour. All of a model’s requests on a given day share all of that model’s compute cost for the day. Matching by day is what lets idle hours be included: an hour with no requests isn’t tracked separately from the rest of the day. The model on the cost row doesn’t have to be an exact copy of themodel in your logs. Vantage compares the App, deployment, or endpoint name on the bill against the models that appear in your logs that day and picks the closest match. For the most reliable matches, name your Modal Apps, Baseten deployments, and Together AI endpoints after the model they serve, and log that same model name on each request.
Work Units and Token Weights
GPU time isn’t measured per request, so Vantage estimates each tag’s share of a model’s compute with work units. A work unit is a stand-in for serving effort: every request takes some effort to handle, and more tokens take more effort, depending on their type. Each request counts as one work unit, plus one weighted unit per 1,000 tokens of each type. You set a separate token weight for each of the four token types: input, output, cache read, and cache write. Token weights reflect how much compute each token type takes to serve on your hardware. They’re unrelated to the model weights in “open-weight models.” Token weights apply to the whole Vantage account. Every source that has inference allocation turned on uses the same token weights.gemma-3-27b deployment is $480 for the day, covering 24 GPU hours. Your token weights are 0.25 for input and 1 for output. These requests have no cache read or cache write tokens, so those two token weights don’t change the result. That day’s requests for the model fall into two tag combinations:$480 row becomes two rows:$480 and 24 hours. Because each request counts as a work unit, a team that sends many short requests can carry a larger share than its token volume alone would suggest.Choose Token Weights
A common starting point is how API-based LLM providers price the same token types:- Output tokens are generated one at a time and usually cost several times more to serve than input tokens. A token weight of
0.25on input and1on output counts each output token as four input tokens. - Cache reads usually cost less than uncached input.
1 for every token type to weight them equally. If your team benchmarks its own hardware, for example by measuring GPU time per token type in load testing, use those ratios instead. You can change token weights at any time, and Vantage re-allocates past periods to match. See Data Freshness and Backfill.
How Cost Rows Are Split
Splits are additive: the allocated rows always add up to the original cost row, so provider-level totals in your existing reports don’t change.- Idle time is included: A model’s full compute cost for the day is split across the requests that model served that day, including hours when the deployment was running but idle. This lets you charge back or show back the full cost of your hourly-billed inference.
- Quiet and busy days: On a quiet day, a small number of requests carry the whole day’s cost. On a busy day, the same cost is spread across many more requests.
- Days without requests aren’t split: If no requests for a model match a cost row on a given day, that row passes through unchanged, with its original cost and usage, the provider’s existing tags, and no request tags. It still counts toward your provider totals, and it appears as untagged when you group by one of your request tag keys.
- Incomplete logs: If your logs cover only part of a model’s traffic on a given day, the logged requests carry that day’s full cost. Send logs for all of a model’s traffic for the most accurate split.
- Leftover rows: If part of a cost row can’t be attributed to matching requests, for example because
resource_account_idlimits the requests to a different provider account, that part stays on a leftover row. The leftover row carries thevntg:ai:modeltag and the provider’s existing tags, but no request tags. - Credits: Credits and other negative charges are allocated in the same proportions as positive charges for the same model and day, and keep their negative sign. Each tag combination’s net cost reflects its share of the credit. If no requests match, the credit passes through unallocated.
- Usage amounts follow the cost: When the provider reports a usage quantity on the cost row, such as Baseten Dedicated Inference hours or Together AI GPU hours, Vantage splits that quantity in the same proportions as the cost. The total stays the same, and each allocated row carries its share. Modal cost rows don’t include a usage quantity, so only their cost is split.
Data Freshness and Backfill
Inference allocation runs as part of each provider’s existing cost import, so it follows that provider’s refresh cadence. Modal and Baseten costs refresh once daily. Recent days are reprocessed within a rolling three-day window, so late-arriving logs are picked up, including logs for a day that initially had no matching requests. See the provider data refresh documentation for per-provider timing. You don’t need to trigger a re-import for past periods. Vantage backfills automatically, back to the start of your account’s data retention, for as long as the matching request logs remain available:- First save: Saving token weights for the first time allocates past billing periods that have matching request logs.
- Token weight changes: Vantage re-allocates only the billing months whose effective token weights changed. Changes that apply only to future dates take effect going forward.
- Allocation selection changes: Switching a cost integration between token and inference allocation, or clearing one of two selected types, re-runs billing periods that already had a successful run for that source.
Use Token and Inference Allocation Together
Some providers bill both ways. A Baseten account, for example, can have Model APIs costs billed per token and Dedicated Inference costs billed by the hour. You can select both Token allocation and Inference allocation for the same cost integration and allocate both kinds of cost from a single stream of request logs. When both are selected, Vantage uses theinference tag on each request to decide where it goes:
- Requests tagged
inference=trueare allocated against time-billed inference costs. - All other requests are allocated against token-billed costs.
Inference Tag Format
Add aninference key to the tags object of each request record, with the string value "true" (case-insensitive) or the boolean true. When both allocation types are selected, requests without the tag, or with any other value, are treated as token requests.
Records follow the Token Cost Allocation Specification. The event_id, timestamp (UTC, with a Z suffix), provider, and model fields are required, along with a usage object that has at least one positive token count. For example, a request served by a Baseten Dedicated Inference deployment looks like this:
tags, such as team and customer, becomes a tag on the allocated cost rows. If you send logs through LiteLLM, Cloudflare AI Gateway, or Bifrost, add inference alongside the other request metadata you already use for tags.
When Only One Type Is Selected
If a cost integration has only one allocation type selected, untagged requests go to that type. Requests explicitly tagged for the other type are skipped:- Only Inference allocation selected: requests tagged
inference=falseare skipped. - Only Token allocation selected: requests tagged
inference=trueare skipped.
inference tag on them matches the allocation type selected for the integration.
Prerequisites
Before you begin, make sure:- Your account has contracted Build Hours. If it doesn’t yet, contact sales@vantage.sh to add them.
- An active Modal, Baseten, Together AI, AWS, Azure, or Google Cloud cost integration exists for the models you host.
- You send request logs for those models through Custom LLM Enrichment, LiteLLM Enrichment, Cloudflare AI Gateway Enrichment, or Bifrost Enrichment, or, for AWS, through AWS Bedrock LLM Enrichment. Each request’s
providermust name the cost provider. - You have a Vantage Organization Owner or Integration Owner role. See Role-Based Access Control.
Set Up Inference Allocation
Setup has two parts: choose which cost integrations use inference allocation, then set the token weights that decide how each day’s cost is split.Select Inference allocation for your cost integration
- Custom LLM Enrichment and Cloudflare AI Gateway: Select Review after a new source’s scan finishes, or open the source’s Manage tab for an existing source.
- LiteLLM: Select Configure Providers on a new collector, or select Edit and then Manage Providers on an existing collector.
- Bifrost: Open the feed’s Manage tab and select Configure Providers.
- AWS Bedrock LLM Enrichment: Open the log bucket’s Manage tab. The page notes, “Add inference allocation for this connected account. Token allocation stays on the existing account.”

inference tag to the requests served by your dedicated deployments.Open Configure Inference Costs

Set your token weights

1 in every field to count each token type equally, and refine later. See Choose Token Weights for common starting points. Vantage re-allocates past months automatically when you change token weights.To apply one set of token weights to all billing dates, use a single period and leave its Date range blank. If your costs changed over time, for example after you moved a model to different hardware, select Add period to add one row per period and set each row’s Date range. A new period copies its token weights from the latest period. Date ranges for the same token type can’t overlap, and a period without a date range can’t be combined with other periods.Save
Change or Turn Off Inference Allocation
Return to the source’s provider selection to change which cost integrations use inference allocation:- Switch or remove an allocation type: Changing the selection re-runs billing periods that already had a successful run for that source, so past costs follow the new selection.
- Clear both allocation types: Clearing both checkboxes for an integration pauses new allocation for it. Vantage remembers the last selection, so earlier periods can still be re-run the same way.
- Stop a source: Select Stop from the source’s ellipses (…) menu. Costs that were already allocated stay in place, and you can select Resume at any time.
View Allocated Costs on Cost Reports
Allocated inference rows appear on your provider costs in any Cost Report. Each row carries:vntg:ai:token_type tag, because each row covers compute time rather than a single token type. See Enrichment Tags for how enrichment tags work with Virtual Tags and allocation chains.
Enrichment tags behave like any other provider tag in the console:
- To group: open the Group By menu, select Tag, and choose the tag key, for example
vntg:ai:model, or one of your own keys such asteam. - To filter: open the Filters menu, click New Rule, select Tag, choose the Tag Key, then pick an operator and one or more values.
Check How Requests Matched
Open the source that carries your request logs and select the Enrichment History tab. The Enrichment Runs table shows each run for each cost integration and billing period. The Enrichment Runs table lists one row per provider cost integration and billing period that ran enrichment, newest first, with columns for the Integration (and its account), Status, Tokens Kept, Log Lines Kept, Bill Match, Billing Period, and Last Run At. Lifecycle changes appear as their own rows: Added when enrichment is first enabled for an integration, and Paused or Resumed when you stop, disable, resume, or re-enable it. The three percentages look similar but measure different things at different stages, so a low number in one column means something very different from a low number in another:LLM Enrichment History

Troubleshooting
My provider costs are not being split
My provider costs are not being split
- Inference allocation is selected for the cost integration on the source’s provider selection. See Step 1.
- You saved token weights on Configure Inference Costs, and all four token weights cover the billing dates you expect. Dated periods with gaps or blank fields leave the uncovered dates unallocated.
- Your requests report a
providerthat names the cost provider, such asmodal,baseten, ortogether. Requests reported under another provider, such as an OpenAI-compatible endpoint, can’t match these costs. - The
modelon your requests resembles the Modal App name, Baseten deployment model name, Together AI endpoint name, or cloud model name on the cost row. If the names don’t share an identifiable model name, Vantage can’t match them. - There were requests for that model on the same UTC day as the cost row. Days without matching requests pass through unchanged.
- The cost row is time-billed. Baseten Model APIs rows and per-token cloud model costs use token allocation instead, and Together AI Serverless Inference rows aren’t allocated by inference.
- If both allocation types are selected for the integration, the requests served by your dedicated deployments carry the
inferencetag.
Requests I expected to allocate were skipped
Requests I expected to allocate were skipped
inference tag. When only Inference allocation is selected, requests tagged inference = false are skipped. When only Token allocation is selected, requests tagged inference = true are skipped. See When Only One Type Is Selected.Part of a cost row stays on a leftover row
Part of a cost row stays on a leftover row
resource_account_id restricts your requests to a different provider account than the cost row. Check that resource_account_id on your requests matches the provider account behind the cost integration, or omit it if one source only carries requests for one account.Allocations changed after I updated my token weights
Allocations changed after I updated my token weights
I can't tell which costs were allocated
I can't tell which costs were allocated
- Check coverage in Enrichment History: Open the source’s Enrichment History and review Tokens Kept (the share of your logged tokens Vantage attached to a cost row) and Bill Match (the share of eligible provider cost rows that received logged usage) for each integration and billing period.
- Check a Cost Report: Filter to the provider, then group by one of your own tag keys. Costs from requests that carried that key show its values. The leftover row (the portion of the bill your usage did not cover) and any usage sent without that key appear as Not tagged with the key name.
inference tag. Allocated inference rows carry inference=true.Frequently Asked Questions
Which providers are supported?
Which providers are supported?
Does inference allocation measure the compute each request used?
Does inference allocation measure the compute each request used?
Is idle time included in the allocated cost?
Is idle time included in the allocated cost?
Can one cost integration use token and inference allocation at the same time?
Can one cost integration use token and inference allocation at the same time?
inference = true. See Use Token and Inference Allocation Together.Can I use different token weights for different sources or providers?
Can I use different token weights for different sources or providers?
How are credits handled?
How are credits handled?
What happens if I turn off inference allocation?
What happens if I turn off inference allocation?
Can I view sources and match results through the Vantage API?
Can I view sources and match results through the Vantage API?
GET /v2/enrichment_sourceslists your sources.GET /v2/enrichment_sources/{enrichment_source_token}returns a single source.GET /v2/enrichment_sources/{enrichment_source_token}/statisticsreturns match statistics for each run, with optionalprovider,page, andlimitquery parameters. Each run includeslog_record_match_rate,bill_match_rate, andtokens_kept_rate.
Can I view sources through Terraform?
Can I view sources through Terraform?
vantage_enrichment_sources and vantage_enrichment_source data sources, which return the same source details as the API. Inference allocation selections and token weights are set in the console.How much does this cost?
How much does this cost?
What Vantage permissions do I need to enable this?
What Vantage permissions do I need to enable this?
Will this change my totals or break existing reports?
Will this change my totals or break existing reports?