Affordable AI API: how to reduce your LLM token costs

There are two levers on any LLM bill: the price you pay per token and the number of tokens you use. Most teams focus on the first and leave large savings on the second. The techniques below are ordered roughly by how much they tend to save, and none of them require you to make your product worse.

1. Match the model to the task

The largest frontier models are extraordinary, and they are priced accordingly. Many requests don't need them. Classifying a support ticket, extracting fields from an invoice, rewriting a sentence or routing a user to the right flow can often be handled by a smaller, faster model at a fraction of the price.

A common pattern is tiering: send every request to an inexpensive model first, and escalate only when the task is hard or the cheap model isn't confident. Because an OpenAI- or Anthropic-compatible gateway gives you many models behind one key, switching models is a one-line change to the model field rather than a new integration.

2. Use prompt caching for repeated context

If every request starts with the same long system prompt, tool definitions or reference document, you are paying full price to send identical tokens again and again. Prompt caching lets the provider store that prefix and bill later reads at a much lower cached-input price.

• Put stable content first: system prompt, tool schemas, then long reference material, with the user's message last.
• Keep the cached prefix byte-for-byte identical between requests; a changed timestamp at the top breaks the cache.
• Remember that writing to the cache has a one-off cost, so caching pays off when the same prefix is read several times.

Caching matters most for agents and coding tools, which send the same instructions and tool definitions with every step of a task. A session of fifty steps that reuses a 10,000-token prefix reads half a million tokens of identical context; billed at a cached rate instead of the full input price, that repeated context becomes one of the cheapest parts of the request.

On GreenTokens, cached input is billed at each model's cached-input price, which you can compare against the normal input price on the models page.

3. Trim the context you resend

Chat applications resend the entire conversation on every turn, so a long session grows more expensive with every message. Keep only what the model needs: summarise older turns, drop tool results once they've been used, and retrieve fewer, more relevant passages instead of attaching whole documents. Cutting a 20,000-token context to 5,000 tokens cuts that request's input cost by three quarters.

4. Cap and shape the output

Output tokens usually cost several times more than input tokens, so verbose answers are expensive answers. Set max_tokens to a sensible ceiling for each use case, ask for concise responses in the system prompt, and request structured output such as JSON when you only need specific fields. For reasoning models, check whether a lower reasoning effort gives acceptable results on simpler tasks.

5. Stop generations you don't need

Streaming lets users see an answer as it's written, and it also lets you stop. If a user navigates away, cancels, or the first lines already show the answer is off track, close the stream. Stopping early means the tokens that would have come after aren't generated.

6. Measure before you send

Guessing token counts leads to surprises. Anthropic's count_tokens endpoint tells you how many input tokens a request will use before you send it, which is useful for enforcing context budgets and estimating costs in advance. It's available on GreenTokens at /v1/messages/count_tokens, and it's free: token counting is never billed.

7. Watch usage by model and by key

You can't reduce what you can't see. Break usage down by model and by API key, and look at it daily rather than at the end of the month. Separate keys for each service or environment make it obvious which part of your product is driving the bill, and per-request logs with tokens, cost and latency show which prompts are heavier than they should be.

The GreenTokens dashboard shows daily totals by model and by key, alongside every individual request, so a spike is easy to trace back to its source.

8. Put limits on every key

A bug that retries in a loop or a key that leaks into a public repository can burn through a budget overnight. Give each key a daily and monthly spend limit that matches what it should reasonably use. When a key reaches its limit, requests are rejected until the period resets, which turns a potential disaster into a minor incident. Pause or revoke a key the moment you suspect a problem.

9. Pay less for the tokens you do send

Once usage is lean, the remaining lever is price. An API gateway that buys capacity in aggregate can offer the same models below the official list price. Look for one that shows the official price next to its own, bills on exact reported usage and doesn't require a subscription. Our guide to getting model tokens at discounted prices covers what to check before switching.

Putting it together

Start with the changes that need no product work, such as spend limits, output caps and caching a stable system prompt, then measure. Most teams find that a few of these, combined with a lower price per token, cut their bill substantially while users notice no difference at all.