What is prompt caching and how does it work?

Look at what you send to a language model on a typical request and most of it is the same as last time: the same system prompt, the same tool definitions, the same reference documents, the same earlier turns of the conversation. Without caching, the model processes all of that from scratch on every call, and you pay full price for it every time. Prompt caching fixes that.

What is prompt caching?

Prompt caching is a feature of LLM APIs that stores the processed form of the beginning of a prompt, so that later requests starting with the exact same content can reuse it. The reused part is billed at a much lower cached-input price and processed faster. Only the part that's new has to be handled at the normal rate.

How it works

Caching works on prefixes. The first time a model sees a prompt, it processes it from start to finish and can save the result up to a given point. When a later request begins with exactly the same tokens up to that point, the model loads the saved work instead of redoing it, and continues from where the cached part ends.

That has one important consequence: the match must be exact and from the very start. Change a single character early in the prompt and everything after it no longer matches. So the order of your prompt matters a great deal.

What prompt caching costs

Caching introduces two new prices alongside normal input. A cache write is charged when content is first stored, and costs a little more than normal input. A cache read is charged whenever stored content is reused, and costs a small fraction of it. Here's Claude Sonnet 5 at GreenTokens prices:

Cached content expires if it isn't used. The two write prices correspond to how long it stays available: about five minutes by default, refreshed each time it's read, or up to an hour at the higher write price. The models page lists cache prices for every model.

How much it saves

Take a 10,000-token system prompt sent with 100 requests in a short burst, on Claude Sonnet 5:

• Without caching: 100 × 10,000 = 1,000,000 input tokens at $1 per million, so $1.00.
• With caching: one 5-minute write of 10,000 tokens ($0.0125), then 99 reads totalling 990,000 tokens at $0.10 per million ($0.099), so about $0.11.

That's roughly 89% less for the repeated part. A cache write pays for itself on the very next request, which is why caching is worth turning on for almost any prompt with a sizeable stable prefix.

Prompt caching in Claude and OpenAI models

The two main API formats handle caching differently:

• Anthropic Messages (Claude): caching is explicit. You mark where the reusable part ends with a cache_control block, and the response reports cache_creation_input_tokens and cache_read_input_tokens.
• OpenAI formats: caching of repeated prefixes is automatic above a minimum prompt length. The response reports cached tokens in the usage details.

Here's a Claude request that caches a long reference document. Through GreenTokens it uses the standard Anthropic SDK; see the text API docs for more:

from anthropic import Anthropic

client = Anthropic(base_url="https://api.greentokens.io", api_key="sk-gt-…")

message = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    system=[{
        "type": "text",
        "text": LONG_REFERENCE_DOCUMENT,        # the stable part
        "cache_control": {"type": "ephemeral"},  # cache everything up to here
    }],
    messages=[{"role": "user", "content": "Summarise section 3."}],
)
print(message.usage.cache_read_input_tokens)   # > 0 once the cache is warm

How to structure prompts for caching

1. Put stable content first: system instructions, tool definitions, then reference documents.
2. Put what changes last: the user's current message and anything request-specific.
3. Keep the stable part byte-for-byte identical. Don't insert timestamps, request IDs or randomly ordered lists near the top.
4. For conversations, append new turns to the end rather than rewriting earlier ones, so the history stays a matching prefix.
5. Check the usage fields. If cached tokens stay at zero, something early in the prompt is changing.

Why it matters most for agents

Agents and coding tools resend a large, growing context on every step: instructions, tool definitions, files they've read and everything that's happened so far. With caching, each step pays full price only for what's new and the cached rate for the rest. Claude Code and Claude Desktop send cache markers automatically, which is one reason long sessions stay affordable when the API passes caching through.

Common questions

Does prompt caching change the model's answers?

No. Caching only skips re-processing content the model has already seen; the model receives exactly the same input either way. Answers are the same as they would be without caching, just cheaper and usually faster to start.

Is prompt caching automatic?

It depends on the format. OpenAI-format requests cache repeated prefixes automatically once they're long enough. Claude requests through the Anthropic Messages API need a cache_control marker to say what to cache, though tools like Claude Code add those markers for you.

When isn't prompt caching worth it?

When nothing repeats. If every request is short and unique, there's no prefix to reuse, and a cache write would cost slightly more than normal input without ever being read. Caching pays off when a sizeable, identical block of content is sent more than once within the cache's lifetime.

In short

Prompt caching stores the processed start of a prompt so identical prefixes can be reused at a fraction of the input price, around a tenth for cache reads. Put stable content first, keep it unchanged, and let the variable part come last. For any application with a long system prompt, shared documents or multi-turn conversations, it's the easiest large saving available.