Add Prompt Caching to Your App

If every request you send starts with the same few thousand tokens of instructions, tools or documents, you're paying full input price for them every time. Prompt caching lets the model reuse that repeated part, and cache reads cost a tenth of normal input. This guide takes an existing app from no caching to verified cache hits.

For the background on how caching works, see what prompt caching is. Here we'll just do it.

1. Find the part of your prompt that repeats

Print or log the full request your app sends and mark which parts are identical from one request to the next. Typical candidates:

• The system prompt: role, rules, tone and format instructions.
• Tool definitions: the same functions on every request.
• Reference material: a product manual, a codebase summary, a policy document.
• In chats: all the earlier turns, which are resent unchanged with each new message.

Caching pays off when that stable part is large. A system prompt of a few hundred tokens isn't worth the effort; several thousand tokens of instructions and documents is.

2. Put the stable part first

Caching matches prompts from the very beginning, so anything that changes must come after everything that doesn't. The most common mistake is a small dynamic value near the top, which breaks the match for everything after it:

system = f"""Today is {today}. The user is {user_name}.
You are a support assistant for Acme…
{LONG_POLICY_DOCUMENT}"""

system = f"""You are a support assistant for Acme…
{LONG_POLICY_DOCUMENT}"""

# Per-request details go in the user message, after the cached part
user_message = f"Today is {today}. The user is {user_name}.\n\n{question}"

Keep the stable part byte-for-byte identical: build it from constants, keep lists in a fixed order, and avoid inserting IDs or timestamps into it.

3. Turn on caching

Claude models (Anthropic Messages API)

Caching is explicit: add a cache_control marker to the last block of the stable part. Everything up to and including that block is cached.

from anthropic import Anthropic

client = Anthropic(base_url="https://api.greentokens.io", api_key="sk-gt-…")

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    system=[{
        "type": "text",
        "text": STABLE_SYSTEM_PROMPT,
        "cache_control": {"type": "ephemeral"},
    }],
    messages=[{"role": "user", "content": user_message}],
)

A cached block expires after about five minutes without use, and each hit refreshes it. If your traffic has longer gaps, use a one-hour cache instead with "cache_control": {"type": "ephemeral", "ttl": "1h"}. Its write price is higher, so choose it only when requests are more than five minutes apart.

GPT and other OpenAI-format models

Caching is automatic for repeated prefixes, so there's no marker to add. Step 2 is all that's needed: once the stable part is first and unchanged, repeated requests hit the cache.

4. Confirm it's working

Send the same request twice within a minute and print the usage from each response:

usage = response.usage
print("written to cache:", usage.cache_creation_input_tokens)
print("read from cache: ", usage.cache_read_input_tokens)
print("uncached input:  ", usage.input_tokens)

• First request: cache_creation_input_tokens is roughly the size of your stable part.
• Second request: cache_read_input_tokens is that size, and input_tokens is only the new part.

For OpenAI-format requests, print response.usage and look for the cached token count in its details. If the cached count stays at zero on repeat requests, something in the stable part is still changing; go back to step 2.

5. Measure the saving

Use your own numbers: the size of the stable part and how many requests reuse it. As an example, an 8,000-token stable prompt sent with 1,000 requests a day on Claude Sonnet 5, at GreenTokens prices:

Add a few cache writes at $1.25 per million for the first request after each gap, and the stable part costs around a tenth of what it did. Your dashboard's Requests log shows the cost of each call, so you can compare a day before and after the change.

Common pitfalls

Next steps

• See caching's price for every model on the models page.
• More on the API side in the text docs.
• Running Claude Code? It caches automatically; trim its bill further.