Prompt Caching

Use prompt caching to cut the cost of repeated content, and keep session routing consistent with request IDs.

What Is Prompt Caching

Many model providers (such as Anthropic Claude and OpenAI) natively support prompt caching. When the same context is sent repeatedly, the provider reuses previously computed results, reducing latency and token costs.

Caching Limitations

Caches cannot be shared across accounts — this is a provider-side restriction, not a platform limitation. As a Tokens Router, we cannot guarantee that two consecutive requests from the same account will be routed to the same backend channel. If two requests land on different channels, the established cache cannot be reused, which may lower cache hit rates and increase costs.

Maintaining Routing Consistency

To solve this, you can carry the ID of the previous request in subsequent requests. The system will try to route the new request to the same channel, reusing the existing cache. Two methods are supported:

Header	Value Source	TTL
X-Request-Id	The X-Calgine-Request-Id header of the previous response	5 minutes
X-Response-Id	The conversation ID in the previous response JSON (e.g., Claude's message.id)	5 minutes

Example

Python

import anthropic

client = anthropic.Anthropic(
    base_url="https://api.calgine.com.cn/anthropic",
    api_key="API_KEY",
)

# First request
response = client.messages.create(
    model="claude-opus-4-5-20251101",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
)

# Get the response ID to keep subsequent requests on the same route
response_id = response.id

# Second request: carry the previous response ID
response2 = client.messages.create(
    model="claude-opus-4-5-20251101",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello!"},
        {"role": "assistant", "content": response.content[0].text},
        {"role": "user", "content": "Let's continue our conversation..."},
    ],
    extra_headers={"X-Response-Id": response_id},
)

cURL

# After the first request, extract the X-Calgine-Request-Id response header
# and carry it in the next request
curl https://api.calgine.com.cn/anthropic/v1/messages \
  -H "x-api-key: API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "X-Request-Id: X-Calgine-Request-Id from the previous response" \
  -H "Content-Type: application/json" \
  -d '{"model": "claude-opus-4-5-20251101", "max_tokens": 1024, "messages": []}'

Sub-Task Extensions

Some models (such as video generation and image editing) support extending an existing task — for example, appending content to an already-generated video or performing a secondary edit on a generated image. These operations typically require the original task_id in the new request; refer to each model's documentation for specific parameters.