Model Studio Explicit Cache Best Practices

Reference, synced 2026-06-13.

flowchart TD
  n0["Models"]
  n1["Overview"]
  n2["Products"]
  n3["Solutions"]
  n4["Pricing"]
  n5["Resources"]
  n6["Partners"]
  n7["Support"]
  n8["Language"]
  n0 --> n1
  n1 --> n2
  n2 --> n3
  n3 --> n4
  n4 --> n5
  n5 --> n6
  n6 --> n7
  n7 --> n8

Free access Accelerate Delivery with Fixed-Cost Agentic CodingWatch how it works

Models

Empowering AI innovation for both enterprises and developers with Alibaba Cloud’s best-in-class Qwen models, AI-native apps, and AI solutions.

Alibaba Cloud Model Studio \ Enterprise-grade large model service and application development platform.

Try Visual Model \ Supports image understanding, image generation, and video generation.

Models

HappyHorse-1.0-T2V \ Cinematic creative generation, ultimate dynamic details Qwen3-VL-Plus \ Native VL, spatial reasoning, 1M-context video analysis Wan2.7-VideoEdit \ Supports both localized and global editing with prompt

Qwen3.6-Plus \ Native multimodal, 1M context, agentic coding Wan2.7-Image-Pro \ Interactive editing, long-text rendering, precise prompt following Qwen-Plus \ Balanced intelligence, efficient inference, production-ready performance

Qwen-Image-2.0 \ Professional infographics, exquisite photorealism Z-Image-Turbo \ Ultra-fast image generation, high throughput, cost-optimized inference Qwen3-Coder-Next \ Multi-turn tool interactions, future-ready development support

Wan2.7-T2V \ High-fidelity T2V, 15s duration, advanced camera control Wan2.7-I2V \ Cinematic I2V with emotional depth and visceral impact Wan2.7-R2V \ Up to 5 mixed image/video inputs and audio timbre cloning

GenAI Application

Qoder \ Intelligent coding assistant, available for enterprise-dedicated deployment. Qoder CN \ AI-powered coding assistant that boosts developer productivity with intelligent code completion, AI chat, multi-file editing, and task automation.

AI Service

Model Experience \ Experience full-scale, multimodal model capabilities online. Platform for AI \ An AI-native algorithm engineering platform for end-to-end modeling, training, and inference service deployment. Fine-tune Video Generation Model \ Customize Wan’s text-to-video capabilities through model fine-tuning to meet your unique requirements.

AI Use Case

AI Savings Plan Hot \ Save up to 47% on AI costs. Limited-time offer tailored to your usage. AI Video Creation \ Elevate your professional video production with Wan 2.6.

AI Token Plan \ One plan. Multiple models. Big Savings with a Fixed Subscription. AI Image Creation \ All-in-one creative suite for copywriting, image generation, and poster design.

Overview

As a global full-stack AI leader, Alibaba Cloud aims to make computing accessible to everyone and help worldwide customers accelerate innovation.

Why Alibaba Cloud

About Alibaba Cloud \ AI Powered Cloud Technology Our Global Network \ Explore our global presence and deployment regions around the world Our Global Offices \ With offices in 4 continents, we're always close to where it matters.

Asia Accelerator \ Accelerate Success in Asia with Alibaba Cloud Go Global \ Benefits of our Global Alliance Trust Center \ Empowering enterprises with a secure, compliant, and globally trusted cloud infrastructure

Customers and Insights

Olympic Games \ Alibaba Cloud Powers Olympic Games with AI-powered cloud technology Case Studies \ Learn how customers are scaling their businesses on Alibaba Cloud Analyst Reports \ Learn what the top industry analyst firms are saying about Alibaba Cloud

What's New

Events and Webinars \ Quick access to upcoming and on-demand events Product Updates \ Stay informed of the latest innovations Press Room \ Latest news and media releases

Products

Featured ProductsAI & Machine Learning Computing Container Storage Networking & CDN Security Middleware Database Analytics ComputingMedia ServicesEnterprise Services & Cloud CommunicationDomain Names and WebsitesEnd User ComputingServerlessDeveloper ToolsMigration & O&M ManagementApsara Stack

Featured Products

Alibaba Cloud Model Studio \ Supercharge your AI journey effortlessly with industry-leading GenAI models ApsaraDB RDS \ Store and manage your business data, with automated monitoring and backups Certificate Management Service (Original SSL Certificate) \ Create a safe and secure connection between your website and users

Elastic Compute Service (ECS) \ Host websites anywhere and scale enterprise workloads Container Service for Kubernetes (ACK) \ Run and scale containerized applications on managed Kubernetes infrastructure Object Storage Service (OSS) \ Store large amounts of data in the cloud and access it anywhere, anytime

Simple Application Server (SAS) \ All-in-one services for fast deployment Elastic IP Address (EIP) \ Manage your public IPs independently to improve internet network quality Domain Names and Website \ Get the perfect domain name to suit your every need

Solutions

Solutions by Industry Technical Solutions AI WebsitesNetworking Security and ComplianceData and AnalyticsEnterprise Service and ApplicationCloud MigrationCloud NativeHybrid CloudSMB solutions

Financial Services \ Innovate faster with Alibaba Cloud Games \ Grow your game rapidly with high global availability

New Retail \ Alibaba Cloud enables digital retail transformation to fuel growth and realize an omnichannel customer experience throughout the consumer journey. Media and Entertainment \ Ready your content for today's media market with a digitalized media journey

Supply Chain \ Power your supply chain with intelligent, efficient, and reliable solutions Sports \ Digitizing the sports industry with intelligent tech

Sustainability \ Achieve a sustainable future with low-carbon and energy-efficient technologies

Pricing

Flexible options like pay-as-you-go and clear billing rules to meet diverse business needs.

Overview & Tools

Pricing Calculator \ Get an instant pricing estimate based on your usage and needs Free Trial \ Try our 80+ cloud products for free.

Pricing Options \ Get the most out of Alibaba Cloud with flexible pricing

Optimize your cost

Migrate & Save \ Superior Performance At Lower Pricing. Save up to 50%. Promotion Center \ Unlock the latest Alibaba Cloud offers & promos

Resources

Official documentation, extensive tools, training resources, and a community to grow and innovate in the cloud.

Technical Resources

Documentation \ Product guides and FAQs Architecture Center \ Design reliable, secure, and efficient cloud architecture. Intelligent Solution Explorer \ Find the right solution for you, powered by AI

Blog \ Latest cloud insights and developer trends Whitepapers \ Research that explores the how and why behind our technology

Training&Certification

Alibaba Cloud Academy \ Build cloud skills and earn certifications with expert-led training.

Developer Hub

Alibaba Cloud Project Hub \ Explore real-world projects built by developers using our platform. Our Developer MVPs \ Celebrating the developers who lead, build, and inspire our community

Partners

Partner-first strategy offering collaborative product, sales, and service models, plus high-quality partner solutions that complement Alibaba Cloud’s capabilities.

Marketplace

AI Alliance for ISVs \ Partner with us to build and grow AI solutions together ISV Benefits \ Unlock resources, market access, and go-to-market support as an ISV partner

Alibaba Cloud Marketplace \ Explore ready-to-deploy solutions from our partners and ISVs

Find a Partner

Partner Hub \ Find your ideal partner in no time

Become a Partner

Partner Network \ A partner portal for Alibaba Cloud Channel, Technology, MSP partner and other partner programs

Support

Full-lifecycle support and expert services, from cloud advisory and migration to operations.

Support & Professional Services

Professional Services \ Expert-led services to design, migrate, and optimize your cloud journey Support Plans \ Flexible support for every stage — from startup to enterprise

Partner Support Program \ Priority technical support for partners, with dedicated managers and faster issue resolution

Contact us

Connect With Us \

Talk to a sales expert and get a custom quote for your business

Language

English
简体中文
繁體中文
日本語
Bahasa Indonesia

Locale

Visit aliyun.com

Documentation

Alibaba Cloud Model Studio

User Guide (Models) User Guide (Application) API Reference (Models) API Reference (Application)

Search for Help Content

Getting Started

The Beginner's Guide

Well-Architected Framework

AI & Machine Learning

Platform For AI

Alibaba Cloud Model Studio

DashVector

Artificial Intelligence Recommendation

OpenSearch

Image Search

Machine Translation

Intelligent Speech Interaction

Optimization Solver

Intelligent Computing LINGJUN

Computing

Elastic Compute Service

Elastic GPU Service

Elastic Container Instance

Dedicated Host

Compute Nest

Simple Application Server

Cloud Box

Auto Scaling

Elastic High Performance Computing

Batch Compute (Deprecated)

Function Compute

Serverless App Engine

ENS

Elastic Desktop Service

App Streaming

WUYING Terminal

Cloud Phone

Edge Network Acceleration

Alibaba Cloud Linux

AgentBay

Container

Container Service for Kubernetes

Container Compute Service

Container Registry

Storage

Object Storage Service

Cloud Parallel File Storage

File Storage NAS

Tablestore

Storage Capacity Unit

Simple Log Service

Cloud Backup

Intelligent Media Management

Drive and Photo Service

Data Transport

Cloud Storage Gateway

Data Online Migration

Hybrid Cloud Storage Array

Storage Services Overview

Backup and Disaster Recovery Center

Networking and CDN

Server Load Balancer

Elastic IP Address

Internet Shared Bandwidth

Data Transfer Plan

Virtual Private Cloud

NAT Gateway

PrivateLink

Alibaba Cloud DNS PrivateZone

Network Intelligence Service

Cloud Data Transfer

IPv6 Gateway

Anycast Elastic IP Address

Cloud Enterprise Network

Global Accelerator

VPN Gateway

Smart Access Gateway

Express Connect

CDN

Edge Security Acceleration

Cloud Network Well-architected Design Guidelines

Security

Anti-DDoS

Web Application Firewall

Cloud Firewall

Security Center

Bastionhost

Secure Access Service Edge

Certificate Management Service

Key Management Service

Data Security Center

Identity as a Service

Fraud Detection

AI Guardrails

Captcha

Blockchain as a Service

ID Verification

Managed Security Service

Middleware

Enterprise Distributed Application Service

Microservices Engine

Alibaba Cloud Service Mesh

SchedulerX

ApsaraMQ for RocketMQ

ApsaraMQ for Kafka

ApsaraMQ for RabbitMQ

ApsaraMQ for MQTT

Simple Message Queue (formerly MNS)

CloudFlow

EventBridge

Application Real-Time Monitoring Service

Managed Service for Prometheus

Managed Service for Grafana

Managed Service for OpenTelemetry

Performance Testing

STAROps

Databases

ApsaraDB Console

PolarDB

ApsaraDB RDS

ApsaraDB for OceanBase (Deprecated)

Tair (Redis® OSS-Compatible)

Lindorm

Time Series Database

ApsaraDB for MongoDB

ApsaraDB for HBase

ApsaraDB for Memcache

ApsaraDB for MyBase

AnalyticDB

ApsaraDB for ClickHouse

ApsaraDB for SelectDB

Data Transmission Service

Database Autonomy Service

Data Management

Database Gateway - Deprecated

ApsaraDB for Cassandra - Deprecated

Analytics Computing

MaxCompute

Hologres

Realtime Compute for Apache Flink

Elasticsearch

Vector Retrieval Service for Milvus

E-MapReduce

Data Lake Formation

DataV

Quick BI

Quick Audience

Quick Tracking

DataWorks

DataHub

Dataphin

Media Services

ApsaraVideo VOD

ApsaraVideo Live

Intelligent Media Services

ApsaraVideo Media Processing

Apsara Video SDK

Enterprise Services & Cloud Communication

Energy Expert

CloudQuotation

Salesforce on Alibaba Cloud

GoChina ICP Filing Assistant

Marketplace

Alibaba Mail

Direct Mail

Short Message Service

Voice Service

Phone Number Verification Service

Cell Phone Number Service

Chat App Message Service

Financial Intelligence Engine

Domain Names and Websites

Domain Names

ICP Filing

Alibaba Cloud DNS

End User Computing

Elastic Desktop Service

App Streaming

WUYING Terminal

Cloud Phone

AgentBay

Internet of Things

IoT Platform

Serverless

Serverless App Engine

CloudFlow

EventBridge

Simple Message Queue (formerly MNS)

Function Compute

Developer Tools

OpenAPI Explorer

Alibaba Cloud SDK

Cloud Shell

Resource Orchestration Service

Alibaba Cloud CLI

BSS OpenAPI

Terraform

Pulumi

Ticket System API

Mobile Platform as a Service

Alibaba Cloud DevOps

API Gateway

Cloud Control API

AI Coding Assistant Lingma

Cloud Skills Portal

Migration & O&M Management

CloudOps Orchestration Service

Cloud Monitor

Intelligent Advisor

Cloud Governance Center

ActionTrail

Cloud Config

Resource Access Management

Resource Management

Cloud Architect Design Tools

Migration Hub

Server Migration Center

Service Catalog

Logic Composer

Quota Center

CloudSSO

HTTPDNS

Solutions

SAP

SuperApp

OpenLake

Membership Service

Expenses and Costs

Account Center

Support

Legal

Tech Share Terms and Conditions

After Sales Support

China Gateway Program

Service Level Objectives

Management Console

Security Control

This topic describes how to use explicit cache and its best practices. By adding cache markers to your requests, explicit cache guarantees deterministic cache hits for identical input content, significantly reducing cost and latency.

When to use explicit cache

You need guaranteed cache hits: Explicit cache delivers 100% deterministic hits regardless of backend resource scheduling. If your application requires stable content reuse, explicit cache is the right choice.
You frequently reuse the same prompt: When identical or highly consistent prompts are submitted repeatedly, explicit cache significantly reduces costs. Creating the cache incurs only a 25% surcharge over the standard input price, while each subsequent hit saves 90%. A single hit is enough to break even.
You manage long contexts in production Agents: In Agent applications, common mechanisms like compression, recap, and system reminders cause the context to change continuously. Explicit cache lets you pin and reuse key context segments so they remain cached even as the surrounding context evolves.

Agent and coding tools

The following Agent and coding tools connect to Model Studio through the Anthropic protocol and natively support explicit cache. Configure them following their respective documentation, and they will automatically leverage explicit cache to optimize context management.

The examples below use the Singapore endpoint. For other regions, replace the base URL with the corresponding regional endpoint.

Claude Code

Open Code

OpenClaw

Hermes

Claude Code v2.x and later automatically includes cache_control markers in requests (system, env, and most recent user message). No additional configuration is needed after connecting to Model Studio's Anthropic-compatible endpoint.

Configuration

Create or edit ~/.claude/settings.json (Windows: C:\Users\<username>\.claude\settings.json) with the appropriate plan settings. Alternatively, connect via environment variables:

bash

export ANTHROPIC_BASE_URL="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/apps/anthropic"
export ANTHROPIC_AUTH_TOKEN="${DASHSCOPE_API_KEY}"
export ANTHROPIC_MODEL="qwen3.7-max"
claude

Set the Anthropic protocol endpoint:

Token Plan (Team): https://token-plan.ap-southeast-1.maas.aliyuncs.com/apps/anthropic
Coding Plan: https://coding-intl.dashscope.aliyuncs.com/apps/anthropic
Pay-as-you-go: https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/apps/anthropic

Replace WorkspaceId with your actual Workspace ID.

For details, see Claude Code.

Optional: Improve cross-session hit rate

By default, Claude Code includes dynamic information in the system prompt (current directory, date, git status), which may reduce cross-session cache hit rates. Add the following flag at startup to move dynamic sections to user messages:

bash

claude --exclude-dynamic-system-prompt-sections

When OpenCode connects to Model Studio's Anthropic-compatible endpoint via @ai-sdk/anthropic, it automatically injects cache_control on the system message and the most recent non-system message.

Installation

bash

npm install -g opencode-ai

Configuration

Create the configuration file ~/.config/opencode/opencode.json (Windows: C:\Users\<username>\.config\opencode\opencode.json):

json

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "bailian": {
      "npm": "@ai-sdk/anthropic",
      "name": "Alibaba Cloud Model Studio",
      "options": {
        "baseURL": "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/apps/anthropic/v1",
        "apiKey": "{env:DASHSCOPE_API_KEY}"
      },
      "models": {
        "qwen3.7-max": { "name": "qwen3.7-max" }
      }
    }
  }
}

Note

The baseURL must end with /v1.

bash

export DASHSCOPE_API_KEY=sk-xxxxx
opencode run -m "bailian/qwen3.7-max" "..."

Other plan base URLs:

Token Plan (Team): https://token-plan.ap-southeast-1.maas.aliyuncs.com/apps/anthropic/v1
Coding Plan: https://coding-intl.dashscope.aliyuncs.com/apps/anthropic/v1

For details, see OpenCode.

When using the Anthropic-compatible endpoint, OpenClaw automatically injects cache_control markers on the system prompt and the most recent user message. No additional configuration is needed — as long as the provider's Base URL points to /apps/anthropic, explicit cache is automatically enabled.

Installation

bash

npm install -g openclaw
# or
curl -fsSL https://openclaw.ai/install.sh | bash

Configuration

Edit the configuration file ~/.openclaw/openclaw.json. Set "api" to "anthropic-messages" and set the base URL:

Token Plan (Team): https://token-plan.ap-southeast-1.maas.aliyuncs.com/apps/anthropic/v1
Coding Plan: https://coding-intl.dashscope.aliyuncs.com/apps/anthropic/v1
Pay-as-you-go: https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/apps/anthropic/v1

For details, see OpenClaw.

Optional: Custom cache boundary

If your system prompt contains both stable template content and dynamic content (timestamps, CWD, etc.), insert `` between them. OpenClaw will apply cache_control only to the stable prefix before the boundary, improving cross-session hit rates:

plaintext

You are a Python engineer following these conventions:
- type hints required
- docstrings in Google format

Current time: 2026-05-25 18:42
Working directory: /Users/.../project

Without this boundary, OpenClaw applies cache_control to the entire system prompt using its built-in strategy, still benefiting from explicit cache.

Configure using the hermes config set command. Set the base URL:

Token Plan (Team): https://token-plan.ap-southeast-1.maas.aliyuncs.com/apps/anthropic/v1
Coding Plan: https://coding-intl.dashscope.aliyuncs.com/apps/anthropic/v1
Pay-as-you-go: https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/apps/anthropic/v1

For details, see Hermes Agent.

API integration

Key points

Add "cache_control": {"type": "ephemeral"} to the message content you want to cache. All content from the beginning of the messages array up to that marker will be cached as a block.
Cached content must be at least 1024 tokens.
A single request supports up to 4 cache markers.
Cache TTL is 5 minutes, automatically renewed on each hit.
Tool definitions are part of the system prompt for caching purposes. If the tools change, the cache will not hit.

Quick start

The following example demonstrates the basic workflow: the first request creates a cache, and the second request hits it.

OpenAI-compatible

Anthropic-compatible

python

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Replace WorkspaceId with your actual Workspace ID.
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)

# Long text to cache (must exceed 1024 tokens)
long_text_content = "<Your Long Text Here>" * 400

def get_completion(user_input):
    messages = [\
        {\
            "role": "system",\
            "content": [\
                {\
                    "type": "text",\
                    "text": long_text_content,\
                    # Cache marker: content from the start of messages to this point will be cached\
                    "cache_control": {"type": "ephemeral"},\
                }\
            ],\
        },\
        {"role": "user", "content": user_input},\
    ]
    completion = client.chat.completions.create(
        model="qwen3.7-max",
        messages=messages,
        extra_body={"enable_thinking": False},
    )
    return completion

# First request: creates cache
first = get_completion("Summarize the key points of this document")
print(f"Cache created: {first.usage.prompt_tokens_details.cache_creation_input_tokens}")
print(f"Cache hit: {first.usage.prompt_tokens_details.cached_tokens}")

# Second request: same system content, different question — hits cache
second = get_completion("What precautions are mentioned in the document?")
print(f"Cache created: {second.usage.prompt_tokens_details.cache_creation_input_tokens}")
print(f"Cache hit: {second.usage.prompt_tokens_details.cached_tokens}")

python

import anthropic
import os

client = anthropic.Anthropic(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/apps/anthropic",
)

# Long text to cache (must exceed 1024 tokens)
long_text_content = "<Your Long Text Here>" * 400

def get_completion(user_input):
    response = client.messages.create(
        model="qwen3.7-max",
        max_tokens=1024,
        system=[\
            {\
                "type": "text",\
                "text": long_text_content,\
                # Cache marker\
                "cache_control": {"type": "ephemeral"},\
            }\
        ],
        messages=[\
            {"role": "user", "content": user_input},\
        ],
    )
    return response

# First request: creates cache
first = get_completion("Summarize the key points of this document")
print(f"Cache created: {first.usage.cache_creation_input_tokens}")
print(f"Cache hit: {first.usage.cache_read_input_tokens}")

# Second request: hits cache
second = get_completion("What precautions are mentioned in the document?")
print(f"Cache created: {second.usage.cache_creation_input_tokens}")
print(f"Cache hit: {second.usage.cache_read_input_tokens}")

Expected output:

plaintext

Cache created: 2005
Cache hit: 0
Cache created: 0
Cache hit: 2005

The first request creates a cache block. The second request hits the cache because the system prompt content is identical. Cached tokens are billed at only 10% of the standard input price.

Verify cache status

Check the usage field in the response to confirm cache behavior:

cache_creation_input_tokens: Number of tokens for which a new cache was created. A value greater than 0 means a new cache block was created.
cached_tokens (OpenAI-compatible) or cache_read_input_tokens (Anthropic-compatible): Number of tokens that hit cache. A value greater than 0 means the cache was successfully hit.

Best practices by scenario

Multi-turn conversations

Characteristics:

Users interact with the model over multiple turns, each request carrying the full conversation history
Typical use cases: customer service, knowledge Q&A, code assistants

Best practice: Add a cache_control marker to the last message in each request. Each turn hits the cache created by the previous turn (the conversation history), while creating a new cache that includes the current turn for the next round.

Example:

python

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Replace WorkspaceId with your actual Workspace ID.
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)

# System prompt: product manual (must exceed 1024 tokens)
product_manual = """You are the support assistant for "BaiLian SmartHome" smart home controller. Here is the complete product manual:

## Product Overview
BaiLian SmartHome is a whole-home smart controller supporting voice control, scene automation, and energy management...

## Installation Guide
1. Install at a central location with good WiFi coverage...
2. Connect the power adapter (5V/2A)...

## FAQ
Q: Cannot connect to WiFi? A: Make sure your router supports 2.4GHz...
""" * 80  # Repeat to exceed 1024 tokens

messages = [{"role": "system", "content": product_manual}]

def chat(user_input):
    # Key: add cache_control to the last user message
    messages.append({
        "role": "user",
        "content": [\
            {\
                "type": "text",\
                "text": user_input,\
                "cache_control": {"type": "ephemeral"},\
            }\
        ],
    })
    completion = client.chat.completions.create(
        model="qwen3.7-max",
        messages=messages,
        extra_body={"enable_thinking": False},
    )
    assistant_msg = completion.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_msg})

    usage = completion.usage
    created = usage.prompt_tokens_details.cache_creation_input_tokens
    cached = usage.prompt_tokens_details.cached_tokens
    print(f"  [Cache] Created: {created} tokens, Hit: {cached} tokens")
    return assistant_msg

# Simulate multi-turn conversation
print("User: What voice assistants does BaiLian SmartHome support?")
print(f"Agent: {chat('What voice assistants does BaiLian SmartHome support?')[:80]}...\n")

print("User: What if I cannot connect to WiFi?")
print(f"Agent: {chat('What if I cannot connect to WiFi?')[:80]}...\n")

print("User: How many devices can it control simultaneously?")
print(f"Agent: {chat('How many devices can it control simultaneously?')[:80]}...")

Expected output:

plaintext

User: What voice assistants does BaiLian SmartHome support?
  [Cache] Created: 8739 tokens, Hit: 0 tokens
Agent: BaiLian SmartHome supports Tmall Genie, XiaoAi, Siri, and other voice assistants...

User: What if I cannot connect to WiFi?
  [Cache] Created: 151 tokens, Hit: 8739 tokens
Agent: For WiFi connectivity issues, try the following: 1. Confirm your router supports 2.4GHz...

User: How many devices can it control simultaneously?
  [Cache] Created: 101 tokens, Hit: 8890 tokens
Agent: BaiLian SmartHome can control up to 256 smart devices simultaneously...

Starting from the second turn, each request hits the cache from the previous turn (the conversation history), while creating a new cache that includes the current turn. The more turns in the conversation, the greater the savings.

Production Agent (multiple cache markers)

Characteristics:

Long multi-turn conversations comprising: system prompt + skills/tools definitions + project context + user messages / tool calls
Different sections change at different frequencies
Typical use cases: AI coding assistants (Claude Code, OpenClaw), RAG-based Q&A systems

Best practice: Use multiple cache markers (up to 4) to pin content at different stability levels. Each marker must be on a separate message (different role) to serve as an independent breakpoint:

System prompt — one marker (rarely changes)
Skills/tools definitions — one marker (may change in combination)
Project context — one marker (may switch or compress)
User messages / tool calls — one marker (grows each turn)

Example: This example simulates a typical Agent architecture with 3 cache markers pinning the system persona and tools (marker 1), knowledge base (marker 2), and conversation history (marker 3). Note that the knowledge base is placed in a user message to ensure it has its own independent cache breakpoint — multiple system messages are merged internally and cannot serve as separate breakpoints:

python

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Replace WorkspaceId with your actual Workspace ID.
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)

# Layer 1: System persona (rarely changes)
system_persona = """You are the senior AI support agent for "Model Studio Electronics". Your guidelines:
1. Answer questions based on the knowledge base
2. For information not in the knowledge base, say "Let me transfer you to a human agent"
3. Maintain a professional and friendly tone
4. If the user is unhappy, apologize first then resolve the issue

Below is your complete service specification and script guide:
""" + "Detailed service specification..." * 200  # Ensure > 1024 tokens

# Layer 2: Tools/skills definitions (changes occasionally, e.g. when new features launch)
tools_description = """### Available Tools
- search_product(query): Search product information
- check_inventory(sku, color): Check stock status
- create_ticket(type, description): Create a support ticket
- transfer_to_human(reason): Transfer to a human agent

### Tool Usage Rules
1. When user asks about product details, use search_product first
2. When user asks about stock/shipping, use check_inventory
3. When user requests return/exchange, use create_ticket
4. When a tool returns an error, apologize and transfer_to_human
""" + "Detailed tool usage examples..." * 150  # Ensure > 1024 tokens

# Layer 3: Project knowledge base (semi-stable, changes when user switches products)
knowledge_base_product_a = """### Current product: Model Studio Pro Max Wireless Earbuds
- SKU: BL-PM-2024
- Price: CNY 599
- Colors: Night Black / Nebula White / Ice Blue
- Battery: 8 hours (ANC on), 12 hours (ANC off)
- Water resistance: IPX5
- Warranty: 1 year, 7-day no-questions-asked return
- Stock: Night Black (in stock) / Nebula White (low) / Ice Blue (out of stock)
""" * 50  # Ensure > 1024 tokens

def ask_agent(user_question, history=None):
    if history is None:
        history = []
    messages = [\
        {\
            "role": "system",\
            "content": [\
                {\
                    "type": "text",\
                    "text": system_persona + "\n\n" + tools_description,\
                    "cache_control": {"type": "ephemeral"},  # Marker 1: system persona + tools\
                }\
            ],\
        },\
        {\
            "role": "user",\
            "content": [\
                {\
                    "type": "text",\
                    "text": f"Here is the knowledge base for the current product:\n{knowledge_base_product_a}",\
                    "cache_control": {"type": "ephemeral"},  # Marker 2: knowledge base\
                }\
            ],\
        },\
        {"role": "assistant", "content": "Got it. I have the product details ready. How can I help you?"},\
    ]
    messages.extend(history)
    # Add current question with marker 3
    messages.append({
        "role": "user",
        "content": [\
            {\
                "type": "text",\
                "text": user_question,\
                "cache_control": {"type": "ephemeral"},  # Marker 3: conversation history\
            }\
        ],
    })

    completion = client.chat.completions.create(
        model="qwen3.7-max",
        messages=messages,
        extra_body={"enable_thinking": False},
    )
    usage = completion.usage
    print(f"  Created: {usage.prompt_tokens_details.cache_creation_input_tokens}, "
          f"Hit: {usage.prompt_tokens_details.cached_tokens}")
    return completion.choices[0].message.content

# First request
print("Q1: Is the Ice Blue color available?")
a1 = ask_agent("Is the Ice Blue color available?")
print(f"A1: {a1}\n")

# Second request: same product (persona + tools + knowledge base all hit)
history = [\
    {"role": "user", "content": "Is the Ice Blue color available?"},\
    {"role": "assistant", "content": a1},\
]
print("Q2: When will it be back in stock?")
a2 = ask_agent("When will it be back in stock?", history)
print(f"A2: {a2}")

Expected output:

plaintext

Q1: Is the Ice Blue color available?
  Created: 7659, Hit: 0
A1: I'm sorry, but the Ice Blue color... is currently out of stock...

Q2: When will it be back in stock?
  Created: 73, Hit: 7659
A2: I don't have access to specific restock dates... Let me transfer you to a human agent...

In Q2, the prefix up to marker 2 (persona + tools + knowledge base = 7,659 tokens) is unchanged, resulting in a full cache hit. Only the new content after marker 2 (conversation history + new question) requires processing.

How multi-marker caching works:

User continues asking about the same product: Persona, tools, and knowledge base are all unchanged, hitting the cache at marker 2 (longest prefix match) for maximum savings.
More conversation turns: Earlier content (persona + tools + knowledge base + history) hits the previous turn's cache; only the new content requires a fresh cache.

Note

Arrange content from most stable to least stable: place content that changes least at the beginning (e.g., system persona) and content that changes most at the end (e.g., current conversation) to maximize cache hit rates.

Batch processing (task completion)

Characteristics:

Single-turn requests, no context memory needed
Fixed long system prompt (task instructions) + variable user input (data to process)
Typical use cases: text classification, intent recognition, data extraction, content moderation

Best practice: Add the cache_control marker only on the system prompt. All subsequent requests hit the cache as long as the system prompt remains unchanged.

Example:

python

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Replace WorkspaceId with your actual Workspace ID.
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)

# Long system prompt: detailed classification rules (must exceed 1024 tokens)
classification_prompt = """You are a product review classifier. Classify each review into one of these categories:
- Positive
- Negative
- Neutral
- Question
- Complaint

Output only the category name, nothing else.

Detailed classification rules and examples:
""" + """Rules:
1. Positive: Contains positive sentiment words (e.g., "great", "excellent", "recommend"), or expresses satisfaction.
2. Negative: Contains negative sentiment words (e.g., "terrible", "disappointed", "return"), or expresses dissatisfaction.
3. Neutral: No clear sentiment, merely states facts.
4. Question: Phrased as a question asking for product information.
5. Complaint: Expresses suggestions for improvement or lodges a complaint.
""" * 100

# Reviews to classify (simulating batch processing)
reviews = [\
    "This product is amazing, great quality, highly recommended!",\
    "Shipping took a week and the packaging was damaged",\
    "Does this come in red? Does it run large or small?",\
    "You should add more size options, medium is too big for me",\
    "It's okay I guess, nothing special, does what it says",\
]

print("=== Batch Classification (Explicit Cache) ===")
for i, review in enumerate(reviews):
    completion = client.chat.completions.create(
        model="qwen3.7-max",
        messages=[\
            {\
                "role": "system",\
                "content": [\
                    {\
                        "type": "text",\
                        "text": classification_prompt,\
                        "cache_control": {"type": "ephemeral"},  # Cache classification rules\
                    }\
                ],\
            },\
            {"role": "user", "content": review},\
        ],
    )
    result = completion.choices[0].message.content
    cached = completion.usage.prompt_tokens_details.cached_tokens
    created = completion.usage.prompt_tokens_details.cache_creation_input_tokens
    print(f"Review {i+1}: \"{review[:40]}...\" -> {result}")
    print(f"  Created: {created}, Hit: {cached}")

Expected output:

plaintext

Review 1: "This product is amazing, great quality, ..." -> Positive
  Created: 10353, Hit: 0
Review 2: "Shipping took a week and the packaging w..." -> Negative
  Created: 0, Hit: 10353
Review 3: "Does this come in red? Does it run large..." -> Question
  Created: 0, Hit: 10353
Review 4: "You should add more size options, medium..." -> Complaint
  Created: 0, Hit: 10353
Review 5: "It's okay I guess, nothing special, does..." -> Neutral
  Created: 0, Hit: 10353

After the first request creates the cache, all subsequent requests hit it. When processing 1,000 items, 999 requests see a 90% reduction in input token cost.

Function Calling with cached tool definitions

Characteristics:

Using Function Calling with a long list of tool definitions
Tool definitions remain unchanged across requests

Best practice: The tools parameter content is part of the system prompt for caching. Ensure tool definitions are exactly identical across requests (same order, same field order, same structure), and add a cache_control marker to the message content.

python

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # Replace WorkspaceId with your actual Workspace ID.
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)

# Long text to meet the 1024-token minimum
long_text_content = "<Your Code Here>" * 400

# Tool definitions: must be exactly identical across requests
tools = [\
    {\
        "type": "function",\
        "function": {\
            "name": "get_weather",\
            "description": "Get the current weather for a given city",\
            "parameters": {\
                "type": "object",\
                "properties": {\
                    "city": {"type": "string", "description": "City name"},\
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}\
                },\
                "required": ["city"]\
            }\
        }\
    },\
    {\
        "type": "function",\
        "function": {\
            "name": "search_flights",\
            "description": "Search flights between two cities",\
            "parameters": {\
                "type": "object",\
                "properties": {\
                    "origin": {"type": "string", "description": "Departure city"},\
                    "destination": {"type": "string", "description": "Destination city"},\
                    "date": {"type": "string", "description": "Departure date in YYYY-MM-DD format"}\
                },\
                "required": ["origin", "destination", "date"]\
            }\
        }\
    }\
]

def ask(user_input):
    messages = [\
        {\
            "role": "system",\
            "content": [\
                {\
                    "type": "text",\
                    "text": long_text_content,\
                    # cache_control can only be added to message content, not to tools\
                    "cache_control": {"type": "ephemeral"},\
                }\
            ],\
        },\
        {"role": "user", "content": user_input},\
    ]
    completion = client.chat.completions.create(
        model="qwen3.7-max",
        messages=messages,
        tools=tools,
        extra_body={"enable_thinking": False},
    )
    usage = completion.usage
    print(f"  Created: {usage.prompt_tokens_details.cache_creation_input_tokens}, "
          f"Hit: {usage.prompt_tokens_details.cached_tokens}")
    tool_calls = completion.choices[0].message.tool_calls
    if tool_calls:
        print(f"  Tools called: {[t.function.name for t in tool_calls]}")
    return completion

# First request: creates cache (includes tool definitions)
print("Q1: What's the weather in Beijing today?")
ask("What's the weather in Beijing today?")

# Second request: hits cache
print("\nQ2: Find flights from Shanghai to Beijing tomorrow")
ask("Find flights from Shanghai to Beijing tomorrow")

Expected output:

plaintext

Q1: What's the weather in Beijing today?
  Created: 1995, Hit: 0
  Tools called: ['get_weather']

Q2: Find flights from Shanghai to Beijing tomorrow
  Created: 0, Hit: 1995
  Tools called: ['search_flights']

Important

Keys to maximizing Function Calling cache hits:

Consistent tool order: Keep the same ordering of tools in the tools array.
Consistent field order: Keep JSON field ordering the same within each tool definition.
Consistent structure: Do not add, remove, or reorder fields between requests, even if they are optional or empty.

Important notes

Content format requirement: When adding cache_control, the content field must be in array form. String-form content does not support cache markers.
Cache marker granularity: Qwen3.5 and later models only support message-level cache breakpoints. Placing multiple cache_control markers within a single message's content array does not create separate breakpoints — the system only stores cache at the last marker position within that message and cannot truncate-match at intermediate content blocks. Additionally, multiple system messages are merged internally into a single segment and cannot serve as separate breakpoints. To create multiple independent breakpoints, distribute cache_control markers across messages with different roles (e.g., one on system, one on user). Models prior to Qwen3.5 support content-level (intra-message) breakpoints.
Mutually exclusive with implicit cache: A request can only use one caching mode. If the request contains a cache_control marker, explicit cache is used; otherwise, the system automatically uses implicit cache.

Supported models

For the list of models that support explicit cache, see the Context cache.

Previous: Best practices for handling rate limitingNext: DeepSeek

Is this page helpful?

When to use explicit cache

Agent and coding tools

API integration

Key points

Quick start

Verify cache status

Best practices by scenario

Multi-turn conversations

Production Agent (multiple cache markers)

Batch processing (task completion)

Function Calling with cached tool definitions

Important notes

Supported models

Sales Support

Live-chat with our sales team or get in touch with a business development professional in your region.

Contact Sales

Technical Support

Open a ticket and get quick help from our technical team.

Open a Ticket >

We look forward to your suggestion.

Post a Suggestion > Report Abuse >

\ \ Hi, I'm Alibaba Cloud AI Assistant!\ \ I can help with questions and solutions.

Why Alibaba Cloud

About Alibaba Cloud

Asia Accelerator

Our Global Network

Global Offices

Trust Center

Case Studies

Analyst Reports

Products & Pricings

Pricing Calculator

ECS

SAS

Model Studio

Database

Security

SMS

Solutions

Financial Services

Retail Services

Media Services

Gaming Services

ISV Solutions

Engage

Developer Community

Partner Network

Startups

Marketplace

Join Alibaba Cloud

Resources & Support

Developer Learning Hub

Documentation Center

Training & Certification

Service Notices

Submit a Ticket

Security Report

Qwen Cloud

Careers About Us Privacy Policy Legal Integrity Compliance Reporting Channel Service Notices Links

Facebook
Linkedin
Twitter
YouTube
TikTok
contact.us@alibabacloud.com
Call Us Now
Discord

Careers About Us Privacy Policy Legal Integrity Compliance Reporting Channel Service Notices Links

Model Studio Explicit Cache Best Practices ​

Models ​

Models ​

GenAI Application ​

AI Service ​

AI Use Case ​

Overview ​

Why Alibaba Cloud ​

Customers and Insights ​

What's New ​

Products ​

Featured Products ​

Related Programs ​

Solutions ​

Pricing ​

Overview & Tools ​

Optimize your cost ​

Resources ​

Technical Resources ​

Training&Certification ​

Developer Hub ​

Partners ​

Marketplace ​

Find a Partner ​

Become a Partner ​

Support ​

Support & Professional Services ​

Contact us ​

Language ​

Locale ​

When to use explicit cache ​

Agent and coding tools ​

API integration ​

Key points ​

Quick start ​

Verify cache status ​

Best practices by scenario ​

Multi-turn conversations ​

Production Agent (multiple cache markers) ​

Batch processing (task completion) ​

Function Calling with cached tool definitions ​

Important notes ​

Supported models ​

Sales Support ​

Technical Support ​

Connect & Report Abuse ​

Model Studio Explicit Cache Best Practices

Models

Models

GenAI Application

AI Service

AI Use Case

Overview

Why Alibaba Cloud

Customers and Insights

What's New

Products

Featured Products

Related Programs

Solutions

Pricing

Overview & Tools

Optimize your cost

Resources

Technical Resources

Training&Certification

Developer Hub

Partners

Marketplace

Find a Partner

Become a Partner

Support

Support & Professional Services

Contact us

Language

Locale

When to use explicit cache

Agent and coding tools

API integration

Key points

Quick start

Verify cache status

Best practices by scenario

Multi-turn conversations

Production Agent (multiple cache markers)

Batch processing (task completion)

Function Calling with cached tool definitions

Important notes

Supported models

Sales Support

Technical Support

Connect & Report Abuse