Appearance
Real-time speech synthesis - CosyVoice/Sambert -
Reference, synced 2026-06-13.
flowchart TD n0["Models"] n1["Overview"] n2["Products"] n3["Solutions"] n4["Pricing"] n5["Resources"] n6["Partners"] n7["Support"] n8["Language"] n0 --> n1 n1 --> n2 n2 --> n3 n3 --> n4 n4 --> n5 n5 --> n6 n6 --> n7 n7 --> n8
Free access Accelerate Delivery with Fixed-Cost Agentic CodingWatch how it works
Models
Empowering AI innovation for both enterprises and developers with Alibaba Cloud’s best-in-class Qwen models, AI-native apps, and AI solutions.
Alibaba Cloud Model Studio \ Enterprise-grade large model service and application development platform.
Try Visual Model \ Supports image understanding, image generation, and video generation.
Models
HappyHorse-1.0-T2V \ Cinematic creative generation, ultimate dynamic details Qwen3-VL-Plus \ Native VL, spatial reasoning, 1M-context video analysis Wan2.7-VideoEdit \ Supports both localized and global editing with prompt
Qwen3.6-Plus \ Native multimodal, 1M context, agentic coding Wan2.7-Image-Pro \ Interactive editing, long-text rendering, precise prompt following Qwen-Plus \ Balanced intelligence, efficient inference, production-ready performance
Qwen-Image-2.0 \ Professional infographics, exquisite photorealism Z-Image-Turbo \ Ultra-fast image generation, high throughput, cost-optimized inference Qwen3-Coder-Next \ Multi-turn tool interactions, future-ready development support
Wan2.7-T2V \ High-fidelity T2V, 15s duration, advanced camera control Wan2.7-I2V \ Cinematic I2V with emotional depth and visceral impact Wan2.7-R2V \ Up to 5 mixed image/video inputs and audio timbre cloning
GenAI Application
Qoder \ Intelligent coding assistant, available for enterprise-dedicated deployment. Qoder CN \ AI-powered coding assistant that boosts developer productivity with intelligent code completion, AI chat, multi-file editing, and task automation.
AI Service
Model Experience \ Experience full-scale, multimodal model capabilities online. Platform for AI \ An AI-native algorithm engineering platform for end-to-end modeling, training, and inference service deployment. Fine-tune Video Generation Model \ Customize Wan’s text-to-video capabilities through model fine-tuning to meet your unique requirements.
AI Use Case
AI Savings Plan Hot \ Save up to 47% on AI costs. Limited-time offer tailored to your usage. AI Video Creation \ Elevate your professional video production with Wan 2.6.
AI Token Plan \ One plan. Multiple models. Big Savings with a Fixed Subscription. AI Image Creation \ All-in-one creative suite for copywriting, image generation, and poster design.
Overview
As a global full-stack AI leader, Alibaba Cloud aims to make computing accessible to everyone and help worldwide customers accelerate innovation.
Why Alibaba Cloud
About Alibaba Cloud \ AI Powered Cloud Technology Our Global Network \ Explore our global presence and deployment regions around the world Our Global Offices \ With offices in 4 continents, we're always close to where it matters.
Asia Accelerator \ Accelerate Success in Asia with Alibaba Cloud Go Global \ Benefits of our Global Alliance Trust Center \ Empowering enterprises with a secure, compliant, and globally trusted cloud infrastructure
Customers and Insights
Olympic Games \ Alibaba Cloud Powers Olympic Games with AI-powered cloud technology Case Studies \ Learn how customers are scaling their businesses on Alibaba Cloud Analyst Reports \ Learn what the top industry analyst firms are saying about Alibaba Cloud
What's New
Events and Webinars \ Quick access to upcoming and on-demand events Product Updates \ Stay informed of the latest innovations Press Room \ Latest news and media releases
Products
Featured ProductsAI & Machine Learning Computing Container Storage Networking & CDN Security Middleware Database Analytics ComputingMedia ServicesEnterprise Services & Cloud CommunicationDomain Names and WebsitesEnd User ComputingServerlessDeveloper ToolsMigration & O&M ManagementApsara Stack
Featured Products
Alibaba Cloud Model Studio \ Supercharge your AI journey effortlessly with industry-leading GenAI models ApsaraDB RDS \ Store and manage your business data, with automated monitoring and backups Certificate Management Service (Original SSL Certificate) \ Create a safe and secure connection between your website and users
Elastic Compute Service (ECS) \ Host websites anywhere and scale enterprise workloads Container Service for Kubernetes (ACK) \ Run and scale containerized applications on managed Kubernetes infrastructure Object Storage Service (OSS) \ Store large amounts of data in the cloud and access it anywhere, anytime
Simple Application Server (SAS) \ All-in-one services for fast deployment Elastic IP Address (EIP) \ Manage your public IPs independently to improve internet network quality Domain Names and Website \ Get the perfect domain name to suit your every need
Related Programs
Solutions
Solutions by Industry Technical Solutions AI WebsitesNetworking Security and ComplianceData and AnalyticsEnterprise Service and ApplicationCloud MigrationCloud NativeHybrid CloudSMB solutions
Financial Services \ Innovate faster with Alibaba Cloud Games \ Grow your game rapidly with high global availability
New Retail \ Alibaba Cloud enables digital retail transformation to fuel growth and realize an omnichannel customer experience throughout the consumer journey. Media and Entertainment \ Ready your content for today's media market with a digitalized media journey
Supply Chain \ Power your supply chain with intelligent, efficient, and reliable solutions Sports \ Digitizing the sports industry with intelligent tech
Sustainability \ Achieve a sustainable future with low-carbon and energy-efficient technologies
Pricing
Flexible options like pay-as-you-go and clear billing rules to meet diverse business needs.
Overview & Tools
Pricing Calculator \ Get an instant pricing estimate based on your usage and needs Free Trial \ Try our 80+ cloud products for free.
Pricing Options \ Get the most out of Alibaba Cloud with flexible pricing
Optimize your cost
Migrate & Save \ Superior Performance At Lower Pricing. Save up to 50%. Promotion Center \ Unlock the latest Alibaba Cloud offers & promos
Resources
Official documentation, extensive tools, training resources, and a community to grow and innovate in the cloud.
Technical Resources
Documentation \ Product guides and FAQs Architecture Center \ Design reliable, secure, and efficient cloud architecture. Intelligent Solution Explorer \ Find the right solution for you, powered by AI
Blog \ Latest cloud insights and developer trends Whitepapers \ Research that explores the how and why behind our technology
Training&Certification
Alibaba Cloud Academy \ Build cloud skills and earn certifications with expert-led training.
Developer Hub
Alibaba Cloud Project Hub \ Explore real-world projects built by developers using our platform. Our Developer MVPs \ Celebrating the developers who lead, build, and inspire our community
Partners
Partner-first strategy offering collaborative product, sales, and service models, plus high-quality partner solutions that complement Alibaba Cloud’s capabilities.
Marketplace
AI Alliance for ISVs \ Partner with us to build and grow AI solutions together ISV Benefits \ Unlock resources, market access, and go-to-market support as an ISV partner
Alibaba Cloud Marketplace \ Explore ready-to-deploy solutions from our partners and ISVs
Find a Partner
Partner Hub \ Find your ideal partner in no time
Become a Partner
Partner Network \ A partner portal for Alibaba Cloud Channel, Technology, MSP partner and other partner programs
Support
Full-lifecycle support and expert services, from cloud advisory and migration to operations.
Support & Professional Services
Professional Services \ Expert-led services to design, migrate, and optimize your cloud journey Support Plans \ Flexible support for every stage — from startup to enterprise
Partner Support Program \ Priority technical support for partners, with dedicated managers and faster issue resolution
Contact us
Connect With Us \
Talk to a sales expert and get a custom quote for your business
Language
- English
- 简体中文
- 繁體中文
- 日本語
- Bahasa Indonesia
Locale
Visit aliyun.com
Documentation
Alibaba Cloud Model Studio
User Guide (Models) User Guide (Application) API Reference (Models) API Reference (Application)
Search for Help Content
Getting Started
The Beginner's Guide
Well-Architected Framework
AI & Machine Learning
Platform For AI
Alibaba Cloud Model Studio
DashVector
Artificial Intelligence Recommendation
OpenSearch
Image Search
Machine Translation
Intelligent Speech Interaction
Optimization Solver
Intelligent Computing LINGJUN
Computing
Elastic Compute Service
Elastic GPU Service
Elastic Container Instance
Dedicated Host
Compute Nest
Simple Application Server
Cloud Box
Auto Scaling
Elastic High Performance Computing
Batch Compute (Deprecated)
Function Compute
Serverless App Engine
ENS
Elastic Desktop Service
App Streaming
WUYING Terminal
Cloud Phone
Edge Network Acceleration
Alibaba Cloud Linux
AgentBay
Container
Container Service for Kubernetes
Container Compute Service
Container Registry
Storage
Object Storage Service
Cloud Parallel File Storage
File Storage NAS
Tablestore
Storage Capacity Unit
Simple Log Service
Cloud Backup
Intelligent Media Management
Drive and Photo Service
Data Transport
Cloud Storage Gateway
Data Online Migration
Hybrid Cloud Storage Array
Storage Services Overview
Backup and Disaster Recovery Center
Networking and CDN
Server Load Balancer
Elastic IP Address
Internet Shared Bandwidth
Data Transfer Plan
Virtual Private Cloud
NAT Gateway
PrivateLink
Alibaba Cloud DNS PrivateZone
Network Intelligence Service
Cloud Data Transfer
IPv6 Gateway
Anycast Elastic IP Address
Cloud Enterprise Network
Global Accelerator
VPN Gateway
Smart Access Gateway
Express Connect
CDN
Edge Security Acceleration
Cloud Network Well-architected Design Guidelines
Security
Anti-DDoS
Web Application Firewall
Cloud Firewall
Security Center
Bastionhost
Secure Access Service Edge
Certificate Management Service
Key Management Service
Data Security Center
Identity as a Service
Fraud Detection
AI Guardrails
Captcha
Blockchain as a Service
ID Verification
Managed Security Service
Middleware
Enterprise Distributed Application Service
Microservices Engine
Alibaba Cloud Service Mesh
SchedulerX
ApsaraMQ for RocketMQ
ApsaraMQ for Kafka
ApsaraMQ for RabbitMQ
ApsaraMQ for MQTT
Simple Message Queue (formerly MNS)
CloudFlow
EventBridge
Application Real-Time Monitoring Service
Managed Service for Prometheus
Managed Service for Grafana
Managed Service for OpenTelemetry
Performance Testing
Databases
ApsaraDB Console
PolarDB
ApsaraDB RDS
ApsaraDB for OceanBase (Deprecated)
Tair (Redis® OSS-Compatible)
Lindorm
Time Series Database
ApsaraDB for MongoDB
ApsaraDB for HBase
ApsaraDB for Memcache
ApsaraDB for MyBase
AnalyticDB
ApsaraDB for ClickHouse
ApsaraDB for SelectDB
Data Transmission Service
Database Autonomy Service
Data Management
Database Gateway - Deprecated
ApsaraDB for Cassandra - Deprecated
Analytics Computing
MaxCompute
Hologres
Realtime Compute for Apache Flink
Elasticsearch
Vector Retrieval Service for Milvus
E-MapReduce
Data Lake Formation
DataV
Quick BI
Quick Audience
Quick Tracking
DataWorks
DataHub
Dataphin
Media Services
ApsaraVideo VOD
ApsaraVideo Live
Intelligent Media Services
ApsaraVideo Media Processing
Apsara Video SDK
Enterprise Services & Cloud Communication
Energy Expert
CloudQuotation
Salesforce on Alibaba Cloud
GoChina ICP Filing Assistant
Marketplace
Alibaba Mail
Direct Mail
Short Message Service
Voice Service
Phone Number Verification Service
Cell Phone Number Service
Chat App Message Service
Financial Intelligence Engine
Domain Names and Websites
Domain Names
ICP Filing
Alibaba Cloud DNS
End User Computing
Elastic Desktop Service
App Streaming
WUYING Terminal
Cloud Phone
AgentBay
Internet of Things
IoT Platform
Serverless
Serverless App Engine
CloudFlow
EventBridge
Simple Message Queue (formerly MNS)
Function Compute
Developer Tools
OpenAPI Explorer
Alibaba Cloud SDK
Cloud Shell
Resource Orchestration Service
Alibaba Cloud CLI
BSS OpenAPI
Terraform
Pulumi
Ticket System API
Mobile Platform as a Service
Alibaba Cloud DevOps
API Gateway
Cloud Control API
AI Coding Assistant Lingma
Cloud Skills Portal
Migration & O&M Management
CloudOps Orchestration Service
Cloud Monitor
Intelligent Advisor
Cloud Governance Center
ActionTrail
Cloud Config
Resource Access Management
Resource Management
Cloud Architect Design Tools
Migration Hub
Server Migration Center
Service Catalog
Logic Composer
Quota Center
CloudSSO
HTTPDNS
Solutions
SAP
SuperApp
OpenLake
Membership Service
Expenses and Costs
Account Center
More
Support
Legal
Tech Share Terms and Conditions
After Sales Support
China Gateway Program
Service Level Objectives
Management Console
Security Control
Real-time speech synthesis (Text-to-Speech, TTS) converts text into natural-sounding speech using machine learning models trained on large speech datasets. Model Studio provides CosyVoice and Sambert models for streaming and non-streaming TTS, with support for voice cloning, voice design, and fine-grained control over speech rate, pitch, and audio format—enabling you to build voice assistants, audiobooks, customer service bots, and other voice-enabled applications.
Core features
Generates high-fidelity speech in real time with multilingual support, including Chinese and English.
Offers two voice customization methods: voice cloning and voice design.
Supports streaming input and output with low first-packet latency, ideal for real-time conversational scenarios.
Provides fine-grained control over speech rate, pitch, volume, and bitrate.
Supports mainstream audio formats (PCM, WAV, MP3, Opus) with output sample rates up to 48 kHz.
Availability
Supported models:
International
Chinese mainland
If you select the International deployment scope, model inference compute resources are dynamically scheduled worldwide, excluding the Chinese mainland. Static data is stored in your selected region. Supported region: Singapore.
When you invoke the following models, select the API key for the Singapore region:
- CosyVoice: cosyvoice-v3-plus, cosyvoice-v3-flash
If you select the Chinese mainland deployment scope, model inference compute resources are restricted to the Chinese mainland. Static data is stored in your selected region. Supported region: China (Beijing).
When you invoke the following models, select an API key for the Beijing region:
- CosyVoice: cosyvoice-v3.5-plus, cosyvoice-v3.5-flash, cosyvoice-v3-plus, cosyvoice-v3-flash, cosyvoice-v2
Model selection
| Scenario | Recommended model | Reason | Notes |
|---|
| Scenario | Recommended model | Reason | Notes |
| Voice customization for brand identity, exclusive voice, or extended system voices (based on text description) | cosyvoice-v3.5-plus | Supports voice design: create custom voices from text descriptions alone, without audio samples. Ideal for building brand-exclusive voices from scratch. | cosyvoice-v3.5-plus is available only in the Beijing region and does not support system voices. |
| Voice customization for brand identity, exclusive voice, or extended system voices (based on audio samples) | cosyvoice-v3.5-plus | Supports voice cloning: quickly replicate voices from real audio samples to create high-fidelity, consistent brand voiceprints. | cosyvoice-v3.5-plus is available only in the Beijing region and does not support system voices. |
| Intelligent customer service / Voice assistant | cosyvoice-v3-flash, cosyvoice-v3.5-flash | Lower cost than Plus models with streaming interaction and emotional expression support. First-packet latency optimized for real-time conversations. | cosyvoice-v3.5-flash is available only in the Beijing region and does not support system voices. |
| Regional dialect broadcasting | cosyvoice-v3.5-plus | Supports multiple Chinese dialects (Northeastern Mandarin, Minnan, and others), ideal for localized content broadcasting. | cosyvoice-v3.5-plus is available only in the Beijing region and does not support system voices. |
| Educational applications (including formula reading) | cosyvoice-v2, cosyvoice-v3-flash, cosyvoice-v3-plus | Supports LaTeX formula-to-speech conversion, ideal for mathematics, physics, and chemistry instruction. | cosyvoice-v2 and cosyvoice-v3-plus have higher costs ($0.286706 per 10,000 characters) |
| Structured voice broadcasting (news/announcements) | cosyvoice-v3-plus, cosyvoice-v3-flash, cosyvoice-v2 | Supports SSML for controlling speech rate, pauses, and pronunciation to enhance broadcast professionalism. | Implement the SSML generation logic independently. This model does not support emotion settings. |
| Precise speech-text alignment for scenarios such as caption generation, lesson playback, and dictation practice | cosyvoice-v3-flash, cosyvoice-v3-plus, cosyvoice-v2 | Supports timestamp output to synchronize the synthesized speech with the original text. | You must explicitly enable the timestamp feature. It is disabled by default. |
| Multilingual international products | cosyvoice-v3-flash, cosyvoice-v3-plus | Supports multiple languages. |
Before selecting a model, review the Compare models section for a full feature comparison by region.
Getting started
Choose the delivery method that fits your use case before writing code:
| Delivery method | Best for | Streaming support |
|---|
| Delivery method | Best for | Streaming support |
| Non-streaming (synchronous) | Batch jobs, short text, complete audio files | No |
| Streaming output (unidirectional) | Real-time applications where first-audio-byte latency matters | Yes |
| Streaming input + output (bidirectional, WebSocket) | Conversational AI, LLM voice output, interactive voice agents | Yes |
For lowest latency in real-time applications, use streaming output with PCM format. PCM requires no encoding overhead and can be played directly on audio devices.
The following examples show how to call the API. For additional code examples covering common scenarios, see GitHub.
Create an API key and export the API key as an environment variable. If you use an SDK to make calls, install the DashScope SDK.
CosyVoice Important The cosyvoice-v3.5-plus and cosyvoice-v3.5-flash models are currently available only in the Beijing region and are designed specifically for voice design and voice cloning scenarios. They do not support system voices. Before using them for speech synthesis, use the CosyVoice voice cloning/design API to create your voice. After the voice is created, simply update the voice field in your code with your voice ID and set the model field to the corresponding model. Use system voices Synthesize speech with a cloned voice Synthesize speech with designed voices The following example demonstrates how to perform speech synthesis using system voices. See the Voice list. Save synthesized audio to a file Convert LLM-generated text to speech in real time and play it through speakers Python Java python # coding=utf-8 import os import dashscope from dashscope.audio.tts_v2 import * # API keys for the Singapore and Beijing regions are different. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key # If you have not configured environment variables, replace the following line with your Model Studio API key: dashscope.api_key = "sk-xxx" dashscope.api_key = os.environ.get('DASHSCOPE_API_KEY') # The following is the URL for the Singapore region. If you use a model in the Beijing region, replace the URL with: wss://dashscope.aliyuncs.com/api-ws/v1/inference dashscope.base_websocket_api_url='wss://dashscope-intl.aliyuncs.com/api-ws/v1/inference' # Model # Different model versions require corresponding voices: # cosyvoice-v3-flash/cosyvoice-v3-plus: Use voices such as longanyang. # cosyvoice-v2: Use voices such as longxiaochun_v2. # Each voice supports different languages. When synthesizing non-Chinese languages such as Japanese or Korean, select a voice that supports the corresponding language. For more information, see the CosyVoice voice list. model = "cosyvoice-v3-flash" # Voice voice = "longanyang" # Instantiate SpeechSynthesizer and pass request parameters such as model and voice in the constructor. synthesizer = SpeechSynthesizer(model=model, voice=voice) # Send the text to be synthesized and get the binary audio. audio = synthesizer.call("How is the weather today?") # The first time you send text, a WebSocket connection is established. The first packet delay includes the connection establishment time. print('[Metric] Request ID: {}, First packet delay: {} ms'.format( synthesizer.get_last_request_id(), synthesizer.get_first_package_delay())) # Save the audio locally. with open('output.mp3', 'wb') as f: f.write(audio) java import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam; import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer; import com.alibaba.dashscope.utils.Constants; import java.io.File; import java.io.FileOutputStream; import java.io.IOException; import java.nio.ByteBuffer; public class Main { // Model // Different model versions require corresponding voices: // cosyvoice-v3-flash/cosyvoice-v3-plus: Use voices such as longanyang. // cosyvoice-v2: Use voices such as longxiaochun_v2. // Each voice supports different languages. When you synthesize non-Chinese languages such as Japanese or Korean, you must select a voice that supports the corresponding language. For more information, see the CosyVoice voice list. private static String model = "cosyvoice-v3-flash"; // Voice private static String voice = "longanyang"; public static void streamAudioDataToSpeaker() { // Request parameters SpeechSynthesisParam param = SpeechSynthesisParam.builder() // API keys for the Singapore and Beijing regions are different. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key // If you have not configured environment variables, replace the following line with your Model Studio API key: .apiKey("sk-xxx") .apiKey(System.getenv("DASHSCOPE_API_KEY")) .model(model) // Model .voice(voice) // Voice .build(); // Synchronous mode: Disable callback (the second parameter is null). SpeechSynthesizer synthesizer = new SpeechSynthesizer(param, null); ByteBuffer audio = null; try { // Block until the audio is returned. audio = synthesizer.call("How is the weather today?"); } catch (Exception e) { throw new RuntimeException(e); } finally { // Close the WebSocket connection after the task is complete. synthesizer.getDuplexApi().close(1000, "bye"); } if (audio != null) { // Save the audio data to the local file "output.mp3". File file = new File("output.mp3"); // The first time you send text, a WebSocket connection must be established. Therefore, the first-packet latency includes the time consumed for connection establishment. // Note: The getFirstPackageDelay() method requires dashscope-sdk-java 2.18.0 or later. System.out.println( "[Metric] Request ID: " + synthesizer.getLastRequestId() + ", First packet delay (ms): " + synthesizer.getFirstPackageDelay()); try (FileOutputStream fos = new FileOutputStream(file)) { fos.write(audio.array()); } catch (IOException e) { throw new RuntimeException(e); } } } public static void main(String[] args) { // The following URL is for the Singapore region. If you use a model in the Beijing region, replace the URL with wss://dashscope.aliyuncs.com/api-ws/v1/inference. Constants.baseWebsocketApiUrl = "wss://dashscope-intl.aliyuncs.com/api-ws/v1/inference"; streamAudioDataToSpeaker(); System.exit(0); } } The following code plays text streamed from the Qwen model (qwen-turbo) on a local device in real time. Important The following sample code must run in an environment with an audio output device, such as speakers or headphones. Running the code on a server without an audio device may cause errors. If you only need to retrieve the synthesized audio data without playback, see the example in Save synthesized audio to a file. Python Java Before you run the Python example, install a third-party audio playback library using pip. python # coding=utf-8 # Installation instructions for pyaudio: # APPLE Mac OS X # brew install portaudio # pip install pyaudio # Debian/Ubuntu # sudo apt-get install python-pyaudio python3-pyaudio # or # pip install pyaudio # CentOS # sudo yum install -y portaudio portaudio-devel && pip install pyaudio # Microsoft Windows # python -m pip install pyaudio import os import pyaudio import dashscope from dashscope.audio.tts_v2 import * from http import HTTPStatus from dashscope import Generation # API keys for the Singapore and China (Beijing) regions are different. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key # If you have not set the environment variable, replace the following line with your Model Studio API key: dashscope.api_key = "sk-xxx" dashscope.api_key = os.environ.get('DASHSCOPE_API_KEY') # The following URL is for the Singapore region. If you use a model in the China (Beijing) region, replace the URL with wss://dashscope.aliyuncs.com/api-ws/v1/inference dashscope.base_websocket_api_url='wss://dashscope-intl.aliyuncs.com/api-ws/v1/inference' # Different model versions require corresponding voices: # cosyvoice-v3-flash/cosyvoice-v3-plus: Use voices such as longanyang. # cosyvoice-v2: Use voices such as longxiaochun_v2. # Each voice supports different languages. When synthesizing non-Chinese languages such as Japanese or Korean, select a voice that supports the corresponding language. For more information, see the CosyVoice voice list. model = "cosyvoice-v3-flash" voice = "longanyang" class Callback(ResultCallback): _player = None _stream = None def on_open(self): print("websocket is open.") self._player = pyaudio.PyAudio() self._stream = self._player.open( format=pyaudio.paInt16, channels=1, rate=22050, output=True ) def on_complete(self): print("speech synthesis task complete successfully.") def on_error(self, message: str): print(f"speech synthesis task failed, {message}") def on_close(self): print("websocket is closed.") # stop player self._stream.stop_stream() self._stream.close() self._player.terminate() def on_event(self, message): print(f"recv speech synthsis message {message}") def on_data(self, data: bytes) -> None: print("audio result length:", len(data)) self._stream.write(data) def synthesizer_with_llm(): callback = Callback() synthesizer = SpeechSynthesizer( model=model, voice=voice, format=AudioFormat.PCM_22050HZ_MONO_16BIT, callback=callback, ) messages = [{"role": "user", "content": "Please introduce yourself"}] responses = Generation.call( model="qwen-turbo", messages=messages, result_format="message", # set result format as 'message' stream=True, # enable stream output incremental_output=True, # enable incremental output ) for response in responses: if response.status_code == HTTPStatus.OK: print(response.output.choices[0]["message"]["content"], end="") synthesizer.streaming_call(response.output.choices[0]["message"]["content"]) else: print( "Request id: %s, Status code: %s, error code: %s, error message: %s" % ( response.request_id, response.status_code, response.code, response.message, ) ) synthesizer.streaming_complete() print('requestId: ', synthesizer.get_last_request_id()) if __name__ == "__main__": synthesizer_with_llm() java import com.alibaba.dashscope.aigc.generation.Generation; import com.alibaba.dashscope.aigc.generation.GenerationParam; import com.alibaba.dashscope.aigc.generation.GenerationResult; import com.alibaba.dashscope.audio.tts.SpeechSynthesisResult; import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisAudioFormat; import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam; import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer; import com.alibaba.dashscope.common.Message; import com.alibaba.dashscope.common.ResultCallback; import com.alibaba.dashscope.common.Role; import com.alibaba.dashscope.exception.InputRequiredException; import com.alibaba.dashscope.exception.NoApiKeyException; import com.alibaba.dashscope.utils.Constants; import io.reactivex.Flowable; import java.nio.ByteBuffer; import java.util.Arrays; import java.util.concurrent.ConcurrentLinkedQueue; import java.util.concurrent.atomic.AtomicBoolean; import javax.sound.sampled.*; public class Main { // Different model versions require corresponding voices: // cosyvoice-v3-flash/cosyvoice-v3-plus: Use voices such as longanyang. // cosyvoice-v2: Use voices such as longxiaochun_v2. // Each voice supports different languages. When synthesizing non-Chinese languages such as Japanese or Korean, select a voice that supports the corresponding language. For more information, see the CosyVoice voice list. private static String model = "cosyvoice-v3-flash"; private static String voice = "longanyang"; public static void process() throws NoApiKeyException, InputRequiredException { // Playback thread class PlaybackRunnable implements Runnable { // Set the audio format. Configure this based on your device, // synthesized audio parameters, and platform. Here, it is set to // 22050 Hz 16-bit single-channel. Choose other // sample rates and formats based on the model sample rate and device // compatibility. private AudioFormat af = new AudioFormat(22050, 16, 1, true, false); private DataLine.Info info = new DataLine.Info(SourceDataLine.class, af); private SourceDataLine targetSource = null; private AtomicBoolean runFlag = new AtomicBoolean(true); private ConcurrentLinkedQueue<ByteBuffer> queue = new ConcurrentLinkedQueue<>(); // Prepare the player. public void prepare() throws LineUnavailableException { targetSource = (SourceDataLine) AudioSystem.getLine(info); targetSource.open(af, 4096); targetSource.start(); } public void put(ByteBuffer buffer) { queue.add(buffer); } // Stop playback. public void stop() { runFlag.set(false); } @Override public void run() { if (targetSource == null) { return; } while (runFlag.get()) { if (queue.isEmpty()) { try { Thread.sleep(100); } catch (InterruptedException e) { } continue; } ByteBuffer buffer = queue.poll(); if (buffer == null) { continue; } byte[] data = buffer.array(); targetSource.write(data, 0, data.length); } // Play all remaining cached audio. if (!queue.isEmpty()) { ByteBuffer buffer = null; while ((buffer = queue.poll()) != null) { byte[] data = buffer.array(); targetSource.write(data, 0, data.length); } } // Release the player. targetSource.drain(); targetSource.stop(); targetSource.close(); } } // Create a subclass that inherits from ResultCallback<SpeechSynthesisResult> // to implement the callback interface. class ReactCallback extends ResultCallback<SpeechSynthesisResult> { private PlaybackRunnable playbackRunnable = null; public ReactCallback(PlaybackRunnable playbackRunnable) { this.playbackRunnable = playbackRunnable; } // Callback for when the service returns the streaming synthesis result. @Override public void onEvent(SpeechSynthesisResult result) { // Get the binary data of the streaming result using getAudio. if (result.getAudioFrame() != null) { // Stream the data to the player. playbackRunnable.put(result.getAudioFrame()); } } // Callback for when the service completes the synthesis. @Override public void onComplete() { // Notify the playback thread to end. playbackRunnable.stop(); } // Callback for when an error occurs. @Override public void onError(Exception e) { // Tell the playback thread to end. System.out.println(e); playbackRunnable.stop(); } } PlaybackRunnable playbackRunnable = new PlaybackRunnable(); try { playbackRunnable.prepare(); } catch (LineUnavailableException e) { throw new RuntimeException(e); } Thread playbackThread = new Thread(playbackRunnable); // Start the playback thread. playbackThread.start(); /******* Call the Generative AI Model to get streaming text *******/ // Prepare for the LLM call. Generation gen = new Generation(); Message userMsg = Message.builder() .role(Role.USER.getValue()) .content("Please introduce yourself") .build(); GenerationParam genParam = GenerationParam.builder() // If you have not configured the API key as an environment variable, uncomment the following line and replace apiKey with your API key. // .apiKey("apikey") .model("qwen-turbo") .messages(Arrays.asList(userMsg)) .resultFormat(GenerationParam.ResultFormat.MESSAGE) .topP(0.8) .incrementalOutput(true) .build(); // Prepare the speech synthesis task. SpeechSynthesisParam param = SpeechSynthesisParam.builder() // To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key // If you have not set the environment variable, replace the following line with your Model Studio API key: .apiKey("sk-xxx") .apiKey(System.getenv("DASHSCOPE_API_KEY")) .model(model) .voice(voice) .format(SpeechSynthesisAudioFormat .PCM_22050HZ_MONO_16BIT) .build(); SpeechSynthesizer synthesizer = new SpeechSynthesizer(param, new ReactCallback(playbackRunnable)); Flowable<GenerationResult> result = gen.streamCall(genParam); result.blockingForEach(message -> { String text = message.getOutput().getChoices().get(0).getMessage().getContent().trim(); if (text != null && !text.isEmpty()) { System.out.println("LLM output: " + text); synthesizer.streamingCall(text); } }); synthesizer.streamingComplete(); System.out.print("requestId: " + synthesizer.getLastRequestId()); try { // Wait for the playback thread to finish playing all audio. playbackThread.join(); } catch (InterruptedException e) { throw new RuntimeException(e); } } public static void main(String[] args) throws NoApiKeyException, InputRequiredException { Constants.baseWebsocketApiUrl = "wss://dashscope-intl.aliyuncs.com/api-ws/v1/inference"; process(); System.exit(0); } } Voice cloning and speech synthesis follow a "create first, then use" workflow: 1. Prepare an audio recording file Upload an audio file that meets the requirements in Voice cloning: Input audio format to a publicly accessible location, such as Alibaba Cloud Object Storage Service (OSS), and ensure the URL is publicly accessible. 2. Create a voice Call the Create voice API. In this step, specifytarget_model ortargetModel to declare which speech synthesis model will drive the created voice. If you already have a voice, skip this step. Check existing voices by calling the Query voice list API. 3. Use the voice for speech synthesis After successfully creating a voice using the Create voice API, the system returns a voice_id or voiceID: - Use this voice_id or voiceID as the voice parameter in the speech synthesis API or language SDKs. - Supports non-streaming, unidirectional streaming, and bidirectional streaming synthesis. - The speech synthesis model specified during synthesis must match the target_model or targetModel used when creating the voice. Otherwise, the synthesis fails. Sample code: python import os import time import dashscope from dashscope.audio.tts_v2 import VoiceEnrollmentService, SpeechSynthesizer # 1. Prepare the environment. # Configure the API key using an environment variable. # The API keys for the Singapore and Beijing regions are different. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key # If an environment variable is not configured, replace the following line with your Model Studio API key: dashscope.api_key = "sk-xxx" dashscope.api_key = os.getenv("DASHSCOPE_API_KEY") if not dashscope.api_key: raise ValueError("DASHSCOPE_API_KEY environment variable not set.") # The following is the WebSocket URL for the Singapore region. If you use a model in the Beijing region, replace the URL with: wss://dashscope.aliyuncs.com/api-ws/v1/inference dashscope.base_websocket_api_url='wss://dashscope-intl.aliyuncs.com/api-ws/v1/inference' # The following is the HTTP URL for the Singapore region. If you use a model in the Beijing region, replace the URL with: https://dashscope.aliyuncs.com/api/v1 dashscope.base_http_api_url = 'https://dashscope-intl.aliyuncs.com/api/v1' # 2. Define the cloning parameters. TARGET_MODEL = "cosyvoice-v3.5-plus" # Give the voice a meaningful prefix. VOICE_PREFIX = "myvoice" # Only digits and lowercase letters are allowed. The prefix must be less than 10 characters long. # A publicly accessible audio URL. AUDIO_URL = "https://dashscope.oss-cn-beijing.aliyuncs.com/samples/audio/cosyvoice/cosyvoice-zeroshot-sample.wav" # This is a sample URL. Replace it with your own. # 3. Create a voice (asynchronous task). print("--- Step 1: Creating voice enrollment ---") service = VoiceEnrollmentService() try: voice_id = service.create_voice( target_model=TARGET_MODEL, prefix=VOICE_PREFIX, url=AUDIO_URL ) print(f"Voice enrollment submitted successfully. Request ID: {service.get_last_request_id()}") print(f"Generated Voice ID: {voice_id}") except Exception as e: print(f"Error during voice creation: {e}") raise e # 4. Poll for the voice status. print("\n--- Step 2: Polling for voice status ---") max_attempts = 30 poll_interval = 10 # seconds for attempt in range(max_attempts): try: voice_info = service.query_voice(voice_id=voice_id) status = voice_info.get("status") print(f"Attempt {attempt + 1}/{max_attempts}: Voice status is '{status}'") if status == "OK": print("Voice is ready for synthesis.") break elif status == "UNDEPLOYED": print(f"Voice processing failed with status: {status}. Please check audio quality or contact support.") raise RuntimeError(f"Voice processing failed with status: {status}") # For intermediate statuses such as "DEPLOYING", continue to wait. time.sleep(poll_interval) except Exception as e: print(f"Error during status polling: {e}") time.sleep(poll_interval) else: print("Polling timed out. The voice is not ready after several attempts.") raise RuntimeError("Polling timed out. The voice is not ready after several attempts.") # 5. Use the cloned voice for speech synthesis. print("\n--- Step 3: Synthesizing speech with the new voice ---") try: synthesizer = SpeechSynthesizer(model=TARGET_MODEL, voice=voice_id) text_to_synthesize = "Congratulations! You have successfully cloned and synthesized your own voice." # The call() method returns binary audio data. audio_data = synthesizer.call(text_to_synthesize) print(f"Speech synthesis successful. Request ID: {synthesizer.get_last_request_id()}") # 6. Save the audio file. output_file = "my_custom_voice_output.mp3" with open(output_file, "wb") as f: f.write(audio_data) print(f"Audio saved to {output_file}") except Exception as e: print(f"Error during speech synthesis: {e}") Voice design and speech synthesis follow a "create first, then use" workflow: 1. Prepare the voice description and preview text required for voice design. - Voice description (voice_prompt): Defines the features of the target voice. See Voice design: Write high-quality voice descriptions. - Preview text (preview_text): The content that the target voice reads for the preview audio, such as "Hello everyone, and welcome." 2. Call the Create voice API to create a custom voice and retrieve the voice name and preview audio. In this step, specifytarget_model to declare which speech synthesis model will drive the created voice. Listen to the preview audio to confirm the voice meets your expectations. If not, redesign the voice. If you already have a voice, skip this step. Check existing voices by calling the Query voice list API. 3. Use the voice for speech synthesis. When a voice is successfully created using the Create voice API, the system returns a voice_id/voiceID. - This voice_id/voiceID can be used as the voice parameter in the speech synthesis API or language SDKs. - Supports non-streaming, unidirectional streaming, and bidirectional streaming synthesis. - The speech synthesis model specified during synthesis must be the same as the target_model/targetModel specified when the voice was created. Otherwise, the synthesis will fail. Sample code: 1. Generate a custom voice and preview the result. If the result meets your expectations, proceed to the next step. Otherwise, regenerate the voice. Python Java python import requests import base64 import os def create_voice_and_play(): # API keys for the Singapore and Beijing regions are different. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key # If you have not configured an environment variable, replace the following line with your Model Studio API key: api_key = "sk-xxx" api_key = os.getenv("DASHSCOPE_API_KEY") if not api_key: print("Error: The DASHSCOPE_API_KEY environment variable is not found. Set the API key first.") return None, None, None # Prepare the request data. headers = { "Authorization": f"Bearer {api_key}", "Content-Type": "application/json" } data = { "model": "voice-enrollment", "input": { "action": "create_voice", "target_model": "cosyvoice-v3.5-plus", "voice_prompt": "A composed middle-aged male announcer with a deep, rich and magnetic voice, a steady speaking speed and clear articulation, is suitable for news broadcasting or documentary commentary.", "preview_text": "Dear listeners, hello everyone. Welcome to the evening news.", "prefix": "announcer" }, "parameters": { "sample_rate": 24000, "response_format": "wav" } } # The following is the URL for the Singapore region. If you use a model in the Beijing region, replace the URL with: https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization url = "https://dashscope-intl.aliyuncs.com/api/v1/services/audio/tts/customization" try: # Send the request. response = requests.post( url, headers=headers, json=data, timeout=60 # Add a timeout setting. ) if response.status_code == 200: result = response.json() # Get the voice ID. voice_id = result["output"]["voice_id"] print(f"Voice ID: {voice_id}") # Get the preview audio data. base64_audio = result["output"]["preview_audio"]["data"] # Decode the Base64 audio data. audio_bytes = base64.b64decode(base64_audio) # Save the audio file to a local file. filename = f"{voice_id}_preview.wav" # Write the audio data to the local file. with open(filename, 'wb') as f: f.write(audio_bytes) print(f"Audio saved to local file: {filename}") print(f"File path: {os.path.abspath(filename)}") return voice_id, audio_bytes, filename else: print(f"Request failed. Status code: {response.status_code}") print(f"Response content: {response.text}") return None, None, None except requests.exceptions.RequestException as e: print(f"A network request error occurred: {e}") return None, None, None except KeyError as e: print(f"The response data format is invalid. A required field is missing: {e}") print(f"Response content: {response.text if 'response' in locals() else 'No response'}") return None, None, None except Exception as e: print(f"An unknown error occurred: {e}") return None, None, None if __name__ == "__main__": print("Creating the voice...") voice_id, audio_data, saved_filename = create_voice_and_play() if voice_id: print(f"\nVoice '{voice_id}' created successfully.") print(f"Audio file saved: '{saved_filename}'") print(f"File size: {os.path.getsize(saved_filename)} bytes") else: print("\nFailed to create the voice.") Import the Gson dependency. If you use Maven or Gradle, add the dependency as follows: Maven Gradle In the pom.xml file, add the following content: xml <dependency> <groupId>com.google.code.gson</groupId> <artifactId>gson</artifactId> <version>2.13.1</version> </dependency> In the build.gradle file, add the following content: gradle // https://mvnrepository.com/artifact/com.google.code.gson/gson implementation("com.google.code.gson:gson:2.13.1") java import com.google.gson.JsonObject; import com.google.gson.JsonParser; import java.io.*; import java.net.HttpURLConnection; import java.net.URL; import java.util.Base64; public class Main { public static void main(String[] args) { Main example = new Main(); example.createVoice(); } public void createVoice() { // API keys for the Singapore and Beijing regions are different. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key // If you have not configured an environment variable, replace the following line with your Model Studio API key: String apiKey = "sk-xxx" String apiKey = System.getenv("DASHSCOPE_API_KEY"); // Create a JSON request body string. String jsonBody = "{\n" + " \"model\": \"voice-enrollment\",\n" + " \"input\": {\n" + " \"action\": \"create_voice\",\n" + " \"target_model\": \"cosyvoice-v3.5-plus\",\n" + " \"voice_prompt\": \"A composed middle-aged male announcer with a deep, rich and magnetic voice, a steady speaking speed and clear articulation, is suitable for news broadcasting or documentary commentary.\",\n" + " \"preview_text\": \"Dear listeners, hello everyone. Welcome to the evening news.\",\n" + " \"prefix\": \"announcer\"\n" + " },\n" + " \"parameters\": {\n" + " \"sample_rate\": 24000,\n" + " \"response_format\": \"wav\"\n" + " }\n" + "}"; HttpURLConnection connection = null; try { // The following is the URL for the Singapore region. If you use a model in the Beijing region, replace the URL with: https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization URL url = new URL("https://dashscope-intl.aliyuncs.com/api/v1/services/audio/tts/customization"); connection = (HttpURLConnection) url.openConnection(); // Set the request method and headers. connection.setRequestMethod("POST"); connection.setRequestProperty("Authorization", "Bearer " + apiKey); connection.setRequestProperty("Content-Type", "application/json"); connection.setDoOutput(true); connection.setDoInput(true); // Send the request body. try (OutputStream os = connection.getOutputStream()) { byte[] input = jsonBody.getBytes("UTF-8"); os.write(input, 0, input.length); os.flush(); } // Get the response. int responseCode = connection.getResponseCode(); if (responseCode == HttpURLConnection.HTTP_OK) { // Read the response content. StringBuilder response = new StringBuilder(); try (BufferedReader br = new BufferedReader( new InputStreamReader(connection.getInputStream(), "UTF-8"))) { String responseLine; while ((responseLine = br.readLine()) != null) { response.append(responseLine.trim()); } } // Parse the JSON response. JsonObject jsonResponse = JsonParser.parseString(response.toString()).getAsJsonObject(); JsonObject outputObj = jsonResponse.getAsJsonObject("output"); JsonObject previewAudioObj = outputObj.getAsJsonObject("preview_audio"); // Get the voice name. String voiceId = outputObj.get("voice_id").getAsString(); System.out.println("Voice ID: " + voiceId); // Get the Base64-encoded audio data. String base64Audio = previewAudioObj.get("data").getAsString(); // Decode the Base64 audio data. byte[] audioBytes = Base64.getDecoder().decode(base64Audio); // Save the audio to a local file. String filename = voiceId + "_preview.wav"; saveAudioToFile(audioBytes, filename); System.out.println("Audio saved to local file: " + filename); } else { // Read the error response. StringBuilder errorResponse = new StringBuilder(); try (BufferedReader br = new BufferedReader( new InputStreamReader(connection.getErrorStream(), "UTF-8"))) { String responseLine; while ((responseLine = br.readLine()) != null) { errorResponse.append(responseLine.trim()); } } System.out.println("Request failed. Status code: " + responseCode); System.out.println("Error response: " + errorResponse.toString()); } } catch (Exception e) { System.err.println("Request error: " + e.getMessage()); e.printStackTrace(); } finally { if (connection != null) { connection.disconnect(); } } } private void saveAudioToFile(byte[] audioBytes, String filename) { try { File file = new File(filename); try (FileOutputStream fos = new FileOutputStream(file)) { fos.write(audioBytes); } System.out.println("Audio saved to: " + file.getAbsolutePath()); } catch (IOException e) { System.err.println("Error saving audio file: " + e.getMessage()); e.printStackTrace(); } } } 2. Use the custom voice generated in the previous step for speech synthesis. This example is based on the non-streaming call sample. Replace the voice parameter with the voice ID from voice design. Key principle: The model used for voice design (target_model) must match the model used for synthesis (model). Mismatches cause synthesis to fail. Python Java python # coding=utf-8 import dashscope from dashscope.audio.tts_v2 import * import os # API keys for the Singapore and Beijing regions are different. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key # If you have not configured an environment variable, replace the following line with your Model Studio API key: dashscope.api_key = "sk-xxx" dashscope.api_key = os.environ.get('DASHSCOPE_API_KEY') # The following is the URL for the Singapore region. If you use a model in the Beijing region, replace the URL with: wss://dashscope.aliyuncs.com/api-ws/v1/inference dashscope.base_websocket_api_url='wss://dashscope-intl.aliyuncs.com/api-ws/v1/inference' # The same model must be used for voice design and speech synthesis. model = "cosyvoice-v3.5-plus" # Replace the voice parameter with the custom voice generated by voice design. voice = "your_voice" # Instantiate SpeechSynthesizer and pass request parameters such as model and voice in the constructor. synthesizer = SpeechSynthesizer(model=model, voice=voice) # Send the text to be synthesized and get the binary audio. audio = synthesizer.call("How is the weather today?") # When you send text for the first time, a WebSocket connection must be established. Therefore, the first-packet latency includes the time taken to establish the connection. print('[Metric] Request ID: {}, First-packet latency: {} ms'.format( synthesizer.get_last_request_id(), synthesizer.get_first_package_delay())) # Save the audio to a local file. with open('output.mp3', 'wb') as f: f.write(audio) java import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam; import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer; import com.alibaba.dashscope.utils.Constants; import java.io.File; import java.io.FileOutputStream; import java.io.IOException; import java.nio.ByteBuffer; public class Main { // The same model must be used for voice design and speech synthesis. private static String model = "cosyvoice-v3.5-plus"; // Replace the voice parameter with the custom voice ID generated by voice design. private static String voice = "your_voice_id"; public static void streamAudioDataToSpeaker() { // Request parameters. SpeechSynthesisParam param = SpeechSynthesisParam.builder() // API keys for the Singapore and Beijing regions are different. To get an API key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key // If you have not configured an environment variable, replace the following line with your Model Studio API key: .apiKey("sk-xxx") .apiKey(System.getenv("DASHSCOPE_API_KEY")) .model(model) // Model .voice(voice) // Voice .build(); // Synchronous mode: Disable the callback (the second parameter is null). SpeechSynthesizer synthesizer = new SpeechSynthesizer(param, null); ByteBuffer audio = null; try { // Block until the audio is returned. audio = synthesizer.call("How is the weather today?"); } catch (Exception e) { throw new RuntimeException(e); } finally { // After the task is complete, close the WebSocket connection. synthesizer.getDuplexApi().close(1000, "bye"); } if (audio != null) { // Save the audio data to the local file "output.mp3". File file = new File("output.mp3"); // When you send text for the first time, a WebSocket connection must be established. Therefore, the first-packet latency includes the time taken to establish the connection. // Note: The getFirstPackageDelay() method requires dashscope-sdk-java 2.18.0 or later. System.out.println( "[Metric] Request ID: " + synthesizer.getLastRequestId() + ", First-packet latency (ms): " + synthesizer.getFirstPackageDelay()); try (FileOutputStream fos = new FileOutputStream(file)) { fos.write(audio.array()); } catch (IOException e) { throw new RuntimeException(e); } } } public static void main(String[] args) { // The following is the URL for the Singapore region. If you use a model in the Beijing region, replace the URL with: wss://dashscope.aliyuncs.com/api-ws/v1/inference Constants.baseWebsocketApiUrl = "wss://dashscope-intl.aliyuncs.com/api-ws/v1/inference"; streamAudioDataToSpeaker(); System.exit(0); } } |
Voice cloning: Input audio format
High-quality input audio produces better cloning results.
| Item | Requirements |
|---|
| Item | Requirements |
| Supported formats | WAV (16-bit), MP3, M4A |
| Audio duration | Recommended: 10 to 20 seconds. Maximum: 60 seconds. |
| File size | ≤ 10 MB |
| Sample rate | ≥ 16 kHz |
| Sound channel | Mono or stereo. For stereo audio, only the first channel is processed. Make sure that the first channel contains a clear human voice. |
| Content | The audio must contain at least 5 seconds of continuous, clear speech without background sound. The rest of the audio can have only short pauses (≤ 2 seconds). The entire audio segment should be free of background music, noise, or other voices to ensure high-quality core speech content. Use normal spoken audio as input. Do not upload songs or singing audio to ensure accuracy and usability of the cloning effect. |
Voice design: Write high-quality voice descriptions
Requirements and limitations
When writing voice descriptions (voice_prompt), follow these technical constraints:
Length limit: The content of
voice_promptmust not exceed 500 characters.Supported languages: The description text supports only Chinese and English.
Core principles
A high-quality voice description (voice_prompt) guides the model to generate a voice with specific characteristics. Follow these principles:
Be specific, not vague: Use words that describe concrete sound qualities, such as "deep," "crisp," or "fast-paced." Avoid subjective, uninformative terms such as "nice-sounding" or "ordinary."
Be multidimensional, not single-dimensional: Excellent descriptions typically combine multiple dimensions, such as gender, age, and emotion. Single-dimensional descriptions, such as "female voice," are too broad to generate distinctive voices.
Be objective, not subjective: Focus on the physical and perceptual characteristics of the sound itself, not your personal preferences. For example, use "high-pitched with energetic delivery" instead of "my favorite voice."
Be original, not imitative: Describe sound characteristics rather than requesting imitation of specific individuals, such as celebrities or actors. Such requests pose copyright risks, and the model does not support direct imitation.
Be concise, not redundant: Ensure every word adds meaning. Avoid repeating synonyms or using meaningless intensifiers, such as "very nice voice."
Description dimension reference
| Dimension | Example |
|---|
| Dimension | Example |
| Gender | Male, female, neutral |
| Age | Child (5-12 years), teenager (13-18 years), young adult (19-35 years), middle-aged (36-55 years), senior (55+ years) |
| Pitch | High, medium, low, slightly high, slightly low |
| Speech rate | Fast, medium, slow, slightly fast, slightly slow |
| Emotion | Cheerful, calm, gentle, serious, lively, cool, soothing |
| Characteristics | Magnetic, crisp, raspy, mellow, sweet, rich, powerful |
| Purpose | News broadcasting, advertisement voice-over, audiobooks, animated characters, voice assistants, documentary narration |
Example comparison
Good cases
- "Young and lively female voice, fast speech rate with noticeable rising intonation, suitable for introducing fashion products."
Analysis: This description combines age, personality, speech rate, and intonation, and specifies the use case, creating a clear voice profile.
- "Calm middle-aged male, slow speech rate, deep and magnetic voice quality, suitable for reading news or documentary narration."
Analysis: This description clearly defines gender, age range, speech rate, voice quality, and intended use.
- "Cute child's voice, approximately 8-year-old girl, slightly childish speech, suitable for animated character dubbing."
Analysis: This description pinpoints the specific age and voice quality (childishness) and has a clear purpose.
- "Gentle and intellectual female, around 30 years old, calm tone, suitable for audiobook narration."
Analysis: This description effectively conveys voice emotion and style through terms such as "intellectual" and "calm."
Bad cases and suggestions
| Poor example | Main issue | Improvement suggestion |
|---|
| Poor example | Main issue | Improvement suggestion |
| 'Nice-sounding voice' | This description is too vague and subjective, and lacks actionable detail. | Add specific dimensions, such as "Clear-toned young female voice with gentle intonation." |
| 'Voice like a celebrity' | This poses a copyright risk. The model does not support direct imitation. | Extract the voice characteristics for the description, such as "Mature, magnetic, steady-paced male voice." |
| 'Very nice female voice' | This description is redundant. Repeating words does not help define the voice. | Remove repetitions and add effective descriptions, such as "A 20- to 24-year-old female voice with a light, cheerful tone, lively pitch, and sweet quality." |
| 123456 | This is an invalid input. It cannot be parsed as voice characteristics. | Provide a meaningful text description. See the examples above. |
API reference
Speech synthesis - CosyVoice API reference
Voice cloning/design - CosyVoice API reference
Compare models
International
Chinese mainland
If you select the International deployment scope, model inference compute resources are dynamically scheduled worldwide, excluding the Chinese mainland. Static data is stored in your selected region. Supported region: Singapore.
| Features | cosyvoice-v3-plus | cosyvoice-v3-flash |
| Supported languages | Varies by system voice: Chinese (Mandarin, Northeastern, Minnan, Shaanxi), English, Japanese, Korean | Varies by system voice: Chinese (Mandarin), English |
| Audio format | pcm, wav, mp3, opus | |
| Audio sample rate | 8 kHz, 16 kHz, 22.05 kHz, 24 kHz, 44.1 kHz, 48 kHz | |
| Voice cloning | Supported > For usage instructions, see CosyVoice voice cloning/design API > The following languages are supported for voice cloning: > cosyvoice-v3-flash: Chinese (Mandarin, Cantonese, Northeastern, Gansu, Guizhou, Henan, Hubei, Jiangxi, Minnan, Ningxia, Shanxi, Shaanxi, Shandong, Shanghai, Sichuan, Tianjin, Yunnan), English, French, German, Japanese, Korean, Russian, Portuguese, Thai, Indonesian, and Vietnamese. > cosyvoice-v3-plus: Chinese (Mandarin), English, French, German, Japanese, Korean, and Russian. | |
| Voice design | Supported > See CosyVoice voice cloning/design API > The following languages are supported for voice design: Chinese and English. | |
| SSML | Supported > This feature applies to cloned voices and system voices in the Voice list marked as supporting SSML. > For usage instructions, see SSML | |
| LaTeX | Supported > For usage instructions, see LaTeX formula-to-speech | |
| Volume adjustment | Supported > See request parameter volume | |
| Speech rate adjustment | Supported > For usage, see request parameter speech_rate > In the Java SDK, this parameter is speechRate | |
| Pitch adjustment | Supported > For usage, see the request parameter pitch_rate > In the Java SDK, this parameter is pitchRate | |
| Bitrate adjustment | Supported > Only the opus audio format supports this feature. > For usage instructions, see the request parameter bit_rate > In the Java SDK, this parameter is pitchRate | |
| Timestamp | SupportedDisabled by default but can be enabled. > This feature applies to cloned voices and system voices in the Voice list marked as supporting timestamps. > For usage instructions, see the request parameter word_timestamp_enabled > In the Java SDK, this parameter is enableWordTimestamp | |
| Instruction control (Instruct) | Not supported | Supported > This feature applies to system voices in the Voice list marked as supporting Instruct > See request parameter instruction |
| Streaming input | Supported | |
| Streaming output | Supported | |
| Rate limiting (RPS) | 3 | |
| Connection type | Java/Python SDK, WebSocket API | |
| Price | $0.26 per 10,000 characters | $0.13 per 10,000 characters |
If you select the Chinese mainland deployment scope, model inference compute resources are restricted to the Chinese mainland. Static data is stored in your selected region. Supported region: China (Beijing).
| Features | cosyvoice-v3.5-plus | cosyvoice-v3.5-flash | cosyvoice-v3-plus | cosyvoice-v3-flash | cosyvoice-v2 |
| Supported languages | No system voices. Cloned voices support the following languages: Chinese (Mandarin, Cantonese, Henan, Hubei, Minnan, Ningxia, Shaanxi, Shandong, Shanghai, Sichuan), English, French, German, Japanese, Korean, Russian, Portuguese, Thai, Indonesian, and Vietnamese. Designed voices support the following languages: Chinese (Mandarin) and English. | System voices (varies by voice): Chinese (Mandarin, Northeastern, Minnan, Shaanxi), English, Japanese, Korean Cloned voices: Chinese (Mandarin), English, French, German, Japanese, Korean, and Russian. | System voices (varies by voice): Chinese (Mandarin), English Cloned voices: Chinese (Mandarin, Cantonese, Northeastern, Gansu, Guizhou, Henan, Hubei, Jiangxi, Minnan, Ningxia, Shanxi, Shaanxi, Shandong, Shanghai, Sichuan, Tianjin, Yunnan), English, French, German, Japanese, Korean, Russian, Portuguese, Thai, Indonesian, and Vietnamese. | System voices (varies by voice): Chinese (Mandarin), English, Korean, Japanese Cloned voices: Chinese (Mandarin) and English. | |
| Audio format | pcm, wav, mp3, opus | ||||
| Audio sample rate | 8 kHz, 16 kHz, 22.05 kHz, 24 kHz, 44.1 kHz, 48 kHz | ||||
| Voice cloning | Supported > For usage instructions, see CosyVoice voice cloning/design API > The following languages are supported for voice cloning: > cosyvoice-v2: Chinese (Mandarin) and English. > cosyvoice-v3-flash: Chinese (Mandarin, Cantonese, Northeastern, Gansu, Guizhou, Henan, Hubei, Jiangxi, Minnan, Ningxia, Shanxi, Shaanxi, Shandong, Shanghai, Sichuan, Tianjin, Yunnan), English, French, German, Japanese, Korean, Russian, Portuguese, Thai, Indonesian, and Vietnamese. > cosyvoice-v3-plus: Chinese (Mandarin), English, French, German, Japanese, Korean, and Russian. > cosyvoice-v3.5-plus, cosyvoice-v3.5-flash: Chinese (Mandarin, Cantonese, Henan, Hubei, Minnan, Ningxia, Shaanxi, Shandong, Shanghai, Sichuan), English, French, German, Japanese, Korean, Russian, Portuguese, Thai, Indonesian, and Vietnamese. | ||||
| Voice design | Supported > For usage instructions, see CosyVoice voice cloning/design API > The following languages are supported for voice design: Chinese and English. | Not supported | |||
| SSML | Supported > This feature applies to cloned voices and system voices in the Voice list marked as supporting SSML. > For usage instructions, see Introduction to SSML | ||||
| LaTeX | Supported > For usage instructions, see LaTeX formula-to-speech | ||||
| Volume adjustment | Supported > For usage instructions, see the request parameter volume | ||||
| Speech rate adjustment | Supported > For usage, see the request parameter speech_rate > In the Java SDK, this parameter is speechRate | ||||
| Pitch adjustment | Supported > See request parameter pitch_rate > In the Java SDK, this parameter is pitchRate | ||||
| Bitrate adjustment | Supported > Only the opus audio format supports this feature. > For usage instructions, see the request parameter bit_rate > In the SDK for Java, this parameter is pitchRate | ||||
| Timestamp | SupportedDisabled by default but can be enabled. > This feature applies to cloned voices and system voices in the Voice list marked as supporting timestamps. > See request parameter word_timestamp_enabled > In the Java SDK, this parameter is enableWordTimestamp | ||||
| Instruction control (Instruct) | Supported > This feature applies to cloned voices and system voices in the Voice list that are marked as supporting Instruct. > Suitable for scenarios that require exaggerated expressiveness, such as video dubbing and audiobook narration. If you want to preserve the original timbre and prosody, you do not need to enable this feature. > Instruct commands may not take effect if they conflict with the inherent style of the voice. For example, applying a sad instruction to a cheerful voice may not produce the expected result. > For usage instructions, see the request parameter instruction | Not supported | Supported > This feature applies to cloned voices and system voices in the Voice list marked as supporting Instruct. > For usage, see the request parameter instruction | Not supported | |
| Streaming input | Supported | ||||
| Streaming output | Supported | ||||
| Rate limiting (RPS) | 3 | ||||
| Connection type | Java/Python SDK, WebSocket API | ||||
| Price | $0.22 per 10,000 characters | $0.116 per 10,000 characters | $0.286706 per 10,000 characters | $0.14335 per 10,000 characters | $0.286706 per 10,000 characters |
System voices
CosyVoice voice list
FAQ
Q: What should I do if speech synthesis produces incorrect pronunciations? How can I control the pronunciation of characters with multiple pronunciations?
Replace characters that have multiple pronunciations with homophones to quickly resolve pronunciation issues.
Use Speech Synthesis Markup Language (SSML) to control pronunciation precisely.
Q: How do I troubleshoot silent audio output from a cloned voice?
- Confirm the voice status.
Call the CosyVoice voice cloning/design API and check whether the voice status is OK.
- Check model version consistency.
Ensure the target_model parameter used for voice cloning exactly matches the model parameter used for speech synthesis. For example:
When cloning, use
cosyvoice-v3-plus.Also use
cosyvoice-v3-plusfor synthesis.
- Verify source audio quality.
Verify that the source audio used for voice cloning meets the requirements in the CosyVoice voice cloning/design API:
Audio duration: 10-20 seconds
Clear audio quality
No background noise
- Check the request parameters.
Confirm that the voice parameter in the speech synthesis request is set to the ID of the cloned voice.
Q: What should I do if the synthesis effect is unstable or the speech is incomplete after voice cloning?
If the synthesized speech after voice cloning has the following issues:
Incomplete playback (only part of the text is spoken)
Inconsistent synthesis quality
Abnormal pauses or silent segments in the speech
Likely cause: The source audio quality does not meet requirements.
Solution: Verify that the source audio meets the following requirements. If not, re-record the audio following the Recording operation guide.
Check audio continuity: Ensure the source audio contains uninterrupted speech with no pauses or silent segments longer than 2 seconds. If the audio contains significant silent gaps, the model may treat the silence or noise as part of the voice profile, degrading output quality.
Check speech activity ratio: Ensure active speech comprises at least 60% of the total audio duration. Excessive background noise or non-speech segments can interfere with voice feature extraction.
Verify the audio quality details:
Audio duration: 10-20 seconds (15 seconds is recommended)
Clear pronunciation and a stable speech rate
No background noise, echo, or static
Consistent speech levels with no long silent gaps
Previous：NoneNext: Product introduction
Is this page helpful?
Core features
Availability
Model selection
Getting started
Voice cloning: Input audio format
Voice design: Write high-quality voice descriptions
Requirements and limitations
Core principles
Description dimension reference
Example comparison
API reference
Compare models
System voices
FAQ
Q: What should I do if speech synthesis produces incorrect pronunciations? How can I control the pronunciation of characters with multiple pronunciations?
Q: How do I troubleshoot silent audio output from a cloned voice?
Q: What should I do if the synthesis effect is unstable or the speech is incomplete after voice cloning?
Contact Us
Sales Support
Live-chat with our sales team or get in touch with a business development professional in your region.
Contact Sales
Technical Support
Open a ticket and get quick help from our technical team.
Open a Ticket >
Connect & Report Abuse
We look forward to your suggestion.
Post a Suggestion > Report Abuse >
Chat now with Alibaba Cloud Customer Service to assist you in finding the right products and services to meet your needs.
\ \ Hi, I'm Alibaba Cloud AI Assistant!\ \ I can help with questions and solutions.
Why Alibaba Cloud
About Alibaba Cloud
Asia Accelerator
Our Global Network
Global Offices
Trust Center
Case Studies
Analyst Reports
Products & Pricings
Pricing Calculator
ECS
SAS
Model Studio
Database
Security
SMS
Solutions
Financial Services
Retail Services
Media Services
Gaming Services
ISV Solutions
Engage
Developer Community
Partner Network
Startups
Marketplace
Join Alibaba Cloud
Resources & Support
Developer Learning Hub
Documentation Center
Training & Certification
Service Notices
Submit a Ticket
Security Report
Qwen Cloud
Careers About Us Privacy Policy Legal Integrity Compliance Reporting Channel Service Notices Links
© 2009-2026 Copyright by Alibaba Cloud All rights reserved
- YouTube
- TikTok
- contact.us@alibabacloud.com
- Call Us Now
- Discord
© 2009-2026 Copyright by Alibaba Cloud All rights reserved
Careers About Us Privacy Policy Legal Integrity Compliance Reporting Channel Service Notices Links