A custom synthetic voice can be built from recordings, created with a voice-design tool, or selected from a library, depending on the job you need it to do. You may be making videos, building an app, or creating audio content. The right path depends on what you want the voice to do.
Making a voice model can mean training a model, cloning a voice with a tool, or using a ready-made voice. These options differ in effort, control, and how closely the result matches a chosen speaker. A simple voice generator may be enough for one project, while a custom build may suit repeated work.
This guide gives you a practical path from choosing the task to testing a responsible result. You will see where audio quality matters, what technology choices change, and how to use the result in content. Start by deciding whether you need a voice to read your words or a model that changes how recorded speech sounds.
Key Takeaways
- Choose speech from text or conversion of recorded speech first.
- Use only audio you own or have clear permission to model.
- Clean, consistent recordings give training a better chance.
- Check hardware needs, license terms, and plan limits before choosing tools.
- Test a short script and review the exported audio before sharing it.
Should You Build TTS, Clone a Voice, or Convert Speech?
If you want typed words spoken aloud, choose TTS; if you want recorded speech to sound like another speaker, choose voice conversion. The task decides which voice technology fits. Prosody means the rhythm, pauses, pitch movement, and emphasis that shape how speech sounds.
Voice AI is a broad label for speech generation and voice-changing systems, not one particular model. Speech can sound natural or stiff depending on the model, the voice, and how you prepare the words. Start with the output, not the name of a tool.
Text-to-speech creates speech from written text
Text-to-speech, or TTS, turns a written script into spoken audio. It can narrate a video, read an app message, or speak a short announcement. A TTS model is a voice model trained from text and audio, and it learns how words and speech patterns connect.
Voice synthesis can use concatenative, formant, or neural network-based methods. Concatenative systems join recorded speech pieces. Formant systems create speech from sound rules, while neural methods learn patterns from examples. Each approach can affect pronunciation, tone, and prosody.
Voice cloning and conversion use recorded speech differently
Voice cloning makes a model resemble a speaker, often by learning from that speaker’s recordings. A permitted likeness might read new lines for a game character or a creator’s own videos. Cloning changes who seems to be speaking, while TTS still generates words from text.
Retrieval-based voice conversion, or RVC, changes a source speaker’s voice toward a target speaker’s voice without changing the speech content. You might convert a sung performance while keeping its words and melody. Research on spectral voice conversion also shows how conversion methods can map one voice’s sound patterns toward another’s.
Choose TTS for script narration, and choose conversion only when you have both source speech and permission to use the target voice.
These methods can sound different even when they share a target voice. A weak source recording can affect conversion, while a script with unfamiliar names can challenge TTS. Choose the workflow that matches the material you already have and the audio you need.
How to Make an AI Voice Model from Scratch
A from-scratch build starts with a clear output target, such as spoken narration from text or converted speech from an audio recording. How to make an AI voice model from scratch means planning the full cycle: data collection, preprocessing, training, fine-tuning, testing, and deployment.
Deep learning learns patterns in speech data rather than relying only on hand-written pronunciation rules. It can learn how sounds, words, and voice traits relate, but the result still depends on the data and the model. Set a narrow target before you begin.
Set the target voice and output task
Write down what the model should make and where you will use it. A neural TTS route generates new speech from text. A voice-conversion route changes recorded speech toward a target voice, so it needs source audio during use.
Neither route is effortless, and neither guarantees natural speech. A voice model can misread names, flatten emphasis, or add odd pauses. Your test should include words, sentence lengths, and speaking styles that match the real task.
Make four decisions before you collect or process data:
- Target use: Choose narration, app speech, character lines, or converted audio.
- Lawful speaker data: Use your own recordings or audio you are allowed to model.
- Model family: Pick neural TTS or a voice-conversion model for the task.
- Available compute: Decide whether you can train locally or need cloud hardware.
Choose a model architecture and computing route
Deep-learning voice systems may use recurrent neural networks, or RNNs, and convolutional neural networks, or CNNs. RNNs process sequences, while CNNs find patterns across parts of data. These are examples, not a guarantee that one architecture suits every project.
A neural TTS model learns a link between text and audio. A voice-conversion model learns how to shift the sound of source speech toward a target speaker. Both routes need suitable training data, a way to tune the model, and tests that reflect real use.
A computer with limited memory may not handle every model or training job. Cloud computing can provide more power, though it may add setup steps or cost. A guide to running AI models locally can help you compare memory needs and offline use before downloading a model.
A 1998 paper on spectral voice conversion describes mapping voice features and adjusting pitch toward a target speaker. That example shows why conversion is more than changing a pitch slider. Write down the intended use and compute you can access before settling on a model architecture.
What Voice Data Makes a Model Sound Natural?
A useful dataset is clean, consistent speech from a speaker whose voice you are authorized to model. Permission and recording quality both matter, and a short or noisy recording does not guarantee a good clone. Keep the voice, microphone setup, and delivery steady across samples.
Record in a quiet room with the microphone at a steady distance. Speak at a consistent volume and use natural conversation rather than a forced announcer voice. Include varied words and speaking styles that match the model’s intended use.
Record a clean, consistent voice sample
An RVC training dataset may start with 10-15 minutes of clean speech, while 30 minutes or more is a stronger target. More audio alone does not ensure quality; clear speech and useful variety matter. Avoid music, competing voices, and large changes in room sound.
WAV files are ideally 16-bit PCM at 44,100 Hz. Keeping individual clips under 10 seconds is a practical preparation tip. Short clips are easier to organize and review, but do not cut words or breathing in ways that change the voice.
Read different sentence types, including questions, statements, and longer lines. Use the accents and words the finished voice needs to handle. A narrow set of repeated phrases may leave the model less prepared for new text or speech.
Preprocess clips without erasing speech detail
Preprocessing makes files easier to train with, but heavy cleanup can harm speech detail. Audacity works for basic editing, while Adobe Audition offers more advanced editing. iZotope RX can help restore damaged audio when simple edits are not enough.
- Trim excess silence without cutting the start or end of speech.
- Normalize levels so clips have a more consistent volume.
- Remove distracting clicks or hiss without damaging speech.
- Name and organize files consistently for easy review.
Listen to processed clips with headphones before you use them. Noise removal can leave watery sounds, and aggressive trimming can make speech feel clipped. Keep an untouched copy of every original recording so you can redo processing if cleanup introduces artifacts.
How Do You Train and Export an RVC Model?
An RVC model is trained by feeding prepared voice clips into a compatible training workflow and checking the run as it learns. Training settings depend on the notebook or interface and the available compute. Google Colab and Google Drive can form one cloud workflow, but neither is required.
Google Colab can provide cloud GPU access and connect to Google Drive. A local interface is another route if your computer can handle the work. Use a trusted workflow and keep track of where it saves each output file.
Train the model in a notebook or local environment
Follow these steps to train and prepare an RVC model:
- Open a trusted RVC notebook or local interface.
- Connect the folder that holds the prepared dataset.
- Set output paths so you can find saved files later.
- Choose training settings for that notebook and monitor the run.
- Save the model files after training finishes.
- Load the files into an RVC-compatible interface.
Example notebook settings include 100-300 epochs and batch sizes of 4 or 8. Those are examples, not universal settings. The right values depend on the notebook and compute, so watch for errors and follow its instructions.
Example training may take 30 minutes to 3 hours or more, depending on dataset size and GPU availability. A run can take longer when compute is limited. Do not judge a model only by how quickly the training screen reaches its final step.
Export files and run a conversion test
Common RVC outputs include model.pth and config.json. The model.pth file contains trained weights, while config.json contains configuration information. The two files work together when an RVC-compatible interface loads the model.
Use an RVC-GUI conversion of a short test recording to check the export. Listen for clear words, stable tone, and rough or metallic sounds. A clean conversion test can show that the files load, but it does not prove the model will suit every recording.
Keep the training notebook or local interface’s output notes with the saved files. If the interface cannot find a file, check the output path before repeating training. Keep the model weights, configuration, and original dataset together, with a clear record of which authorized speaker they represent.
Which Free or Open-Source Route Fits Your Setup?
A free route is most practical when its hardware needs, cloning features, and license fit your project. Free can mean no software license fee, browser-based generation, or a free allowance, and those options are not the same. Check the exact model terms before use.
Kokoro and Piper can generate speech without cloning a voice. Chatterbox-Turbo, Qwen3-TTS, and Fish Audio S2 support cloning, with different model sizes and hardware needs. Match the model to your computer, not just the feature list.
Choose a lightweight model for local speech generation
Kokoro is an 82-million-parameter open-source TTS model under Apache 2.0. It can run on a CPU and has 54 preset voices, but it does not clone voices. Piper is a small MIT-licensed model for a CPU or Raspberry Pi, and it also does not support voice cloning.
SoundTools runs Kokoro in a browser through ONNX Runtime Web. It says text is not uploaded to a server and offers WAV and MP3 downloads without a watermark or character limit. That browser speech generator is not the same as training a custom voice model.
Local AI model setup advice can help you check memory needs before you download weights. The Apache 2.0 license allows use, changes, and sharing under stated conditions, but it does not grant trademark rights. Read the Apache 2.0 license terms and the specific model license.
Check cloning support, hardware, and commercial terms
These five options differ in size, cloning support, and hardware needs:
| Model | Size and license | Voice cloning | Hardware or notable limit |
|---|---|---|---|
| Kokoro | 82M, Apache 2.0 | No | Can run on a CPU |
| Piper | Small, MIT | No | CPU or Raspberry Pi |
| Chatterbox-Turbo | 350M, MIT | Yes, 7-10 seconds | About 6GB VRAM |
| Qwen3-TTS | 0.6B or 1.7B, Apache 2.0 | Yes, 3 seconds | 4-8GB VRAM |
| Fish Audio S2 | 4B, open-weight | Yes | 12-24GB VRAM; paid commercial license |
Qwen3-TTS is listed under Apache 2.0, while Fish Audio S2 has a separate research license and requires a paid commercial license. Chatterbox-Turbo uses an MIT license and can clone from 7-10 seconds of audio. A short sample does not guarantee a close match.
Chatterbox Multilingual is a 0.5B-parameter MIT model that clones from 7-10 seconds across 23 languages. CosyVoice 3.0 is a 0.5B-parameter Apache 2.0 model with zero-shot cloning, and F5-TTS is a roughly 0.3B-parameter MIT model with cloning support.
A vendor-run blind test preferred Chatterbox’s cloning model over ElevenLabs Turbo by 65%. Treat that result cautiously, since one test does not prove universal quality. Confirm the model’s license and your computer’s memory needs before downloading weights.
Which Online Voice Generators and TTS APIs Fit Your Budget?
An online voice generator is easiest to start using, but the right option depends on whether you need a cloned voice, a preset voice, or an API. Compare billing units and limits before picking a plan. A Google AI voice generator can refer to Google Cloud Text-to-Speech.
How to make an AI voice model online may mean cloning a voice, designing one, or generating speech with a preset voice. Those workflows are not interchangeable. Choose for your actual output, then estimate the text or audio use you expect.
Compare free tiers, character billing, and API limits
Google Cloud Text-to-Speech offers monthly free character allowances for these voice types:
| Google Cloud voice type | Free monthly characters | Price per character after allowance |
|---|---|---|
| Chirp 3: HD | 1 million | US$0.00003 |
| WaveNet | 4 million | US$0.000004 |
| Neural2 | 1 million | US$0.000016 |
Google Cloud counts spaces, newline characters, and SSML tags other than <mark> for billing. Billing must be enabled, and usage above the free character allowance is charged automatically. Check the character total, not just the number of words in a script.
ElevenLabs’ free plan provides 10,000 credits per month. Starter costs $6 per month for 30,000 characters, Creator costs $22 for 121,000 characters, Pro costs $99 for 600,000 credits, and Scale costs $299 for 1,800,000 credits.
Unused ElevenLabs character credits do not roll over monthly. Its API caps each request at 40,000 characters for Flash v2.5, 30,000 for Turbo v2.5, and 10,000 for Multilingual V2. The units and limits differ, so do not treat character billing and credits as equal measures.
Pick a hosted route for convenience or custom voices
ElevenLabs Creator and Pro include commercial rights, while Starter commercial rights are limited. Professional Voice Cloning requires Creator or higher. Instant Voice Cloning uses 30 seconds of vocal files, so the two cloning features have different access needs.
Google Cloud Instant custom voice costs US$0.00006 per character, with no free allowance listed. Gemini 3.8 Flash TTS Preview costs $0.50 per million text tokens and $9 per million audio tokens through December 31, 2026. Starting January 1, 2027, those rates become $1 and $18.
Gemini 3.8 Flash-Lite TTS Preview costs $0.50 per million text tokens and $6 per million audio tokens through December 31, 2026. Starting January 1, 2027, those rates become $1 and $12. Token-based prices are separate from character-based voice pricing.
A free allowance can lower a bill, but it does not decide whether a plan grants the rights you need. Compare billing units, free allowances, usage limits, and terms before choosing. Estimate a typical month of text before choosing a paid tier, since unused ElevenLabs character credits do not roll over.
How Do You Add AI Voice to Canva, CapCut, and TikTok Videos?
For a short video, generate narration in the editor, preview it, then align the audio with the visuals on the timeline. Canva AI Voice allows up to 1,000 characters per conversion, while CapCut offers a separate TTS workflow. Check timing before export, especially when narration names on-screen items.
Canva AI Voice is available on the free plan, subject to fair-use limits, and does not count toward its monthly AI allowance. Canva Pro unlocks more AI Voice options, but the conversion cap remains. Its separate Text to Speech page states a 2,000-character limit, not the 1,000-character AI Voice limit.
Generate and place narration in Canva
Enter up to 1,000 characters in Canva AI Voice, choose a voice, and generate the audio. Preview the result, then add it to your Canva design and place it on the timeline. Trim or move the clip so speech matches the scene.
Canva’s separate voice-cloning feature copies a real voice from an audio sample across 130-plus languages. That is different from the AI Voice generator, which creates speech with its available voices. Murf’s Canva app lets you choose a language and voice, enter text, generate and preview audio, then add it to a design.
You can adjust Murf audio on the Canva timeline. Murf PRO voices are unavailable on FREE and BASIC plans. Keep scripts short enough to review each conversion, especially when your video has several scenes.
Create a voiceover in CapCut and export it
CapCut offers more than 200 AI voices and multiple languages, including English, Spanish, Japanese, Chinese, Malay, Vietnamese, and Thai. On desktop, add or paste your script, open Text to speech, select a voice, and choose Generate speech.
CapCut online lets you preview a voice for five seconds before generation. Its online controls include speed, pitch, and volume, and you can download the audio for use in other apps. Core TTS is free, while some premium voices may require Pro.
An AI voiceover for TikTok videos can come from CapCut or another editor. Time the voice against the video, then review the words with the visuals. A separate free AI video generator guide covers other creator tools, and an image-to-video ad workflow includes voiceover review. Play the exported video with sound on to catch timing, pronunciation, and abrupt cuts.
How Can You Use a Cloned Voice Responsibly?
Use a voice clone only when you have clear permission for the voice and a legitimate purpose for the output. Ask for explicit permission before cloning someone else, and confirm it covers model creation, intended uses, sharing, and commercial work.
Permission does not remove every privacy or legal duty. Service terms and rules that apply to your project still matter. Right-of-publicity and privacy rules can differ across U.S. states, so avoid treating one person’s approval as a complete answer.
The Federal Trade Commission warns that scammers can use cloned voices in impersonation schemes. Label synthetic narration when an audience could mistake it for a real person. Limit access to recordings and model files, especially when they represent someone else.
Before publishing, check both the speaker’s permission and the service terms that govern the model.
Keep a record of the permission and the use it allows. Do not share a model file just because you can share the audio it creates. Protecting the files can reduce misuse, but it cannot prevent every copy or unauthorized use.
Conclusion
The best first move is to match the model type to the speech task you actually need. TTS reads a script aloud, voice conversion changes recorded speech, and a custom model can represent a particular permitted voice.
Start with your own voice if you want to test a custom result. Record a short, clean sample and use a small script with familiar words and one name that needs careful pronunciation.
Listen for clear speech and steady tone before you spend time on a larger build. Start with a short sample of your own voice, then keep only the workflow that produces clear, usable audio.
FAQ
Can I train an AI voice model on my phone?
You can record and edit samples on a phone, but training is usually more practical on a computer or cloud GPU. Model training can need more memory and computing power than a phone can provide. A lightweight CPU model is a separate option, since using a model is not the same as training one. Check the model’s system needs before downloading it.
Does an AI voice model need an internet connection?
No, a downloaded local model can run offline after setup. You need an internet connection to download its files, and some tools may need online activation or updates. Hosted APIs and cloud voice tools do need a connection while you use them. Store local model files safely so you can access them without downloading them again.
Can I use one voice model for both TTS and voice conversion?
Not automatically: TTS generates speech from text, while voice conversion transforms recorded speech. The two tasks use different inputs and model functions. Some toolkits offer both workflows, but that does not mean one trained model handles both. Check the toolkit’s task support and test each workflow with a short sample before building around it.
How do I fix a model mispronouncing a name?
Try a pronunciation guide or phonetic spelling, then test the name in a short script. Some TTS tools accept pronunciation controls, while others respond better to a spelling that shows the sounds. Keep the test sentence simple so you can hear the change. If the name still sounds wrong, try a different voice or edit the final audio.
Is it legal to clone a celebrity’s voice?
Do not assume public availability means you have permission to clone or share a celebrity voice. Rights and privacy rules vary, and a use that seems harmless can still raise concerns. Get authorization and check the rules that apply before making or sharing a clone. Avoid presenting synthetic speech in a way that could make listeners think the celebrity spoke those words.
Can I use an AI voice commercially?
Sometimes, but check the specific plan’s commercial terms and the speaker’s permission first. A free tier does not automatically grant every commercial right, and a tool’s terms may limit certain uses. Permission from a speaker does not replace the service terms. Keep records of the allowed use, especially when you publish paid content or use a custom voice.
