Open Source ElevenLabs Alternatives: Self-Hosted Text-to-Speech and Voice Cloning

What can open source TTS replace today?

Open text-to-speech models have closed much of the gap with commercial voice APIs for narration, assistants and accessibility features. Several run in real time on a consumer GPU, and some run on a CPU or even a Raspberry Pi.

Where commercial services still tend to lead is polish at scale: dozens of languages with consistent quality, fine control over emotion and pacing, a large library of licensed voices and an API that never needs a GPU on your side.

The strongest reason to self-host is control. Audio never leaves your servers, costs scale with hardware rather than characters, and you can run fully offline, which matters for devices, call centres and privacy-sensitive products.

It also removes a dependency. A voice feature built on a hosted API breaks when pricing, rate limits or terms change; a model you host keeps working exactly as it did on the day you shipped.

Which open source TTS models should you compare?

This field moves quickly. New models appear every few months, so treat the table as a map of approaches rather than a permanent ranking, and read the latest model card before you commit.

Why do weight licences matter so much?

In TTS, the code licence and the model weight licence are often different. A repository can be MIT while the downloadable weights are restricted to non-commercial use, often because of the training data.

XTTS-v2 is the best-known example: the Coqui model licence prohibits commercial use, and the company behind it has shut down, so there is no one to buy a commercial licence from. Community forks continue the code but cannot change the weight licence.

For a commercial product, shortlist models whose weights are permissively licensed, and keep a copy of the licence and model card version you relied on.

Voices are a second layer. Piper’s voices, for example, are trained on different datasets and each carries its own licence, so a permissive engine does not automatically mean every voice is usable commercially.

Is open source voice cloning safe and legal to use?

Cloning a voice from a few seconds of audio is now easy, which makes consent the central issue. Only clone voices of people who have explicitly agreed, in writing, to that specific use.

Many jurisdictions protect a person’s voice and likeness, and deceptive use such as impersonation or fraud is illegal almost everywhere. Some platforms and regulations also expect synthetic audio to be disclosed or watermarked.

Build safeguards into your product: consent records, restrictions on uploading voices of public figures, audit logs and, where possible, audio watermarking. This is not legal advice; check the rules where you operate.

Consent should also be revocable. If a voice actor or employee withdraws permission, you need to be able to delete their reference audio and any fine-tuned model derived from it, so store those assets in a way you can find and remove.

How do you choose a TTS model?

• Commercial product: filter out any model with non-commercial weights first.
• Offline or on-device: start with Piper or another small model that runs on CPU.
• Natural narration on a budget: test Kokoro and similar compact models.
• Custom brand voice: either fine-tune a permissive model on licensed recordings or use a cloning model whose licence allows it.
• Real-time agents: measure time to first audio chunk, not only total generation time.
• Languages: test every language you need with native speakers; quality varies a lot.

What hardware do you need?

Small models like Piper run comfortably on a CPU and are fast enough for interactive use. Compact neural models such as Kokoro can also run on CPU, with a GPU making batch generation much faster.

Zero-shot cloning models are heavier and usually need a GPU for real-time output. Memory needs vary by model, so check the repository’s requirements and test on the hardware you actually plan to use.

For production, wrap the model in a small HTTP service, cache repeated phrases such as greetings and menu prompts, and stream audio in chunks so users hear the start while the rest is generated.

Measure real-time factor, the ratio of generation time to audio length, on your target machine. Anything comfortably below one means the model can keep up with playback, which is the minimum for conversational use.

Where does self-hosted TTS break?

• Numbers, dates and abbreviations: text normalisation errors are the most common complaint. Normalise text before synthesis.
• Long texts: some models drift or repeat on long inputs. Split by sentence and stitch the audio.
• Pronunciation of names and brands: keep a pronunciation dictionary or phoneme overrides.
• Latency spikes under load: a single GPU serving many requests needs queueing and batching.
• Inconsistent voices across updates: pin model versions so your product voice does not change overnight.

How does TTS fit into a voice agent stack?

A typical self-hosted voice agent chains speech-to-text, an LLM and text-to-speech, with a real-time layer handling turn-taking and interruptions. TTS latency is often the part users notice most, because silence after they stop speaking feels broken.

Streaming matters at every step. Send LLM tokens to TTS sentence by sentence rather than waiting for the full answer. RepoLoot’s catalog includes voice and audio projects with notes on difficulty, which helps when assembling this pipeline.

Keep a commercial API as a fallback during early launches. You can route traffic to your own model gradually as you confirm its quality and stability.

Voice agents also need barge-in: when the user starts talking, playback must stop immediately. That logic lives in the real-time layer, but your TTS service should support cancelling a generation mid-stream so GPU time is not wasted on audio nobody hears.

When is ElevenLabs still the better choice?

Pay for a commercial API when you need many languages at consistent quality, a large library of ready voices, or you have no one to run GPU infrastructure. Dubbing and audiobook projects with tight deadlines often fall here.

Self-host when volume is high and predictable, when audio must stay private, when you need offline operation, or when a small set of voices covers your product. Many teams mix both: open models for bulk and on-device speech, a commercial service for premium voices.

Whichever route you take, keep your text preparation, pronunciation dictionary and caching layer independent of the engine. Those pieces carry over when you switch models, and they are where much of the perceived quality comes from.

Frequently asked questions

What is the best open source ElevenLabs alternative?

There is no single best model. Piper is excellent for fast offline speech, Kokoro for natural narration with a small footprint, and Chatterbox or XTTS-v2 for voice cloning. Choose by licence first, then test quality on your own texts and languages.

Can I use XTTS-v2 commercially?

The XTTS-v2 weights are released under the Coqui Public Model License, which does not allow commercial use. Because Coqui has shut down, buying a commercial licence is not an option. For commercial products, choose a model with permissive weight licensing instead.

Does open source TTS run without a GPU?

Yes, some models do. Piper is designed to run on CPUs and small devices, and compact models like Kokoro can run on a CPU at usable speeds. Voice cloning models usually need a GPU for real-time output.

Is it legal to clone someone’s voice?

Cloning a voice without consent can violate privacy, publicity or fraud laws, depending on where you are. Get explicit written consent for each use, avoid cloning public figures, and disclose synthetic audio where required. Check local law with a lawyer for commercial projects.