Getting Began with OmniVoice-Studio – KDnuggets

# Introduction

 
You paste a paragraph of textual content into ElevenLabs, press Generate, and watch the character counter tick down. The free tier is gone earlier than you end testing. The Creator plan is $22 a month. The Professional plan is $99. And each audio file you generate leaves your machine and finally ends up on their servers, which issues the second your content material is delicate, proprietary, or just yours.

OmniVoice Studio is constructed on a unique premise: all the pieces runs in your {hardware}. Voice cloning, video dubbing, real-time dictation, voice design — all of it native, all of it free for private use, no API key required, no utilization counter. The venture describes itself as “the open-source ElevenLabs different,” and that is correct, although the language protection alone makes the comparability fascinating: ElevenLabs helps 32 languages. OmniVoice Studio helps 646.

The venture has collected 7.1k GitHub stars and 1.1k forks. The newest launch, v0.2.7, shipped Might 3, 2026, consists of pre-built installers for macOS, Home windows, and Linux. This text covers the complete path from set up to your first generated audio.

Beta discover: OmniVoice Studio is in lively beta. Issues can break between releases. For probably the most present fixes, cloning from supply and operating bun run desktop-prod is the really helpful path over pre-built installers.

# What OmniVoice Studio Is and Why It Was Constructed

 
The best framing is that this: OmniVoice Studio provides you an expert voice AI desktop app that by no means telephones dwelling. No accounts, no subscriptions, no cloud calls throughout inference. Your reference audio, your scripts, your generated information — they keep in your machine.

Right here is how the function set and pricing examine on to ElevenLabs:

Underneath the hood, OmniVoice Studio is a Tauri desktop utility — a Rust-based framework that wraps a React frontend and a FastAPI backend with 97 API endpoints. Persistent state lives in SQLite. The AI pipeline is constructed on 4 open-source parts that do the precise work:

1. WhisperX handles transcription, word-level speech recognition, and alignment.
2. Demucs (from Meta) handles vocal isolation, separating speech from music and background noise.
3. OmniVoice from k2-fsa is the zero-shot diffusion text-to-speech (TTS) engine — the mannequin that makes cloning work from a 3-second clip throughout 646 languages.
4. Pyannote handles speaker diarization, figuring out who stated what in a multi-speaker recording, which is what makes computerized voice task within the dubbing pipeline doable.

GPU acceleration is auto-detected at launch. You do not configure something: OmniVoice reads your {hardware} and routes accordingly to CUDA (NVIDIA), MPS (Apple Silicon), ROCm (AMD), or CPU. When you have underneath 8 GB VRAM, the TTS mannequin offloads to the CPU robotically throughout transcription. The pipeline nonetheless runs, simply slower.

# System Necessities

 
Earlier than putting in, test that your machine meets the minimal specs. The app will run under these, however you’ll discover it.

One factor value realizing: you do not want a GPU to make use of OmniVoice Studio. Your complete pipeline runs on CPU. TTS synthesis is roughly 3x slower with no GPU, and transcription of lengthy movies will take longer, however for brief voice clones and dictation, the CPU path is completely usable. Apple Silicon Macs are the candy spot for GPU-less customers; the app robotically picks MLX-optimized Whisper and TTS backends that use the Apple Neural Engine and Metallic Efficiency Shaders, giving roughly 2x the throughput of the CPU path.

# Putting in OmniVoice Studio

 
Choose the part on your working system and comply with it from prime to backside. The set up sequence is identical throughout platforms: clone, set up frontend dependencies with Bun, and launch. The variations are within the conditions.

// Putting in on macOS

Stipulations:

• macOS 12 (Monterey) or newer — Apple Silicon or Intel
• Python 3.11+
• Bun (the JavaScript runtime used to construct the frontend)
• Xcode Command Line Instruments
• FFmpeg

Set up them so as:

# 1. Set up Python by way of Homebrew (or use pyenv should you handle a number of variations)
brew set up python@3.11

# 2. Set up Bun
curl -fsSL https://bun.sh/set up | bash

# 3. Set up Xcode Command Line Instruments
xcode-select --install

# 4. Set up FFmpeg (utilized by the dubbing and seize pipelines)
brew set up ffmpeg

Then clone and run:

# Clone the repository
git clone https://github.com/debpalash/OmniVoice-Studio.git
cd OmniVoice-Studio

# Set up frontend dependencies
bun set up

# Launch the app
bun run desktop-prod

The primary launch is slower than each subsequent one. It builds the Tauri shell, creates the Python digital setting by way of uv, syncs all Python dependencies, and downloads mannequin weights — roughly 2.4 GB. The splash display screen reveals reside progress for every step. As soon as it completes, the complete UI opens.

Pre-built DMG customers: Obtain the newest DMG from the Releases web page, mount it, and drag OmniVoice Studio into /Functions. If the primary launch reveals “app is broken and cannot be opened,” that’s macOS Gatekeeper reacting to an unsigned app. The developer-ID signing and notarization pipeline is tracked for v0.4. For now, clear the quarantine attribute with a single terminal command:

# Take away the Gatekeeper quarantine attribute.
# Run this as soon as after putting in. The app is open supply -- confirm the
# SHA-256 checksum on the Releases web page towards the .dmg.sha256 file
# earlier than operating this if you wish to verify the obtain is clear.
xattr -cr "/Functions/OmniVoice Studio.app"

// Putting in on Home windows

Stipulations:

• Home windows 10 (21H2 or newer) or Home windows 11, x64
• Python 3.11+
• Microsoft C++ Construct Instruments (required by pyannote.audio and occasional torch wheel rebuilds)
• Bun
• FFmpeg

Set up them from an everyday (non-admin) PowerShell:

# Set up Python by way of winget
winget set up Python.Python.3.11

# Set up Microsoft C++ Construct Instruments
# Obtain from https://visualstudio.microsoft.com/visual-cpp-build-tools/
# Choose "Desktop growth with C++" workload throughout set up

# Set up Bun
powershell -c "irm bun.sh/set up.ps1 | iex"

# Set up FFmpeg by way of winget
winget set up Gyan.FFmpeg

Then clone and run (nonetheless in PowerShell):

git clone https://github.com/debpalash/OmniVoice-Studio.git
cd OmniVoice-Studio
bun set up
bun run desktop-prod

Home windows-specific be aware (Triton/torch.compile OOM): On Home windows, sure TTS engines (notably CosyVoice paths) set off torch.compile kernel compilation on the primary synthesis name. On machines with underneath 16 GB VRAM, this may OOM earlier than any audio renders, surfacing as OutOfMemoryError: CUDA out of reminiscence. The repair is in Settings → Efficiency: toggle “Disable torch.compile (Home windows)” on. From the command line, set the setting variable earlier than launching:

# Disable torch.compile to keep away from OOM on first synthesis (Home windows solely).
# This falls again to the eager-mode kernel path -- barely slower peak
# throughput, however the engine truly masses on low-VRAM machines.
$env:TORCH_COMPILE_DISABLE = "1"
bun run desktop-prod

Pre-built MSI customers: Obtain the newest MSI from the Releases web page, run the installer, and discover OmniVoice Studio within the Begin menu.

// Putting in on Linux

Stipulations (Debian/Ubuntu):

# Set up Python
sudo apt set up python3.11

# Set up Bun
curl -fsSL https://bun.sh/set up | bash

# Set up FFmpeg
sudo apt set up ffmpeg

# Set up GTK/WebKit dependencies required by the Tauri desktop shell
sudo apt set up 
  libwebkit2gtk-4.1-dev 
  libayatana-appindicator3-dev 
  librsvg2-dev 
  libssl-dev 
  libxdo-dev 
  build-essential

Stipulations (Fedora):

sudo dnf set up python3.11 ffmpeg-free
curl -fsSL https://bun.sh/set up | bash
sudo dnf set up webkit2gtk4.1-devel libappindicator-gtk3-devel librsvg2-devel openssl-devel

Clone and run:

git clone https://github.com/debpalash/OmniVoice-Studio.git
cd OmniVoice-Studio
bun set up
bun run desktop-prod

Pre-built AppImage customers:

# Obtain the AppImage from the Releases web page, then:
chmod +x OmniVoice.Studio_*.AppImage
./OmniVoice.Studio_*.AppImage

# For those who see a white display screen on Fedora 44+ or Ubuntu 24.04, set this:
WEBKIT_DISABLE_COMPOSITING_MODE=1 ./OmniVoice.Studio_*.AppImage

The white display screen on newer distros is a compositing regression in WebKitGTK 2.44/2.46. v0.3+ of the AppImage autodetects this and units the flag robotically. The handbook setting variable path is the fallback for supply installs.

Pre-built .deb customers:

sudo apt set up ./OmniVoice.Studio_*.amd64.deb
omnivoice-studio

Docker (backend solely):

For headless server use or staff deployments the place the desktop GUI is not wanted, OmniVoice Studio ships a Docker path that runs simply the FastAPI backend and its 97 API endpoints:

# Clone the repo if you have not already
git clone https://github.com/debpalash/OmniVoice-Studio.git
cd OmniVoice-Studio

# Construct and begin the backend container
docker compose -f deploy/docker-compose.yml up

The backend API is then out there at http://localhost:8000. The complete API reference lives within the repo’s docs/ listing. This path is beneficial when integrating OmniVoice capabilities right into a pipeline with out operating a desktop session.

# Setting Up Your Hugging Face Token

 
This step is non-compulsory for primary use, however required for 2 options: speaker diarization (the pyannote/speaker-diarization-3.1 mannequin is gated on Hugging Face) and the bigger voice-design engines.

You want a free Hugging Face account and a learn token. After you have one:

Choice 1 — By means of the app (really helpful): Open Settings → API Keys, paste your hf_... token, and save. The app writes it to OmniVoice’s encrypted SQLite retailer and to the canonical huggingface_hub location, so each subprocess the app spawns picks it up robotically.

Choice 2 — Atmosphere variable:

# macOS / Linux -- add to ~/.zshrc or ~/.bashrc
export HF_TOKEN=hf_your_token_here
supply ~/.zshrc

# Home windows PowerShell -- writes to user-scope setting
# Use this, not setx. setx truncates values over 1024 chars and
# does not propagate to the present shell session.
[Environment]::SetEnvironmentVariable("HF_TOKEN", "hf_your_token_here", "Consumer")

You additionally want to just accept the mannequin phrases on the Hugging Face mannequin web page earlier than downloading. Go to pyannote/speaker-diarization-3.1 and settle for the gated mannequin entry request. It is a one-time step.

# Cloning a Voice

 
Voice cloning is the core function and the one most individuals set up OmniVoice for. The mannequin powering it’s OmniVoice from k2-fsa — a diffusion-based TTS system educated on 646 languages that operates zero-shot, that means there isn’t a fine-tuning step. You present a reference clip at inference time, and the mannequin adapts to the speaker’s voice on the fly.

Methods to clone a voice:
Navigate to the Voice Clone tab. You’ve gotten two choices for the reference audio: file instantly within the app by clicking the microphone button, or add an current audio file. Both method, 3 to 10 seconds of clear speech is sufficient.

Then:

1. Add or file your reference audio clip.
2. Choose the goal language from the dropdown (646 out there).
3. Kind or paste the textual content you need synthesized within the textual content discipline.
4. Click on Generate.

OmniVoice processes regionally, generates the audio, and performs it again for preview. You may export to MP3, WAV, or FLAC from the export button.

What makes a superb reference clip: Background noise is the most important high quality killer. A clip recorded in a quiet room, with the speaker talking naturally, will clone higher than a loud excerpt from a cellphone name. Keep away from clips with background music; if that is all you will have, run it by the Vocal Isolation tab first (lined under) to strip the background earlier than utilizing it as a reference.

If the output sounds barely off — robotic consonants, improper rhythm — attempt an extended or completely different reference clip earlier than assuming an engine problem. The zero-shot mannequin is delicate to the standard of the reference audio.

# Dubbing a Video

 
The dubbing pipeline is probably the most complicated factor OmniVoice Studio does, and watching it run end-to-end is genuinely spectacular. You give it a video — both a YouTube URL or a neighborhood file — and it transcribes the speech, interprets it to your goal language, clones the unique speaker voices, synthesizes the dubbed audio within the cloned voices, and muxes all the pieces again into an MP4. Regionally. No add.

The pipeline makes use of WhisperX for transcription, Pyannote for speaker diarization (figuring out which voice belongs to which speaker), the OmniVoice mannequin for synthesis, and Demucs to separate the unique speech from background audio so the background might be preserved underneath the dubbed monitor.

Methods to dub a video:

1. Navigate to the Dub tab. Paste a YouTube URL or click on the add button to pick out a neighborhood file. Select the goal language. Click on Begin Dub.
2. The progress bar reveals every stage because it runs: obtain (for YouTube), transcription, diarization, translation, synthesis, and mux. For a 5-minute video on a machine with a GPU, the complete pipeline sometimes takes 8 to 12 minutes. CPU-only will take longer.
3. When full, the dubbed MP4 is obtainable within the Initiatives panel alongside the SRT subtitle file, the remoted stems, and the unique transcription.
4. Batch queue: When you have a number of movies, drop all of them into the queue by clicking Add to Queue for each, then click on Run All. The conductor processes them sequentially utilizing GPU execution with a reside progress bar per job. You may add extra jobs whereas the queue is operating.

# Designing a Voice

 
Voice design is for creating a brand new voice from scratch when you do not have a reference clip. Navigate to the Voice Design tab, and you will find sliders and controls for gender, age, accent, pitch, velocity, emotion, and dialect.

The design course of is iterative: alter the controls, hit Preview, hear, alter once more. The A/B Comparability button enables you to lock one voice configuration as model A, tune the controls additional to create model B, and toggle between them whereas the identical pattern textual content performs, so that you’re evaluating voices instantly relatively than counting on reminiscence.

When you’re glad with a voice, reserve it to your Voice Gallery with a reputation and tags. Saved voices are then out there as targets within the Voice Clone tab — you’ll be able to synthesize new audio in a saved designed voice with no need a reference clip every time.

# Utilizing the Dictation Widget

 
The dictation widget is a system-wide transcription device that works from any utility with out switching home windows. The worldwide hotkey is Cmd+Shift+House on macOS and Ctrl+Shift+House on Home windows and Linux.

Press the hotkey from any app — your code editor, a browser textual content discipline, a notes app, wherever. A small frameless floating window seems. Begin talking. The widget streams your speech by WhisperX’s ASR engine over a neighborhood WebSocket connection, transcribes in actual time, auto-pastes the consequence into no matter utility was centered earlier than the widget opened, and disappears.

Your complete circulation — set off, converse, paste — takes just a few seconds. There is no such thing as a window switching, no copy-paste step.

1. Configuring the hotkey: If Cmd+Shift+House conflicts with one other utility in your system, open Settings → Dictation and alter the hotkey binding to any mixture that does not conflict. The brand new binding takes impact instantly with out restarting the app.
2. What the widget doesn’t do: It doesn’t hold a transcription historical past. Every activation transcribes and pastes, then discards the audio. For longer transcription periods the place you wish to assessment and edit a full transcript, use the principle Transcription tab as a substitute — that information to a file and reveals a full editable transcript.

# Selecting a TTS Engine

 
OmniVoice Studio ships six TTS engines. The default, OmniVoice, covers 600+ languages and handles voice cloning and instructed era. The others exist for particular causes, and switching takes ten seconds by way of Settings → TTS Engine or the OMNIVOICE_TTS_BACKEND setting variable.

For English-only use on a machine with no GPU, KittenTTS and MOSS-TTS-Nano run close to real-time on CPU. For Apple Silicon, switching to MLX-Audio provides you the quickest inference out there on M-series {hardware} utilizing the Apple Neural Engine instantly. CosyVoice 3 is the selection if you need instructed era — describing the voice type in pure language relatively than dialing sliders.

Change by way of the setting variable if you wish to set it system-wide:

# Run OmniVoice Studio with CosyVoice 3 because the lively TTS backend.
# Legitimate values: omnivoice, cosyvoice, mlx-audio, voxcpm2, moss-tts-nano, kittenTTS
export OMNIVOICE_TTS_BACKEND=cosyvoice
bun run desktop-prod

# Home windows equal
$env:OMNIVOICE_TTS_BACKEND = "cosyvoice"
bun run desktop-prod

Including a customized engine: OmniVoice makes use of a built-in backend registry. To plug in your personal TTS engine, subclass TTSBackend in backend/providers/tts_backend.py and add it to the _REGISTRY dictionary on the backside of that file. The README paperwork the interface as roughly 50 strains of Python. The CONTRIBUTING information covers the complete growth setup.

# Utilizing OmniVoice Studio by way of the MCP Server

 
OmniVoice Studio ships a Mannequin Context Protocol (MCP) server, which suggests you’ll be able to name its TTS and dubbing capabilities from Claude Desktop, Cursor, or any MCP-compatible shopper — with out opening the desktop app in any respect.

That is helpful if you wish to generate voice audio from inside an AI coding session or automate voice era as a part of a pipeline that is already operating in an MCP-capable device.

The MCP server is uncovered on localhost:8765 when OmniVoice Studio is operating. To attach it to Claude Desktop, add the next to your claude_desktop_config.json:

{
  "mcpServers": {
    "omnivoice": {
      "command": "npx",
      "args": ["-y", "@omnivoice/mcp-server"],
      "env": {
        "OMNIVOICE_API_URL": "http://localhost:8765"
      }
    }
  }
}

As soon as linked, Claude Desktop can name OmniVoice instruments instantly. For instance, you’ll be able to sort “Generate audio of this paragraph in a feminine voice with a British accent,” and Claude will route the request to OmniVoice’s native API, synthesize the audio, and return the file path.

The MCP server exposes the core capabilities — TTS era, voice cloning with a reference file, and dubbing job creation — as named instruments that any MCP shopper can uncover and invoke. See the docs/ listing within the repo for the complete device schema.

# Conclusion

 
OmniVoice Studio makes a sensible case for local-first voice AI. Not as a result of cloud instruments are unhealthy, however as a result of 646 languages, no utilization meter, and audio that by no means leaves your machine add as much as one thing genuinely completely different. The setup — one set up sequence, a 2.4 GB mannequin obtain, and an non-compulsory Hugging Face token — is a one-time funding. Every little thing after that’s simply utilizing the device.

It is in lively beta, and a few edges are tough. However the core pipeline — cloning, dubbing, dictation, design — works, the neighborhood is responsive, and releases have been coming frequently since launch. For builders, content material creators, and researchers who work with audio and care about the place their information goes, it is value having regionally.
 
 

Shittu Olumide is a software program engineer and technical author enthusiastic about leveraging cutting-edge applied sciences to craft compelling narratives, with a eager eye for element and a knack for simplifying complicated ideas. It’s also possible to discover Shittu on Twitter.