Microsoft VibeVoice Complete Installation Guide 2026

Microsoft VibeVoice is an open-source voice AI family covering speech recognition and speech generation.

What can VibeVoice do?

Real-world examples of VibeVoice in action

Transcription & Summaries

**Meeting → Full Notes:** Record your one-hour office meeting. VibeVoice can tell you who spoke, what they said, and when they said it.

**YouTube Video → Transcript:** Give it a one-hour podcast or YouTube video and get a speaker-by-speaker transcript with timestamps.

**Lecture Notes for Students:** Record your one-hour college lecture and automatically convert it into transcript, timestamps, summary and important points.

Live AI & Voice Bots

**Talk to Your Own AI:** You speak to your laptop, AI understands you, ChatGPT generates the answer, and VibeVoice speaks back almost instantly.

**Customer Support Voice Bot:** A customer calls your business, talks naturally, AI understands the problem and answers them back with a human-like voice.

Private & Autonomous AI

**Offline/Local Voice Assistant:** You can even build your own private AI voice assistant where your audio doesn't need to depend completely on paid voice APIs.

**AI Podcast From One Prompt:** Imagine you type one topic, and AI creates a 30-minute podcast where 3 different people are talking naturally.

Audio Content Generation

**Turn a Blog Into an Audiobook:** You have a 50-page PDF or blog? Give it to AI and turn the whole thing into natural audio instead of reading it yourself.

**AI Interview:** Give AI a topic like 'Will AI replace developers?' and create two AI speakers debating each other like a real interview.

**Dub Content Into Another Language:** Take your English content, translate it, then generate voice for another language without recording everything again.

The Current Project Includes

The official repository is here: [Microsoft VibeVoice GitHub repository](https://github.com/microsoft/VibeVoice?utm_source=chatgpt.com)

Important: VibeVoice Changed

Legacy TTS Removed If you have seen older VibeVoice videos showing 90-minute podcast generation, 4 speakers, voice cloning, VibeVoice 1.5B / Large, or long-form multi-speaker TTS, those tutorials refer to the original 2025 release. Microsoft removed the original VibeVoice-TTS code from the repository on September 5, 2025 after reporting misuse concerns. The current repository continues with ASR and the newer realtime TTS model. The old TTS model is still documented as supporting up to 90 minutes, four speakers, expressive conversation and multilingual generation, but its current "Quick Try" is disabled.

1. VibeVoice-ASR

VibeVoice-ASR is the main long-form speech recognition model. **Features** - Up to 60 minutes of audio in one pass - Speaker diarization - Speaker identification in the transcript - Timestamps - "Who said what and when" structured output - Customized hotwords - More than 50 languages - Code-switching between languages - Gradio interface - File transcription - LoRA fine-tuning - vLLM deployment - OpenAI-compatible serving through the vLLM integration Microsoft's current ASR checkpoint is approximately 17.3 GB. **Links:** - [VibeVoice-ASR on Hugging Face](https://huggingface.co/microsoft/VibeVoice-ASR?utm_source=chatgpt.com) - [Official VibeVoice ASR Playground](https://aka.ms/vibevoice-asr?utm_source=chatgpt.com)

2. 2. Requirements

The current Python package requires: `Python >= 3.10` `PyTorch` `Transformers >= 4.51.3 and < 5` `Accelerate` `Diffusers` `Gradio` `FastAPI` `Uvicorn` `Librosa` `AV` `AIORTC` `NumPy` `SciPy` `FFmpeg` for audio decoding. Microsoft recommends an NVIDIA Deep Learning Container for the GPU versions. You will also need: `docker --version` `nvidia-smi` `python3 --version` If Docker cannot see your GPU, fix NVIDIA Container Toolkit support before continuing.

3. 3. Clone VibeVoice

With GitHub CLI:

gh repo clone microsoft/VibeVoice
cd VibeVoice

If you do not use GitHub CLI, open the official repository above and use its Clone button.

4. 4. Recommended NVIDIA Docker Setup

From inside the VibeVoice directory:

sudo docker run \
  --privileged \
  --net=host \
  --ipc=host \
  --ulimit memlock=-1:-1 \
  --ulimit stack=-1:-1 \
  --gpus all \
  --rm -it \
  -v "$PWD":/app \
  -w /app \
  nvcr.io/nvidia/pytorch:25.12-py3

Inside the container:

pip install -e .
apt update
apt install -y ffmpeg
ffmpeg -version
python --version

The NVIDIA 24.07 through 25.12 containers are listed as verified for the ASR setup.

5. 5. Start the VibeVoice ASR Web Interface

python demo/vibevoice_asr_gradio_demo.py \
  --model_path microsoft/VibeVoice-ASR \
  --share

The first launch downloads the model, so expect a large download. Once Gradio starts, open the generated local or public address. You can then upload audio and receive a structured transcription. Microsoft also provides direct file inference:

python demo/vibevoice_asr_inference_from_file.py \
  --model_path microsoft/VibeVoice-ASR \
  --audio_files ./audio.wav

6. VibeVoice Streaming ASR

This is one of the most interesting additions to VibeVoice. Microsoft released VibeVoice-ASR-Streaming on September 3, 2026. Instead of waiting until the entire recording finishes, the model continuously returns transcription chunks while someone is speaking. **Features** - Real-time transcription - Speaker-attributed transcription - Customized hotwords - 10 supported languages - WebSocket streaming - FastAPI demo - File transcription - vLLM deployment - REST API - OpenAI-compatible endpoint - Prefix/KV-cache reuse for streaming sessions The currently listed languages are: Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian, Spanish. The model download is approximately 17.4 GB. Model: [VibeVoice-ASR-Streaming-7B model](https://huggingface.co/microsoft/VibeVoice-ASR-Streaming-7B?utm_source=chatgpt.com)

7. 7. Install Streaming ASR

Use the same repository and environment:

pip install -e .
apt update
apt install -y ffmpeg

Launch the FastAPI demonstration:

python demo/vibevoice_asr_streaming_fastapi_demo.py \
  --model_path microsoft/VibeVoice-ASR-Streaming-7B

Open: `http://localhost:7870` You can speak through the microphone or upload an audio file. Text should start appearing while speech is still arriving. For direct streaming inference:

python demo/vibevoice_asr_streaming_inference_from_file.py \
  --model_path microsoft/VibeVoice-ASR-Streaming-7B \
  --audio_files ./audio.wav

You can also bias transcription toward technical names (behaves similarly to customized hotwords):

--context_info "Microsoft,VibeVoice,RexxuLabs,Django,FastAPI"

8. 8. Production vLLM Streaming API

If you want to build an application around VibeVoice rather than simply use the demo, the vLLM integration is much more interesting. Start the server:

docker run -d \
  --gpus all \
  --name vibevoice-vllm-streaming \
  --ipc=host \
  -p 8000:8000 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v "$PWD":/app \
  -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_streaming_server.py"

Watch startup: `docker logs -f vibevoice-vllm-streaming` The streaming server exposes endpoints including: `WS /v1/stream`, `POST /v1/transcribe`, `POST /v1/transcribe_batch`, `POST /v1/chat/completions`, `GET /v1/config`. The `/v1/chat/completions` endpoint is designed for compatibility with existing OpenAI-style clients. This makes VibeVoice particularly interesting for: meeting transcription, AI meeting assistants, live subtitles, interviews, podcast transcription, customer-support calls, voice agents, lecture transcription, video transcription, and live content indexing.

9. VibeVoice Realtime 0.5B TTS

This is the current officially supported VibeVoice text-to-speech model. Instead of turning audio into text, it turns text into speech. **Features** - 0.5B model - About 2.04 GB model weight - Streaming text input - Around 200 to 300 ms first audible output depending on hardware - Long-form speech - Approximately 10-minute context - Single-speaker generation - WebSocket demo - Direct file generation - Multiple provided voices - Experimental multilingual voices Microsoft says NVIDIA T4 and Mac M4 Pro achieved real-time performance in its tests. Model: [VibeVoice-Realtime-0.5B on Hugging Face](https://huggingface.co/microsoft/VibeVoice-Realtime-0.5B?utm_source=chatgpt.com) Colab: [Official VibeVoice Realtime Colab](https://colab.research.google.com/github/microsoft/VibeVoice/blob/main/demo/vibevoice_realtime_colab.ipynb?utm_source=chatgpt.com)

10. 10. Install Realtime TTS

Inside the repository:

pip install -e '.[streamingtts]'

The streaming-TTS extra currently pins Transformers 4.51.3. Start the realtime interface:

python demo/vibevoice_realtime_demo.py \
  --model_path microsoft/VibeVoice-Realtime-0.5B

Direct text-file generation:

python demo/realtime_model_inference_from_file.py \
  --model_path microsoft/VibeVoice-Realtime-0.5B \
  --txt_path demo/text_examples/1p_vibevoice.txt \
  --speaker_name Carter

Additional experimental voices can be installed with `bash demo/download_experimental_voices.sh`. Microsoft currently describes the model as primarily English, with experimental support for: German, French, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish. Those languages have not been tested as extensively as English.

11. CPU-Only VibeVoice: VibeVoice ASR BitNet

If you have no NVIDIA GPU, this is the version I would put prominently in the guide. Microsoft released a separate CPU runtime called VibeASR.cpp. It compresses the model to approximately 1.58 GB and can reach real-time transcription with three or more CPU threads on supported hardware. Official repository: [Microsoft VibeASR.cpp](https://github.com/microsoft/VibeASR.cpp?utm_source=chatgpt.com)

gh repo clone microsoft/VibeASR.cpp
cd VibeASR.cpp
git submodule update --init --recursive
pip install -r requirements.txt
python setup_env.py

# Manual build
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

# Run transcription
./build/bin/asr_infer \
  --vae-model models/vibeasr/vibeasr-vae-encoder-i8_s.gguf \
  --lm-model models/vibeasr/vibeasr-lm-i2_s-embed-q6_k.gguf \
  --audio input.wav \
  -t 4

It supports optimized CPU execution on x86 AVX2 and ARM NEON hardware. Windows users should note that the official runtime says MSVC is not supported for the build. GCC/Clang, with MinGW-w64 recommended on Windows, is the supported direction.

12. Which Version Should You Install?

• **Want speech-to-text from meetings or videos?** Use: `VibeVoice-ASR`
• **Want live transcription while someone is talking?** Use: `VibeVoice-ASR-Streaming-7B`
• **No NVIDIA GPU?** Use: `VibeVoice-ASR-BitNet` / `VibeASR.cpp`
• **Want an AI voice that speaks generated text?** Use: `VibeVoice-Realtime-0.5B`
• **Want the old 4-person, 90-minute AI podcast generation?** That refers to the original VibeVoice-TTS release. Microsoft removed that TTS code from the official repository.

Money-making ideas

The last two are especially interesting for you because you already work with Python/backend/cloud deployment.

Project ideas

1. PDF → AI Podcast

This is probably my favorite. **User uploads:** PDF / blog / research paper **Your system does:** PDF → LLM understands it → Creates conversation → Speaker 1 + Speaker 2 → VibeVoice → 🎧 Podcast Imagine a student uploads: “Operating System Chapter 5” And gets: **Host:** “So what exactly is deadlock?” **Expert:** “Imagine four cars stuck at an intersection…” Much easier than reading a 50-page PDF. **Monetization** - **Free:** 5-minute podcast - **Pro:** 30–60 minutes, multiple speakers, download, summaries (₹199–₹499/month)

2. 🎓 Lecture → Study Pack

Student uploads lecture recording. **VibeVoice:** Audio → Speaker + timestamp + transcript **Then LLM generates:** Summary, Important questions, MCQs, Flashcards, Exam notes, Definitions, Revision sheet. Microsoft's ASR is specifically designed for long recordings and can generate structured who / when / what transcription. This could be huge for students.

3. 🤯 “Talk To Any PDF”

Imagine uploading a 500-page book, then pressing 🎙️ Talk. **User:** “Explain chapter 7.” **AI:** “Chapter 7 mainly discusses…” (And it speaks the response aloud.) **Architecture:** Microphone → VibeVoice ASR → RAG → LLM → VibeVoice Realtime → 🔊 That's a genuinely impressive portfolio/SaaS project.

4. 🧑💼 AI Meeting Brain

Connect Zoom / Meet / Jitsi recording. **After meeting:** Meeting duration: 54 min **Haresh:** Finish API by Friday, Deploy staging **John:** Test UI, Contact client **Then:** AI Summary, Decisions, Action Items, Deadlines, Who said what. And users can ask: “What did John say about AWS?” Your system jumps to 32:14. That is a real business product.

5. ☎️ AI Sales Call Analyzer

Upload sales calls. **AI detects:** **Customer:** “I think ₹50,000 is expensive.” **Your system identifies:** Objection: PRICE **Then provides:** Suggested response “Instead of positioning the full package…” **Dashboard:** Customer interest, Objections, Competitor mentioned, Next action, Buying probability, Salesperson talk ratio. Companies will pay much more than consumers for this.

6. 🎙️ AI Podcast Studio

User enters: “Make a podcast about whether AI will replace programmers.” **Select:** Host: Female, Guest 1: Developer, Guest 2: AI Researcher **AI writes:** Host: Today we're discussing... Developer: I don't think developers will disappear... Researcher: I actually disagree... Then generate the voices. VibeVoice's long-form TTS design supports up to four distinct speakers and long conversational speech.

7. 📱 Reel/Shorts Voice Generator

This matches your content work especially well. Creator enters: “Explain this GitHub repo in Hindi.” **AI generates:** Hook → Script → Voice → Captions → MP3 → MP4 **Example:** GitHub URL → AI reads repo → Creates: “Microsoft ka ye AI tool literally crazy hai…” → VibeVoice narration → Captions → Ready Instagram Reel This becomes a GitHub → Reel Generator. That's actually something I'd strongly consider for RexxuLabs.

8. 🔥 YouTube → Complete Content Machine

**Input:** 1-hour podcast **Output:** Transcript, Summary, 10 reel ideas, LinkedIn post, Twitter thread, Blog, Newsletter, Quotes, Chapters, Titles, Description. Then optionally generate voice versions too. One piece of content becomes 20 pieces. Sell it to creators/agencies.

9. 🗣️ Personal AI Voice Assistant

Something similar to Jarvis. **User:** “What meetings do I have tomorrow?” (Assistant speaks.) **User:** “Read my latest important email.” (Assistant speaks.) **User:** “Reply saying I'll send it tomorrow.” You could connect: Gmail, Calendar, WhatsApp/business systems, Notion, Drive. **Architecture:** Voice → ASR → AI Agent → Tools → LLM → Realtime TTS VibeVoice-Realtime is designed for streaming text input with roughly 300 ms first-audible latency, which makes this kind of experiment interesting.

10. 🏢 Private Company Voice AI

This one has higher-ticket potential. Businesses sometimes don't want customer calls going to a third-party transcription service. You deploy transcription infrastructure on their own AWS / Azure / GPU server / local machine. Then: “Your recordings stay within your infrastructure.” **Sell:** Installation (₹20k–₹1L+), Maintenance (₹5k–₹25k/month). VibeVoice's newer BitNet ASR is particularly interesting here because Microsoft says it compresses the model from 4.62 GB to 1.58 GB and can achieve realtime inference using 3+ CPU threads without a GPU.

11. 🩺 Doctor Consultation → Clinical Draft Notes

Doctor records a consultation. **AI creates:** Transcript, Symptoms mentioned, Discussion summary, Follow-up points, Draft notes. But this becomes a high-stakes medical product, so you'd need serious privacy, accuracy, regulatory, and human-review safeguards. I wouldn't choose this as your first product.

12. ⚖️ Lawyer / Client Meeting Search

Record 1-hour consultation. **Later:** “When did the client mention the contract termination?” **AI returns:** 38:22 transcript section. Great use of speaker diarization + timestamps.

13. 🎧 Podcast Search Engine

Upload hundreds of podcasts. **Then search:** “Find every time Elon Musk discussed AGI.” **Results:** Episode 37 01:12:44 "..." Episode 89 33:18 "..." Click → audio jumps directly there. This could also work for company calls, university lectures, YouTube channels, interviews.

14. 🌎 Automatic Dubbing Studio

**Workflow:** English video → Transcription → Translation → New voice → Subtitle → Dubbed video **Possible creator service:** Turn your English YouTube video into Hindi + Spanish versions. But voice cloning/impersonation needs explicit consent and careful safeguards.

15. 🧠 “Second Brain for Voice Notes”

User keeps sending voice notes: “Remember I need to redesign homepage…” “Client wants Jitsi autoscaling…” “Follow up with Mike tomorrow…” **Your app organizes everything into:** Tasks, Ideas, People, Projects, Reminders, Searchable transcript. **Then user asks:** “What ideas did I have last month for RexxuLabs?” That feels much more valuable than a normal recorder.

💎 My top 5 for YOU

• **#1 GitHub → Reel Generator 🔥** GitHub URL → explanation → Hindi script → AI voice → captions → reel. Revenue: creator subscription.
• **#2 PDF → Podcast** Very easy concept to market virally. “I turned a 100-page PDF into a 20-minute podcast.” 🔥 Great Reel hook.
• **#3 AI Meeting Brain** More difficult, but companies actually pay. Potential: ₹999 → ₹10,000+/month rather than trying to get ₹99 from consumers.
• **#4 Private VibeVoice Deployment** Don't even build a SaaS initially. Put on RexxuLabs: We deploy private AI transcription infrastructure on your server. Charge for installation + maintenance. This probably gets you to revenue faster than waiting for a SaaS to grow.
• **#5 Voice AI Agent** Combine VibeVoice + LLM + RAG + Django/FastAPI and build: Talk to your business knowledge. Customer: “What's your return policy?” AI answers verbally based on the company's actual documents.

🚀 One idea I'd actually build first

**RexxuVoice:** A simple page - Turn anything into audio intelligence. Four buttons: 🎧 Audio → Notes 📄 PDF → Podcast 🎙 Meeting → Summary 💬 Talk to Document Then underneath: *Powered by open-source AI.* Instead of making another VibeVoice clone, you're using VibeVoice as just one component. **Your stack could be:** `Next.js ↓ FastAPI / Django ↓ VibeVoice ASR / LLM / RAG / VibeVoice Realtime ↓ PostgreSQL + S3` And then your Instagram content becomes the acquisition channel: “I found this crazy Microsoft repo.” → Show VibeVoice → “So I built THIS with it.” → Show your product → “Try it free. Link in bio.” That is much stronger than just getting views from GitHub-repo reels because now every viral repo you cover can potentially funnel users into your own tools, consulting services, or SaaS.