Microsoft VibeVoice Complete Installation Guide 2026

Microsoft VibeVoice is an open-source voice AI family covering speech recognition and speech generation.

What can VibeVoice do?

Real-world examples of VibeVoice in action

Transcription & Summaries

**Meeting → Full Notes:** Record your one-hour office meeting. VibeVoice can tell you who spoke, what they said, and when they said it.

**YouTube Video → Transcript:** Give it a one-hour podcast or YouTube video and get a speaker-by-speaker transcript with timestamps.

**Lecture Notes for Students:** Record your one-hour college lecture and automatically convert it into transcript, timestamps, summary and important points.

Live AI & Voice Bots

**Talk to Your Own AI:** You speak to your laptop, AI understands you, ChatGPT generates the answer, and VibeVoice speaks back almost instantly.

**Customer Support Voice Bot:** A customer calls your business, talks naturally, AI understands the problem and answers them back with a human-like voice.

Private & Autonomous AI

**Offline/Local Voice Assistant:** You can even build your own private AI voice assistant where your audio doesn't need to depend completely on paid voice APIs.

**AI Podcast From One Prompt:** Imagine you type one topic, and AI creates a 30-minute podcast where 3 different people are talking naturally.

Audio Content Generation

**Turn a Blog Into an Audiobook:** You have a 50-page PDF or blog? Give it to AI and turn the whole thing into natural audio instead of reading it yourself.

**AI Interview:** Give AI a topic like 'Will AI replace developers?' and create two AI speakers debating each other like a real interview.

**Dub Content Into Another Language:** Take your English content, translate it, then generate voice for another language without recording everything again.

The Current Project Includes

Model	Purpose	Hardware
VibeVoice-ASR	Long-form speech-to-text	NVIDIA GPU recommended
VibeVoice-ASR-Streaming	Live speech-to-text	NVIDIA GPU recommended
VibeVoice-ASR-BitNet	Lightweight speech-to-text	CPU
VibeVoice-Realtime-0.5B	Real-time text-to-speech	GPU / supported Apple Silicon
VibeVoice-TTS-1.5B	Legacy long-form TTS	Official code removed

The official repository is here: [Microsoft VibeVoice GitHub repository](https://github.com/microsoft/VibeVoice?utm_source=chatgpt.com)

Important: VibeVoice Changed

Legacy TTS Removed If you have seen older VibeVoice videos showing 90-minute podcast generation, 4 speakers, voice cloning, VibeVoice 1.5B / Large, or long-form multi-speaker TTS, those tutorials refer to the original 2025 release. Microsoft removed the original VibeVoice-TTS code from the repository on September 5, 2025 after reporting misuse concerns. The current repository continues with ASR and the newer realtime TTS model. The old TTS model is still documented as supporting up to 90 minutes, four speakers, expressive conversation and multilingual generation, but its current "Quick Try" is disabled.

1. VibeVoice-ASR

VibeVoice-ASR is the main long-form speech recognition model. **Features** - Up to 60 minutes of audio in one pass - Speaker diarization - Speaker identification in the transcript - Timestamps - "Who said what and when" structured output - Customized hotwords - More than 50 languages - Code-switching between languages - Gradio interface - File transcription - LoRA fine-tuning - vLLM deployment - OpenAI-compatible serving through the vLLM integration Microsoft's current ASR checkpoint is approximately 17.3 GB. **Links:** - [VibeVoice-ASR on Hugging Face](https://huggingface.co/microsoft/VibeVoice-ASR?utm_source=chatgpt.com) - [Official VibeVoice ASR Playground](https://aka.ms/vibevoice-asr?utm_source=chatgpt.com)

2. 2. Requirements

The current Python package requires: `Python >= 3.10` `PyTorch` `Transformers >= 4.51.3 and < 5` `Accelerate` `Diffusers` `Gradio` `FastAPI` `Uvicorn` `Librosa` `AV` `AIORTC` `NumPy` `SciPy` `FFmpeg` for audio decoding. Microsoft recommends an NVIDIA Deep Learning Container for the GPU versions. You will also need: `docker --version` `nvidia-smi` `python3 --version` If Docker cannot see your GPU, fix NVIDIA Container Toolkit support before continuing.

3. 3. Clone VibeVoice

With GitHub CLI:

gh repo clone microsoft/VibeVoice
cd VibeVoice

If you do not use GitHub CLI, open the official repository above and use its Clone button.

4. 4. Recommended NVIDIA Docker Setup

From inside the VibeVoice directory:

sudo docker run \
  --privileged \
  --net=host \
  --ipc=host \
  --ulimit memlock=-1:-1 \
  --ulimit stack=-1:-1 \
  --gpus all \
  --rm -it \
  -v "$PWD":/app \
  -w /app \
  nvcr.io/nvidia/pytorch:25.12-py3

Inside the container:

pip install -e .
apt update
apt install -y ffmpeg
ffmpeg -version
python --version

The NVIDIA 24.07 through 25.12 containers are listed as verified for the ASR setup.

5. 5. Start the VibeVoice ASR Web Interface

python demo/vibevoice_asr_gradio_demo.py \
  --model_path microsoft/VibeVoice-ASR \
  --share

The first launch downloads the model, so expect a large download. Once Gradio starts, open the generated local or public address. You can then upload audio and receive a structured transcription. Microsoft also provides direct file inference:

python demo/vibevoice_asr_inference_from_file.py \
  --model_path microsoft/VibeVoice-ASR \
  --audio_files ./audio.wav

6. VibeVoice Streaming ASR

This is one of the most interesting additions to VibeVoice. Microsoft released VibeVoice-ASR-Streaming on September 3, 2026. Instead of waiting until the entire recording finishes, the model continuously returns transcription chunks while someone is speaking. **Features** - Real-time transcription - Speaker-attributed transcription - Customized hotwords - 10 supported languages - WebSocket streaming - FastAPI demo - File transcription - vLLM deployment - REST API - OpenAI-compatible endpoint - Prefix/KV-cache reuse for streaming sessions The currently listed languages are: Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian, Spanish. The model download is approximately 17.4 GB. Model: [VibeVoice-ASR-Streaming-7B model](https://huggingface.co/microsoft/VibeVoice-ASR-Streaming-7B?utm_source=chatgpt.com)

7. 7. Install Streaming ASR

Use the same repository and environment:

pip install -e .
apt update
apt install -y ffmpeg

Launch the FastAPI demonstration:

python demo/vibevoice_asr_streaming_fastapi_demo.py \
  --model_path microsoft/VibeVoice-ASR-Streaming-7B

Open: `http://localhost:7870` You can speak through the microphone or upload an audio file. Text should start appearing while speech is still arriving. For direct streaming inference:

python demo/vibevoice_asr_streaming_inference_from_file.py \
  --model_path microsoft/VibeVoice-ASR-Streaming-7B \
  --audio_files ./audio.wav

You can also bias transcription toward technical names (behaves similarly to customized hotwords):

--context_info "Microsoft,VibeVoice,RexxuLabs,Django,FastAPI"

8. 8. Production vLLM Streaming API

If you want to build an application around VibeVoice rather than simply use the demo, the vLLM integration is much more interesting. Start the server:

docker run -d \
  --gpus all \
  --name vibevoice-vllm-streaming \
  --ipc=host \
  -p 8000:8000 \
  -e VIBEVOICE_FFMPEG_MAX_CONCURRENCY=64 \
  -e PYTORCH_ALLOC_CONF=expandable_segments:True \
  -v "$PWD":/app \
  -w /app \
  --entrypoint bash \
  vllm/vllm-openai:v0.14.1 \
  -c "python3 /app/vllm_plugin/scripts/start_streaming_server.py"

Watch startup: `docker logs -f vibevoice-vllm-streaming` The streaming server exposes endpoints including: `WS /v1/stream`, `POST /v1/transcribe`, `POST /v1/transcribe_batch`, `POST /v1/chat/completions`, `GET /v1/config`. The `/v1/chat/completions` endpoint is designed for compatibility with existing OpenAI-style clients. This makes VibeVoice particularly interesting for: meeting transcription, AI meeting assistants, live subtitles, interviews, podcast transcription, customer-support calls, voice agents, lecture transcription, video transcription, and live content indexing.

9. VibeVoice Realtime 0.5B TTS

This is the current officially supported VibeVoice text-to-speech model. Instead of turning audio into text, it turns text into speech. **Features** - 0.5B model - About 2.04 GB model weight - Streaming text input - Around 200 to 300 ms first audible output depending on hardware - Long-form speech - Approximately 10-minute context - Single-speaker generation - WebSocket demo - Direct file generation - Multiple provided voices - Experimental multilingual voices Microsoft says NVIDIA T4 and Mac M4 Pro achieved real-time performance in its tests. Model: [VibeVoice-Realtime-0.5B on Hugging Face](https://huggingface.co/microsoft/VibeVoice-Realtime-0.5B?utm_source=chatgpt.com) Colab: [Official VibeVoice Realtime Colab](https://colab.research.google.com/github/microsoft/VibeVoice/blob/main/demo/vibevoice_realtime_colab.ipynb?utm_source=chatgpt.com)

10. 10. Install Realtime TTS

Inside the repository:

pip install -e '.[streamingtts]'

The streaming-TTS extra currently pins Transformers 4.51.3. Start the realtime interface:

python demo/vibevoice_realtime_demo.py \
  --model_path microsoft/VibeVoice-Realtime-0.5B

Direct text-file generation:

python demo/realtime_model_inference_from_file.py \
  --model_path microsoft/VibeVoice-Realtime-0.5B \
  --txt_path demo/text_examples/1p_vibevoice.txt \
  --speaker_name Carter

Additional experimental voices can be installed with `bash demo/download_experimental_voices.sh`. Microsoft currently describes the model as primarily English, with experimental support for: German, French, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish. Those languages have not been tested as extensively as English.

11. CPU-Only VibeVoice: VibeVoice ASR BitNet

If you have no NVIDIA GPU, this is the version I would put prominently in the guide. Microsoft released a separate CPU runtime called VibeASR.cpp. It compresses the model to approximately 1.58 GB and can reach real-time transcription with three or more CPU threads on supported hardware. Official repository: [Microsoft VibeASR.cpp](https://github.com/microsoft/VibeASR.cpp?utm_source=chatgpt.com)

gh repo clone microsoft/VibeASR.cpp
cd VibeASR.cpp
git submodule update --init --recursive
pip install -r requirements.txt
python setup_env.py

# Manual build
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

# Run transcription
./build/bin/asr_infer \
  --vae-model models/vibeasr/vibeasr-vae-encoder-i8_s.gguf \
  --lm-model models/vibeasr/vibeasr-lm-i2_s-embed-q6_k.gguf \
  --audio input.wav \
  -t 4

It supports optimized CPU execution on x86 AVX2 and ARM NEON hardware. Windows users should note that the official runtime says MSVC is not supported for the build. GCC/Clang, with MinGW-w64 recommended on Windows, is the supported direction.

12. Which Version Should You Install?

**Want speech-to-text from meetings or videos?** Use: `VibeVoice-ASR`
**Want live transcription while someone is talking?** Use: `VibeVoice-ASR-Streaming-7B`
**No NVIDIA GPU?** Use: `VibeVoice-ASR-BitNet` / `VibeASR.cpp`
**Want an AI voice that speaks generated text?** Use: `VibeVoice-Realtime-0.5B`
**Want the old 4-person, 90-minute AI podcast generation?** That refers to the original VibeVoice-TTS release. Microsoft removed that TTS code from the official repository.

13. Official Demo Videos

Microsoft includes several actual VibeVoice-generated examples in the repository: - [Official VibeVoice ASR demo MP4](https://github.com/user-attachments/assets/acde5602-dc17-4314-9e3b-c630bc84aefa?utm_source=chatgpt.com) - [English VibeVoice demo MP4](https://github.com/user-attachments/assets/0967027c-141e-4909-bec8-091558b1b784?utm_source=chatgpt.com) - [Chinese VibeVoice demo MP4](https://github.com/user-attachments/assets/322280b7-3093-4c67-86e3-10be4746c88f?utm_source=chatgpt.com) - [Cross-lingual VibeVoice demo MP4](https://github.com/user-attachments/assets/838d8ad9-a201-4dde-bb45-8cd3f59ce722?utm_source=chatgpt.com) - [VibeVoice spontaneous singing demo MP4](https://github.com/user-attachments/assets/6f27a8a5-0c60-4f57-87f3-7dea2e11c730?utm_source=chatgpt.com) - [VibeVoice four-speaker long conversation MP4](https://github.com/user-attachments/assets/a357c4b6-9768-495c-a576-1618f6275727?utm_source=chatgpt.com) - [Current VibeVoice Realtime demo MP4](https://github.com/user-attachments/assets/9aa8ab3c-681d-4a02-b9ea-3f54ffd180b2?utm_source=chatgpt.com)

14. Common Problems

CUDA Out of Memory

For vLLM deployments: Reduce GPU memory utilization. Reduce max model length. Reduce concurrent sequences. Use tensor parallelism if multiple GPUs are available. The vLLM integration supports tensor parallelism for splitting one model across multiple GPUs. Non-streaming ASR also supports data-parallel replicas.

FFmpeg Errors

Check `ffmpeg -version`. On Ubuntu/Debian: `apt update && apt install -y ffmpeg`.

Package Installation Problems

Use a clean environment (`python3 -m venv .venv`). Using a virtual environment can also avoid system-Python package conflicts on newer Ubuntu/Debian installations.

vLLM Plugin Not Found

Check `pip show vibevoice` and `pip show -f vibevoice | grep entry`.

Streaming Session Stops or Runs Out of Cache

For vLLM Streaming ASR: Increase `mm-processor-cache-gb` or reduce the number of concurrent sessions.

Responsible Usage Realtime voice generation can potentially be used to produce misleading or impersonated audio. Microsoft recommends disclosing AI-generated content, checking outputs carefully, and thoroughly testing VibeVoice before deploying it in real-world or commercial environments.

15. One Prompt to Install Everything with Claude Code or Codex

Agentic Installation Prompt

Copy the following prompt into Claude Code or Codex from the directory where you want the project installed. That single prompt is suitable for both Claude Code and Codex because it tells the coding agent to detect the machine first instead of blindly assuming CUDA/Linux.

Act as a senior ML engineer, Python engineer and DevOps engineer.
Your task is to install, configure, test and document the CURRENT official Microsoft VibeVoice project on this machine.
Official source:
GitHub repository: microsoft/VibeVoice

Do not use an outdated fork unless the official Microsoft repository explicitly requires it.

IMPORTANT: The current repository has changed from the original 2025 VibeVoice release. Do not assume that the old 90-minute multi-speaker VibeVoice-TTS code is still available. Inspect the current repository and README first.

GOAL
Automatically inspect this machine and install the most appropriate current VibeVoice components.
I want support for:
1. VibeVoice-ASR - long-form speech-to-text - speaker diarization - timestamps - customized hotwords - file transcription - Gradio demo
2. VibeVoice-ASR-Streaming - live transcription - speaker-attributed transcription - microphone/file demo - customized hotwords - FastAPI/WebSocket service
3. VibeVoice-Realtime-0.5B - real-time text-to-speech - websocket demo - direct text-file inference - available built-in voices
4. vLLM APIs when the hardware supports them.
5. If no compatible GPU exists, install the official Microsoft VibeASR.cpp / VibeVoice-ASR-BitNet CPU implementation instead and verify CPU transcription.

DO EVERYTHING AGENTICALLY.

STEP 1: Inspect the environment
Check and report:
- operating system and version
- CPU architecture
- RAM
- available disk space
- NVIDIA GPU and VRAM if present
- nvidia-smi
- CUDA availability
- Docker
- NVIDIA Container Toolkit
- Python
- pip
- Git
- GitHub CLI
- FFmpeg
- CMake
- compiler
- Apple Silicon / Metal availability when applicable
Do not install blindly before determining the environment.

STEP 2: Select installation path
For a compatible NVIDIA Linux environment:
Prefer Microsoft's documented NVIDIA Docker setup.
For Apple Silicon:
Determine which current VibeVoice components are actually supported locally by the upstream repository. Use the native PyTorch/MPS path only where compatible. Prefer the official Colab route if the local installation is unsupported.
For a CPU-only machine:
Use Microsoft's VibeASR.cpp BitNet implementation for ASR.
For Windows:
Use WSL2 + NVIDIA Docker when appropriate for GPU models. For VibeASR.cpp follow the project's compiler requirements and use MinGW/GCC/Clang where required instead of unsupported MSVC configurations.
Explain which path you selected.

STEP 3: Clone safely
Clone the current official Microsoft repository. Do not delete or overwrite unrelated files.
If VibeVoice already exists:
- inspect it
- check git status
- preserve user changes
- update only when safe

STEP 4: Create an isolated environment
Use an appropriate Python virtual environment when installing outside Docker.
Upgrade pip/setuptools/wheel. Respect the package versions in the current pyproject.toml. Do not randomly upgrade Transformers beyond the version range supported by VibeVoice.
For VibeVoice Realtime install the project's streamingtts extra.

STEP 5: Install system dependencies
Install required dependencies including FFmpeg where necessary. Install Flash Attention only when it is actually required and compatible with the current CUDA/GPU environment. Do not force Flash Attention on unsupported hardware.

STEP 6: Install VibeVoice-ASR
Install the official package and prepare: microsoft/VibeVoice-ASR
Create a reusable launch script for the Gradio ASR demo.
Also create a reusable command/script for direct audio-file transcription.

STEP 7: Install VibeVoice-ASR-Streaming
Prepare: microsoft/VibeVoice-ASR-Streaming-7B
Create a reusable launcher for the FastAPI streaming demo.
Verify microphone/file streaming support where the environment permits it.
Also prepare a direct file streaming inference command.

STEP 8: Install Realtime TTS
Prepare: microsoft/VibeVoice-Realtime-0.5B
Create: scripts/start_tts.sh scripts/tts_file.sh or Windows equivalents.
Use one of the official bundled voices for the initial verification test.

STEP 9: Configure vLLM when supported
If this system has enough compatible NVIDIA GPU resources, configure the official VibeVoice vLLM integration.
Prepare launchers for: Non-streaming ASR API and Streaming ASR API
Verify the documented endpoints including:
/v1/stream
/v1/transcribe
/v1/transcribe_batch
/v1/chat/completions
/v1/config
Do not install vLLM if the current hardware/platform does not support it reliably.

STEP 10: Download models
Download models only when needed. Before downloading very large checkpoints, print:
- model name
- approximate download size
- available disk space
Continue automatically if enough disk space exists. Avoid downloading the same checkpoint multiple times. Use the Hugging Face cache properly.

STEP 11: Verify
Create a small test directory: test_data/
If there is an upstream example audio/text sample, use it.
Verify:
- Python imports
- model loading
- ASR inference
- streaming ASR server startup where available
- TTS generation where available
- generated audio file
- Gradio/FastAPI startup
- vLLM health/API where enabled
Do not claim success only because pip installation succeeded. Actually run a minimal inference test.

STEP 12: Create convenient launchers
Create a scripts directory with platform-appropriate scripts such as:
scripts/start_asr scripts/transcribe scripts/start_streaming_asr scripts/start_tts scripts/tts_generate scripts/start_vllm_asr scripts/start_vllm_streaming scripts/stop_services
Only create launchers for features supported by this machine.

STEP 13: Create documentation
Create VIBEVOICE_SETUP.md containing:
- detected system configuration
- installation method selected
- installed components
- model locations
- model sizes
- commands to start each service
- URLs/ports used locally
- how to transcribe an audio file
- how to use hotwords
- how to generate TTS
- how to stop services
- troubleshooting
- how to update VibeVoice later
- limitations
- disk usage

STEP 14: Error handling
If any command fails:
1. inspect the complete error
2. determine the real root cause
3. search/inspect the current upstream repository documentation if internet access is available
4. apply the smallest safe fix
5. retry
6. record the fix in VIBEVOICE_SETUP.md
Do not repeatedly run the same failing command. Do not install random packages just to make errors disappear.

STEP 15: Final verification report
At the end show me:
SYSTEM OS: CPU: RAM: GPU: VRAM: CUDA: Python: Docker:
INSTALLED VibeVoice package: ASR: Streaming ASR: Realtime TTS: BitNet CPU: vLLM:
MODELS List every downloaded model and location.
TEST RESULTS ASR: Streaming ASR: TTS: API:
SERVICES List commands required to launch every working component.
FILES CREATED List every configuration, launcher and documentation file you created.
ISSUES List anything that could not be installed and the exact reason.

Do not stop after giving me instructions. Execute the setup, fix errors, test the installation and leave the machine in a working state.