OpenMOSS-Team/MOSS-Transcribe-Diarize

Overview

MOSS-Transcribe-Diarize is a 0.9B end-to-end speech-to-text model for long-form multi-speaker transcription. It uses a Whisper-style audio encoder and a Qwen3-style causal decoder to generate structured transcripts with text, timestamps, and speaker labels in a single pass.

Example output:

[0.11][S01] Good morning! [1.03]
[1.11][S02] Morning, let's start the meeting. [2.42]

Key capabilities:

• End-to-end multi-speaker transcription without a separate ASR, diarization, or alignment pipeline.
• Structured text output with speaker IDs and timestamps.
• Long-audio transcription for meetings, interviews, podcasts, and customer support recordings.
• Prompt-based hotwords for names, organizations, product names, project codes, and domain vocabulary.

Prerequisites

Install vLLM with audio dependencies.

uv venv
source .venv/bin/activate

uv pip install -U "vllm[audio]" --torch-backend auto

Launching with vLLM

vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --trust-remote-code \
  --served-model-name moss-mtd

The model uses custom Hugging Face processor code, so --trust-remote-code is required.

Client Usage

OpenAI Transcription API

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

with open("meeting.wav", "rb") as audio_file:
    transcription = client.audio.transcriptions.create(
        model="moss-mtd",
        file=audio_file,
        response_format="diarized_json",
        temperature=0,
    )

print(transcription.text)

for segment in transcription.segments:
    print(
        f"[{segment.start:.2f} - {segment.end:.2f}] "
        f"{segment.speaker}: {segment.text}"
    )

cURL

curl http://localhost:8000/v1/audio/transcriptions \
  -F "model=moss-mtd" \
  -F "file=@meeting.wav" \
  -F "response_format=diarized_json" \
  -F "temperature=0"

Output Format

With response_format=diarized_json, vLLM returns an OpenAI-compatible response with structured speaker segments:

{
  "task": "transcribe",
  "duration": 5.1,
  "text": "Let's review the deployment plan. The vLLM server is already running.",
  "segments": [
    {
      "type": "transcript.text.segment",
      "id": "seg_0",
      "start": 0.41,
      "end": 2.8,
      "text": "Let's review the deployment plan.",
      "speaker": "S01"
    },
    {
      "type": "transcript.text.segment",
      "id": "seg_1",
      "start": 3.12,
      "end": 5.1,
      "text": "The vLLM server is already running.",
      "speaker": "S02"
    }
  ],
  "usage": {
    "type": "duration",
    "seconds": 6
  }
}

Speaker IDs such as S01, S02, and S03 are generated by the model and are intended to remain consistent within the audio.

Use response_format=json if you want the model's native timestamped transcript instead.

[0.41][S01] Let's review the deployment plan. [2.80]
[3.12][S02] The vLLM server is already running. [5.10]

References

• Model card
• GitHub repository
• vLLM audio support