> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modellix.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# GPT Realtime Whisper

> Stream PCM audio to openai/gpt-realtime-whisper over WebSocket and receive partial and final text transcripts, billed by input audio seconds.

GPT Realtime Whisper converts streaming speech into text. Use it for live captions and dictation. Your application decides where each turn ends and commits the audio.

> [card] WSS
>
> `wss://api.modellix.ai/api/v1/live/openai/gpt-realtime-whisper`

| Item | Value |
| - | - |
| Model ID | `openai/gpt-realtime-whisper` |
| Input | Raw 16-bit, little-endian, mono PCM audio at 24 kHz |
| Output | Partial and final text transcripts |
| Auth | `Authorization: Bearer <Modellix API Key>` on the handshake |

For authentication, handshake errors, and close codes shared by all Live models, see the [Live Overview](https://docs.modellix.ai/live/overview.md).

## Quick Start

Prepare a short mono, 16-bit PCM WAV file at 24 kHz and set `MODELLIX_API_KEY`. Save the following code as `whisper.py` and run `python whisper.py speech.wav`. The script commits the whole file as one turn.

```python Python theme={null}
# pip install "websockets>=14"
import asyncio
import base64
import json
import os
import sys
import wave

import websockets

URL = "wss://api.modellix.ai/api/v1/live/openai/gpt-realtime-whisper"


async def main(path):
    headers = {"Authorization": f"Bearer {os.environ['MODELLIX_API_KEY']}"}
    async with websockets.connect(URL, additional_headers=headers) as ws:
        # 1. Wait for session.created, then configure transcription.
        created = json.loads(await ws.recv())
        if created["type"] != "session.created":
            raise RuntimeError(created)
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "type": "transcription",
                "audio": {
                    "input": {
                        "format": {"type": "audio/pcm", "rate": 24000},
                        "transcription": {"model": "gpt-realtime-whisper"},
                        "turn_detection": None,
                    }
                },
            },
        }))

        # 2. Stream audio in real time, then commit the turn.
        with wave.open(path, "rb") as audio:
            if (audio.getnchannels(), audio.getsampwidth(), audio.getframerate()) != (1, 2, 24000):
                raise ValueError("Use mono, 16-bit PCM WAV audio at 24 kHz")
            while chunk := audio.readframes(4800):  # 200 ms at 24 kHz
                await ws.send(json.dumps({
                    "type": "input_audio_buffer.append",
                    "audio": base64.b64encode(chunk).decode("ascii"),
                }))
                await asyncio.sleep(len(chunk) / 48000)
        await ws.send(json.dumps({"type": "input_audio_buffer.commit"}))

        # 3. Match events to the committed item by item_id.
        item_id = None
        async for raw in ws:
            event = json.loads(raw)
            kind = event["type"]
            if kind == "input_audio_buffer.committed":
                item_id = event["item_id"]
            elif kind == "conversation.item.input_audio_transcription.delta":
                print("Partial:", event["delta"])
            elif kind == "conversation.item.input_audio_transcription.completed":
                if event["item_id"] == item_id:
                    print("Final:", event["transcript"])
                    print("Usage:", event["usage"])
                    return
            elif kind in ("conversation.item.input_audio_transcription.failed", "error"):
                raise RuntimeError(event)
    raise RuntimeError("Connection ended before the transcript completed")


if len(sys.argv) != 2:
    raise SystemExit("Usage: python whisper.py speech.wav")

asyncio.run(main(sys.argv[1]))
```

## Client Messages

### session.update

Send `session.update` after `session.created`. The following fields belong inside `session`.

**type** `string` *required*

Allowed value: `transcription`.

**audio.input.format** `object`

Use `{"type": "audio/pcm", "rate": 24000}`.

**audio.input.transcription.model** `string` *required*

Use `gpt-realtime-whisper`. The URL selects the model, and Modellix replaces this value with the upstream model identifier.

**audio.input.transcription.language** `string`

The input language as an ISO 639-1 code, such as `en`.

**audio.input.transcription.delay** `string`

Allowed values: `minimal`, `low`, `medium`, `high`, `xhigh`. Lower values give earlier partial text; higher values usually improve accuracy.

**audio.input.turn\_detection** `null` *required*

Must be `null`. This model does not support voice activity detection, so your application detects the end of speech and sends `input_audio_buffer.commit`.

This model does not support `audio.input.transcription.prompt`.

### Audio Input

**input\_audio\_buffer.append** `event`

Appends base64-encoded 24 kHz PCM16 mono audio in the `audio` field. Chunks of 9,600 bytes hold 200 ms of audio.

**input\_audio\_buffer.commit** `event`

Ends the current audio turn and starts its final transcription.

## Server Messages

Server events are forwarded unchanged. Match transcript events to their turn by `item_id`.

**session.created** `event`

The first event on a connection. Wait for it before sending `session.update`.

**input\_audio\_buffer.committed** `event`

The turn was committed. `item_id` identifies it.

**conversation.item.input\_audio\_transcription.delta** `event`

Newly available text in `delta`, with `item_id` and `content_index`.

**conversation.item.input\_audio\_transcription.completed** `event`

The final text in `transcript`, with `item_id` and `content_index`. `usage` reports `type: "duration"` and `seconds`.

**conversation.item.input\_audio\_transcription.failed** `event`

Transcription failed for the item in `item_id`.

**error** `event`

An upstream error. `error.message` describes it.

## Billing

Each completed turn produces one request record, based on `usage.seconds` from its `completed` event. Seconds are charged from the per-minute price without rounding up to a whole minute. Unit prices and invoices follow the console.

* **Silence counts.** Silence in a committed turn is part of the reported audio seconds.
* **Uncommitted audio.** If you disconnect before committing, Modellix commits the buffered audio once and waits up to 2 seconds for the result. If no usage arrives in that time, there is no charge for that audio.
* **Disconnect after commit.** A committed turn is billed if its result arrives within the same 2-second window. To keep the transcript as well, wait for `completed` before closing.

You can inspect usage and the charged amount in the console's Live request records.

## Sessions and Limits

* This model is available only through its own Live endpoint. Other Live models cannot use it for attached transcription.
* WebSocket only. WebRTC and other upstream HTTP endpoints are not available through Modellix.
* Modellix closes any connection at its maximum connection duration with code `4408`.

See the [OpenAI model page](https://developers.openai.com/api/docs/models/gpt-realtime-whisper) for model details.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.