> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modellix.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# GPT Realtime Translate

> Stream speech to openai/gpt-realtime-translate over WebSocket and receive translated audio and transcripts, billed by the audio seconds you send.

GPT Realtime Translate turns a live speech stream into speech in another language, with text transcripts alongside. Each session has one output language. It translates only: it has no conversations, tools, or `response.create`.

> [card] WSS
>
> `wss://api.modellix.ai/api/v1/live/openai/gpt-realtime-translate`

| Item | Value |
| - | - |
| Model ID | `openai/gpt-realtime-translate` |
| Input | Raw 16-bit, little-endian, mono PCM audio at 24 kHz, as text JSON frames |
| Output | Translated audio (PCM16 at 24 kHz by default) and transcript deltas |
| Output Languages | One per session |
| Auth | `Authorization: Bearer <Modellix API Key>` on the handshake |

For authentication, handshake errors, and close codes shared by all Live models, see the [Live Overview](https://docs.modellix.ai/live/overview.md).

## Quick Start

Prepare a mono, 16-bit PCM WAV file at 24 kHz with at least a few seconds of speech and set `MODELLIX_API_KEY`. Save the following code as `translate.py` and run `python translate.py speech.wav es`. The script prints the translated text and saves the translated speech as raw PCM in `translated.pcm`.

```python Python theme={null}
# pip install "websockets>=14"
import asyncio
import base64
import json
import os
import sys
import wave

import websockets

URL = "wss://api.modellix.ai/api/v1/live/openai/gpt-realtime-translate"


async def send_audio(ws, path):
    with wave.open(path, "rb") as audio:
        if (audio.getnchannels(), audio.getsampwidth(), audio.getframerate()) != (1, 2, 24000):
            raise ValueError("Use mono, 16-bit PCM WAV audio at 24 kHz")
        while chunk := audio.readframes(4800):  # 200 ms at 24 kHz
            await ws.send(json.dumps({
                "type": "session.input_audio_buffer.append",
                "audio": base64.b64encode(chunk).decode("ascii"),
            }))
            await asyncio.sleep(len(chunk) / 48000)
    # The service flushes remaining output, then sends session.closed.
    await ws.send(json.dumps({"type": "session.close"}))


async def receive_events(ws):
    with open("translated.pcm", "wb") as output:
        async for raw in ws:
            event = json.loads(raw)
            if event["type"] == "session.output_audio.delta":
                output.write(base64.b64decode(event["delta"]))
            elif event["type"] == "session.output_transcript.delta":
                print(event["delta"], end="", flush=True)
            elif event["type"] == "error":
                raise RuntimeError(event["error"])
            elif event["type"] == "session.closed":
                print()
                return
    raise RuntimeError("Connection ended before session.closed")


async def main(path, language):
    headers = {"Authorization": f"Bearer {os.environ['MODELLIX_API_KEY']}"}
    async with websockets.connect(URL, additional_headers=headers, max_size=16 * 1024 * 1024) as ws:
        # 1. Wait for session.created, then set the output language.
        created = json.loads(await ws.recv())
        if created["type"] != "session.created":
            raise RuntimeError(created)
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {"audio": {"output": {"language": language}}},
        }))

        # 2. Stream audio and read events at the same time.
        await asyncio.gather(send_audio(ws, path), receive_events(ws))


if len(sys.argv) != 3:
    raise SystemExit("Usage: python translate.py speech.wav es")

asyncio.run(main(sys.argv[1], sys.argv[2]))
```

## Client Messages

Send every message as a text JSON frame. Binary frames, non-JSON text, and non-object JSON close the connection with `1008` and reason `invalid translation audio`.

### session.update

Send `session.update` after `session.created`. The session type and model cannot change. The following fields belong inside `session`.

**audio.output.language** `string`

Target language for translated audio and transcripts, such as `es`. Create one session per output language. OpenAI does not publish a full list of language codes in its translation guide.

**audio.input.noise\_reduction** `object | null`

Noise reduction with `type` (`near_field` for close-talking microphones, `far_field` for laptop or room microphones) and `strength` (0 to 1, default 1). Use `null` to turn it off.

**audio.input.transcription** `null`

Attached input transcription is not available through Modellix. Only `null` is accepted; any other value closes the connection with `1008`. To transcribe speech, use [GPT Live Transcribe](https://docs.modellix.ai/live/models/gpt-live-transcribe.md) or [GPT Realtime Whisper](https://docs.modellix.ai/live/models/gpt-realtime-whisper.md) on their own endpoints.

### session.input\_audio\_buffer.append

Appends source audio. Keep sending throughout the session, including silence between phrases.

**audio** `string` *required*

Base64-encoded raw 24 kHz, PCM16, mono, little-endian audio, with no container header. The decoded length must be even. Invalid base64 or an odd length closes the connection with `1008` and reason `invalid translation audio`; the audio is not forwarded.

OpenAI recommends 200 ms chunks, which is 9,600 bytes before encoding.

### session.close

Tells the service to flush pending input and output. Stop sending audio first, then keep reading until `session.closed`. Closing the WebSocket earlier can drop output, and Modellix does not forward audio sent after `session.close`.

## Server Messages

Server events are forwarded unchanged.

**session.created** `event`

The first event on a connection. Wait for it before sending `session.update`.

**session.updated** `event`

The session configuration after a `session.update`.

**session.output\_audio.delta** `event`

Translated audio. `delta` is base64-encoded. `format` is `pcm16`, `sample_rate` defaults to 24000, `channels` defaults to 1, and `elapsed_ms` is optional.

**session.output\_transcript.delta** `event`

Translated text in `delta`, with optional `elapsed_ms`. A very short clip can return translated audio with no transcript text.

**session.closed** `event`

The final event after `session.close`. It carries no usage. If the socket closes before this event, treat the session as incomplete.

**error** `event`

An upstream error with `error.message`, `error.type`, `error.code`, and `error.param`.

## Billing

This model reports no usage in its events, so Modellix bills the audio you send. Seconds are the decoded bytes successfully forwarded upstream, divided by 48,000, and are charged from the per-minute price without rounding up to a whole minute. Unit prices and invoices follow the console.

* **Silence counts, gaps do not.** Silence you send is audio. Time when you send nothing is free, even with the connection open.
* **Not billed.** Audio rejected with `1008`, a frame that fails to reach the upstream, audio sent after `session.close`, and sessions with no audio.
* **Not used for billing.** `elapsed_ms`, output audio length, and connection time.
* **Multiple records per session.** Modellix seals a billing record for every 60 seconds of input audio. The remainder is settled when the session closes. If the session ends without `session.closed`, that remainder is recorded as incomplete and is still billed.

You can inspect usage and the charged amount in the console's Live request records.

## Sessions and Limits

* WebSocket only. WebRTC, browser client secrets, and other upstream HTTP endpoints are not available through Modellix.
* Rooms with several speakers need about one session per active speaker track and target language.
* Other event types are forwarded unchanged and validated by the upstream. Only audio in `session.input_audio_buffer.append` counts toward billing.
* Modellix closes any connection at its maximum connection duration with code `4408`.

See the [OpenAI translation guide](https://developers.openai.com/api/docs/guides/realtime-translation), the [client events reference](https://developers.openai.com/api/reference/resources/realtime/translation-client-events), and the [server events reference](https://developers.openai.com/api/reference/resources/realtime/translation-server-events) for protocol details.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.