Skip to main content

WSS

wss://api.modellix.ai/api/v1/live/google/gemini-3.8-live
For the session flow, billing, close codes, and handshake errors shared by all Live models, see the Live Overview.

Quick Start

Set MODELLIX_API_KEY first. Both examples send one text turn and print the spoken reply as text through output transcription.

Configuration Examples

Each example is a complete setup frame. Pick one and adjust.

Sending Audio and Video

Send video as a stream, not a single frame. A single JPEG followed immediately by a question may be ignored by the model, and no image tokens are counted. Send several frames at up to 1 frame per second, then ask your question. Video frames are reported and billed as image input tokens.
Stream a 16 kHz PCM recording in 100 ms chunks, followed by silence so the server detects the end of speech.

Client Messages

Every frame is a JSON text message with exactly one top-level field. Send setup first and wait for setupComplete. Parameters are optional unless marked required. Where a default is not listed, the upstream default applies.

setup

First frame. It must arrive within 30 seconds and cannot be changed afterwards. Every field is optional. Other upstream fields, such as tools, are forwarded unchanged. See the Gemini Live API reference.
string
Ignored. The model is selected by the URL, and Modellix replaces this field.
object
Generation settings.
object
Instruction that stays in effect for the whole session. Allowed values: {"parts": [{"text": "..."}]}, text parts only. It can only be set here; to add context later, use clientContent.
object
default:"off"
Allowed values: {}. Enables transcription of the audio you send, returned in serverContent.inputTranscription.
object
default:"off"
Allowed values: {}. Enables the text of the spoken reply, returned in serverContent.outputTranscription.
object
Server-side voice activity detection (VAD). While it is on, the user can interrupt the model by speaking.
object
default:"off"
Compresses the context so a session can run past the audio (15 minutes) and audio plus video (2 minutes) limits, and keeps per-turn cost bounded.
object
default:"off"
Allowed values: {}. Enables sessionResumptionUpdate messages.
Do not set generationConfig.thinkingConfig.thinkingLevel; it is not supported for this model. Sampling parameters (temperature, topP, topK) and penalties (presencePenalty, frequencyPenalty) are not documented here because recent Gemini models ignore or reject them.

realtimeInput

Streams user input. Send exactly one of the following fields per message. Real-time input can be sent continuously without interrupting the model.
string
Text input. Triggers a model reply. Allowed values: non-empty text.
object
Raw 16-bit PCM, 16 kHz, little-endian, mono. Send in small chunks, for example 100 ms.
object
One JPEG frame. Send at most 1 frame per second, and keep sending while the user talks about the scene. A single frame followed immediately by a question may be ignored.
boolean
Allowed values: true. The audio stream ended, for example the microphone was turned off. Send audio again to resume. Use only while VAD is on.
object
Allowed values: {}. Marks the start of user speech. Use only while VAD is off.
object
Allowed values: {}. Marks the end of user speech. Use only while VAD is off.

clientContent

Appends turns to the conversation history. Use it to restore earlier context or to send text turns with explicit roles.
object[]
Turns appended in order.
boolean
default:"false"
Allowed values: true, false. If true, the model starts replying now and interrupts any generation in progress. Otherwise the server waits for more input.

Server Messages

object
The session is ready for input. Always the first message after setup.
object
Model output, transcriptions, and turn signals. Output streams across many frames and is generated faster than real time; buffer audio and play it back at 24 kHz.
object
Token usage for the turn, sent at the end of each turn. Billing uses it, split by modality. Video frames are reported as IMAGE input.
object
Sent periodically when sessionResumption is set. Treat the handle as a credential and do not log it.
object
Sent about 60 seconds before the connection ends.
Server frames can arrive as text or binary WebSocket messages. Both carry UTF-8 JSON.

Example Frame Sequence

A text question with a spoken reply and transcription. Audio data is shortened.

Voices

Set voiceName to one of these 30 voices: The model switches language with the conversation. To keep replies in one language, say so in systemInstruction, for example "Respond only in Japanese."

Sessions and Limits

Upstream limits apply to this model, independent of Modellix: To go beyond these limits, configure the following in your setup message. Modellix forwards them untouched:
  • contextWindowCompression: keeps the context bounded with a sliding window, which also controls cost.
  • sessionResumption: the server sends sessionResumptionUpdate messages containing a handle. After goAway or a disconnect, open a new connection and pass the latest handle in setup.sessionResumption.handle to continue with the previous context.
Each new connection is a new Modellix session. Turn numbering and billing records are independent per connection. Modellix also closes any connection at its maximum connection duration with code 4408.

FAQ

Set setup.generationConfig.speechConfig.voiceConfig.prebuiltVoiceConfig.voiceName to one of the 30 voices. If you omit it, the upstream default voice is used.
The model replies in audio only. Add outputAudioTranscription: {} to setup and read serverContent.outputTranscription.text.
Add inputAudioTranscription: {} to setup and read serverContent.inputTranscription.text.
The model follows the conversation language. To fix it, say so in systemInstruction, for example "Respond only in Japanese."
While voice activity detection is on, the user can interrupt by speaking. You can also send a clientContent message with turnComplete set to true. The server sends serverContent.interrupted; stop playback and discard buffered audio.
Yes. Send JPEG frames through realtimeInput.video at up to 1 frame per second. See Sending Audio and Video.
Enable contextWindowCompression and sessionResumption, then reconnect with the latest handle after goAway. See Sessions and Limits.
Read usageMetadata at the end of each turn. Billing uses it, split by modality.
setup.tools and the related messages are forwarded unchanged. See the upstream Gemini Live API reference for their format.