WSS
wss://api.modellix.ai/api/v1/live/google/gemini-3.8-live
For the session flow, billing, close codes, and handshake errors shared by all Live models, see the Live Overview.
Quick Start
MODELLIX_API_KEY first. Both examples send one text turn and print the spoken reply as text through output transcription.
Configuration Examples
Each example is a completesetup frame. Pick one and adjust.
- Audio Only
- Transcription
- Voice
- Long Session
- Manual VAD
- Context
Sending Audio and Video
- Audio Input
- Video Input
Stream a 16 kHz PCM recording in 100 ms chunks, followed by silence so the server detects the end of speech.
Client Messages
Every frame is a JSON text message with exactly one top-level field. Sendsetup first and wait for setupComplete. Parameters are optional unless marked required. Where a default is not listed, the upstream default applies.
setup
First frame. It must arrive within 30 seconds and cannot be changed afterwards. Every field is optional. Other upstream fields, such astools, are forwarded unchanged. See the Gemini Live API reference.
string
Ignored. The model is selected by the URL, and Modellix replaces this field.
object
Generation settings.
object
Instruction that stays in effect for the whole session. Allowed values:
{"parts": [{"text": "..."}]}, text parts only. It can only be set here; to add context later, use clientContent.object
default:"off"
Allowed values:
{}. Enables transcription of the audio you send, returned in serverContent.inputTranscription.object
default:"off"
Allowed values:
{}. Enables the text of the spoken reply, returned in serverContent.outputTranscription.object
Server-side voice activity detection (VAD). While it is on, the user can interrupt the model by speaking.
object
default:"off"
Compresses the context so a session can run past the audio (15 minutes) and audio plus video (2 minutes) limits, and keeps per-turn cost bounded.
object
default:"off"
Allowed values:
{}. Enables sessionResumptionUpdate messages.Do not set
generationConfig.thinkingConfig.thinkingLevel; it is not supported for this model. Sampling parameters (temperature, topP, topK) and penalties (presencePenalty, frequencyPenalty) are not documented here because recent Gemini models ignore or reject them.realtimeInput
Streams user input. Send exactly one of the following fields per message. Real-time input can be sent continuously without interrupting the model.string
Text input. Triggers a model reply. Allowed values: non-empty text.
object
Raw 16-bit PCM, 16 kHz, little-endian, mono. Send in small chunks, for example 100 ms.
object
One JPEG frame. Send at most 1 frame per second, and keep sending while the user talks about the scene. A single frame followed immediately by a question may be ignored.
boolean
Allowed values:
true. The audio stream ended, for example the microphone was turned off. Send audio again to resume. Use only while VAD is on.object
Allowed values:
{}. Marks the start of user speech. Use only while VAD is off.object
Allowed values:
{}. Marks the end of user speech. Use only while VAD is off.clientContent
Appends turns to the conversation history. Use it to restore earlier context or to send text turns with explicit roles.object[]
Turns appended in order.
boolean
default:"false"
Allowed values:
true, false. If true, the model starts replying now and interrupts any generation in progress. Otherwise the server waits for more input.Server Messages
object
The session is ready for input. Always the first message after
setup.object
Model output, transcriptions, and turn signals. Output streams across many frames and is generated faster than real time; buffer audio and play it back at 24 kHz.
object
Token usage for the turn, sent at the end of each turn. Billing uses it, split by modality. Video frames are reported as
IMAGE input.object
Sent periodically when
sessionResumption is set. Treat the handle as a credential and do not log it.object
Sent about 60 seconds before the connection ends.
Example Frame Sequence
A text question with a spoken reply and transcription. Audio data is shortened.Voices
SetvoiceName to one of these 30 voices:
The model switches language with the conversation. To keep replies in one language, say so in
systemInstruction, for example "Respond only in Japanese."
Sessions and Limits
Upstream limits apply to this model, independent of Modellix:
To go beyond these limits, configure the following in your
setup message. Modellix forwards them untouched:
contextWindowCompression: keeps the context bounded with a sliding window, which also controls cost.sessionResumption: the server sendssessionResumptionUpdatemessages containing a handle. AftergoAwayor a disconnect, open a new connection and pass the latest handle insetup.sessionResumption.handleto continue with the previous context.
4408.
FAQ
How do I choose a voice?
How do I choose a voice?
Set
setup.generationConfig.speechConfig.voiceConfig.prebuiltVoiceConfig.voiceName to one of the 30 voices. If you omit it, the upstream default voice is used.How do I get the reply as text?
How do I get the reply as text?
The model replies in audio only. Add
outputAudioTranscription: {} to setup and read serverContent.outputTranscription.text.How do I get a transcript of what the user said?
How do I get a transcript of what the user said?
Add
inputAudioTranscription: {} to setup and read serverContent.inputTranscription.text.How do I make the model reply in a specific language?
How do I make the model reply in a specific language?
The model follows the conversation language. To fix it, say so in
systemInstruction, for example "Respond only in Japanese."How do I interrupt the model?
How do I interrupt the model?
While voice activity detection is on, the user can interrupt by speaking. You can also send a
clientContent message with turnComplete set to true. The server sends serverContent.interrupted; stop playback and discard buffered audio.Does it support video?
Does it support video?
Yes. Send JPEG frames through
realtimeInput.video at up to 1 frame per second. See Sending Audio and Video.How do I keep a session running longer than 10 minutes?
How do I keep a session running longer than 10 minutes?
Enable
contextWindowCompression and sessionResumption, then reconnect with the latest handle after goAway. See Sessions and Limits.How do I see token usage?
How do I see token usage?
Read
usageMetadata at the end of each turn. Billing uses it, split by modality.Can I use tools or function calling?
Can I use tools or function calling?
setup.tools and the related messages are forwarded unchanged. See the upstream Gemini Live API reference for their format.