Voice Input

Supported providers

A third option, Platform, is available when connected to the Gamut platform. It uses Deepgram Nova 3 via the platform connection and requires no separate API key.

Setting up voice input

1. Open Settings > Voice.
2. Select a Speech-to-Text Provider from the dropdown.
3. Enter the API key for the selected provider (not needed for Platform).
4. Click Validate & Save. Gamut verifies the key against the provider's API before saving.
5. Use the Test section to confirm the microphone and transcription work.

Deepgram

Deepgram provides the lowest-latency transcription with the Nova 3 model. Gamut connects to Deepgram's WebSocket API for real-time streaming transcription.

API key requirements

The Deepgram API key must have at least Member-level access to create temporary (ephemeral) tokens. Gamut validates this during setup by:

1. Checking that the key can access the Deepgram projects API.
2. Verifying the key can create ephemeral tokens via the /v1/auth/grant endpoint.

If the key passes the first check but fails the second, the error reads "API key is valid but lacks permission to create temporary tokens." Upgrade the key's access level in the Deepgram Console.

How it works

When a voice recording starts:

1. Gamut requests a short-lived ephemeral token from Deepgram (valid for 10 minutes) using the stored API key. Only the ephemeral token goes to the browser; the long-lived key never leaves the server.
2. The browser opens a WebSocket connection to wss://api.deepgram.com/v1/listen with the ephemeral token.
3. Microphone audio streams in real time (16kHz, 16-bit linear PCM, mono).
4. Deepgram returns interim transcripts (displayed while speaking) and final transcripts (used as the message text).

Deepgram also supports batch transcription of audio files, used when audio must be transcribed server-side rather than over the real-time WebSocket.

OpenAI

OpenAI provides transcription through the Whisper and GPT-4o Mini Transcribe models. Gamut uses OpenAI's Realtime API for streaming transcription in the browser.

API key requirements

A standard OpenAI API key is sufficient. Gamut validates it by checking access to the OpenAI models endpoint.

How it works

When a voice recording starts:

1. Gamut requests a client secret from OpenAI's Realtime API (/v1/realtime/client_secrets) using the stored API key. The short-lived secret is passed to the browser.
2. The browser establishes a WebSocket connection to OpenAI's Realtime API with the client secret.
3. Audio streams and is transcribed in real time, as with Deepgram.

OpenAI also supports batch audio file transcription via the Whisper API (/v1/audio/transcriptions), used for server-side transcription of recorded audio.

API key management

STT provider keys follow the same pattern as LLM provider keys:

• Settings UI: Enter the key in Settings > Voice. It is stored locally in settings.json with restricted file permissions.
• Environment variables: Set DEEPGRAM_API_KEY or OPENAI_API_KEY before starting Gamut. Saved keys take precedence over environment variables.

A badge shows the current key's source. Remove a saved key to revert to the environment variable, or save a new key to override it.

Voice input in the UI

Once voice input is configured, a microphone button appears in the message composer throughout the app:

1. Click the microphone button (or use the keyboard shortcut).
2. Grant microphone access if the browser prompts for it.
3. Speak the message. Interim transcripts appear in real time.
4. Click the button again (or stop speaking) to finish recording.
5. The final transcript lands in the message input, ready to send.

Voice input is available wherever a message can be typed to an agent, including the main chat and the agent creation prompt.

Voice Agent

Both Deepgram and OpenAI support Voice Agent sessions, a more interactive mode where the agent responds with voice as well. When a voice agent session is active, a separate token is minted for the voice agent endpoint.

Troubleshooting

• No microphone button visible: Verify that a provider is selected and its API key is configured in Settings > Voice.
• "API key lacks permission to create temporary tokens" (Deepgram): The key needs Member-level access. Check its permissions in the Deepgram Console.
• "OpenAI API quota exceeded": Check the account balance and billing settings at platform.openai.com.
• Transcription is inaccurate: Speak clearly and reduce background noise. Confirm the microphone works with the test tool in Settings > Voice > Test.
• Connection timeout: The browser waits up to 10 seconds to connect to the STT provider's WebSocket. If this fails, check the network connection and try again.