How a WebSocket Speech Recognition Relay Is Built

Why relay at all?

Streaming speech recognition APIs have a shape that is hostile to client-side integration: the credential is long-lived and billable, the wire protocol carries sequence numbers and per-frame headers, and connection setup is stateful. A browser or desktop client should have to know none of that.

A relay inverts it. The client speaks one contract - binary audio in, text out - and the relay owns everything stateful behind it.

• The upstream API key stays server-side, encrypted at rest, and never crosses the network to a device you do not control.
• Rotating, rate-limiting or disabling the upstream account becomes a server-side operation, not a client rollout.
• Usage can be metered by something the client cannot under-report: the audio bytes that actually crossed the socket.

Frame-level passthrough, not file upload

The important design decision is what the relay does with your audio. The cheap implementation buffers a session, writes it to disk, and hands a finished file to the model. The relay implementation streams: each 100-200 ms frame is forwarded upstream as it arrives.

Streaming costs a little protocol complexity and buys three things that matter:

• No storage surface. There is no buffer to breach, no retention policy to get wrong, and no file to subpoena. The audio genuinely does not exist at rest.
• Real latency. The first text arrives after roughly one frame, not after the recording ends.
• Honest metering. Duration is derived from bytes actually relayed, so a client cannot claim less than it sent.

Short-lived, single-use session tokens

A WebSocket handshake cannot carry an Authorization header from a browser, so the token has to travel in the URL. Anything long-lived in a URL leaks: URLs land in proxy logs, browser history, referrer headers and crash reports.

The pattern that works:

• The client authenticates normally and gets a bearer access token.
• It exchanges that for a WebSocket token valid for 5 minutes and bound to one connection attempt.
• The relay validates it once, on the handshake, and then forgets about it.

The blast radius of a leaked URL token is therefore a single five-minute window, not a standing credential. The handshake is the only place authorisation needs to happen, because once the socket is open the connection itself is the authorisation.

Speaking the upstream protocol

Streaming recognizers generally expect a sequence counter on every frame, a session-init message with stream metadata, and an explicit end-of-stream marker. Different providers spell this differently, and the details are exactly what breaks when you assume they are standard.

The relay absorbs this. Your client sends frames; the relay assigns the upstream sequence numbers, opens the upstream session with the right parameters, translates upstream messages back into a small result envelope, and signals end-of-stream when your client closes.

This is also where the "independent service" boundary matters. Dictate relays to the Volcano Engine Doubao streaming recognition model (bigmodel) through the Volcano Engine Open Speech Platform. The upstream model is not ours, and accuracy, language support and model behaviour are properties of that model.

Atomic quota accounting

Naive quota checks race. Two sessions start, both read the same remaining balance, both pass the check, and both proceed - the limit is exceeded by the amount the concurrency allowed.

The fix is to make the check and the decrement a single database operation, inside one transaction, and to fail the session rather than retry if the commit loses the race:

UPDATE usage_records
   SET used = used + :duration_ms
 WHERE user_id = :user_id
   AND period_start = :period_start
   AND used + :duration_ms <= included_duration_ms
RETURNING remaining_ms

If the statement returns no row, the quota is genuinely exhausted. If it returns a row, the decrement is already durable and no other session can double-spend it. There is no read-then-write window to lose.

Backpressure and abuse resistance

A WebSocket with no send limit is a free bandwidth and compute meter. A relay needs to bound:

• Frame size and rate. Reject oversized frames and clients that exceed a sane frames-per-second ceiling.
• Session duration. Cap a single session's wall time so a client cannot hold a connection open indefinitely.
• Concurrent sessions per account. Otherwise one key can fan out across hundreds of sockets and exhaust upstream capacity for everyone.
• Total payload. Track bytes relayed per session and reconcile against the metered duration at close.

On a SQLite deployment this last part needs one more thing: WAL mode with a busy timeout. Without it, concurrent writes fail with "database is locked" under exactly the load you are trying to protect against.

What this buys the client

A client integrating Dictate needs a bearer token, a five-minute socket token, and a loop that writes PCM frames and reads text. It does not need to know the upstream provider, hold a billable credential, implement sequence numbering, or reason about retention - because there is nothing retained.

Next: the integration guide, or what "we do not store your audio" means in practice.