Assistant App — runnable, bring your own keys

It runs before you have any keys

Most boilerplate hands you a dead app until you have registered for six services. This one ships realistic mocks for every stage, so the first thing you see is a working assistant — streaming tokens, a tool call, a dangerous action held at an approval gate, and a live spend meter. Then you light stages up individually.

| Stage | Set this | Get this | | --- | --- | --- | | Speech-to-text | `GROQ_API_KEY` | Groq Whisper — fastest and cheapest | | | `OPENAI_API_KEY` | OpenAI Whisper | | Model | `ANTHROPIC_API_KEY` | Best tool-calling; the default when the assistant must *act* | | | `GROQ_API_KEY` | Cheapest and fastest; weaker at multi-step tool use | | Speech | `DEEPGRAM_API_KEY` | Deepgram Aura | | | *(nothing)* | Your browser's own synthesis — free, but audibly synthetic |

The app always shows which stages are real. A mock that is indistinguishable from the real thing is how a demo reaches production by accident.

You are not locked into realtime APIs. The expensive all-in-one realtime voice APIs are one arrangement among several, and usually not the cheapest. Composing cheap transcription + a cheap model + browser speech costs a fraction, and switching is a config change here rather than a rewrite.

What actually takes the time

The three API calls are the easy part. The orchestration between them is where assistants feel broken, and the turn orchestrator exists entirely to get it right:

1. Barge-in. You talk while it's talking. Speech stops *immediately* — not at the end of the sentence — and the new utterance supersedes the old turn. An assistant that talks over you is unusable, and this is the most common omission. 2. Cancellation cascades. One cancel aborts transcription, the model stream, *and* audio playback. Cancel only the outer promise and the voice keeps speaking a turn you abandoned. 3. Cost survives failure. A turn that dies mid-stream still cost money for the transcription that completed. Spend is charged per stage, never per turn — bill on success only and your tracking under-reports while budgets silently blow. 4. Budget is checked at every stage boundary. Running out mid-turn stops cleanly with a stated reason, rather than being discovered after the expensive part already ran. 5. User errors are not provider errors. Silence is yours; a 503 is theirs. Only the latter triggers failover — retrying silence across three providers spends three times as much to fail identically. 6. Approval pauses the voice. When a tool needs a human, the assistant stops talking and waits. It does not narrate over its own permission prompt.

Each is a test. All six were verified by mutation — the guard removed one at a time, the suite confirmed to fail each time. A test never seen to fail is a decoration.

25 tests passing · typecheck clean · 10/10 mutations caught.

What's inside

• The turn orchestrator and session. Pure: providers are injected, so it is testable without audio hardware, network, or keys.
• Key detection and the live-vs-mock report.
• Provider adapters, plus the mocks that make the zero-key path real.
• Voice capture with hysteresis VAD and the browser traps handled, and the INSTRUMENT design system for the interface — including hold-to-confirm on irreversible actions, because approval fatigue is the last defense and a click is one motor action.

Delivery

Private repository invite within 24 hours. Single-product commercial license: use and modify in one shipped product; no redistribution or resale of the source.

Interface

What you call, and what comes back. Types and signatures only — the implementation ships with the source.

  export function resolveConfig(env: Env = {}): AppConfig;
  export function describeConfig(cfg: AppConfig): string[];
  export function mockProviders(): TurnProviders;

  export type StageName = "stt" | "llm" | "tts";
  export type TurnStage = "idle" | "transcribing" | "thinking" | "awaiting_approval" | "speaking" | "done";
  export type TurnEvent = | { type: "stage";
  export type TurnEndReason = Extract<TurnEvent, { type: "done" }>["reason"];