Real-time voice translation under 500ms across six platforms, with voice cloning and intelligent agents. That is the brief we shipped on the AI Communication Platform. It is also a brief that breaks the moment you let a coding model touch the latency-critical path or invent the rubric that decides what "good" means.
We built it anyway, and we used AI heavily to do it. The trick was knowing where to point the model and where to keep our hands on the keyboard. This post is the build-log of that boundary: by our internal estimate, roughly 70 percent of the implementation scaffolding was AI-drafted and human-reviewed, while the architecture, latency path, security policy, eval rubrics, and release gates stayed human-owned.
What sub-500ms across six surfaces actually demands
The platform spans web, desktop, iOS, Android, Slack, and Teams, six clients, one media pipeline behind them. Here, <500ms means first translated audio byte on the optimized same-region streaming path, measured from the point where the platform has enough transcript to start translation. It is not p95 full call round-trip latency across arbitrary regions, and it is not a promise that every language or model route has the same profile.
Voice flows through streaming speech-to-text, an LLM translation step, and streaming text-to-speech, with WebRTC for media transport and WebSockets for session control. LiveKit's current guidance puts cascaded STT -> LLM -> TTS streaming pipelines in the sub-second range: STT partials around 100-200ms, LLM time-to-first-token around 200-400ms, TTS first audio around 100-300ms, plus transport and turn detection. Getting a measured path below 500ms means overlapping stages, co-locating inference, controlling turn detection, and treating first audio byte, p50, and p95 as separate metrics.
One caveat matters. ElevenLabs Flash v2.5 is the low-latency 32-language TTS path. Eleven v3 is the 70+ language model. The fastest path and the widest-language path are not the same route, so we benchmark and route by language instead of pretending one model gives you everything.
None of those decisions are prompts. They are layout decisions, where compute lives, what runs in parallel, which buffer gets flushed first. A coding model can write a WebSocket handler that compiles and passes a unit test. We do not let it own the latency topology or the rollback criteria when your TTS first byte slips from 75ms to 240ms after a region move. That gap, between "code that runs" and "code that runs inside a real-time budget", is where the build-log starts.
Imagine you are a CTO shipping a voice agent into a regulated industry. The demo works on your laptop. Then you put it in front of a customer in Frankfurt talking to an agent in Sao Paulo, and the round-trip jumps to 1.4 seconds because your inference is in us-east-1 and your TTS is in us-west-2. That is an illustrative topology failure, not a product benchmark. No prompt fixes it. A senior engineer redrawing the topology does.
Why "vibe coding" the whole thing fails in production
Simon Willison drew the cleanest line on this in October 2025: vibe coding is irresponsibly building software through dice rolls without caring what code is produced; vibe engineering is when senior practitioners use AI tools responsibly to accelerate their work. The 2025-2026 incident catalog from Crackr, Autonoma, and Kognitos is the negative case for the first half of that distinction.
The recurring failure modes when AI authors the wrong layer are familiar to anyone who has cleaned up after a vibe-coded prototype:
- Lovable-generated apps shipping with missing or insufficient Postgres Row-Level Security policies, leaving private data reachable by unauthenticated users.
- 1.5M API keys exposed across documented vibe-coded repos.
- Buried business logic with no canonical place for the rule to live.
- Success-path-only error handling, no audit trail, logic drift across deployments as prompts evolve.
- Zero institutional knowledge captured in the codebase, only the model knew why something worked, and the model is non-deterministic.
For a CRUD app, those are bugs. For a communication platform that handles conversations, billing meters, voice clone consent, and PII across languages and locales, they are existential. A translation that hallucinates a clause in a legal call is not a bad UX. It is a liability with a name on it.
The honest version of the failure is what one developer in the catalog wrote about their own vibe-coded launch: "random things are happening, maxed out usage on api keys, people bypassing the subscription, creating random shit on db." That is what happens when you let a model author code in layers it cannot evaluate.
The appssemble rule: AI ships code that evals cover
Our AI service follows a one-line process, Define → Prototype → Harden → Ship, and inside that process there is a single rule that decides where AI is allowed to write code. AI ships code that evals cover. Humans write the code, the rubrics, and the policies that decide what "covered" means.
That rule is not a moral position. It is operational. Hamel Husain makes the same point in his field guide: unsuccessful AI products usually fail because the team never builds a reliable evaluation system. If we cannot point at an automated eval that catches a regression in a given layer, we do not let the model author that layer. The eval is the seatbelt. No seatbelt, no AI-authored code.
This split lines up with what Sourcetoad, CIO, and Mabl have been writing for the last year about senior teams: AI handles routine implementation work, boilerplate, CRUD, scaffolding, glue, test fixtures, while the critical layer stays human. Architecture, security, performance, business logic, and the rubrics that score everything else. Our 70/30 number is an internal estimate of first-draft implementation scaffolding, not a line-count audit or a claim that AI owned production accountability.
Five places the boundary actually lives
Here is where AI authored code in the AI Communication Platform, and where it did not.
1. Latency budget owned by humans. Sub-500ms is not a prompt; it is a layout decision. The streaming topology, the regional pinning, the preemptive generation that starts the LLM response before the user's end of turn is confirmed, all of it hand-shaped. We pick the predicates. We measure the hops. The model can write a Deepgram client adapter. It does not get to decide where the inference lives.
2. Eval rubrics owned by humans. The rubric is the asset, not the judge. In MT-Bench and Chatbot Arena, strong LLM judges exceeded 80% agreement with human preferences in those benchmark settings. That is useful, but it is not a blank check. For translation quality, voice-clone consent, and refusal behavior, the rubric is hand-written, the judge is calibrated against human labels, and deterministic checks catch what should never be subjective. Braintrust runs the regressions; LangSmith captures the traces. Neither tool decides what good means.
3. Glue and scaffolding owned by AI. Six clients, one backend, fast iteration. Swift view controllers for iOS, Kotlin fragments for Android, an Electron wrapper for desktop, and the Slack and Teams integration apps, all of them have idiomatic patterns the model is well-trained on. AI writes the first draft of the UI scaffolding, the DTOs, the parsers, the mock-driven Jest suites, the protobuf adapters between client and gateway. Humans review every diff. The acceptance bar is the test suite, and the test suite is the eval. The speedup is real by our internal delivery notes, but it only compounds because a senior engineer is reading every line before it merges.
4. Boring-but-critical owned by humans. Idempotency keys on every external tool call, durable state in Postgres instead of the context window, cheap models tried first with expensive models on retry, validation steps before every irreversible action, and per-user rate limits on top of global ones. Five patterns, one rule: written by hand, reviewed twice. A multi-tenant cost meter that attaches { customer_id, workspace_id, feature, model, environment } to every token event is not the kind of thing you want a model to invent on a Tuesday afternoon. We wrote the schema. We wrote the rollup. We wrote the dead-letter queue for when the rollup falls behind.
5. Prompts versioned like code. Every prompt is in the repo with a version tag. Every change runs against a regression bank, typically 100+ items the previous version handled correctly. If the new version fails even one previously safe item, it is a regression and the version is rejected. New prompts ship as canary deployments: 10 percent of traffic goes to variant B, the other 90 percent stays on the control, metrics are compared, and if the canary holds we promote. This is the same release discipline we use for code, because prompts are code in this product. Treating them otherwise is how logic drift creeps in.
How the handoff to a human actually fires
One detail worth making concrete, because it sits right on the AI/human authorship line. The platform supports handoff to a human agent: a translator stepping in on a sensitive call, a support rep taking over from an AI agent that has drifted. The trigger conditions are not LLM judgments. They are explicit predicates we wrote and version: confidence score below a threshold, frustration sentiment detected by a sentiment classifier with a published rubric, regulated-topic match against a keyword list, three failed attempts at resolution, or an explicit ask from the user.
Each predicate is testable. Each one has a fixture set in the regression bank. When a predicate fires, the platform packages a structured context: a summary of the issue, account data, identified intent, sentiment indicators, prior attempts, and routes the call. The summary is LLM-generated. The decision to route is not. That distinction is a one-line policy in code, and it is the kind of thing we will not let drift into a prompt.
"Can't AI eventually do all of this?"
Yes, when the rubric, the latency budget, and the policy are themselves machine-readable. They are not yet, and pretending they are is how you ship the incident catalog above.
The honest answer is that AI does more of the work each quarter, and the boundary moves. Two years ago we were still writing native client UI by hand; now the model writes most first drafts and we review. A year ago we were hand-writing the structured-output parsers for tool use; now Instructor and function calling cover that surface. The 70/30 split is a snapshot from this build, not a constant. What does not move is the principle: a layer becomes safe for AI authorship when there is an automated eval that can catch its failures. Until that eval exists, a senior engineer writes the code.
There is a related objection worth answering. Founders sometimes ask whether evals are themselves vendor lock-in, once your rubrics live in Braintrust, are you stuck? No. The rubric is plain text and JSON; the eval runner is a thin layer on top. We have rebuilt this stack on Promptfoo, on Langfuse, on Helicone-plus-LangSmith. The point is to own the rubric, not the tool. Tools come and go.
What it adds up to
The AI Communication Platform, sub-500ms first translated audio on the low-latency path, six platforms, separate routes for wider language coverage, is roughly 70 percent AI-drafted scaffolding and 100 percent human-owned architecture. That is not a contradiction. It is the only ratio that lets you move fast on the parts a model is good at while keeping the parts that decide whether the product survives in production firmly in human hands.
Vibe coding ships demos. Vibe engineering ships products. The difference is the eval, the rubric, and the discipline to draw a line between "AI can write this" and "I have to write this myself", and to redraw it every time the tooling improves.
If you are a CTO or a founder weighing whether to own the AI work in your product or hand it off, this is the conversation we like to have. See how we build production AI without the demo-to-disaster gap. Book a call.
Built by developers. Accelerated by AI.