Overview

Principal Machine Learning Engineer (Speech AI and LLM Agents)

Responsibilities

• Own the full realtime speech loop: streaming ASR, LLM inference and streaming TTS, with hard p50 and p95 latency budgets for every hop.
• Design, train and ship turn-taking, endpointing, barge-in and audio emotion models where off-the-shelf systems fall short.
• Build agent systems that compile a natural-language brief into a tested production agent: tool use, structured state graphs, automated simulation, scoring and self-repair.
• Architect multi-party, multilingual realtime audio: per-speaker streams, live speech translation, and floor control between humans and agents.
• Implement agent-to-agent communication on open protocols (A2A, MCP) with verified identity, scoped permissions and loop protection.
• Build the evaluation and data flywheel: golden sets per language, LLM-as-judge calibrated against human labels, regression gates in CI, consent-aware labelling and PII redaction.
• Fine-tune, distil, quantise and serve open-weight LLM and speech models, on cloud GPUs and on edge hardware that runs with no network.
• Automate MLOps end to end: training pipelines, model registry, canary releases, auto rollback, and cost and latency guardrails per tenant.
• Hire, mentor and lead a team of 3 to 8 ML engineers.

Qualifications

• 7+ years building ML systems, with 3+ years shipping LLM or speech systems to real users at scale.
• A latency-critical streaming system you shipped, and the numbers to prove it: p50, p95, cost per unit, failure modes.
• Expert Python and PyTorch. Strong command of transformer internals: attention variants, KV cache, decoding and sampling strategies.
• Hands-on speech experience: Conformer and Whisper-class ASR, neural codec or flow-matching TTS, VAD, audio front ends, Opus and telephony sample rates.
• LLM serving and optimisation: vLLM, TensorRT-LLM or equivalent; continuous batching, speculative decoding, INT8 and INT4 quantisation (AWQ, GPTQ).
• Training and alignment: SFT, LoRA and QLoRA, DPO or other preference optimisation, distillation; evals that predict production outcomes.
• Production infrastructure: Kubernetes, AWS, GPU scheduling, tracing and observability.
• You write clearly, turn ambiguity into a spec, and ship.
• WebRTC, SIP and RTP, or telephony at scale.
• Multilingual and Indic speech, including code-switched speech.
• Multi-agent systems in production; A2A or MCP.
• On-device inference: llama.cpp, ONNX Runtime, TensorRT on Jetson, Core ML.
• Audio watermarking, voice anti-spoofing or deepfake detection.
• Publications at Interspeech, ICASSP, ACL, EMNLP or NeurIPS, or widely used open-source work.
• Has hired and led ML engineers.
• 30 min intro screen
• 45 min conversation with the Founder
• 60 min technical deep dive: realtime speech and LLM agents
• 90 min live system design
• Paid take-home (4 to 6 hours) or an ML depth session, your choice
• References
• First principal ML seat, with real ownership of architecture and hiring.
• Direct line to the Founder; no layers.
• Hard, unsolved problems in realtime speech and multi-agent systems.
• Freedom to publish and open-source non-core work.
• Top-of-market compensation for the role.