Overview

Company Description

Responsibilities

• You'll own features end to end, from prototype to production. And you'll be the person who knows whether the system actually works because you built the evals that prove it'.
• Design, build, and ship LLM-backed product features: RAG systems, agentic workflows, tool and function calling, structured extraction, and multi-step reasoning pipelines.
• Build evaluation harnesses offline eval sets, LLM-as-judge rubrics, regression suites, online A/B tests so prompt and model changes ship on evidence rather than intuition.
• Own context engineering: chunking strategy, embedding and reranking choices, hybrid search, caching, and context-window budgeting.
• Integrate models into production services with sane latency, cost, and failure behavior streaming, retries, timeouts, cross-provider fallback, and token and cost instrumentation.
• Instrument and monitor AI in production: tracing, prompt and response logging, quality and drift dashboards, and incident response when output quality regresses.
• Implement safety and reliability guardrails input and output validation, PII handling, prompt-injection defenses, jailbreak resistance, and human-in-the-loop escalation.
• Fine-tune or adapt models (LoRA, SFT, distillation to smaller models) where it beats prompting on cost or quality.
• Work with product, design, and domain experts to define what 'good output' means before the prompt gets written.
• Write clear technical docs and post-mortems, and help teammates new to LLM systems get productive.

Qualifications

• 2–5 years of professional software engineering experience, including at least one year shipping LLM features to production users .
• Strong Python , Solid fundamentals: data structures, concurrency, API design, testing.
• Hands-on experience with modern LLM APIs and SDKs Anthropic Claude, OpenAI, Gemini, or open-weight models served through vLLM, Ollama, Bedrock, or Vertex.
• Practical command of prompt and context engineering : system prompts, few-shot design, structured and JSON output, tool use, and reflection or self-critique patterns.
• Real RAG experience: embeddings, vector stores (pgvector, Pinecone, Weaviate, Qdrant, Milvus), chunking strategy, hybrid and BM25 search, and reranking.
• Demonstrated ability to evaluate AI systems. You can describe an eval set you built, the metrics you chose, and a change you rejected because the numbers said so.
• Experience with agent orchestration LangGraph, LlamaIndex, the Vercel AI SDK, DSPy, the Claude Agent SDK, or a hand-rolled equivalent and with MCP or a comparable tool-integration standard.
• Production engineering skills: Docker, CI/CD, a major cloud, observability, and cost awareness at scale.
• Fluency in AI-specific failure modes hallucination, prompt injection, data leakage, non-determinism and concrete mitigations for each.
• Clear written communication. You can explain a tradeoff to a PM and a stack trace to an engineer.

Nice to have

• Fine-tuning experience (LoRA/QLoRA, SFT, DPO or RLHF) with PyTorch or Hugging Face.
• Inference optimization: quantization, batching, KV caching, speculative decoding, GPU serving with vLLM, TGI, or TensorRT-LLM.
• LLM observability tooling LangSmith, Langfuse, Braintrust, Arize, or W&B Weave.
• Multimodal work: vision, audio, document understanding, or OCR pipelines.
• Voice and realtime AI, streaming architectures, or other low-latency interactive systems.
• Classical ML background feature engineering, gradient boosting, ranking or recommender systems.
• Data engineering: dbt, Airflow or Dagster, Spark, warehouse modeling.
• Working knowledge of AI governance and regulation (EU AI Act, NIST AI RMF, SOC 2, GDPR) and how it shapes system design.