Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Back to Browse Jobs
Weekday AI
Software Engineer - (Machine learning Platform)
Weekday AI
Bengaluru•Hybrid•5-8 Yrs•3 days ago
Job description
Hybrid
Overview
This role is for one of the Weekday's clients
Responsibilities
- Design and operate an enterprise-grade LLM gateway and control plane featuring intelligent routing, rate limiting, failover, token management, and cost optimization.
- Develop unified APIs and SDKs (REST/gRPC) with standardized schemas, structured outputs, caching, and comprehensive observability.
- Implement security and privacy controls including content moderation, prompt validation, response filtering, and PII protection.
- Enable seamless multi-model and multi-provider LLM integration with automated versioning, canary deployments, and rollback mechanisms.
- Build and maintain the runtime environment for AI agents, including tool registries, permissions, function calling, retrieval, and grounding capabilities.
- Design orchestration patterns for sequential, planner-executor, streaming, and long-running workflows.
- Integrate human-in-the-loop approval mechanisms and safe execution controls before automated actions interact with production systems.
- Monitor agent performance, reliability, and operational health through telemetry and evaluation frameworks.
- Develop scalable infrastructure supporting training, deployment, and inference for classical machine learning models and deep learning workloads.
- Build standardized pipelines for experiment tracking, model packaging, deployment, and lifecycle management.
- Implement automated monitoring for model performance, drift detection, retraining, and continuous optimization.
- Support online and batch inference workflows for production ML applications.
- Build and evolve knowledge graph infrastructure and entity resolution systems.
- Develop reliable, scalable data ingestion and transformation pipelines.
- Deliver contextual information to AI systems through secure access controls, lineage tracking, and governance.
- Implement hybrid retrieval systems combining graph databases, vector search, and keyword search for high-quality context retrieval.
- Design continuous evaluation frameworks measuring model quality, factual accuracy, bias, safety, and performance.
- Define and monitor Service Level Objectives (SLOs) for latency, uptime, throughput, and infrastructure cost.
- Manage model registries, versioning, approvals, audit trails, and reproducible deployment workflows.
- Build autoscaling strategies, monitoring dashboards, and cost optimization mechanisms.
- Create reusable SDKs, templates, developer tools, CLIs, documentation, and sandbox environments.
- Improve platform usability by establishing best practices for ML engineering, MLOps, AI safety, and deployment workflows.
- Mentor engineering teams and promote scalable engineering standards across the organization.
- Knowledge Graph & Data Platform
Qualifications
- 5–11+ years of experience building large-scale machine learning platforms, data platforms, or distributed software systems.
- Strong software engineering skills with expertise in distributed systems, concurrency, and scalable API design.
- Production experience with Python and at least one of Java, Go, or Scala.
- Hands-on experience building microservices and REST/gRPC APIs.
- Strong understanding of cloud-native architecture, containers, and Kubernetes.
- Practical experience deploying and operating production machine learning systems.
- Knowledge of AI agent architectures, orchestration frameworks, tool calling, safety mechanisms, and online evaluation techniques.
- Strong cloud experience on AWS or equivalent cloud platforms.
- Familiarity with vector databases such as pgvector, Milvus, Qdrant, or similar technologies.
- Background in retrieval-augmented generation (RAG), AI agents, and enterprise AI platforms.
- Thinks of infrastructure as a product and prioritizes developer experience.
- Designs systems with reliability, observability, scalability, and security from the outset.
- Is passionate about AI platforms, LLMs, machine learning infrastructure, and agentic systems.
- Balances performance, latency, reliability, and operational cost when designing solutions.
- Prefers vendor-agnostic architectures that maximize flexibility and portability.
- Enjoys documenting complex systems and mentoring engineering teams.
- Python
- Machine Learning Platform
- MLOps
- Distributed Systems
- Kubernetes
- REST APIs
- AI/LLM Infrastructure
- ML Infrastructure
- Java
- Graph Databases
- Vector Databases
- RAG
- AI Agents
- Kafka