Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Senior Data Engineer II
MX
Chennai•On-site•5-8 Yrs•1 hour ago
Job description
Overview
We are fueled by a moral imperative to advance mankind, and it all begins with our people, our product, and our purpose.
Responsibilities
- Architect ML Infrastructure: Design and maintain scalable data pipelines and MLOps workflows specifically within the Google Cloud Platform (GCP) ecosystem.
- Model Productionization: Deploy, monitor, and optimize machine learning models as production-ready Vertex AI and Ray endpoints.
- Cluster Management: Orchestrate and fine-tune Kubernetes (GKE) clusters to support high-throughput data processing and real-time model serving.
- CI/CD for ML: Collaborate with Data Scientists to automate the entire ML lifecycle—from training and evaluation to seamless deployment—using Docker and modern orchestration tools.
- Real-Time Data Engineering: Build and optimize streaming pipelines (utilizing Apache Flink) and implement advanced analytical structures like Data Sketches for high-speed probabilistic analysis.
- Environment Standardization: Develop and maintain specialized Docker images to ensure consistent, reproducible environments across the full ML development lifecycle.
Qualifications
- Cloud Mastery: Extensive, hands-on experience with Google Cloud Platform (GCP) is mandatory.
- Vertex AI Deep Dive: Proven expertise in the Vertex AI suite, including Pipelines, Model Registry, Feature Store, and Endpoints.
- Containerization & Orchestration: Deep technical knowledge of Kubernetes architecture and Docker best practices for production workloads.
- Programming & Data: Expert proficiency in SQL and Python is required for complex data manipulation and SDK integration.
- Streaming & Scalability: Experience handling large-scale, real-time datasets. Familiarity with Apache Flink and event-driven architectures is highly desirable.
- Fintech Mindset: Understanding of the rigor required for financial data, including idempotency, auditability, and security.
- Collaborative Leader: A track record of working effectively across multi-disciplinary teams (Data Science, DevOps, and Product).
- Problem Solver: Ability to leverage approximate computing (Data Sketches) and other advanced techniques to solve performance bottlenecks.
Education
- Bachelor’s or Master’s degree in Computer Science, Data Science, Statistics, or a related technical field.
- Please note applicants applying for this position must have the legal right to work in India without the need for sponsorship. We are unable to provide work sponsorship for this role, and candidates should be able to verify their eligibility to work in the country independently. Proof of eligibility to work in India will be required as part of the hiring process.