Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Back to Browse Jobs
Turing
Remote Data Engineer - 78243
Turing
New Delhi•Hybrid•1-3 Yrs•Jun 26
Job description
Hybrid
Overview
ABOUT THE ROLE: The TDD's multi-modal expertise capture pipeline and Graph RAG branch of the Hybrid architecture require a dedicated data engineer who builds and operates the document ingestion pipeline, chunking engine, vector database indexing, retrieval/re-ranking pipeline, golden dataset management system, and data pool routing infrastructure.
Responsibilities
- Implement data quality checks, validation rules, and monitoring for pipeline health Write and maintain data transformation logic in Python with comprehensive test coverage B. PROJECT-SPECIFIC RESPONSIBILITIES: Build the batch extraction pipeline that ingests historical expertise artifacts in multiple formats (PDF, audio, video, text) using Apache Tika and PyMuPDF for document parsing, stores them in S3 with metadata in PostgreSQL, and routes them through the chunking engine
- Implement the semantic chunking engine that processes different document types with type-specific strategies (semantic chunking for transcripts, structural chunking for reports, function-level chunking for code) and generates embeddings for vector database indexing
- Implement the synthetic augmentation data pipeline that takes validated human examples, generates synthetic variants via LLM, and routes them through the human review queue with "syntheticpending validation" status
- Build the Graph Context Retriever that extracts entity subgraphs from the knowledge graph via read-only API for the evaluation scenario context