Back to Browse Jobs
Altraize
Create a free account
One signup unlocks everything for this job:
- Apply to this job and track your application
- See your personalized profile match score
- Generate an AI cover letter for this role
Coimbatore · On-site · 10-13 Yrs · Aug 30
1 open role0 posted in 7d1 in 30d
Job description
data engineerLeadGenAILLM
Overview
Lead Data Engineer with expertise in Databricks, GenAI/LLM concepts, and cloud data platforms.
Responsibilities
- Technical leadership of data platform initiatives
- Designing and implementing the medallion architecture for data ingestion, curation, and consumption
- Building ETL/ELT pipelines for batch and streaming data at scale
- Building and managing data lakes and data warehouses
- Working with LLMs and GenAI techniques such as prompts, embeddings, vector databases, and RAG
- Leading large-scale data migrations from on-premise to modern architectures
- Implementing data security, governance, lineage, and access controls
- Experience with Agile/Scrum delivery methodologies
- Mentoring engineers
Qualifications
- 10+ years of hands-on data engineering experience
- Strong expertise with Databricks Lakehouse Platform
- Proven experience with Delta Lake, Delta Live Tables, Databricks Workflows, and Unity Catalog
- Experience with distributed data processing using Apache Spark (PySpark and Spark SQL)
- Expert-level proficiency in Python and SQL
- Experience building data lakes and data warehouses
- Knowledge of LLMs and GenAI concepts
- Hands-on cloud platform experience (AWS, Azure, or GCP)
- Proven experience leading large-scale migrations
- Understanding of performance optimization techniques
Verify Spark SQL for this role
Free PRACTICE for Spark SQL adds a verified skill signal that can boost how roles like this rank for you — optional, not required to browse or apply.
Sign up — verify Spark SQL free →Want match scores and deeper insight? Create a free candidate profile