Overview

NTT DATA is seeking a Data Engineer Advisor to join their team in Chennai, Tamil Nādu, India. The role involves working with the Databricks ecosystem including Unity Catalog, Delta Lake, and Delta Live Tables. The ideal candidate will have expertise in Python (PySpark), SQL, Terraform, CI/CD tools, and cloud platforms.

Responsibilities

• Architect and provision the core Databricks ecosystem across multi-workspace environments using cloud-native services (Azure, AWS, or GCP).
• Establish shared data structures and optimized storage strategies utilizing the Delta Lake format, Auto Loader, and Delta Live Tables (DLT).
• Implement cost-management strategies by configuring optimal cluster sizing, autoscaling, and serverless SQL warehouses to balance performance against cloud spend.

Qualifications

• Mastery of Databricks Ecosystem including Unity Catalog, Delta Lake, Photon engine, Serverless Compute, DABs, and DLT.
• Advanced proficiency in Python (PySpark), SQL, and standard shell scripting languages (Bash / PowerShell).
• Strong experience with Terraform, git workflows, and CI/CD tools (e.g., Azure DevOps, GitHub Actions, or GitLab CI).
• Hands-on experience administering data infrastructure on at least one cloud provider (AWS, Azure, or GCP).
• Complete understanding of Apache Spark core concepts, memory management, optimizations (Z-order, data skipping), and tuning.

Benefits

• Inclusive, adaptable, and forward-thinking organization.
• Opportunity to work on innovative and exciting projects.
• Global Top Employer with experts in over 50 countries.
• Access to a robust ecosystem of innovation centers and partners.