Back to Browse Jobs
enGen Global
Create a free account
One signup unlocks everything for this job:
- See your personalized profile match score
- See the skills this role is missing for you
- Apply to this job and track your application
- Generate an AI cover letter for this role
Chennai · On-site · 8-11 Yrs · Oct 7
1 views
Energy & Utilities · 1001 to 5000 Employees · Cape Town, South Africa
1 open role0 posted in 7d1 in 30d
Work-life balance 3.3Compensation & benefits 3.6Culture & values 3.8
Hiring: Engineering
What employees say
+ National Company Great place to work - a lot of interaction
− BEE dominant Travel Challenging environment
More reviews on the company profile.
Job description
Overview
Lead PySpark Cloud Data Engineer: This job involves understand the overall requirement of the enterprise Data need and design develop the robust, highly scalable resilient data pipeline using PySpark, Dataproc, open source table formats and other Google services.
Responsibilities
- Design, develop, and implement highly scalable, reliable, and performant data pipelines using PySpark, Dataproc, and other Google cloud-native technologies
- Create pipelines for data lakehouse architectures utilizing Apache Iceberg for open table formats and BigLake Metastore for unified metadata management
- Develop and maintain data transformation logic using Pyspark and / or dbt to create clean, consistent, and production-ready data models
- Work extensively with Google Cloud Platform (GCP) services, including Dataproc, BigQuery, Cloud Storage, and other relevant data services
- Optimize data pipelines and queries for performance, efficiency, and cost-effectiveness
- Implement and enforce data quality checks, monitoring, and goverce best practices to ensure data integrity and reliability
- Provide technical guidance and mentorship to junior data engineers, fostering a culture of continuous learning and improvement
- Create and maintain comprehensive technical documentation for data pipelines, data models, and platform architecture
- Provide regular updates on the tasks, status and risks to project manager
Qualifications
- Bachelors degree or higher from a reputed university
- 8 to 12 years total experience with majority of that experience related to building high performant data pipelines batch and streaming
- Strong Hands on experience / expert-level proficiency in PySpark for data processing, transformation, and analysis
- Extensive experience in implementing large scale data ingestion and curation solutions
- Hands-on experience with cloud-based data platforms, preferably Google Cloud Platform (GCP) and services like Dataproc, BigQuery, and Cloud Storage
- Knowledge in Apache Iceberg for open table formats
- Advanced SQL skills for data querying, manipulation, and optimization
- Proficiency with Git and collaborative development workflows
- Excellent analytical and problem-solving skills with a keen attention to detail
- Strong communication and interpersonal skills, with the ability to explain complex technical concepts to both technical and non-technical audiences
- Expertise in Google Cloud services
- Proven experience with Apache Iceberg for open table formats and BigLake Metastore for unified metadata management
- Strong experience with dbt (data build tool) for data modeling, transformation, and orchestration
- Experience in Data Goverce and Data quality tools like Atlan, Monte Carlo etc.
- Experience in processing streaming data using Kafka / Pub-Sub
- Healthcare industry experience
- Experience in Agile
Verify SQL for this role
Free PRACTICE for SQL adds a verified skill signal that can boost how roles like this rank for you — optional, not required to browse or apply.
Sign up — verify SQL free →Want match scores and deeper insight? Create a free candidate profile
Job application FAQ
Last updated 28 October 2026
- How do I apply for Senior Consultant - Cloud Data Engineer at enGen Global?
- Open this job on Feedback AI and sign up free (or sign in). You can apply from the job detail page; match scores and skill gaps appear when your profile is complete. enGen Global may attach structured assessments to interview rounds after you apply.
- Does Feedback AI show a match score for this Senior Consultant - Cloud Data Engineer role?
- Yes after you add a resume or skills. Match score ranks this role against your profile — skills met, gaps, level, and location — so you prioritise applications where fit is strongest.
- Is Senior Consultant - Cloud Data Engineer remote or on-site in Chennai, Tamil Nadu?
- This listing is marked on_site with location Chennai, Tamil Nadu. Confirm details on the job description before applying.
- What skills matter for this Senior Consultant - Cloud Data Engineer role at enGen Global?
- This is a senior Senior Consultant - Cloud Data Engineer role. Key skills include skills listed in the job description. Feedback AI highlights matched vs missing skills on your profile after signup.