Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Back to Browse Jobs
Astrazeneca India Private Limited
Data Engineer Consultant
Astrazeneca India Private Limited
Chennai•On-site•5-8 Yrs•Jul 7
Job description
Data EngineeringETL/ELTPlatform AdministrationSREGovernanceAWSDatabricksRedshift Serverless
Overview
Data Engineer Consultant at AstraZeneca India Private Limited, responsible for running high-impact data platforms at scale using Databricks and Redshift Serverless on AWS. Administrative and evolution of data platforms to ensure reliability and efficiency, coordination of ETL/ELT end-to-end, maintenance of strong governance, and enabling colleagues to move from proof of concept to enterprise-grade solutions.
Responsibilities
- Administer and evolve Databricks and Redshift Serverless on AWS
- Coordinate ETL/ELT end-to-end
- Maintain strong governance for data platforms
- Provision and lead Databricks workspaces, cluster policies, and Workflows
- Administer Redshift Serverless workgroups and name spaces
- Implement IAM/RBAC and Unity Catalog permissions
- Monitor and tune PySpark/Spark SQL and Redshift Serverless
- Use Bash/Python and CI/CD to codify infrastructure and deployments
- Orchestrate and support ingestion and transformation pipelines across S3, Databricks, and Redshift
- Lead incident response and root-cause analysis
- Own GitHub strategy, tagging, and protection rules
- Produce clear documentation and handovers
Qualifications
- B.E/B.Tech/MCA/MSc (or equivalent experience)
- 5 - 6 years of overall experience
- Databricks: PySpark and Spark SQL, Delta Lake guidelines, Workflows/Jobs, cluster policies, Unity Catalog governance, and cost controls
- Redshift Serverless: Workgroup administration, RPU management, query monitoring rules, performance tuning, Spectrum/external tables, COPY/UNLOAD to S3, snapshots/restore, security and permissions
- Unix/Linux: System administration, Bash scripting, log management, OS hardening
- AWS: S3, IAM, CloudWatch/Cloud Trail, basic VPC concepts, budgeting and tagging
- DevOps/IaC: Git and GitHub (branching, PRs, Actions), CI/CD pipelines and deployments
- Strong troubleshooting experience across Spark/SQL/OS; on-call readiness
Nice to have
- Amazon EC2, S3, exposure to PostgreSQL
- Jira and ServiceNow for ITSM
- Airflow or similar orchestration tooling
- Fast learner, proactive problem solver able to grasp new technical concepts
Education
- B.E/B.Tech/MCA/MSc with 5-6 years experience
Benefits
- Competitive compensation packages
- Learning and development opportunities
- Industry leading benefits and wellness programs
- Flexible working hours within office requirements