Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Back to Browse Jobs
Allianz Insurance
Senior Consultant-Data Engineer
Allianz Insurance
Pune•On-site•10-13 Yrs•2 days ago
Job description
Overview
Role Overview We are looking for a highly experienced Senior Data Engineer to lead the re-engineering of an existing enterprise data platform built on Azure Synapse Analytics.
Responsibilities
- Lead the technical assessment and re-engineering of an existing enterprise data platform, spanning all layers from source ingestion through to data consumption
- Reverse-engineer, document, and validate existing pipeline logic, data models, transformation frameworks, and data governance controls
- Identify gaps, defects, and technical debt across the platform and remediate where implementations are incorrect or sub-optimal
- Ensure correctness of data processing patterns including change data capture, slowly changing dimensions, deduplication, and business reconciliation
- Design and implement target-state architectures aligned to modern lakehouse principles, ensuring feature parity and business logic fidelity during transitions
- Manage platform evolution initiatives, including parallel-run phases where multiple implementations operate simultaneously, validating output consistency before cutover
- Define and execute migration strategies for existing workloads to modern data platforms, preserving existing governance and control framework semantics
- Re-implement ingestion, transformation, and orchestration pipelines on target platforms, maintaining audit, quality, and reconciliation standards
- Collaborate with business, data governance, and architecture stakeholders to validate embedded business rules and data quality requirements
- Provide technical leadership across re-engineering and migration workstreams, contributing to decommission planning for legacy components
- Azure Synapse & Data Platform Mandatory hands-on expertise with:
- Azure Synapse Analytics (Pipelines, Spark Pool, Dedicated SQL Pool)
- Azure Data Lake Storage Gen2 (ADLS Gen2)
- Delta Lake on Azure (Synapse Lakehouse patterns)
- Oracle Golden Gate Replication for real-time source integration
- Azure Analysis Services and Power BI consumption layer patterns
- Deep understanding of medallion architecture: Raw / Harmonized / Conformed / Consumption layers
- Strong knowledge of SCD Type 0/1/2, CDC patterns, soft/hard delete, and retroactive change processing
Qualifications
- Familiarity with config-driven and automation-first pipeline patterns (YAML, PySpark, SQL-driven generation from mapping documents)
- Hands-on experience with Azure Databricks (Delta Live Tables, Unity Catalog preferred)
- Strong Apache Spark skills (PySpark / Spark SQL)
- Ability to re-implement governance and control frameworks natively in Databricks (audit logging, reconciliation, DQ checks)
- Data Engineering & Development
- Strong Python and SQL programming skills
- Strong data modelling skills: relational, dimensional, and lakehouse-oriented
- CI/CD pipelines for data engineering (Azure DevOps / GitHub Actions)
- Infrastructure as Code (Terraform or ARM)
- Containerization (Docker)
- Familiarity with Azure Purview for data cataloguing and governance
- Exposure to real-time and streaming pipelines (Event Hub / Kafka / Kinesis)
- Familiarity with monitoring and observability tools (e.g., Dynatrace)
- Exposure to BI tools (Power BI, Tableau)
- 10+ years of experience in Data Engineering, with significant platform migration or re-engineering experience
- Proven track record auditing and taking ownership of existing, complex enterprise data platforms — not just building from scratch
- Deep knowledge of enterprise data governance patterns: audit trails, reconciliation, data quality controls, SCD versioning
- Strong analytical mindset: ability to read existing implementations, identify intent versus defect, and make sound re-engineering decisions
- Comfortable operating across both hands-on engineering and technical architecture
- Strong communication skills — able to engage business, governance, and engineering stakeholders with clarity
- [please translate into your local language]
- 93014 | Data & AI | Professional | Non-Executive | Allianz Services | Full-Time | Permanent
- Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges, is what makes us a unique employer. Together we can build an environment where everyone feels empowered and has the confidence to explore, to grow and to shape a better future for our customers and the world around us.
- At Allianz, we stand for unity: we believe that a united world is a more prosperous world, and we are dedicated to consistently advocating for equal opportunities for all. And the foundation for this is our inclusive workplace, where people and performance both matter, and nurtures a culture grounded in integrity, fairness, inclusion and trust.
- We therefore welcome applications regardless of ethnicity or cultural background, age, gender, nationality, religion, social class, disability or sexual orientation, or any other characteristics protected under applicable local laws and regulations.
- Great to have you on board. Let's care for tomorrow.
- Note: Having different strengths, experiences, perspectives and approaches is an integral part of Allianz‘ company culture. One means to achieve this is a regular rotation of Allianz Executive employees across functions, Allianz entities and geographies. Therefore, the company expects from its employees a general openness and a high motivation to regularly change positions and collect experiences across Allianz Group.