Overview

Senior Associate - Azure Data Engineer with experience in data architecture, developing scalable cloud-based data solutions, ETL/ELT pipelines, and working within agile environments.

Responsibilities

• Analyses current business practices, processes, and procedures as well as identifying future business opportunities for leveraging Microsoft Azure Data & Analytics Services
• Provide technical leadership and thought leadership in areas such as data access & ingestion, data processing, data integration, data modeling, database design & implementation, data visualization, and advanced analytics
• Engage and collaborate with customers to understand business requirements/use cases and translate them into detailed technical specifications
• Develop best practices including reusable code, libraries, patterns, and consumable frameworks for cloud-based data warehousing and ETL
• Designing and implementing highly performant data pipelines from multiple sources using Apache Spark and/or Azure Databricks
• Integrating the end-to-end data pipeline to take data from source systems to target data repositories ensuring quality and consistency of data is always maintained
• Working with other members of the project team to support delivery of additional project components (API interfaces)
• Evaluating the performance and applicability of multiple tools against customer requirements
• Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints
• Integrate Databricks with other technologies (Ingestion tools, Visualization tools)

Qualifications

• Proven experience working as a data engineer
• Highly proficient in using the spark framework (python and/or Scala)
• Extensive knowledge of Data Warehousing concepts, strategies, methodologies
• Direct experience of building data pipelines using Azure Data Factory and Apache Spark (preferably in Databricks)
• Hands on experience designing and delivering solutions using Azure including Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics
• Experience in designing and hands-on development in cloud-based analytics solutions
• Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required
• Designing and building of data pipelines using API ingestion and Streaming ingestion methods

Nice to have

• Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential
• Thorough understanding of Azure Cloud Infrastructure offerings
• Strong experience in common data warehouse modeling principles including Kimball
• Working knowledge of Python is desirable
• Experience developing security models
• Databricks & Azure Big Data Architecture Certification would be plus

Education

• BE, B.Tech, MCA, M.Tech
• MBA (Master of Business Administration), Master of Engineering, Bachelor of Engineering
• Certifications: ADE, ADB, ADF

Benefits

• At PwC, you will be part of a vibrant community of solvers that leads with trust and creates distinctive outcomes for our clients and communities
• Rewarded for contributions, supported in wellbeing, offer inclusive benefits, flexibility programmes and mentorship that will help you thrive in work and life