Overview

Senior Associate - Azure Data Engineer in Data & Analytics Advisory sector. Location: PAN India.

Responsibilities

• Analyses current business practices, processes, and procedures as well as identifying future business opportunities for leveraging Microsoft Azure Data & Analytics Services
• Provide technical leadership and thought leadership as a senior member of the Analytics Practice in areas such as data access & ingestion, data processing, data integration, data modeling, database design & implementation, data visualization, and advanced analytics.
• Engage and collaborate with customers to understand business requirements/use cases and translate them into detailed technical specifications.
• Develop best practices including reusable code, libraries, patterns, and consumable frameworks for cloud-based data warehousing and ETL.
• Maintain best practice standards for the development or cloud-based data warehouse solutioning including naming standards.
• Designing and implementing highly performant data pipelines from multiple sources using Apache Spark and/or Azure Databricks
• Integrating the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained
• Working with other members of the project team to support delivery of additional project components (API interfaces)
• Evaluating the performance and applicability of multiple tools against customer requirements
• Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.
• Integrate Databricks with other technologies (Ingestion tools, Visualization tools).

Qualifications

• Proven experience working as a data engineer
• Highly proficient in using the spark framework (python and/or Scala)
• Extensive knowledge of Data Warehousing concepts, strategies, methodologies.
• Direct experience of building data pipelines using Azure Data Factory and Apache Spark (preferably in Databricks).
• Hands on experience designing and delivering solutions using Azure including Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics
• Experience in designing and hands-on development in cloud-based analytics solutions.
• Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required.
• Designing and building of data pipelines using API ingestion and Streaming ingestion methods.
• Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential.
• Thorough understanding of Azure Cloud Infrastructure offerings.
• Strong experience in common data warehouse modeling principles including Kimball.
• Working knowledge of Python is desirable.
• Experience developing security models.

Nice to have

• Databricks & Azure Big Data Architecture Certification
• Certified in Android Debug Bridge (ADB)
• Certified in Microsoft Azure
• Python proficiency
• Scala proficiency
• Spark proficiency

Education

• BE
• B.Tech
• MCA
• M.Tech

Benefits

• Equal employment opportunities without discrimination
• Inclusive benefits
• Flexibility programmes
• Mentorship