Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Back to Browse Jobs
BlackRock
VP - Senior Data Engineer (Python/Snowflake/Apache Airflow)
BlackRock
Mumbai•Hybrid•10-13 Yrs•Jul 7
Job description
Hybrid
Overview
• Actively participate in Agile ceremonies, including requirements refinement, sprint planning, effort estimation, sprint reviews, and retrospectives, to support successful delivery of platform capabilities.
Responsibilities
- Actively participate in Agile ceremonies, including requirements refinement, sprint planning, effort estimation, sprint reviews, and retrospectives, to support successful delivery of platform capabilities.
- Participate in technical design discussions, analyze requirements, and translate complex business needs into scalable data platform solutions.
- Design, develop, and optimize core enterprise data platform capabilities, including data acquisition, ingestion, transformation, workflow orchestration, and reusable processing frameworks, leveraging Python, SQL, cloud-native technologies, and cloud data warehouse platforms to support large-scale, high-performance data processing.
- Build and optimize large-scale data pipelines supporting data ingestion, transformation, validation, and distribution across enterprise cloud environments.
- Implement metadata-driven processing frameworks and reusable engineering patterns that improve automation, standardization, scalability, and reuse.
- Build and enhance platform capabilities related to metadata management, lineage, dependency tracking, data quality, governance, and workflow control.
- Incorporate Agentic AI capabilities into platform services to enhance automation, intelligent decision-making, developer productivity, and operational efficiency.
- Partner with cross-functional teams to support collaborative development, system integration, and end-to-end testing across platform components and dependent systems.
- Develop and maintain system, integration, regression, and test automation frameworks to ensure platform quality, reliability, and production readiness.
- Support deployment and release activities, ensuring successful rollout of platform enhancements and seamless integration with dependent systems and downstream consumers.
- Contribute to engineering standards, architectural patterns, operational best practices, and framework conventions that improve platform maintainability, consistency, scalability, and supportability.
- Provide L2/L3 production support for data platforms and products, perform root cause analysis, and implement corrective actions to ensure platform stability and operational excellence.
- Drive continuous improvements in platform performance, scalability, resilience, observability, and operational efficiency across data workflows and framework components.
- Lead Proof of Concepts (POCs) and technology evaluations to assess emerging technologies, validate architectural approaches, and identify opportunities for platform modernization and innovation.
- Support product documentation by providing technical expertise, implementation details, and content validation to ensure accuracy, completeness, and alignment with platform capabilities.
- Conduct live product demonstrations and participate in user workshops to showcase platform capabilities, gather feedback, validate solutions, and support platform adoption
Qualifications
- Bachelor’s degree in Computer Science, Information Systems, or a related technical field.
- 10–14 years of experience in Data Engineering, Data Platform Engineering, or a related software engineering role.
- Hands-on experience with Apache Airflow for orchestrating complex, dependency-driven data pipelines and workflows.
- Strong proficiency in Python, with hands-on experience developing scalable, production-grade data pipelines and frameworks.
- Hands-on experience designing and building ETL/ELT frameworks, reusable data pipeline components, and workflow-driven data systems.
- Hands-on experience with Snowflake or equivalent cloud-native analytical data platforms.
- Strong hands-on experience writing and optimizing complex SQL, including stored procedures, UDFs, and performance-critical queries.
- Proven experience in query optimization, performance tuning, and workload management for large-scale data environments.
- Extensive hands-on experience with dbt, including data modeling, transformation, testing, documentation, and deployment within modern data platforms.
- Proven track record of improving the performance, reliability, scalability, and maintainability of enterprise data platforms and pipelines.
- Hands-on experience with streaming technologies (e.g., Snowpipe, Kafka) and messaging platforms for building scalable, real-time data processing solutions.
- Hands-on experience building solutions in cloud-native and distributed environments, leveraging object storage, cloud data services, and major cloud platforms such as Azure, AWS, or GCP.
- Exposure to Apache Iceberg, leveraging schema evolution, partition evolution, and time-travel capabilities to build scalable and reliable data lake solutions.
- Hands-on experience designing and developing cloud-native data platforms, including Data Lake and Data Warehouse architectures, using modern data engineering best practices.
- Exposure to AI-native development tools such as Windsurf, Cursor, and Antigravity, leveraging AI-assisted coding, testing, and automation to improve engineering productivity and software delivery.
- Understanding of Agentic AI architectures, including Skills, Model Context Protocol (MCP), Vector Databases, Retrieval-Augmented Generation (RAG), and AI Agent orchestration concepts.
- Strong understanding of software engineering principles, scalable system design, and modern development practices, including Agile methodologies, Git-based source control, CI/CD pipelines, code reviews, and production-grade development standards.
- Excellent verbal and written communication skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.
- Strong analytical, critical-thinking, and problem-solving skills, with a proactive, solution-oriented mindset and the ability to apply innovative, out-of-the-box thinking to solve complex challenges.
- Adaptable and collaborative team player with a continuous learning mindset and passion for emerging technologies.
- Proven ability to mentor and guide engineers, fostering technical excellence, knowledge sharing, and engineering best practices across teams.