Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Back to Browse Jobs
NielsenIQ
Senior Data Engineer (Pyspark, AWS EMR, Airflow, SQL, Data Warehousing, ETL, Spark)
NielsenIQ
Pune•On-site•6-9 Yrs•Jul 9
Job description
Overview
• Become a essential part of the Data Refinement team, which provides data integration services for most downstream Media Measurement departments.
Responsibilities
- Become a essential part of the Data Refinement team, which provides data integration services for most downstream Media Measurement departments.
- Design, develop, test and deploy big data applications and pipelines.
- Ensure that you and the team follow best practices for data pipeline design with attention to the scalability, efficiency and maintainability.
- Collaborate in a distributed, cross-functional agile team of Data Engineers, Data Scientists and Cloud Engineers to reach shared goals.
- Take ownership of technical implementations and architecture together with your team and in alignment with communities of practices.
- Communicate with data producers & consumers to align on data exchange formats and interfaces.
- Provide suggestions to business in the Media Measurement context on how existing and new data sources can be utilized to enhance our existing product portfolio.
- Recommend different ways to constantly improve data reliability and quality.
- Improve ways of working with integrating AI based workflows in the engineering domain.
- Take ownership of the components, pipelines and the overall system so that you contribute to improve things over time.
Qualifications
- +6 years of relevant experience in Data Engineering using Spark.
- Solid hands-on experience with Python but also experience with Java or Scala.
- Knowledge in how to develop data-intensive scalable applications using Spark.
- Knowledge in writing SQL queries to wrangle data from relational databases.
- Knowledge in how to orchestrate data pipelines with Airflow or any other similar data scheduling and orchestration tool.
- Knowledge with Public Cloud providers (ideally AWS with services like S3, Lambda, EMR).
- You are following best practices in software engineering, like pair programming, testing, clean code and agile software development frameworks.
- Enjoy a flexible and rewarding work environment with peer-to-peer recognition platforms.
- Recharge and revitalize with help of wellness plans made for you and your family.
- Plan your future with financial wellness tools.
- Stay relevant and upskill yourself with career development opportunities.
- Flexible working environment
- Volunteer time off
- LinkedIn Learning
- Employee-Assistance-Program (EAP)