Overview

Grade Level (for internal use): 10S&P Dow Jones Indices The Role: Lead Software Engineer (Data Engineer)

Responsibilities

• Assist in onboarding new data assets to the enterprise platform following established procedures.
• Support existing data pipelines by monitoring, troubleshooting, and performing routine maintenance.
• Help ensure data quality and consistency during onboarding and ongoing operations.
• Follow documented processes and checklists to support data ingestion and transformation tasks.
• Support data storage and processing using Amazon S3 as the core cloud storage layer, ensuring data is organized, secure, and accessible.
• Support metadata management and data cataloging using AWS Glue Data Catalog to facilitate discoverability and governance.
• Assist in implementing data validation, quality checks, and lineage documentation.
• Write and maintain Python scripts and Big data processing engines transformations for data ingestion, processing, and automation tasks.
• Support the team in maintaining data pipelines, fixing issues, and implementing small enhancements.
• Assist with data validation, metadata tagging, and basic lineage documentation.
• Help monitor pipeline health and escalate issues as needed.
• Support operational activities such as deployment, testing, and routine monitoring.
• Contribute to automation of deployment and operational workflows using AWS CloudFormation, or similar tools.
• Work with senior engineers to understand platform standards and best practices.
• Document onboarding steps, pipeline configurations, and operational procedures.
• Communicate effectively with team members and stakeholders about progress and issues.
• Participate in production incident response and troubleshooting.
• Support routine deployment, testing, and monitoring activities.
• Contribute to team knowledge sharing and documentation efforts.

Qualifications

• 5+ years of experience in data engineering, data support, or related roles.
• Basic knowledge of SQL, Python, and Big data processing engines.
• Hands-on experience with programming in Python or similar languages for data processing and automation.
• Familiarity with cloud data platforms such as Databricks or similar.
• Understanding of data onboarding, data quality, metadata, and lineage concepts.
• Hands-on experience with AWS services such as Amazon S3, AWS Glue Data Catalog, AWS IAM, and AWS CloudWatch.
• Good problem-solving skills and attention to detail.
• Ability to work collaboratively in a team environment.

Nice to have

• Exposure to data pipelines, data onboarding, or data migration projects.
• Knowledge of Delta Lake, Iceberg, or similar storage formats.
• Basic understanding of data governance and security principles.
• Any relevant certifications or training in data engineering or AWS.

Benefits

• Health & Wellness: Health care coverage designed for the mind and body.
• Flexible Downtime: Generous time off helps keep you energized for your time on.
• Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.
• Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.
• Family Friendly Perks: It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families.
• Beyond the Basics: From retail discounts to referral incentive awards—small perks can make a big difference.
• ---------------------------------------------------------
• ---------------------------------------------------------