Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Back to Browse Jobs
Warner Bros. Discovery
Staff Data Engineer - Product Engagement(Data Platform Team) Hyderabad
Warner Bros. Discovery
Hyderabad•On-site•5-8 Yrs•Jul 3
Job description
Overview
Welcome to Warner Bros. Discovery… the stuff dreams are made of.
Responsibilities
- Own the architecture of the multi-tenant product engagement data platform end to end, from streaming event ingestion and normalization through silver sessionization and the gold attribution, journey, and component-performance layers that downstream analytics depend on.
- Set technical direction and standards for high-performance Spark applications, partitioning and data-modeling conventions, idempotent backfills, schema evolution, and release management across a shared library and multiple tenant pipelines.
- Lead the design and delivery of streaming and batch processing in Scala, Spark, and SQL on Databricks and AWS, optimizing for performance, cost, and reliability at scale.
- Drive engineering excellence in data quality and observability, including monitoring, alerting, validation, and lineage frameworks that protect the integrity of business-critical engagement metrics.
- Resolve the hardest technical problems, including distributed-systems failure modes, session split and late-arrival handling, skew and shuffle optimization, cross-catalog dependencies, and correctness of attribution logic.
- Align engineering, analytics, data science, and platform stakeholders on shared contracts such as event schemas, gold-table grains, and semantic-layer mappings, and shepherd new metrics from requirement to reporting surface.
- Mentor and grow senior and mid-level engineers through design reviews, code review, pairing, and documentation, and build a culture of rigor, ownership, and continuous improvement.
- Evaluate emerging tools and patterns, manage technical debt deliberately, and contribute to the multi-year roadmap for the engagement data platform.
Qualifications
- Programming: Expert-level proficiency in Scala (strongly preferred) and/or Python, with a strong software-engineering foundation including testing, modularity, versioned shared libraries, and CI/CD.
- Distributed computing: Deep expertise with Apache Spark (structured streaming and batch), including performance tuning, skew and shuffle optimization, and operating at petabyte scale.
- Lakehouse and storage: Advanced experience with Delta Lake and lakehouse or medallion architectures, including schema evolution, partitioning strategy, and idempotent and incremental processing.
- Data platforms: Hands-on experience with Databricks (jobs, workflows, clusters) and AWS (S3 and related services).
- SQL: Advanced SQL for large-scale transformation, validation, and analytical workloads.
- DevOps: Proficiency with CI/CD (GitHub Actions), artifact and version management, and modern build tooling such as sbt.
- Architecture and leadership: Demonstrated ownership of architectural decisions for scaling and reliability, plus a record of mentoring engineers and driving engineering best practices.
- Familiarity with multi-tenant data platforms and semantic layers such as Looker and LookML, and the contracts between engineering and reporting.
- Familiarity with data quality frameworks such as Deequ, and with data governance, lineage, and catalog tooling.
- Exposure to supporting ML and data science consumers such as feature data and experimentation datasets.
- Data volume: 30+ billion events processed daily.
- Infrastructure: Databricks on AWS (S3, Delta Lake).
- Primary languages: Scala, Python, SQL.
- Processing: Apache Spark (structured streaming and batch) on a medallion bronze, silver, and gold architecture with real-time and batch layers.
- Domain: Product engagement analytics, including sessionization, pathing, user-journey attribution, component (page, rail, tile) performance, search, and weekly business-review metrics.
- Platform model: A shared Scala and Spark library of reusable processors plus tenant-specific workflows, configurations, and DDLs serving multiple streaming products.
- Engineering practices: Versioned artifact releases, CI/CD via GitHub Actions, unit and integration testing, partition-convention and schema-evolution discipline, and pipeline monitoring and data-quality checks.
Nice to have
- Background in streaming media, entertainment, or high-traffic consumer applications.
Benefits
- The opportunity to set technical direction for large-scale data systems serving millions of users worldwide.
- A collaborative environment with talented engineers, analysts, and data scientists.
- A modern data stack and high technical autonomy.
- Direct impact on product strategy and decision-making through trusted engagement data.
- A great place to work and an equal opportunity employer.