Overview

We are looking for a hands-on Data Engineer II with strong Google Cloud Platform experience to build scalable data pipelines, transformations, and reusable data-platform capabilities.

Responsibilities

• Design, develop, and maintain production-grade data pipelines on Google Cloud Platform.
• Build scalable batch and incremental data-processing solutions.
• Develop complex transformations using BigQuery, SQL, and Dataform/dbt.
• Build and maintain data-processing workflows using Cloud Composer / Apache Airflow.
• Develop reusable transformation components, orchestration patterns, libraries, and templates.
• Implement metadata-driven and configuration-driven processing where appropriate.
• Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
• Design data models for analytical and downstream data-product requirements.
• Implement incremental and idempotent processing patterns.
• Define and maintain schemas and data contracts.
• Implement automated data-quality checks, validation, and reconciliation.
• Capture and integrate metadata and lineage into data-processing workflows.
• Optimize BigQuery queries and pipelines for performance and cloud cost.
• Implement automated testing and integrate data workloads with CI/CD pipelines.
• Build monitoring and operational controls for production pipelines.
• Troubleshoot production issues and perform root-cause analysis.
• Contribute to reusable data-platform capabilities and engineering standards.
• Collaborate with data engineers, platform engineers, DevOps, architects, and business stakeholders.

Qualifications

• 4+ years of hands-on data engineering experience building production data solutions.
• Strong practical experience with Google Cloud Platform (GCP).
• Strong hands-on experience with BigQuery.
• Advanced SQL skills, including complex joins, CTEs, window functions, and query tuning.
• Strong Python development skills for data processing, automation, and testing.
• Hands-on experience with Cloud Composer / Apache Airflow.
• Strong understanding of data modelling, including normalization, denormalization, and dimensional modelling.
• Understanding of data contracts and schema evolution.
• Understanding of metadata and data lineage.
• Ability to independently design solutions rather than only implement predefined specifications.
• Cloud and Storage: Google Cloud Platform, BigQuery, Google Cloud Storage.
• Transformation and Orchestration: Advanced SQL, Dataform/dbt, Cloud Composer/Apache Airflow.
• Data Processing: Dataflow / Apache Beam, Dataproc / Spark / PySpark.
• Programming and Engineering: Python, Git, automated testing, CI/CD, monitoring, and troubleshooting.
• Data Engineering Capabilities: ETL/ELT, batch processing, incremental and idempotent pipelines, data modelling, data contracts, data quality and reconciliation, metadata and lineage, query performance optimization, cloud-cost optimization, and reusable data-platform components.
• Practical understanding of Data Product / Data Mesh principles.
• Familiarity with Terraform / Infrastructure as Code.
• Strong ownership mindset and ability to take engineering work through production.
• Strong analytical, debugging, and problem-solving skills.
• Ability to communicate technical decisions clearly.
• Comfortable participating in design and code reviews.
• Ability to collaborate with engineering and business stakeholders.
• Ability to work independently while seeking guidance for complex architectural decisions.
• Comfortable working within distributed and multicultural teams.
• Strong Interpersonal Skills