Overview

Department: Platform Engineering / Data Platform

Qualifications

• Design, build, and maintain batch and real-time data pipelines that ingest data from transactional systems, third-party APIs, event streams, and operational databases.
• Build robust, idempotent, and observable ETL/ELT workflows using Apache Airflow , dbt , or equivalent orchestration and transformation tooling.
• Ensure pipelines are fault-tolerant, recoverable, and instrumented with data quality checks at every stage.
• Design and maintain dimensional and analytical data models (star schema, OBT, wide tables) optimized for BI reporting and ML feature consumption.
• Collaborate with analysts to translate business questions into clean, reusable, and well-tested data models.
• Build and maintain real-time data pipelines for use cases such as live inventory updates, order event processing, and customer behaviour streams.
• Implement stream processing logic using Apache Flink, Spark Streaming, or ksqlDB for low-latency data delivery.
• Design Kafka topic schemas, partition strategies, and consumer group management for high-throughput e-commerce event flows.
• Build and manage a well-structured data lake on GCS
• Work with cloud-native data services:
• Collaborate with the Platform/DevOps team on data infrastructure provisioning using Terraform and containerized data workloads on Kubernetes.
• Implement data quality frameworks using Great Expectations, dbt tests, or Monte Carlo to catch anomalies, nulls, duplicates, and schema drift before they reach downstream consumers.
• Build and maintain data observability dashboards — pipeline SLAs, freshness SLOs, row count anomalies, and schema change alerts.
• Build and maintain feature pipelines for ML models powering personalisation, recommendations, fraud detection, and demand forecasting.
• 3–8 years of hands-on data engineering experience, ideally in a high-transaction e-commerce, fintech, or consumer tech environment.
• Strong proficiency in SQL — complex analytical queries, window functions, CTEs, query optimisation, and warehouse-specific dialects (BigQuery / Redshift).
• Hands-on experience with Apache Airflow for pipeline orchestration and dbt for transformation layer management.
• Working knowledge of Apache Kafka or equivalent event streaming platforms for real-time data ingestion.
• Proficiency in Python for pipeline development, data transformation, and automation.
• Hands-on experience with cloud platforms — GCP (BigQuery, Cloud Composer, Dataproc) and/or AWS (Glue, Athena, Kinesis, S3).
• Understanding of data modelling principles : normalisation, dimensional modelling, and analytical patterns.
• Good to Have
• Familiarity with Apache Spark (PySpark) for large-scale distributed data processing.
• Exposure to feature store design and ML pipeline integration.
• Knowledge of data mesh or data product principles and federated ownership models.
• Familiarity with infrastructure-as-code (Terraform) and deploying data workloads on Kubernetes.
• Relevant certifications: Google Professional Data Engineer, AWS Data Analytics Specialty, dbt Analytics Engineering.
• The Data Problems
• You'll Solve at Styli
• Personalisation at scale — building event pipelines that capture every click, view, and purchase to power real-time recommendations for millions of customers.