Create a free account
One signup unlocks everything for this job:
- See your personalized profile match score
- See the skills this role is missing for you
- Apply to this job and track your application
- Generate an AI cover letter for this role
Chennai · On-site · 4-7 Yrs · 5 days ago
Investment & Asset Management · 10000+ Employees · New York, US
Citi is more than a global financial services company. It’s an engine for progress. Join us, and you’ll have the chance to get involved in progress in all its shapes and forms, right across the world. From the micro to the macro, from Australia to Zambia, the work we do has real positive impact. Wherever you are and whatever you do, progress starts here. We’ve built a world famous, trusted brand over 200 years of continuously evolving financial services. And today, our influence continues to grow: with financial operations in 160 countries and 100 currencies. We have 200 million different clients, including some of the biggest names in the industries, sectors, businesses and governments that we serve every day. It all adds up to an exciting place to be for talented, ambitious people who want to build a truly remarkable career.
Hiring: Engineering
What employees say
+ good medical insurance good pay
− micromanagement old school management office politics
More reviews on the company profile.
Interview experience · Operations Associate
Interview was slow . The interviewee scheduled me interview and didn't join the call. After waiting for 30 minutes, i sent a mail to hr and then she called the technician team and then they had joined the call. Also seem
Average difficulty · Chennai · 965 reports
More interview insightsJob description
Overview
We are seeking an experienced and results-driven PySpark / Big Data Engineer to design, develop, and maintain large-scale batch and real-time data pipelines.
Responsibilities
- Architect, build, and maintain robust, scalable ETL/ELT pipelines using PySpark and Python to ingest data from diverse batch and streaming sources (Kafka, relational databases, REST APIs, Object Storage).
- Design and implement Lakehouse architectures using formats such as Delta Lake, Apache Iceberg, or Apache Hudi.
- Build reusable frameworks and data ingestion libraries to standardize enterprise data processing.
- Diagnose and tune Spark application performance, managing memory allocation, partitions, execution plans, broadcast joins, data skews, and garbage collection.
- Optimize query execution plans using Catalyst Optimizer and Tungsten execution engine.
- Monitor and troubleshoot production Spark jobs across cluster management environments (YARN, Kubernetes, or Databricks).
- Collaborate with Data Architects to design dimensional (Star/Snowflake), normalized, and Data Vault schemas optimized for analytical workloads.
- Ensure efficient data storage, partitioning strategies, and file format selection (Parquet, ORC, Avro).
- Schedule and orchestrate complex workflow DAGs using workflow orchestrators such as Apache Airflow, Prefect, or Dagster.
- Implement CI/CD pipelines (Git, Jenkins, GitLab CI, or GitHub Actions) for automated testing, packaging, and deployment of data pipelines.
- Write unit tests, integration tests, and data validation routines (e.g., Great Expectations, PyTest).
- Implement enterprise data governance, lineage tracking, metadata management, and data quality frameworks.
- Ensure strict compliance with data security, masking, tokenization, RBAC, and regulatory privacy standards (e.g., GDPR, CCPA).
- Core Technologies - Apache Spark, PySpark (Spark SQL, DataFrames, Datasets, Structured Streaming), Python 3.x
- Database & Querying - Advanced SQL, Query optimization, Indexing, Window functions
- Data Lake & Lakehouse - Delta Lake, Apache Iceberg, Apache Hudi, Parquet, Avro, ORC
- Workflow & Orchestration - Apache Airflow, Luigi, Prefect, or Databricks Workflows
- Cloud Platforms - AWS (EMR, Glue, S3, Redshift), Azure (Synapse, ADLS, HDInsight), or GCP (Dataproc, BigQuery, GCS)
- Streaming Technologies - Apache Kafka, Spark Structured Streaming, Event Hubs, or Kinesis
- Distributed Computing Fundamentals: In-depth knowledge of Spark internals (RDD, Catalyst Optimizer, DAG execution, Shuffling, Narrow vs. Wide dependencies).
- Production Experience: Minimum 4+ years of hands-on experience building and deploying production-grade PySpark applications at enterprise scale.
- Problem Solving: Proven track record in debugging memory bottlenecks, Out-Of-Memory (OOM) errors, and resolving data skew issues.
- Data Engineering Best Practices: Experience implementing modular code architectures, logging, monitoring (Prometheus, Grafana, CloudWatch), and alerting.
Nice to have
- Hands-on experience with Databricks platform (Delta Lake, Unity Catalog, Photon engine, MLflow).
- Familiarity with Scala/Java in addition to Python.
- Knowledge of DataOps and Infrastructure as Code (Terraform, CloudFormation).
- Experience with modern analytical data warehouses (Snowflake, Google BigQuery, Amazon Redshift).
- Relevant certifications: Databricks Certified Data Engineer Associate / Professional AWS Certified Data Engineer / Azure Data Engineer Associate (DP-203)CCA Spark and Hadoop Developer (or equivalent)
- Educational Requirements
- Bachelor’s or Master’s degree in Computer Science, Software Engineering, Information Technology, Data Science, or a related quantitative field (or equivalent practical experience).
- ----------------------------------------------------
- ----------------------------------------------------
- ----------------------------------------------------
- ----------------------------------------------------
- ----------------------------------------------------
- ----------------------------------------------------
Verify Python for this role
Free PRACTICE for Python adds a verified skill signal that can boost how roles like this rank for you — optional, not required to browse or apply.
Sign up — verify Python free →Want match scores and deeper insight? Create a free candidate profile
Job application FAQ
Last updated 27 October 2026
- How do I apply for Pyspark Engineer at CITI?
- Open this job on Feedback AI and sign up free (or sign in). You can apply from the job detail page; match scores and skill gaps appear when your profile is complete. CITI may attach structured assessments to interview rounds after you apply.
- Does Feedback AI show a match score for this Pyspark Engineer role?
- Yes after you add a resume or skills. Match score ranks this role against your profile — skills met, gaps, level, and location — so you prioritise applications where fit is strongest.
- Is Pyspark Engineer remote or on-site in Chennai, Tamil Nadu?
- This listing is marked on_site with location Chennai, Tamil Nadu. Confirm details on the job description before applying.
- What skills matter for this Pyspark Engineer role at CITI?
- This is a mid-level Pyspark Engineer role. Key skills include skills listed in the job description. Feedback AI highlights matched vs missing skills on your profile after signup.