Overview

You will design and implement data engineering and analytics solutions using Databricks and AWS, operating in the C Shift (1 PM - 11 PM).

Responsibilities

• Build and operate large-scale data warehouses and data lakes.
• Design, code, and tune big data processes using Apache Spark and PySpark.
• Develop data pipelines and applications to stream and process datasets at low latencies.
• Optimize ETL processes and manage data lineage, quality, and discoverability.
• Collaborate with client stakeholders to deliver technical solutions.

Qualifications

• Minimum 5 years of experience in ETL, Big Data/Hadoop, and data warehouse architecture.
• Minimum 2 years of experience with Databricks engineering on AWS using PySpark, Databricks SQL, and Delta Lake.
• Minimum 3 years of experience in real-time streaming using Kafka or Kinesis.
• Minimum 4 years of experience in Python, Java, or Scala.
• 2 years of experience developing CI/CD pipelines using GIT, Jenkins, Docker, Kubernetes, Shell Scripting, and Terraform.
• Hands-on experience using Airflow for data pipelines in at least one project.
• Strong knowledge of Data Warehousing principles.
• Certifications in Databricks, Python, or AWS.
• Healthcare domain knowledge.