Overview

We are seeking a skilled Azure Databricks Data Engineer to join our data engineering team.

Responsibilities

• As an Azure Databricks Data Engineer, your responsibilities include:
• Technical Requirements Gathering and Development of Functional Specifications.
• Designing, developing, and maintaining scalable data pipelines and ETL processes using Azure Databricks, Data Factory, and other Azure services.
• Implementing and optimizing Spark jobs, data transformations, and data processing workflows in Databricks.
• Developing and integrating custom machine learning models using Azure Machine Learning, MLflow, and other relevant libraries.
• Leveraging Azure DevOps and CI/CD best practices to automate the deployment and management of data pipelines and infrastructure.
• Conducting troubleshooting on data models.
• Working with Agile multicultural teams in Asia, the EU, Canada, and the USA.
• Profile Requirements
• For this position of Azure Databricks Data Engineer, we are looking for someone with:
• (Required) At least 5 years of experience in developing and maintaining data pipelines using Azure Databricks, Azure Data Factory, and Spark.
• (Required) Hands-on experience with Unity Catalog.
• (Required) Fluent English communication and soft skills.
• (Required) Knowledge and experience in CI/CD using tools such as Terraform, ARM, and Bicep Script.
• (Required) Solid technical skills in Python and SQL.
• (Required) Familiarity with machine learning concepts, tools, and libraries (e.g., TensorFlow, PyTorch, Scikit-learn, MLflow).
• (Required) Solid problem-solving, communication, and analytical skills.
• (Optional) Willingness to learn and expand technical skills in other areas.