Short Job Description

We need a strong Azure Data Engineer with expertise in Databricks, including Unity Catalog experience), strong Pyspark and understanding of CI/CD and Infrastructure as Code (IaC) using Terraform.

Requirements

7+ years of experience as a Data Engineer

Hands-on experience with Azure Databricks, Spark, and Python

Experience with Delta Live Tables (DLT) and Databricks SQL

Strong SQL and database background

Experience with Azure Functions, messaging services, or orchestration tools

Familiarity with data governance, lineage, or cataloging tools (e.g., Purview, Unity Catalog)

Experience monitoring and optimizing Databricks clusters or workflows

Experience working with Azure cloud data services and understanding how they integrate with Databricks and enterprise data platforms

Experience with Terraform for cloud infrastructure provisioning

Experience with GitHub and GitHub Actions for version control and CI/CD automation

Strong understanding of distributed computing concepts (partitions, joins, shuffles, cluster behavior)

Familiarity with SDLC and modern engineering practices

Ability to balance multiple priorities, work independently, and stay organized