Our Client is looking for a Data Engineer specializing in Azure and Databricks. You will architect and build secure, scalable data pipelines and a cloud-native data platform that powers real-time compliance analytics and regulatory reporting across global markets. Work collaboratively as a team player on mission-critical projects involving complex financial datasets, leveraging Azure Data Factory, Databricks, Delta Lake, and Python to transform raw data into trusted, governed insights. This is a high-impact role where a motivated self-starter owns, end-to-end data engineering initiatives, collaborates with governance teams, and influence the architecture of our client's next-generation compliance systems.

PRINCIPAL DUTIES & RESPONSIBILITIES:

• Own the design and implementation of ETL workflows using Azure Data Factory and Databricks for compliance-related data ingestion and transformation.
• Integrate and normalize data from multiple structured and unstructured sources, ensuring adherence to strict data quality governance framework.
• Develop secure, auditable, and optimized pipelines for performance and reliability across sensitive financial datasets.
• Automate ingestion and transformation processes enable real-time compliance analytics and reporting.
• Implement efficient storage formats and partitioning strategies for fast loading and retrieval.
• Utilize Databricks and Delta Lake for distributed processing and versioned data storage.
• Enforce data retention, archiving, and purging policies aligned with global regulatory requirements.
• Migrate legacy application logic into modern Azure Databricks/Data Lake and SQL environments.
• Maintain comprehensive data lineage, metadata, and audit trails using Azure Purview or custom frameworks.
• Collaborate with governance teams and compliance analysts to define requirements and manage access controls.
• Participate in Agile development cycles, sprint planning, and compliance reviews.
• Review code changes and support production deployments.

JOB SPECIFICATIONS:

• 10+ years of experience in data engineering and over 5 years of financial services, preferably compliance/regulatory.
• Advanced proficiency in SQL, Python, and Azure Databricks.
• Hands-on experience with Azure Data Factory, Databricks notebooks, and ADLS.
• Expertise in data wrangling, modeling, and analytics enablement for compliance use cases.
• Experience integrating Microsoft ADF and Databricks for complex ETL development.
• Familiarity with data governance, quality frameworks, and audit processes.
• Strong communication and collaboration skills.
• Experience with Git/GitLab and CI/CD tools such as Azure DevOps.
• Preferred: Certifications in Azure Data Engineering, Databricks, or Financial Compliance Technology.

Preferred: Bachelor's or Master's in Computer Science, Data Engineering, or related field