Key Responsibilities

Stakeholder Engagement & Solution Leadership

• Act as the primary technical liaison between business stakeholders and engineering teams.

• Lead requirement discovery sessions and translate business needs into scalable data architecture solutions.

• Present architecture approaches, trade-offs, and roadmaps to leadership and decision-makers.

• Provide technical leadership across design, implementation, and major delivery milestones.

• Collaborate with distributed and offshore teams to ensure alignment and quality execution.

Data & Platform Architecture

• Design and implement end-to-end Azure-based data platforms leveraging Azure Databricks and Azure Data Lake.

• Define and implement lakehouse architecture patterns supporting batch, streaming, and near real-time processing.

• Establish data modeling standards including layered architecture (bronze/silver/gold) and curated data models.

• Design integration patterns with upstream and downstream systems including APIs, event streaming platforms, and analytics tools.

• Support cloud modernization and legacy platform transformation initiatives.

• Ensure data governance, security, scalability, and cost optimization best practices are applied.

Technical Governance & Delivery Oversight

• Review and validate development deliverables to ensure adherence to architecture standards.

• Provide guidance on Spark performance optimization, data engineering best practices, and reusable framework design.

• Establish CI/CD processes and DevOps enablement using Azure DevOps and Git.

• Implement monitoring, logging, and performance optimization strategies across the data platform.

• Drive continuous improvement in architecture, performance, and cost efficiency.

Required Technical Skills

• Strong hands-on expertise in Azure Databricks (Spark, Delta Lake, Unity Catalog).

• Deep experience with:

  • Azure Data Lake Storage (ADLS Gen2)

  • Azure Data Factory or Synapse Pipelines

  • Event-driven or streaming architectures (Kafka, Event Hub, Structured Streaming)

  • Python, SQL, Spark optimization techniques

• Solid understanding of modern data warehousing and lakehouse architectures.

• Experience integrating BI tools such as Power BI or similar analytics platforms.

• Exposure to ML/AI integration within Databricks environments is a plus.

Experience & Qualifications

• 10+ years of experience in data engineering or data architecture roles.

• 5+ years designing and implementing solutions on Azure data platforms.

• 3+ years in a technical lead or solution architect capacity.

• Azure and/or Databricks certifications preferred.