Databricks Lead

Job Title: Databricks Lead
Location: Minneapolis, MN / Denver, CO
Hire Type: Full time

Must Have Technical/Functional Skills

• Databricks Lakehouse Platform
• Delta Lake
• Unity Catalog
• Databricks Workflows
• Databricks SQL
• Delta Live Tables
• MLflow
• Apache Spark
• PySpark
• Data Modeling
• ETL/ELT Architecture
• Streaming Data Processing
• Data Quality Frameworks
• Event-Driven Architectures

Programming

• Python
• SQL

Roles & Responsibilities

• Define and execute the enterprise data platform strategy leveraging Databricks and cloud-native technologies.
• Establish reference architectures, standards, and engineering best practices.
• Lead technical design reviews and architecture governance forums.
• Create reusable frameworks and accelerators to improve productivity and quality.
• Design and oversee implementation of scalable batch, streaming, and real-time data pipelines.
• Lead development of data products supporting analytics, reporting, data science, and AI initiatives.
• Ensure high standards for reliability, scalability, observability, security, and maintainability.
• Drive adoption of DataOps, DevOps, and CI/CD practices across engineering teams.
• Define enterprise Lakehouse architecture patterns.
• Establish data models, storage strategies, and data lifecycle management processes.
• Lead implementation of data governance, metadata management, lineage, and access controls.
• Ensure platform compliance with organizational security and regulatory requirements.
• Partner with data science and AI teams to operationalize machine learning workloads.
• Enable model training, deployment, monitoring, and lifecycle management.
• Support GenAI, predictive analytics, and intelligent automation initiatives.
• Build scalable data foundations for AI-driven business capabilities.
• Lead and mentor data engineering teams.
• Manage technical delivery across multiple workstreams and vendors.
• Collaborate with business stakeholders to translate requirements into scalable solutions.
• Provide regular updates to leadership on delivery progress, risks, and architectural decisions.

Generic Managerial Skills

• 12+ years of experience in Data Engineering and Analytics.
• 5+ years of hands-on Databricks experience.
• 3+ years leading enterprise-scale data engineering teams.
• Proven experience building large-scale cloud data platforms.
• Experience leading distributed teams in Agile delivery environments