Lead Data Engineer

Lead Data Engineer

Frisco, TX - Bellevue, WA
Data Platform & Near Real-Time Analytics Role Overview We are seeking a Lead Data Engineer to design and build a scalable, high-quality data platform that ingests data from multiple sources, ensures data quality and governance, and delivers near real-time insights (15-minute SLA) through Power BI / Microsoft Fabric dashboards.
This role will provide technical leadership and drive end-to-end data engineering architecture and delivery.
Key Responsibilities
Architecture & Platform Design Design and implement end-to-end data platform architecture (ingestion processing storage serving)
Define batch and near real-time data pipelines ensuring low-latency and high reliability Make technology decisions across Databricks, Delta Lake, Snowflake, and Azure ecosystem
Data Engineering & Pipelines Build scalable pipelines using: Databricks / Spark for large-scale processing Delta Lake for ACID-compliant data storage Snowflake for data warehousing and analytics Implement ETL/ELT pipelines with strong data modeling practices Streaming & Real-Time Processing
Design and implement real-time pipelines using Kafka / Azure Event Hubs Ensure data freshness within ~15-minute SLA
Enable incremental processing and efficient data updates Data Quality & Governance Establish data quality frameworks (validation, completeness, consistency checks) Implement monitoring, alerting, and data observability
Define and enforce data governance, lineage, and metadata standards Data Serving & Analytics Enable optimized data layers for Power BI / Microsoft Fabric dashboards Design semantic models and curated data layers for business consumption
Ensure consistent, accurate, and high-performance reporting Performance & Scalability Optimize pipelines and storage for large-scale datasets (TB/PB)
Ensure low-latency query performance and efficient compute usage Implement partitioning, indexing, caching, and optimization strategies Leadership & Collaboration Lead and mentor a team of data engineers Collaborate with Technical Product Managers, BI teams, and business stakeholders Drive best practices in coding, architecture, and delivery Manage technical risks, dependencies, and roadmap execution
Required Skills
Strong experience in data engineering and platform architecture (Lead level)
Expertise in: Databricks, Spark, Delta Lake Snowflake or similar cloud data warehouses Hands-on with streaming technologies (Kafka / Event Hubs)
Strong knowledge of data modeling, ETL/ELT, and pipeline design Experience with data quality frameworks and monitoring tools
Familiarity with Power BI / Microsoft Fabric Strong programming skills (Python, SQL) Experience with Azure ecosystem (ADF, ADLS, AKS - preferred)
Nice to Have Experience with real-time analytics platforms Exposure to data governance / MDM frameworks
Familiarity with CI/CD and DevOps practices for data platforms
Key Expectations
Own and deliver a robust, scalable data platform
Ensure high data quality and near real-time availability (15 min SLA) Drive standardization, reusability, and performance optimization
Enable business-ready, trusted data for analytics and decision-making
Business Impact Build and scale a modern data platform that delivers trusted, near real-time insights, enabling faster decisions and powering analytics across the organization.