Job DescriptionJob DescriptionDescriptionWe are seeking a Lead - Data Engineer to lead the hands-on design, development, and operation of key components of Astrana’s enterprise data platform, including the Constellation Lakehouse, Connect API (GraphQL) layer, and Lakebase-backed applications.\nThis role will build scalable data pipelines, integrate healthcare data sources, and develop unified data models that support analytics, reporting, AI and machine learning, and operational applications. The position combines advanced data engineering with responsibility for platform reliability, performance, security, and automation within an Azure Databricks environment.\nAs a hands-on technical lead, you will translate complex requirements into reliable production solutions, establish engineering standards, and mentor other engineers. You will collaborate with analytics, software engineering, infrastructure, security, business stakeholders, and external vendors to advance the platform and support its growing adoption.Our Values\n\nPut Patients First\nEmpower Entrepreneurial Provider and Care Teams\nOperate with Integrity & Excellence\nBe Innovative\nWork As One Team\nWhat You'll Do\nLead the design, development, and delivery of core platform components, including the Constellation Lakehouse in Databricks, Connect API (GraphQL) layer, and Lakebase-backed applications.\nBuild and maintain scalable batch and incremental data pipelines using Databricks, Apache Spark, PySpark, SQL, and Delta Lake.\nIntegrate data from transactional databases, APIs, files, and external partners into governed datasets that support analytics and operational applications.\nDesign and maintain unified healthcare data models spanning membership, providers, claims, encounters, clinical data, and other business domains.\nDevelop reliable ingestion and transformation processes, including change data capture, schema evolution, deduplication, historical tracking, and reconciliation.\nBuild and optimize data access layers and APIs in partnership with software engineering teams, supporting consistent business definitions and reliable downstream consumption.\nDevelop and support Lakebase-backed applications and PostgreSQL data structures, including integration with lakehouse datasets and application services.\nMaintain and optimize Databricks workspaces, compute resources, job orchestration, and environment configurations in collaboration with infrastructure and security teams.\nImplement data governance and access controls through Unity Catalog and established organizational standards, including permissions, lineage, and auditability.\nEstablish automated data quality checks, source-to-target reconciliation, freshness monitoring, and alerting to ensure datasets are accurate, complete, and usable.\nApply DevOps practices, including Git, automated testing, CI/CD, infrastructure as code, and repeatable deployment across development, testing, and production environments.\nTune pipelines, Spark workloads, SQL queries, and compute configurations to improve performance, scalability, and cost efficiency.\nOwn the investigation and resolution of platform incidents and pipeline failures; perform root-cause analysis and implement preventive improvements.\nPartner with infrastructure teams on backup, recovery, and business continuity procedures, and maintain operational documentation and runbooks.\nConduct code and design reviews, establish engineering standards, and mentor engineers across internal and vendor teams.\nSkills, Knowledge & Expertise\nBachelor’s degree in Computer Science, Engineering, Data Science, or a related field.\n5+ years of data engineering experience building and supporting enterprise data platforms and pipelines.\n3+ years of hands-on experience with Databricks in production environments.\nStrong proficiency in Python, PySpark, SQL, and Apache Spark.\nExperience with Delta Lake and modern lakehouse architectures.\nHands-on experience with Azure data services and cloud-based data platforms.\nStrong knowledge of data modeling, relational databases, and ETL/ELT best practices.\nExperience integrating enterprise databases and APIs into scalable data solutions.\nExperience with Git, CI/CD pipelines, and automated deployment practices.\nProven ability to monitor, troubleshoot, and optimize data pipelines and platform performance.\nDemonstrated leadership in solution design, code reviews, mentoring, and cross-functional collaboration.\nStrong problem-solving and communication skills.\n\nExperience administering and optimizing Databricks environments, including Unity Catalog.\nExperience with PostgreSQL, GraphQL APIs, and operational applications leveraging lakehouse data.\nHealthcare data experience, including claims, eligibility, provider, or clinical datasets.\nFamiliarity with healthcare interoperability standards such as HL7, FHIR, and X12.\nExperience with Terraform or other infrastructure-as-code tools.\nExperience with streaming data pipelines and event-driven architectures.\nExperience supporting AI, machine learning, or NLP data workloads.\nFamiliarity with regulated data environments and governance best practices.\nExperience with cloud data platform migrations and modernization initiatives.\nAzure, Databricks, or AWS certifications preferred.\nJob Benefits\nThis position is remotely based in the U.S.\nThe total compensation target pay range for this role is: $170,000 - $240,000. Actual compensation will be determined based on geographic location (current or future), experience, and other job-related factors.\nAstrana Health is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based on race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. All employment is decided based on qualifications, merit, and business need. If you require assistance in applying for open positions due to a disability, please email us at humanresourcesdept@astranahealth.com to request an accommodation. Additional Information:The job description does not constitute an employment agreement between the employer and employee and is subject to change by the employer as the needs of the employer and requirements of the job change.