Job Description Must Have Technical/Functional Skills We are seeking a skilled Data Engineer to design, build, and maintain high-quality, scalable data pipelines with a strong focus on Silver and Gold layer transformations, data quality automation, and CI/CD-driven deployments. The ideal candidate will have hands‑on experience with Python, PySpark, Git‑based CI/CD pipelines, ARM templates, and data quality libraries such as Great Expectations. Roles & Responsibilities Design and implement Silver and Gold layer data models following medallion architecture best practices. Perform data cleansing, standardization, enrichment, and aggregation to support analytics and reporting. Build optimized PySpark-based transformations for large-scale data processing. Ensure data consistency, performance, and scalability across datasets. Build and maintain CI/CD pipelines using Git-based workflows (Azure DevOps / GitHub). Use ARM templates (or IaC equivalents) for automated infrastructure provisioning. Enable automated deployment of data pipelines, notebooks, and configurations. Follow DevOps best practices for version control, branching, and release management. Develop reusable Python-based automation scripts for pipeline orchestration, validation, and monitoring. Create modular, maintainable, and testable Python code. Support automation of metadata, logging, alerting, and operational tasks. Implement data quality libraries such as Great Expectations. Define and automate data quality rules (completeness, accuracy, freshness, consistency). Integrate data quality checks into data pipelines and CI/CD workflows. Monitor, log, and troubleshoot data quality issues proactively. Work closely with data architects, analysts, QA, and business stakeholders. Translate business and analytical requirements into robust data engineering solutions. Participate in Agile ceremonies and support sprint-based delivery. Provide production support, performance tuning, and troubleshooting. Salary Range $120,000-$130,000 per year. Benefits Discretionary Annual Incentive. Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans. Family Support: Maternal & Parental Leaves. Insurance Options: Auto & Home Insurance, Identity Theft Protection. Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement. Time Off: Vacation, Time Off, Sick Leave & Holidays. Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing. Qualifications BACHELOR OF COMPUTER SCIENCE #J-18808-Ljbffr