Lead Data Engineer / Architect

Location - Chicago, IL / Remote (should be local to Chicago and be able to go into office if needed) Our client is looking to add a Lead Data Engineer / Architect to their team that will have full authority over data architecture decisions and the engineering roadmap. You will stand up a Databricks (or Snowflake, or Hybrid)-based Data Lakehouse from greenfield, define the architectural patterns the team will build on for years, and grow a small high-caliber team. Key Responsibilities Data Architecture & Lakehouse Design Design and own the end-to-end data architecture. Example: Databricks(or Snowflake, or Hybrid)-— Unity Catalog, Delta Lake, medallion layer structure (Bronze / Silver / Gold) Define data modeling standards: Example: star schema, dimensional modeling, and semantic layer design using dbt Establish data governance frameworks including lineage, cataloging, and access controls Own the technology selection process in partnership with the COO, consultants, and other stakeholders. Ensure the architecture can support investment analytics, portfolio reporting, and operational workflows at scale Build and own the ingestion layer: Example: Fivetran/Airbyte connectors for custodians, Bloomberg, and internal systems Develop and maintain dbt transformation models, tests, and documentation Configure and manage pipeline orchestration using Airflow or Prefect Write clean, production-grade Python for data pipelines, custom connectors, and analytical tooling Build internal dashboards and data applications using for example Plotly Dash or Streamlit Integrate Claude Enterprise (Anthropic) into data workflows for AI-assisted analysis and automation Recruit, interview, onboard, and mentor direct reports — you define who joins this team Define scope, manage delivery, and sign off on all work from external consultants (pipeline specialists, viz specialists) Ensure all consultant engagements include knowledge transfer and documentation before close Build a team culture grounded in engineering excellence, documentation discipline, and continuous improvement Stakeholder Communication & COO Reporting Report directly to the COO with regular updates on build progress, risks, and resource decisions Translate complex technical decisions into clear business terms for investment and operations leadership Collaborate with portfolio managers, operations, and compliance teams to understand and prioritize data needs Own the data roadmap and communicate trade-offs transparently Required Qualifications Experience 7+ years in data engineering, data architecture, or a combined role Demonstrable experience designing and building a modern Data Lakehouse or cloud data warehouse from the ground up — not just maintaining one 3+ years working with cloud data platforms (Databricks or snowflake strongly preferred) Strong hands-on dbt experience: model design, testing, documentation, and semantic layer development Experience managing or mentoring engineers — you have grown people, not just pipelines Track record of working directly with senior business stakeholders and translating requirements into technical solutions Technical Skills Orchestration: Apache Airflow or Prefect (managed or self-hosted) Python: production-grade scripting, pipeline development, Plotly/Dash or Streamlit SQL: expert-level; complex transformations, window functions, query optimization Git, CI/CD, and engineering best practices Domain Knowledge Financial services or asset management experience strongly preferred Familiarity with custodian data feeds, Bloomberg API, or investment data sources is a significant advantage Working knowledge of data governance, lineage, and quality frameworks Preferred Qualifications Direct experience with Databricks Unity Catalog rollout Hands-on experience with data catalog tooling (Atlan, Monte Carlo, or similar) Familiarity with Claude Enterprise or Anthropic AI tools in a production context Experience with SOC 2 data environments and security-conscious engineering practices Exposure to portfolio management systems, order management systems (OMS), or similar investment platforms Experience standing up a data team from zero — first hire, first pipeline, first architecture decision For all non-bonus, non-commission direct hire positions: The anticipated salary range for this position is ($180,000 - $200,000). Actual salary will be based on a variety of factors including relevant experience, knowledge, skills and other factors permitted by law. A range of medical, dental, vision, retirement, paid time off, and/or other benefits are available. #J-18808-Ljbffr