Lead Data Engineer

About the Role Data Operations team plays a vital role in setting PitchBook apart from competitors, providing top-quality data to customers while being accountable, collaborative, and receptive to feedback. Team members are highly engaged, detail-oriented, and eager to learn and grow as the industry evolves. The team offers specialized training and a mentorship program tailored to its needs. Primary Job Responsibilities Lead the design and development of scalable data pipelines and workflows that support Data Operations Define and implement best practices for data modeling, pipeline orchestration, testing, and monitoring Drive improvements in data architecture, including warehousing, transformation layers, and data access patterns Identify opportunities to modernize tooling, improve performance, and reduce operational overhead Act as a technical mentor and coach a team of data engineers with diverse experience levels Uplevel the team's capabilities in SQL, Python, data modeling, and system design Establish coding standards, review processes, and develop best practices Help bridge the gap between operationally minded team members and software engineering rigor Serve as the primary technical point of contact for Data Operations leadership and key stakeholders Partner with Product, Engineering, and Data Platform teams to align architecture and shared capabilities Translate business and operational needs into clear technical requirements and solutions Influence prioritization and roadmap decisions within Data Operations initiatives Build and maintain data solutions that integrate human-driven workflows with automated systems Develop reporting and monitoring solutions to ensure data quality, pipeline reliability, and operational visibility Deeply understand Data Operations processes and identify opportunities for automation and optimization Balance speed and quality in a fast-moving, ambiguity-rich environment Support the vision and values of the company through role modeling and encouraging desired behaviors Participate in various company initiatives and projects as requested Skills and Qualifications Bachelor's degree in computer science, engineering, statistics, information systems, or related field 5+ years of experience in data engineering or related roles Advanced SQL skills and strong experience working with large, complex datasets Strong Python experience for data processing, automation, and pipeline development Proven experience building and maintaining ETL/ELT pipelines (batch and event-driven) and data workflows Experience with data warehousing concepts and modern data stacks (e.g., Snowflake or similar) Ability to design scalable, maintainable data models and systems Strong communication skills, with the ability to work effectively with both technical and non-technical stakeholders Demonstrated experience mentoring or guiding other engineers (formally or informally) Experience working with large-scale proprietary data environments Familiarity with orchestration tools (e.g., Airflow) and modern data tooling Experience with BI tools such as Tableau or Power BI Exposure to cloud platforms (AWS, Azure, or GCP) Experience improving data quality, observability, and governance practices Experience applying AI/ML or automation techniques to improve operational efficiency Must be authorized to work in the United States without the need for visa sponsorship now or in the future Benefits & Compensation Physical Health Comprehensive health benefits Additional medical wellness incentives STD, LTD, AD&D, and life insurance Emotional Health Paid sabbatical program after four years Paid family and paternity leave Annual educational stipend Ability to apply for tuition reimbursement CFA exam stipend Robust training programs on industry and soft skills Employee assistance program Generous allotment of vacation days, sick days, and volunteer days Social Health Matching gifts program Employee resource groups Subsidized emergency childcare Dependent Care FSA Company-wide events Employee referral bonus program Quarterly team building events Financial Health 401k match Shared ownership employee stock program Monthly transportation stipend Compensation Annual base salary: $175,000 - $220,000 Target annual bonus percentage: 15% *Please be aware the above PitchBook benefit and perk offerings are subject to corresponding plan and policy documents and may change during the course of your employment. Working Conditions The role is expected to be in the office 5 days a week. Employees will use PC and phone on an ongoing basis throughout the day. Limited corporate travel may be required to remote offices or other business meetings and events. We are excited to get to know you and your background. Concerned that you might not meet every requirement? We encourage you to still apply as you might be the right candidate for the role or other roles at PitchBook. #J-18808-Ljbffr