Data Engineer Apply
Dear Partner, Good Morning!
Greetings from SnapRecruit LLC. We do have urgent requirement for Data Engineer.
Job Summary & Responsibilities:
The Investment Banking Information Technology team has the mission of becoming a true business partner and we are searching for a Senior Data Engineer to help execute on this mission. The Senior Data Engineer will work with data scientists and business users to architect, develop, and manage data pipelines across proprietary and 3rd party data sources that will power analytics and machine learning models that drive growth and achieve productivity gains. Ideal candidates love breaking down problems, building solutions, delivering actionable and data-driven insights, and working in a fast-paced, dynamic environment.
- Design, build, and manage data pipelines to support ETL processes and data integration tasks
- Develop and deploy robust data architectures (data lake, data warehouse, etc.) to handle large-scale datasets, ensuring data quality and integrity
- Manage and optimize data processing workflows, ensuring timely and accurate data availability
- Optimize data retrieval processes through database tuning, query optimization, and ensuring scalable infrastructures
- Work with data scientists, analysts, and business teams to understand data requirements and deliver scalable data solutions
- Collaborate with IT, security, and compliance teams to ensure adherence to data management and protection standards
- Manage and optimize cloud-based data solutions (preferably Azure: including Synapse, Databricks, ADF, and Azure Data Lake)
- Ensure robustness, scalability, and sustainability of data infrastructure in the cloud environment
- Maintain comprehensive documentation of data models, pipelines, and ETL processes
- Share knowledge and mentor junior engineers and stakeholders on best practices in data engineering
Qualifications:
- Bachelor's degree in Computer Science, Engineering, Data Science, or a related field
- 4+ years of experience as a data engineer, with a proven track record in developing ETL processes, data pipeline architecture, and data modeling
- Strong proficiency in Python for data processing and manipulation
- Experience with SQL and Spark to handle data extraction, transformation, and loading of big data
- Demonstrable expertise in designing and implementing efficient data models to support ETL processes and data analytics
- Extensive experience managing and optimizing Azure cloud data technologies (Synapse, Databricks, ADF, or Azure Data Lake)
- Solid experience in data warehouse management, ensuring data quality and accessibility
- Hands-on experience with API utilization, development, and management
- Practical experience with event-driven architecture and real-time data processing
- Ability to effectively communicate technical concepts to both technical and non-technical stakeholders
- Self-starter, comfortable with ambiguity, ability to initiate and drive projects with minimal oversight and guidance
- Proven track record of leveraging data to deliver business value and present data-driven insights to business audiences
- Experience building cross functional relationships across a variety of functions to launch projects to drive business value
- Certifications in Azure Data Engineering, Spark, or other relevant technologies
- Familiarity or certification with Agile methodologies and experience in an Agile working environment
- Familiarity with PowerBI for developing interactive reports and data visualizations
- Experience in the investment banking or financial sector
- A record of continuous learning and adaptation to stay updated with the latest in data engineering, cloud technologies, and data compliance standards
