Job Description:

Senior Data Engineering Specialist to join our growing team. The ideal candidate will have extensive experience in designing, building, and optimizing scalable data pipelines, as well as managing and deploying data infrastructure in cloud environments. You will work closely with our data science, analytics, and DevOps teams to ensure seamless data flow and integration across our systems, and you will mentor junior team members to elevate the overall technical capability of the team.

Principal Responsibilities:

Lead the design, development, and maintenance of complex, scalable data pipelines
Optimize and troubleshoot data pipelines for performance, reliability, and scalability
Collaborate with engineers and analysts to understand data requirements and deliver high-quality data solutions
Implement and manage ETL processes using advanced tools like Apache Airflow, Spark, or similar
Ensure data quality and consistency through rigorous testing, validation, and governance practices
Deploy, monitor, and maintain data infrastructure in cloud environments (AWS, GCP, Azure)
Implement DevOps practices to streamline data workflows, including CI/CD pipelines, Docker, and Kubernetes
Mentor and guide junior data engineers, fostering a culture of continuous improvement and collaboration
Stay up-to-date with industry trends and emerging technologies to drive innovation within the data engineering team

Minimum Requirements:
7+ years of experience in data engineering or related roles
Advanced proficiency in SQL and experience with relational databases (e.g., PostgreSQL, MySQL)
Extensive experience with big data technologies (e.g., Hadoop, Spark)
Deep understanding of cloud platforms (AWS, GCP, Azure) and their data services
Strong understanding and practical experience with DevOps practices and tools (CI/CD, Docker, Kubernetes)
Proven ability to optimize and troubleshoot complex data systems and pipelines
Excellent problem-solving skills and attention to detail
Strong communication and leadership abilities, with experience mentoring junior engineers
Experience with source configuration management tools such as Git
CI & CD development experience with technologies like Bash, Jenkins, or GitLab
Experience with automated deployment technologies such as Cloud Formation, Ansible, Terraform
Cloud Technologies deployment experience
Open-source application deployments and maintenance
Custom source application deployments and maintenance
LINUX system administration experience (RedHat and/or CentOS).
Working knowledge of Linux and Windows operating systems, web services, and SQL databases
Experience working in an Agile environment
Must be willing to work on-site

An Excellent Candidate for this position will meet most of the following experience requirements:
Experience with NoSQL databases (e.g., MongoDB)
Proficiency in programming languages such as Python, Java, or Kotlin
Experience with data visualization tools (e.g., Tableau, Power BI)
Strong understanding of data governance and security practices
Experience working in Agile/Scrum environments
Ability to work collaboratively in a team environment
Strong problem-solving skills and attention to detail
Eagerness to learn and adapt to new technologies and methodologies

General personal traits we know will connect well with the team:
Superior communication skills
A positive, willing attitude
An ability to think on your feet and solve problems quickly
Adaptability to learn new methodologies and technologies
Comfortable working in an agile team environment
Technology and methodology agnostic, but accepts using tools needed for the requirement
Enjoys coaching and teaching