Data engineer with databricks and sql

Location Dallas, Texas

Project Description:

Work closely with source data application teams and product owners to design, implement and support analytics solutions that provide insights to make better decisions

Implement data migration and data engineering solutions using Azure products and services: (Azure Data Lake Storage, Azure Data Factory, Azure Functions, Event Hub, Azure Stream Analytics, Azure Databricks, etc.) and traditional data warehouse tools

Perform multiple aspects involved in the development lifecycle - design, cloud engineering (Infrastructure, network, security, and administration), ingestion, preparation, data modeling, testing, CICD pipelines, performance tuning, deployments, consumption, BI, alerting, prod support

Provide technical leadership and collaborate within a team environment as well as work independently

Be a part of a DevOps team that completely owns and supports their product

Implement batch and streaming data pipelines using cloud technologies

Understands and follow development of coding standards, best practices and privacy and security guidelines

Responsibilities:

Work closely with source data application teams and product owners to design, implement and support analytics solutions that provide insights to make better decisions

Implement data migration and data engineering solutions using Azure products and services: (Azure Data Lake Storage, Azure Data Factory, Azure Functions, Event Hub, Azure Stream Analytics, Azure Databricks, etc.) and traditional data warehouse tools

Perform multiple aspects involved in the development lifecycle - design, cloud engineering (Infrastructure, network, security, and administration), ingestion, preparation, data modeling, testing, CICD pipelines, performance tuning, deployments, consumption, BI, alerting, prod support

Provide technical leadership and collaborate within a team environment as well as work independently

Be a part of a DevOps team that completely owns and supports their product

Implement batch and streaming data pipelines using cloud technologies

Understands and follow development of coding standards, best practices and privacy and security guidelines

Mandatory Skills Description:

Top required skills:

Databricks, Python, SQL Cloud engineering

Bachelor's degree in Computer Science, Engineering, Information Systems, or related field (or equivalent experience).

3 to 5+ years of experience in software development (Agile, DevOps), designing, developing, and implementing large-scale applications or data engineering solutions.

3 to 5+ years of data analytics experience using SQL and working with data platforms like Teradata, Cassandra, MongoDB, Oracle, SQL Server, Snowflake, ADLS, and Azure Data Explorer.

3 to 5 years of cloud development and data lake experience, primarily in Microsoft Azure (EventHub, Data Factory, Databricks, Blob Storage, Power BI, Azure Functions, etc.).

Nice-to-Have Skills Description :Airline experience