Houston United States

Industry: Oil and Gas/ Energy

Job Title: Data Engineer

Duration: 6+ Months

Location: Houston, Texas/ San Ramon, California

Summary:

As an Azure Data Engineer you should have subject matter expertise integrating, transforming, and consolidating data from various structured and unstructured data systems into structures that are suitable for building integration and analytics solutions.

Responsibilities for this role include helping stakeholders understand the data through exploration, building and maintaining secure and compliant data processing pipelines by using different tools and techniques. This professional uses various Azure data services and languages to store and produce cleansed and enhanced data for consumption by analytical or application solutions.

An Azure data engineer also helps ensure that data pipelines and data stores are high-performing, efficient, organized, and reliable, given a specific set of business requirements and constraints. This professional deals with unanticipated issues swiftly and minimizes data loss. An Azure data engineer also designs, implements, monitors, and optimizes data platforms to meet the data pipeline needs.

responsible for the design of solutions using Azure Data Integration Services. This will include

• Solution design using MS Azure services and related tools.
• Design of enterprise data models and Data Warehouse solutions.
• Specification of ETL pipelines, data integration and data migration design.
• Design & implementation of Master data management solutions.
• Specification of Data Quality Management methodology and supporting technology tools.
• Working within a project management/agile delivery methodology in a leading role as part of a wider team.
• Deal with other stakeholders/ end users in the software development lifecycle - Scrum Masters, Product Owners, Data Architects, and testing teams.

Specific Technologies within the MS Azure Cloud stack at Client include:

• API Development and APIM registration of REST/SOAP API's.
• Azure Service Bus Messaging and Subscribing solutions.
• Azure Databricks, Azure Cosmos DB, Azure Data Factory, Azure Logic Apps, Azure Functions.
• Azure Storage, Azure SQL Data Warehouse/Synapse, Azure Data Lake.
• Azure Active Directory and Enterprise/Application registrations.
• Experience in the design of reporting & data visualization solutions such as Power BI or Tableau.
• CI/CD Process, Ansible Knowledge.
• Data validation, Basic Data Modeling, SQL Expertise.