Loading, Please wait..!!
Data Engineer
-
Posted on: Nov 18, 2026
-
, District of Columbia
-
Salary: $70 per hour
-
Remote
-
Full-time
Curious about compensation?
Explore the historical salary trends, average pay, and estimated compensation for Data Engineer roles in District of Columbia.
View Salary Guide →Job Title :
Data Engineer
Job Type :
Full-Time
Job Location :
District of Columbia United States
Remote :
Yes
Job Description :
Position: Data Engineer
Hybrid or Remote?: Open to fully remote but will prioritize local
Data Engineer
Role Summary:
The Client has an opening for a Data Engineer in the Office of Technology (OT). this role will be responsible for designing, developing, implementing, and maintaining scalable, secure, and reliable data pipelines and data products within the Client modern cloud-based data environment. The Data Engineer will work closely with Data Architects, Data Scientists, business stakeholders, application teams, and analytics teams to ingest, integrate, transform, govern, and deliver trusted enterprise data for analytics, reporting, operational, and artificial intelligence use cases.
The role will leverage technologies including Microsoft Azure, Azure SQL, Databricks, Delta Lake, Unity Catalog, PySpark, Python, SQL and Databricks AI/BI, aligning directly with the Client technology landscape.
Key Responsibilities:
- Design, develop, deploy, maintain, and optimize scalable data pipelines supporting batch and near-real-time data processing.
- Collaborate with Client Divisions and Offices to understand business and technical requirements and translate them into reliable data engineering solutions.
- Work closely with Data Architects to implement data solutions consistent with PCAOB enterprise data architecture, standards, patterns, and strategic roadmaps.
- Develop all aspects of ETL and ELT processes necessary to ingest, validate, cleanse, transform, enrich, store, and deliver data for operational, reporting, analytics, and AI use cases.
- Build and maintain data pipelines using Databricks, Python, PySpark, and SQL, as appropriate.
- Design and implement scalable data processing solutions using Databricks, Apache Spark, Delta Lake, and Azure cloud services.
- Develop and maintain curated enterprise data products and reusable datasets for consumption by reporting, analytics, data science, application, and AI workloads.
- Implement data models and physical database structures based on architectural and analytical requirements.
- Build pipelines to integrate data from enterprise applications, relational databases, APIs, flat files, cloud storage, SaaS platforms, and external data sources.
- Implement reusable patterns and frameworks for data ingestion, data transformation, change data capture, schema evolution, orchestration, and data delivery.
- Develop automated data validation, reconciliation, and data quality processes to ensure accuracy, completeness, consistency, and reliability.
- Implement monitoring, logging, alerting, error handling, and recoverability capabilities for production data pipelines.
- Troubleshoot and resolve performance, reliability, data quality, and integration issues across the data platform.
- Optimize storage, Spark workloads, SQL queries, pipeline execution, and compute utilization to improve platform performance and cost efficiency.
- Support the rationalization and consolidation of data sources to reduce redundant data movement, replication, and organizational data silos.
- Implement enterprise data governance capabilities addressing metadata, lineage, access control, security, data classification, and lifecycle management.
- Implement and administer appropriate data access controls and governance patterns using Databricks Unity Catalog and related enterprise security capabilities.
- Develop and maintain technical documentation covering data sources, mappings, transformations, schemas, lineage, dependencies, pipeline design, and operational support procedures.
- Partner with Data Scientists to develop data pipelines, features, and datasets required for machine learning, advanced analytics, Generative AI, and AI agent workloads.
- Partner with reporting and analytics teams to provide curated and optimized datasets for Power BI dashboards and other analytical applications.
- Participate in solution design and code reviews to ensure engineering implementations conform to PCAOB architecture, security, development, and operational standards.
- Support CI/CD, source control, automated testing, infrastructure automation, and DevOps practices for data engineering solutions.
- Research emerging data engineering technologies and recommend improvements to data platform capabilities and engineering practices.
Qualifications:
- Bachelor's degree in Computer Science or a related field, or equivalent practical experience.
- Minimum of 4 years of experience supporting technology organizations, IT leadership teams, or government agencies.
- Professional certification in Databricks Data Engineering, Microsoft Azure, cloud data engineering, or related technologies a plus.
- Deep hands-on expertise with SQL and Python, including practical experience developing production data pipelines.
- Strong hands-on experience with Databricks, Apache Spark, PySpark, Delta Lake, and Unity Catalog.
- Experience developing enterprise-scale data solutions on Microsoft Azure.
- Hands-on experience with data integration such as Lakeflow connect for data ingestion and Spark Declarative Pipelines.
- Strong knowledge of relational databases, data warehouses, data lakes, and Lakehouse architectures.
- Experience working with database technologies such as Azure SQL, SQL Server, Databricks SQL, and NoSQL technologies.
- Experience developing ETL/ELT pipelines for structured, semi-structured, and unstructured data.
- Strong understanding of data modeling concepts, including dimensional, relational, normalized, denormalized, and analytical models.
- Experience integrating data using REST APIs, JDBC/ODBC, files, messaging, and cloud-based integration patterns.
- Experience implementing data quality, data lineage, metadata management, data governance, and security controls.
- Understanding of cloud identity, role-based access control, secrets management, encryption, and secure data access patterns.
- Experience with Git, CI/CD, automated deployment, testing, and DevOps practices for data solutions.
- Ability to diagnose complex data processing, performance, and integration problems.
- Ability to manage multiple assignments simultaneously, including unstructured assignments requiring autonomy and independent judgment.
- Ability to clearly articulate technical designs, trade-offs, risks, and recommendations using sound logic and supporting evidence.
- Strong interpersonal and communication skills with the ability to collaborate with architects, developers, analysts, data scientists, cybersecurity teams, and business stakeholders.
- Ability to effectively represent OCOO-OT to internal and external constituencies.
- Experience implementing enterprise Lakehouse or medallion architectures using Databricks.
- Experience with Databricks serverless compute, workflows/jobs, Delta Live Tables or Lakeflow, Unity Catalog, Databricks SQL, and related platform capabilities.
- Experience supporting datasets and engineering pipelines for machine learning, Generative AI, RAG, and AI applications.
- Familiarity with Informatica Cloud Data Governance, Microsoft Purview or comparable metadata and data governance platforms.
- Experience with infrastructure-as-code technologies and automated cloud provisioning.
- Experience working in regulated, government, financial, audit, or other highly governed environments.
- Familiarity with agile methods and processes and experience supporting agile delivery teams.
Similar Jobs You May Be Interested In
Principal Engineer – Data & Microservices Architecture
Snowflake Data Engineer
Senior Data Engineer, Azure Databricks (remote)
Snowflake ETL/ELT Data Engineer (Hybrid - Must be local to Denver CO)
Databricks Data Engineer
[C]-Sr. Data Engineer Level 4
Data Engineer IV
Data Engineer (Azure/Databricks)
Data Governance Architect (Data Catalog Engineer Role)
Data Engineer Ab initio
Senior Analytics Engineer - Marketing Data
Azure Data Engineer
Position Details
Posted:
Nov 18, 2026
Reference Number:
8291-8310
Employment:
Full-time
Salary:
$70 per hour
City:
Job Origin:
CIEPAL_ORGANIC_FEED
Share this job:
Please wait..!!

