Skip to main content
Loading
loadingbar
Loading, Please wait..!!

Data Engineer - GCP

  • Data Engineer - GCP job type Posted on: Nov 03, 2026
  • Data Engineer - GCP experience level Core AI Consulting Inc
  • Data Engineer - GCP employment type Phoenix, Arizona
  • Data Engineer - GCP employment type Onsite
  • Data Engineer - GCP salary Full-time

Point Apply Here APPLY LATER

Curious about compensation?

Explore the historical salary trends, average pay, and estimated compensation for Data Engineer - GCP roles in Arizona.

View Salary Guide →

Job Title :

Data Engineer - GCP

Job Type :

Full-Time

Job Location :

Phoenix Arizona United States

Remote :

No

Jobcon Logo Job Description :

We are seeking a highly skilled GCP Data Engineer with 5+ years of experience designing and building scalable, production-grade data platforms. This role requires deep expertise in Google Cloud Platform, Python, PySpark, SQL, and distributed data processing across the complete data lifecycle—including ingestion, transformation, orchestration, storage, quality, and serving.The ideal candidate is a hands-on engineer with strong experience building batch and real-time streaming pipelines, developing reusable Python data frameworks, optimizing large-scale workloads, and designing high-performance data models in BigQuery. The engineer will help establish reliable, secure, observable, and cost-efficient data solutions using modern GCP architecture and engineering practices.Key ResponsibilitiesDesign, develop, and maintain enterprise-scale batch and streaming data pipelines using Python, PySpark, Dataflow, Dataproc, Pub/Sub, BigQuery, and Cloud ComposerBuild modular, reusable, and testable Python frameworks for data ingestion, transformation, validation, and pipeline automationDevelop high-performance data-processing solutions using PySpark, Spark SQL, and Apache BeamIntegrate data from relational and NoSQL databases, REST APIs, flat files, cloud storage, third-party platforms, and event streamsBuild real-time streaming pipelines using Pub/Sub and Dataflow, including windowing, triggers, watermarking, late-arriving data, deduplication, and delivery semanticsDesign event-driven architectures supporting real-time analytics, operational reporting, and downstream data productsCreate and maintain workflow orchestration using Cloud Composer and Apache Airflow, including reusable operators, dependency management, retries, alerting, and recoveryDesign scalable BigQuery schemas using dimensional, normalized, denormalized, and lakehouse modeling patternsOptimize BigQuery workloads through effective partitioning, clustering, materialized views, query tuning, and cost-management strategiesTune Spark workloads by resolving data skew, excessive shuffling, inefficient joins, memory constraints, partitioning issues, and small-file problemsDevelop incremental loading, change data capture, and high-volume data-processing solutionsImplement automated data-quality checks, reconciliation, schema validation, lineage, logging, monitoring, and alertingDevelop Python-based data services and APIs using frameworks such as FastAPIApply security best practices using IAM, service accounts, encryption, secret management, and least-privilege accessBuild and maintain CI/CD pipelines for data applications using Cloud Build, GitHub Actions, or equivalent toolsProvision and manage GCP data infrastructure using TerraformCollaborate with architects, analysts, data scientists, and business stakeholders to translate requirements into scalable data solutionsCreate technical documentation, data mappings, operational runbooks, and architecture diagramsParticipate in code reviews and promote Python, SQL, GCP, and data-engineering best practicesRequired Qualifications5+ years of hands-on experience in data engineering, including building production-grade data pipelines and platformsStrong programming expertise in Python, including object-oriented programming, reusable modules, exception handling, logging, testing, packaging, and performance optimizationAdvanced experience with PySpark, Spark SQL, and Apache SparkStrong knowledge of GCP data services, particularly BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Composer, and Cloud StorageAdvanced SQL skills, including complex joins, window functions, CTEs, query optimization, and large-scale data transformationDeep expertise in BigQuery, including schema design, partitioning, clustering, performance tuning, workload management, and cost optimizationHands-on experience building batch and streaming pipelines with Dataflow and Apache Beam using PythonStrong experience designing and operating Spark workloads on DataprocExperience developing and managing Airflow DAGs using Cloud ComposerExperience designing Pub/Sub topics, subscriptions, schemas, retry policies, dead-letter queues, and deduplication strategiesStrong understanding of distributed processing concepts, including partitioning, shuffling, parallelism, fault tolerance, and delivery guaranteesSolid understanding of Spark internals, including execution plans, DAGs, Catalyst optimization, memory management, and Spark UI troubleshootingExperience with dimensional data modeling, including star and snowflake schemas and SCD Type 1 and Type 2Experience processing large-scale partitioned datasets and implementing incremental data loadsExperience with data formats such as Parquet, Avro, ORC, JSON, and DeltaStrong understanding of streaming concepts, including event time, processing time, windowing, triggers, watermarking, and exactly-once versus at-least-once processingExperience implementing data-quality, observability, reconciliation, and production-monitoring frameworksProficiency with Git, automated testing, Agile delivery, and CI/CD practicesPreferred QualificationsExperience building Python APIs and microservices using FastAPI or FlaskWorking knowledge of Terraform and infrastructure-as-code practicesExperience with CDC technologies and patternsFamiliarity with Dataform or dbt for SQL-based data transformationsKnowledge of GCP security, networking, IAM, Secret Manager, and data-governance practicesGoogle Cloud Professional Data Engineer certification is preferred

Jobcon Logo Position Details

Posted:

Nov 03, 2026

Reference Number:

10440_4479067478

Employment:

Full-time

City:

Phoenix

Job Origin:

APPCAST_CPC

Share this job:

  • linkedin

Jobcon Logo
A job sourcing event
In Dallas Fort Worth
Aug 19, 2017 9am-6pm
All job seekers welcome!

Data Engineer - GCP    Apply

Click on the below icons to share this job to Linkedin, Twitter!

We are seeking a highly skilled GCP Data Engineer with 5+ years of experience designing and building scalable, production-grade data platforms. This role requires deep expertise in Google Cloud Platform, Python, PySpark, SQL, and distributed data processing across the complete data lifecycle—including ingestion, transformation, orchestration, storage, quality, and serving.The ideal candidate is a hands-on engineer with strong experience building batch and real-time streaming pipelines, developing reusable Python data frameworks, optimizing large-scale workloads, and designing high-performance data models in BigQuery. The engineer will help establish reliable, secure, observable, and cost-efficient data solutions using modern GCP architecture and engineering practices.Key ResponsibilitiesDesign, develop, and maintain enterprise-scale batch and streaming data pipelines using Python, PySpark, Dataflow, Dataproc, Pub/Sub, BigQuery, and Cloud ComposerBuild modular, reusable, and testable Python frameworks for data ingestion, transformation, validation, and pipeline automationDevelop high-performance data-processing solutions using PySpark, Spark SQL, and Apache BeamIntegrate data from relational and NoSQL databases, REST APIs, flat files, cloud storage, third-party platforms, and event streamsBuild real-time streaming pipelines using Pub/Sub and Dataflow, including windowing, triggers, watermarking, late-arriving data, deduplication, and delivery semanticsDesign event-driven architectures supporting real-time analytics, operational reporting, and downstream data productsCreate and maintain workflow orchestration using Cloud Composer and Apache Airflow, including reusable operators, dependency management, retries, alerting, and recoveryDesign scalable BigQuery schemas using dimensional, normalized, denormalized, and lakehouse modeling patternsOptimize BigQuery workloads through effective partitioning, clustering, materialized views, query tuning, and cost-management strategiesTune Spark workloads by resolving data skew, excessive shuffling, inefficient joins, memory constraints, partitioning issues, and small-file problemsDevelop incremental loading, change data capture, and high-volume data-processing solutionsImplement automated data-quality checks, reconciliation, schema validation, lineage, logging, monitoring, and alertingDevelop Python-based data services and APIs using frameworks such as FastAPIApply security best practices using IAM, service accounts, encryption, secret management, and least-privilege accessBuild and maintain CI/CD pipelines for data applications using Cloud Build, GitHub Actions, or equivalent toolsProvision and manage GCP data infrastructure using TerraformCollaborate with architects, analysts, data scientists, and business stakeholders to translate requirements into scalable data solutionsCreate technical documentation, data mappings, operational runbooks, and architecture diagramsParticipate in code reviews and promote Python, SQL, GCP, and data-engineering best practicesRequired Qualifications5+ years of hands-on experience in data engineering, including building production-grade data pipelines and platformsStrong programming expertise in Python, including object-oriented programming, reusable modules, exception handling, logging, testing, packaging, and performance optimizationAdvanced experience with PySpark, Spark SQL, and Apache SparkStrong knowledge of GCP data services, particularly BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Composer, and Cloud StorageAdvanced SQL skills, including complex joins, window functions, CTEs, query optimization, and large-scale data transformationDeep expertise in BigQuery, including schema design, partitioning, clustering, performance tuning, workload management, and cost optimizationHands-on experience building batch and streaming pipelines with Dataflow and Apache Beam using PythonStrong experience designing and operating Spark workloads on DataprocExperience developing and managing Airflow DAGs using Cloud ComposerExperience designing Pub/Sub topics, subscriptions, schemas, retry policies, dead-letter queues, and deduplication strategiesStrong understanding of distributed processing concepts, including partitioning, shuffling, parallelism, fault tolerance, and delivery guaranteesSolid understanding of Spark internals, including execution plans, DAGs, Catalyst optimization, memory management, and Spark UI troubleshootingExperience with dimensional data modeling, including star and snowflake schemas and SCD Type 1 and Type 2Experience processing large-scale partitioned datasets and implementing incremental data loadsExperience with data formats such as Parquet, Avro, ORC, JSON, and DeltaStrong understanding of streaming concepts, including event time, processing time, windowing, triggers, watermarking, and exactly-once versus at-least-once processingExperience implementing data-quality, observability, reconciliation, and production-monitoring frameworksProficiency with Git, automated testing, Agile delivery, and CI/CD practicesPreferred QualificationsExperience building Python APIs and microservices using FastAPI or FlaskWorking knowledge of Terraform and infrastructure-as-code practicesExperience with CDC technologies and patternsFamiliarity with Dataform or dbt for SQL-based data transformationsKnowledge of GCP security, networking, IAM, Secret Manager, and data-governance practicesGoogle Cloud Professional Data Engineer certification is preferred

Loading
Please wait..!!