Job Description:

• Build and maintain batch data ingestion and transformation pipelines using Databricks and Apache Spark.

• Implement data transformations using SQL and Spark in line with defined medallion (Bronze/Silver/Gold) patterns.

• Apply data quality checks, validations, and basic monitoring to data pipelines.

• Support the onboarding of new data sources following existing ingestion frameworks.

• Work with Senior Data Engineers to understand requirements and deliver domain-aligned datasets.

• Participate in code reviews, testing, and deployment activities.

• Monitor and support production data pipelines, helping to diagnose and resolve issues.

• Document data pipelines, transformations, and datasets to support transparency and reuse.