Sr Technical Lead Data Bricks

Position : Sr Technical Lead Data Bricks

Location: Dallas, TX (Must be Onsite on needed basis)

Duration : Long Term Contract

Mandatory skills: AWS, Databricks, Python, Spark & Pyspark

Key Responsibilities:

Experience: 6 to 10 years

Realtime experience on databricks is must. Collaborate as part of a development team to design and enhance large scale applications developed using Python, Spark & Pyspark . Realtime experience on databricks is must. Evaluates and plans software designs, test results and technical manuals using AWS. Confer with business units and development staff to understand both the business and technical requirements for producing technical solutions. Create and review technical and user-focused documentation for data solutions (data models, data dictionaries, business glossaries, process and data flows, architecture diagrams, etc.). Extend and enhance the business Data Lake. Create or implement solutions for metadata management. Solve for complex data integrations across multiple systems. Design and execute strategies for real-time data analysis and decisioning. Build robust data processing pipelines using AWS Services and integrate with multiple data sources. Translating client user requirements into data flows, data mapping, etc. Analyses and determines data integration needs and follows Agile practices.

Required Skills:

• 4+ years experience designing and developing Data Pipelines for Data Ingestion or Transformation using Scala or Python
• 4 years experience with Python, Spark & Pyspark
• 3 years experience working on AWS technologies
• Experience designing, building, and deploying production-level data pipelines using AWS Glue, Lambda, Kinesis with Aurora and Redshift
• Spark programming experience (Pyspark or Scala)
• Hands on with AWS components EMR, S3, Redshift, Lambda, API Gateway, Kinesis in production
• Strong analytical skills and advanced SQL knowledge, indexing, query optimization techniques
• Experience using ETL tools for data ingestion
• Experience with Change Data Capture (CDC) technologies and relational databases such as MS SQL, Oracle and DB
• Ability to translate data needs into detailed functional and technical designs

Thanks & Regards OpenKyber.

For applications and inquiries,