14+ years of experience in Enterprise Data Architecture, Open-Source Data Platforms, Distributed Data Processing, and Data Engineering.
Proven experience in Enterprise Solution architecture using Open source technology stack across the Data to Insights end to end value chain.
Strong hands-on expertise in Apache Spark, PySpark, Kafka, Flink, Airflow, NiFi, Trino, SQL, Python, Scala, and Java.
Proven experience in designing and implementing open-source data platform architectures, including component selection, integration, scalability, interoperability, and platform modernization.
Deep understanding of batch, streaming, and real-time data processing architectures, including performance tuning, workload optimization, and failure recovery.
Experience building modern Data Lakehouse architectures using Apache Iceberg, Hudi, Delta Lake, object storage, and open table formats.
Strong knowledge of event-driven architectures, CDC, APIs, schema management, and enterprise integration patterns.
Expertise in DataOps practices, including Airflow orchestration, CI/CD, automated testing, observability, metadata management, lineage, and platform operations.
Experience with PostgreSQL, Cassandra, MongoDB, OpenSearch, and enterprise analytics platforms, including data modeling, indexing, workload management, and availability design.
Strong understanding of security, governance, and compliance, including RBAC, identity management, encryption, privacy controls, auditability, metadata governance, and vulnerability management.
Proven ability to lead architecture definition, HLD/LLD creation, NFR assessment, migration planning, technical governance, and production readiness activities.
Experience supporting large-scale data platform modernization, cloud migration, solutioning, RFP responses, effort estimation, and client-facing architecture discussions.
Good understanding of BFSI regulatory, security, resilience, privacy, and operational requirements.
Bachelor's degree in computer science, Engineering, Information Systems, Data Management, or a related discipline.
Preferred certifications in Cloud Data Engineering, Kubernetes, Enterprise Architecture, Data Management, Security, DevOps, or Open-Source Data Platforms.
Experience assessing business use cases and selecting the appropriate approach across BI/KPI reporting, rules, statistical methods, forecasting, machine learning, and GenAI, with clear baselines, acceptance criteria, and measurable business KPIs.
Understanding of AI/ML platform architecture supporting feature engineering, model training, batch and real-time inference, and integration of ML and GenAI workloads with enterprise open-source data platforms.
Understanding of MLOps and LLMOps capabilities, including experiment tracking, model registry, feature stores, pipeline orchestration, automated quality gates, and versioning of data, models, and prompts using tools such as MLflow and Kubeflow.
Exposure to AI observability and monitoring for data drift, model drift, LLM quality, tracing, latency, availability, alerting, and operational runbooks.
Roles & Responsibilities
Engage with business and technology stakeholders to understand data strat egy, platform requirements, architecture goals, and transformation priorities.
Design and define end-to-end architecture for enterprise open-source data platforms supporting batch, streaming, analytics, and AI workloads.
Lead modernization initiatives covering data lakes, lakehouse platforms, distributed processing frameworks, and enterprise data ecosystems.
Architect scalable, secure, highly available, and resilient data platforms using open-source technologies and industry best practices.
Design data ingestion, integration, streaming, storage, and consumption patterns using technologies such as Spark, Kafka, Flink, NiFi, and Trino.
Establish data governance, security, metadata, lineage, privacy, and observability standards across the platform landscape.
Evaluate and recommend open-source technologies based on business requirements, scalability, supportability, interoperability, and total cost of ownership.
Provide technical leadership across data modeling, data engineering, platform implementation, migration, testing, deployment, and production readiness activities.
Conduct architecture reviews, identify risks early, and guide delivery teams on mitigation strategies and best practices.
Support solutioning, RFP responses, effort estimation, technical presentations, and client workshops for strategic opportunities.
Collaborate with engineering teams, platform vendors, open-source communities, and client stakeholders to deliver scalable and sustainable solutions.
Mentor architects and engineers, contribute to technical capability development, and establish reusable accelerators, playbooks, and best practices.
Support hiring, technical assessments, and knowledge-sharing initiatives across Data Engineering, Open-Source Platforms, and Modern Data Architecture.
Assess candidate use cases with business, technology, data, and risk stakeholders; recommend BI, rules, analytics, ML, or GenAI based on feasibility, risk, cost, and measurable business outcomes.
Integrate data, ML, and GenAI reference architectures covering data sources, feature pipelines, models or LLMs, vector stores, serving layers, consuming applications, and enterprise integration points.
Partner with platform and operations teams to productionize AI ML and LLM workloads on Kubernetes.
Generic Managerial Skills, If any
Strong leadership and stakeholder management skills with the ability to work effectively across business, architecture, engineering, operations, and executive teams.
Excellent communication and presentation skills, with the ability to simplify complex technical concepts for both technical and non-technical audiences.
Strong consulting and problem-solving mindset with the ability to evaluate alternatives, challenge assumptions, and recommend practical solutions.
Proven ability to lead distributed, cross-functional teams and drive alignment across multiple stakeholders and workstreams.
High level of ownership, accountability, and attention to detail with a focus on quality delivery and customer success.
Strong planning, prioritization, and decision-making skills with the ability to manage multiple initiatives in a fast-paced environment.
Experience mentoring architects and engineers, fostering collaboration, knowledge sharing, and technical excellence.
Ability to manage risks, dependencies, and escalations proactively while ensuring successful project outcomes.
Click on the below icons to share this job to Linkedin, Twitter!
Must Have Technical/Functional Skills
14+ years of experience in Enterprise Data Architecture, Open-Source Data Platforms, Distributed Data Processing, and Data Engineering.
Proven experience in Enterprise Solution architecture using Open source technology stack across the Data to Insights end to end value chain.
Strong hands-on expertise in Apache Spark, PySpark, Kafka, Flink, Airflow, NiFi, Trino, SQL, Python, Scala, and Java.
Proven experience in designing and implementing open-source data platform architectures, including component selection, integration, scalability, interoperability, and platform modernization.
Deep understanding of batch, streaming, and real-time data processing architectures, including performance tuning, workload optimization, and failure recovery.
Experience building modern Data Lakehouse architectures using Apache Iceberg, Hudi, Delta Lake, object storage, and open table formats.
Strong knowledge of event-driven architectures, CDC, APIs, schema management, and enterprise integration patterns.
Expertise in DataOps practices, including Airflow orchestration, CI/CD, automated testing, observability, metadata management, lineage, and platform operations.
Experience with PostgreSQL, Cassandra, MongoDB, OpenSearch, and enterprise analytics platforms, including data modeling, indexing, workload management, and availability design.
Strong understanding of security, governance, and compliance, including RBAC, identity management, encryption, privacy controls, auditability, metadata governance, and vulnerability management.
Proven ability to lead architecture definition, HLD/LLD creation, NFR assessment, migration planning, technical governance, and production readiness activities.
Experience supporting large-scale data platform modernization, cloud migration, solutioning, RFP responses, effort estimation, and client-facing architecture discussions.
Good understanding of BFSI regulatory, security, resilience, privacy, and operational requirements.
Bachelor's degree in computer science, Engineering, Information Systems, Data Management, or a related discipline.
Preferred certifications in Cloud Data Engineering, Kubernetes, Enterprise Architecture, Data Management, Security, DevOps, or Open-Source Data Platforms.
Experience assessing business use cases and selecting the appropriate approach across BI/KPI reporting, rules, statistical methods, forecasting, machine learning, and GenAI, with clear baselines, acceptance criteria, and measurable business KPIs.
Understanding of AI/ML platform architecture supporting feature engineering, model training, batch and real-time inference, and integration of ML and GenAI workloads with enterprise open-source data platforms.
Understanding of MLOps and LLMOps capabilities, including experiment tracking, model registry, feature stores, pipeline orchestration, automated quality gates, and versioning of data, models, and prompts using tools such as MLflow and Kubeflow.
Exposure to AI observability and monitoring for data drift, model drift, LLM quality, tracing, latency, availability, alerting, and operational runbooks.
Roles & Responsibilities
Engage with business and technology stakeholders to understand data strat egy, platform requirements, architecture goals, and transformation priorities.
Design and define end-to-end architecture for enterprise open-source data platforms supporting batch, streaming, analytics, and AI workloads.
Lead modernization initiatives covering data lakes, lakehouse platforms, distributed processing frameworks, and enterprise data ecosystems.
Architect scalable, secure, highly available, and resilient data platforms using open-source technologies and industry best practices.
Design data ingestion, integration, streaming, storage, and consumption patterns using technologies such as Spark, Kafka, Flink, NiFi, and Trino.
Establish data governance, security, metadata, lineage, privacy, and observability standards across the platform landscape.
Evaluate and recommend open-source technologies based on business requirements, scalability, supportability, interoperability, and total cost of ownership.
Provide technical leadership across data modeling, data engineering, platform implementation, migration, testing, deployment, and production readiness activities.
Conduct architecture reviews, identify risks early, and guide delivery teams on mitigation strategies and best practices.
Support solutioning, RFP responses, effort estimation, technical presentations, and client workshops for strategic opportunities.
Collaborate with engineering teams, platform vendors, open-source communities, and client stakeholders to deliver scalable and sustainable solutions.
Mentor architects and engineers, contribute to technical capability development, and establish reusable accelerators, playbooks, and best practices.
Support hiring, technical assessments, and knowledge-sharing initiatives across Data Engineering, Open-Source Platforms, and Modern Data Architecture.
Assess candidate use cases with business, technology, data, and risk stakeholders; recommend BI, rules, analytics, ML, or GenAI based on feasibility, risk, cost, and measurable business outcomes.
Integrate data, ML, and GenAI reference architectures covering data sources, feature pipelines, models or LLMs, vector stores, serving layers, consuming applications, and enterprise integration points.
Partner with platform and operations teams to productionize AI ML and LLM workloads on Kubernetes.
Generic Managerial Skills, If any
Strong leadership and stakeholder management skills with the ability to work effectively across business, architecture, engineering, operations, and executive teams.
Excellent communication and presentation skills, with the ability to simplify complex technical concepts for both technical and non-technical audiences.
Strong consulting and problem-solving mindset with the ability to evaluate alternatives, challenge assumptions, and recommend practical solutions.
Proven ability to lead distributed, cross-functional teams and drive alignment across multiple stakeholders and workstreams.
High level of ownership, accountability, and attention to detail with a focus on quality delivery and customer success.
Strong planning, prioritization, and decision-making skills with the ability to manage multiple initiatives in a fast-paced environment.
Experience mentoring architects and engineers, fostering collaboration, knowledge sharing, and technical excellence.
Ability to manage risks, dependencies, and escalations proactively while ensuring successful project outcomes.
Adaptability and resilience to work across evolving technologies, business priorities, and transformation programs.
yes
Send Us Your Feedback
Sign
In
Please check your email. We have sent you a password reset link.
This link will expire in one hour.
Please input your account's email
Get $5 added to your wallet for registering an account. Refer others and get $1 for each successful referral. Earn while you search for jobs and redeem in gift cards!Register to search from millions of jobs across the United States. Upload your resume and get matched with top employers today!
Welcome to SnapRecruit! Your account has been created via Google.
Get $5.00 Added to Your Wallet!
Upload your resume below and apply to this job to have $5.00 added directly to your wallet, activate your account, and qualify for high-priority employer matches.
Perks & Rewards You Receive with SnapRecruit:
$5.00 Resume Bonus
Added automatically to your wallet when you apply with your resume.
$1.00 Per Referral
Earn $1.00 cash rewards for every friend who joins SnapRecruit in the US.
Earn While You Search
Get credited $0.02 - $0.05 every time you explore sponsored job listings.
Amazon eGift Cards
Redeem your earned wallet funds for Amazon eGift Cards delivered to your email.
Get $5 added to your wallet for registering anRegister to search from millions of jobs across the United States. Upload your resume and get matched with top employers today!
account.
Refers others and get $1 for each successful referral.
Earn while you search for jobs and redeem in
gift cards!
Upload your resume for a free professional assessment
Loading, Please wait..!!
Send this job to my email
Please wait..!!
Confirmation
Are you sure?
List of resumes
Confirmation
It is working for Chrome Mozilla only for now
We use cookies and similar technologies to analyze site traffic and improve your experience.
We use cookies to improve your experience. You can customize your preferences below.
Privacy Policy |
Terms of Service