Kafka Engineer

Fine With Relocation Role : Core Platform Engineer Location : Charlotte-Concord-Gastonia, NC-SC (Hybrid) Contarct End Client - Banking Industry Experience - 8+ Years Core Platform Engineer Primary Role: Build and maintain secure, scalable infrastructure and services. Responsibilities: Support a highly available and scalable infrastructure containing Object storage, Openshift, Spark, Iceberg, Yunikorn, Trino Monitor for configuration drift and enforce infrastructure policies. Configure and monitor Big Data ecosystem components with various BI tools, observability tools etc Build automated regression and performance test suite to ensure health checks of all components of the platform Tasks: Monitor system health and enforce runtime policies. Implement and manage security protocols, including Oauth authentication, TLS encryption, and role-based access control (RBAC). Conduct regular maintenance, including cluster scaling, perform regular security audits. Programming & Scripting Languages: Python, Bash, Shell, SQL, Java (basic), Scala (for big data, good to have) Automation & Scripting: Python scripting for automation, Linux shell scripting Operating Systems & Containers System programing, performance tuning, networking OCP, Kubernetes (K8s), Helm, Terraform, container orchestration and deployment Big Data & Data Engineering Frameworks: NexusOne, Apache Spark, Hadoop, Hive, Trino, Iceberg ETL Tools: Apache Airflow, NiFi (good to have) Data Pipelines: Batch and streaming (Kafka, Flink) Object Storage: S3, NetApp StorageGrid Data Formats: Parquet/Avro, ORC, JSON, CSV AI/ML & MTC (Model Training & Consumption) (Nice to have) Frameworks or LLM modeling Model Ops: MLflow, Kubeflow, SageMaker Data Science: Feature engineering, model deployment, inference pipelines Security & Access Control Access Models : RBAC (Role-Based Access Control), ABAC (Attribute-Based Access Control) Data Protection : Encryption at rest and in transit, TLS/SSL, KMS (Key Management Services) Compliance : GDPR, HIPAA (if applicable), IAM policies System Design & Architecture (good to have, at least at a conceptual level) Design Principles : Microservices, Event-driven architecture, Serverless Scalability : Load balancing, caching (Redis, Memcached), horizontal scaling High Availability : Failover strategies, disaster recovery, monitoring (Prometheus, Grafana)

For applications and inquiries,