Loading, Please wait..!!
AI Monitoring Engineer
-
Posted on: Oct 28, 2026
-
New York, New York
-
Onsite
-
Full-time
Curious about compensation?
Explore the historical salary trends, average pay, and estimated compensation for AI Monitoring Engineer roles in New York.
View Salary Guide →Find your match !
Job Title :
AI Monitoring Engineer
Job Type :
Full-Time
Job Location :
New York New York United States
Remote :
No
Job Description :
Job Description
We are seeking an AI Monitoring Engineer to design, implement, and manage monitoring solutions for AI, machine learning, and Generative AI systems. The ideal candidate will ensure the availability, reliability, performance, and quality of AI services by developing observability frameworks, monitoring model behavior, detecting anomalies, and automating alerts and incident response. This role collaborates closely with AI Engineers, ML Engineers, Data Scientists, DevOps Engineers, and Platform Engineers to maintain high-performing AI systems in production.
Key ResponsibilitiesDesign and implement monitoring frameworks for AI applications, machine learning models, and AI infrastructure.
Monitor model performance, prediction quality, latency, throughput, and availability.
Detect and analyze data drift, model drift, concept drift, and inference anomalies.
Build dashboards and alerts for AI system health using observability tools.
Develop Service Level Indicators (SLIs), Service Level Objectives (SLOs), and operational metrics for AI services.
Implement logging, distributed tracing, and telemetry for AI applications.
Configure automated alerting and incident response workflows.
Monitor GPU, CPU, memory, storage, and network utilization for AI workloads.
Analyze AI system performance trends and recommend optimization strategies.
Collaborate with AI Engineers and Data Scientists to improve model reliability and operational performance.
Support root cause analysis (RCA) for production incidents and implement preventive measures.
Maintain monitoring documentation, operational runbooks, and reporting dashboards.
Ensure compliance with monitoring, security, governance, and operational standards.
Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.
3 years of experience in Monitoring Engineering, Site Reliability Engineering (SRE), DevOps, Platform Engineering, MLOps, or Cloud Operations.
Experience monitoring production AI or machine learning systems.
Strong programming skills in Python and scripting with Bash.
Experience with Linux administration.
Hands-on experience with Docker and Kubernetes.
Experience with AWS, Microsoft Azure, or Google Cloud Platform.
Strong understanding of distributed systems, cloud infrastructure, and networking.
Experience with observability tools such as Prometheus, Grafana, OpenTelemetry, ELK Stack, Datadog, Splunk, or New Relic.
Experience with SQL and time-series databases for operational reporting.
Experience with machine learning monitoring platforms such as Evidently AI, Arize AI, Fiddler AI, WhyLabs, or Azure AI monitoring capabilities.
Experience monitoring Large Language Models (LLMs) and Generative AI applications.
Knowledge of AI evaluation metrics, prompt monitoring, hallucination detection, and Retrieval-Augmented Generation (RAG) observability.
Experience with MLOps platforms such as MLflow, Kubeflow, SageMaker, Vertex AI, or Azure Machine Learning.
Familiarity with vector databases and model serving platforms.
Experience monitoring GPU-enabled infrastructure and AI inference systems.
Knowledge of Responsible AI, fairness monitoring, and AI governance.
Python
Bash
SQL
Linux
Docker
Kubernetes
AWS / Azure / Google Cloud Platform
Prometheus
Grafana
OpenTelemetry
ELK Stack
Datadog
Splunk
New Relic
MLflow
Kubeflow
SageMaker
Vertex AI
Azure Machine Learning
Evidently AI
Arize AI
Fiddler AI
WhyLabs
PostgreSQL
Redis
Strong analytical and troubleshooting skills
Excellent communication and collaboration
Attention to detail
Problem-solving mindset
Ownership and accountability
Continuous learning and improvement
Experience monitoring LLMs, AI agents, and Retrieval-Augmented Generation (RAG) systems
Knowledge of AI evaluation frameworks and model quality assessment
Experience implementing automated anomaly detection and predictive alerting
Familiarity with FinOps and AI infrastructure cost monitoring
Cloud, observability, or Kubernetes certifications
AI service availability and uptime
Model inference latency and throughput
Alert accuracy and mean time to detect (MTTD)
Mean time to resolve (MTTR) production incidents
Model drift detection and response time
Dashboard coverage for AI services
Reduction in production incidents and false alerts
Infrastructure resource utilization and optimization
Hybrid / Remote / On-site (as applicable)
Employment TypeFull-time
Position Details
Posted:
Oct 28, 2026
Reference Number:
b3f667e9
Employment:
Full-time
City:
New York
Job Origin:
ziprecruiter
Share this job:
Please wait..!!

