Skip to main content
Loading
loadingbar
Loading, Please wait..!!

AI Monitoring Engineer

  • AI Monitoring Engineer job type Posted on: Oct 28, 2026
  • AI Monitoring Engineer experience level Ova Technologies
  • AI Monitoring Engineer employment type New York, New York
  • AI Monitoring Engineer employment type Onsite
  • AI Monitoring Engineer salary Full-time

Point Apply Here APPLY LATER

Curious about compensation?

Explore the historical salary trends, average pay, and estimated compensation for AI Monitoring Engineer roles in New York.

View Salary Guide →

Job Title :

AI Monitoring Engineer

Job Type :

Full-Time

Job Location :

New York New York United States

Remote :

No

Jobcon Logo Job Description :

Job Description

Job Description
Job Title: AI Monitoring EngineerJob Summary

We are seeking an AI Monitoring Engineer to design, implement, and manage monitoring solutions for AI, machine learning, and Generative AI systems. The ideal candidate will ensure the availability, reliability, performance, and quality of AI services by developing observability frameworks, monitoring model behavior, detecting anomalies, and automating alerts and incident response. This role collaborates closely with AI Engineers, ML Engineers, Data Scientists, DevOps Engineers, and Platform Engineers to maintain high-performing AI systems in production.

Key Responsibilities
  • Design and implement monitoring frameworks for AI applications, machine learning models, and AI infrastructure.

  • Monitor model performance, prediction quality, latency, throughput, and availability.

  • Detect and analyze data drift, model drift, concept drift, and inference anomalies.

  • Build dashboards and alerts for AI system health using observability tools.

  • Develop Service Level Indicators (SLIs), Service Level Objectives (SLOs), and operational metrics for AI services.

  • Implement logging, distributed tracing, and telemetry for AI applications.

  • Configure automated alerting and incident response workflows.

  • Monitor GPU, CPU, memory, storage, and network utilization for AI workloads.

  • Analyze AI system performance trends and recommend optimization strategies.

  • Collaborate with AI Engineers and Data Scientists to improve model reliability and operational performance.

  • Support root cause analysis (RCA) for production incidents and implement preventive measures.

  • Maintain monitoring documentation, operational runbooks, and reporting dashboards.

  • Ensure compliance with monitoring, security, governance, and operational standards.

Required Qualifications
  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.

  • 3 years of experience in Monitoring Engineering, Site Reliability Engineering (SRE), DevOps, Platform Engineering, MLOps, or Cloud Operations.

  • Experience monitoring production AI or machine learning systems.

  • Strong programming skills in Python and scripting with Bash.

  • Experience with Linux administration.

  • Hands-on experience with Docker and Kubernetes.

  • Experience with AWS, Microsoft Azure, or Google Cloud Platform.

  • Strong understanding of distributed systems, cloud infrastructure, and networking.

  • Experience with observability tools such as Prometheus, Grafana, OpenTelemetry, ELK Stack, Datadog, Splunk, or New Relic.

  • Experience with SQL and time-series databases for operational reporting.

Preferred Qualifications
  • Experience with machine learning monitoring platforms such as Evidently AI, Arize AI, Fiddler AI, WhyLabs, or Azure AI monitoring capabilities.

  • Experience monitoring Large Language Models (LLMs) and Generative AI applications.

  • Knowledge of AI evaluation metrics, prompt monitoring, hallucination detection, and Retrieval-Augmented Generation (RAG) observability.

  • Experience with MLOps platforms such as MLflow, Kubeflow, SageMaker, Vertex AI, or Azure Machine Learning.

  • Familiarity with vector databases and model serving platforms.

  • Experience monitoring GPU-enabled infrastructure and AI inference systems.

  • Knowledge of Responsible AI, fairness monitoring, and AI governance.

Technical Skills
  • Python

  • Bash

  • SQL

  • Linux

  • Docker

  • Kubernetes

  • AWS / Azure / Google Cloud Platform

  • Prometheus

  • Grafana

  • OpenTelemetry

  • ELK Stack

  • Datadog

  • Splunk

  • New Relic

  • MLflow

  • Kubeflow

  • SageMaker

  • Vertex AI

  • Azure Machine Learning

  • Evidently AI

  • Arize AI

  • Fiddler AI

  • WhyLabs

  • PostgreSQL

  • Redis

Soft Skills
  • Strong analytical and troubleshooting skills

  • Excellent communication and collaboration

  • Attention to detail

  • Problem-solving mindset

  • Ownership and accountability

  • Continuous learning and improvement

Nice to Have
  • Experience monitoring LLMs, AI agents, and Retrieval-Augmented Generation (RAG) systems

  • Knowledge of AI evaluation frameworks and model quality assessment

  • Experience implementing automated anomaly detection and predictive alerting

  • Familiarity with FinOps and AI infrastructure cost monitoring

  • Cloud, observability, or Kubernetes certifications

Key Performance Indicators (KPIs)
  • AI service availability and uptime

  • Model inference latency and throughput

  • Alert accuracy and mean time to detect (MTTD)

  • Mean time to resolve (MTTR) production incidents

  • Model drift detection and response time

  • Dashboard coverage for AI services

  • Reduction in production incidents and false alerts

  • Infrastructure resource utilization and optimization

Location

Hybrid / Remote / On-site (as applicable)

Employment Type

Full-time

Jobcon Logo Position Details

Posted:

Oct 28, 2026

Reference Number:

b3f667e9

Employment:

Full-time

City:

New York

Job Origin:

ziprecruiter

Share this job:

  • linkedin

Jobcon Logo
A job sourcing event
In Dallas Fort Worth
Aug 19, 2017 9am-6pm
All job seekers welcome!

AI Monitoring Engineer    Apply

Click on the below icons to share this job to Linkedin, Twitter!

Job Description

Job Description
Job Title: AI Monitoring EngineerJob Summary

We are seeking an AI Monitoring Engineer to design, implement, and manage monitoring solutions for AI, machine learning, and Generative AI systems. The ideal candidate will ensure the availability, reliability, performance, and quality of AI services by developing observability frameworks, monitoring model behavior, detecting anomalies, and automating alerts and incident response. This role collaborates closely with AI Engineers, ML Engineers, Data Scientists, DevOps Engineers, and Platform Engineers to maintain high-performing AI systems in production.

Key Responsibilities
  • Design and implement monitoring frameworks for AI applications, machine learning models, and AI infrastructure.

  • Monitor model performance, prediction quality, latency, throughput, and availability.

  • Detect and analyze data drift, model drift, concept drift, and inference anomalies.

  • Build dashboards and alerts for AI system health using observability tools.

  • Develop Service Level Indicators (SLIs), Service Level Objectives (SLOs), and operational metrics for AI services.

  • Implement logging, distributed tracing, and telemetry for AI applications.

  • Configure automated alerting and incident response workflows.

  • Monitor GPU, CPU, memory, storage, and network utilization for AI workloads.

  • Analyze AI system performance trends and recommend optimization strategies.

  • Collaborate with AI Engineers and Data Scientists to improve model reliability and operational performance.

  • Support root cause analysis (RCA) for production incidents and implement preventive measures.

  • Maintain monitoring documentation, operational runbooks, and reporting dashboards.

  • Ensure compliance with monitoring, security, governance, and operational standards.

Required Qualifications
  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.

  • 3 years of experience in Monitoring Engineering, Site Reliability Engineering (SRE), DevOps, Platform Engineering, MLOps, or Cloud Operations.

  • Experience monitoring production AI or machine learning systems.

  • Strong programming skills in Python and scripting with Bash.

  • Experience with Linux administration.

  • Hands-on experience with Docker and Kubernetes.

  • Experience with AWS, Microsoft Azure, or Google Cloud Platform.

  • Strong understanding of distributed systems, cloud infrastructure, and networking.

  • Experience with observability tools such as Prometheus, Grafana, OpenTelemetry, ELK Stack, Datadog, Splunk, or New Relic.

  • Experience with SQL and time-series databases for operational reporting.

Preferred Qualifications
  • Experience with machine learning monitoring platforms such as Evidently AI, Arize AI, Fiddler AI, WhyLabs, or Azure AI monitoring capabilities.

  • Experience monitoring Large Language Models (LLMs) and Generative AI applications.

  • Knowledge of AI evaluation metrics, prompt monitoring, hallucination detection, and Retrieval-Augmented Generation (RAG) observability.

  • Experience with MLOps platforms such as MLflow, Kubeflow, SageMaker, Vertex AI, or Azure Machine Learning.

  • Familiarity with vector databases and model serving platforms.

  • Experience monitoring GPU-enabled infrastructure and AI inference systems.

  • Knowledge of Responsible AI, fairness monitoring, and AI governance.

Technical Skills
  • Python

  • Bash

  • SQL

  • Linux

  • Docker

  • Kubernetes

  • AWS / Azure / Google Cloud Platform

  • Prometheus

  • Grafana

  • OpenTelemetry

  • ELK Stack

  • Datadog

  • Splunk

  • New Relic

  • MLflow

  • Kubeflow

  • SageMaker

  • Vertex AI

  • Azure Machine Learning

  • Evidently AI

  • Arize AI

  • Fiddler AI

  • WhyLabs

  • PostgreSQL

  • Redis

Soft Skills
  • Strong analytical and troubleshooting skills

  • Excellent communication and collaboration

  • Attention to detail

  • Problem-solving mindset

  • Ownership and accountability

  • Continuous learning and improvement

Nice to Have
  • Experience monitoring LLMs, AI agents, and Retrieval-Augmented Generation (RAG) systems

  • Knowledge of AI evaluation frameworks and model quality assessment

  • Experience implementing automated anomaly detection and predictive alerting

  • Familiarity with FinOps and AI infrastructure cost monitoring

  • Cloud, observability, or Kubernetes certifications

Key Performance Indicators (KPIs)
  • AI service availability and uptime

  • Model inference latency and throughput

  • Alert accuracy and mean time to detect (MTTD)

  • Mean time to resolve (MTTR) production incidents

  • Model drift detection and response time

  • Dashboard coverage for AI services

  • Reduction in production incidents and false alerts

  • Infrastructure resource utilization and optimization

Location

Hybrid / Remote / On-site (as applicable)

Employment Type

Full-time

Loading
Please wait..!!