Skip to main content
Loading
loadingbar
Loading, Please wait..!!

AL / ML Engineer

  • AL / ML Engineer job type Posted on: Oct 28, 2026
  • AL / ML Engineer experience level Diversity Nexus
  • AL / ML Engineer employment type New York, New York
  • AL / ML Engineer employment type Onsite
  • AL / ML Engineer salary Full-time

Point Apply Here APPLY LATER

Curious about compensation?

Explore the historical salary trends, average pay, and estimated compensation for AL / ML Engineer roles in New York.

View Salary Guide →

Job Title :

AL / ML Engineer

Job Type :

Full-Time

Job Location :

New York New York United States

Remote :

No

Jobcon Logo Job Description :

Job Description

Job Description
J ob Description :

Job Requirements About the Role
We are seeking a Lead AI/ML Engineer to architect, build, and operationalize a full-lifecycle, automated evaluation harness for non-deterministic Generative AI agents deployed across our Cloud GTM space (e.g., deep research, conversational analytics, pipeline forecasting).

In this role, you will transition our AI evaluations from manual human reviews to high-throughput automated pipelines backed by LLM-as-a-Judge and targeted human audits. You will work at the intersection of enterprise software engineering, distributed computing, and cutting-edge LLM evaluation frameworks to ensure data integrity, eliminate regressions, and scale our GTM agent capabilities.

Key Responsibilities
Foundations & Local Sandbox DevelopmentBuild robust golden datasets extracted from UAT logs and utilize frontier models to synthetically generate variations (typos, phrasing, syntax) for robust testing.Production Pipeline Automation & System Architecture
Architect and deploy language-agnostic RPC endpoints to systematically invoke GTM agents within the ecosystem.
Build resilient production pipelines supporting parallel inference execution across 1,000 trajectory datasets in under 15 minutes.
Stand up a centralized Model Context Protocol (MCP) logging server to capture raw prompts, tool trajectories, SQL queries, and token costs using strict JSON schemas.Implement asynchronous message queues (Pub/Sub) for rate-limiting/backpressure, along with retry policies for network and generation failures.

Skill Benchmarking & Trajectory Validation
Author eval test suites to isolate specific agent competencies (e.g., CRM writes, SQL analytics).Inject sandboxed mocks to validate tool-calling logic without producing live CRM side effects or executing heavy database reads.
Stream execution logs for baseline delta analysis and capability scoring.

Enterprise Analytics, Governance & Security (Phase 4)Build low-latency Hydra ETL pipelines to stream structured evaluation JSON records into data warehouses and construct Plx analytics dashboards.

Work Experience 5-7Years

Jobcon Logo Position Details

Posted:

Oct 28, 2026

Reference Number:

468e7b6b

Employment:

Full-time

City:

New York

Job Origin:

ziprecruiter

Share this job:

  • linkedin

Jobcon Logo
A job sourcing event
In Dallas Fort Worth
Aug 19, 2017 9am-6pm
All job seekers welcome!

AL / ML Engineer    Apply

Click on the below icons to share this job to Linkedin, Twitter!

Job Description

Job Description
J ob Description :

Job Requirements About the Role
We are seeking a Lead AI/ML Engineer to architect, build, and operationalize a full-lifecycle, automated evaluation harness for non-deterministic Generative AI agents deployed across our Cloud GTM space (e.g., deep research, conversational analytics, pipeline forecasting).

In this role, you will transition our AI evaluations from manual human reviews to high-throughput automated pipelines backed by LLM-as-a-Judge and targeted human audits. You will work at the intersection of enterprise software engineering, distributed computing, and cutting-edge LLM evaluation frameworks to ensure data integrity, eliminate regressions, and scale our GTM agent capabilities.

Key Responsibilities
Foundations & Local Sandbox DevelopmentBuild robust golden datasets extracted from UAT logs and utilize frontier models to synthetically generate variations (typos, phrasing, syntax) for robust testing.Production Pipeline Automation & System Architecture
Architect and deploy language-agnostic RPC endpoints to systematically invoke GTM agents within the ecosystem.
Build resilient production pipelines supporting parallel inference execution across 1,000 trajectory datasets in under 15 minutes.
Stand up a centralized Model Context Protocol (MCP) logging server to capture raw prompts, tool trajectories, SQL queries, and token costs using strict JSON schemas.Implement asynchronous message queues (Pub/Sub) for rate-limiting/backpressure, along with retry policies for network and generation failures.

Skill Benchmarking & Trajectory Validation
Author eval test suites to isolate specific agent competencies (e.g., CRM writes, SQL analytics).Inject sandboxed mocks to validate tool-calling logic without producing live CRM side effects or executing heavy database reads.
Stream execution logs for baseline delta analysis and capability scoring.

Enterprise Analytics, Governance & Security (Phase 4)Build low-latency Hydra ETL pipelines to stream structured evaluation JSON records into data warehouses and construct Plx analytics dashboards.

Work Experience 5-7Years

Loading
Please wait..!!