Skip to main content
Loading
loadingbar
Loading, Please wait..!!

Machine Learning Engineer - GPU Performance & Low-Latency Inference

  • Machine Learning Engineer - GPU Performance & Low-Latency Inference job type Posted on: Nov 16, 2026
  • Machine Learning Engineer - GPU Performance & Low-Latency Inference experience level Fintal Partners
  • Machine Learning Engineer - GPU Performance & Low-Latency Inference employment type New York, New York
  • Machine Learning Engineer - GPU Performance & Low-Latency Inference employment type Onsite
  • Machine Learning Engineer - GPU Performance & Low-Latency Inference salary Full-time

Point Apply Here APPLY LATER

Curious about compensation?

Explore the historical salary trends, average pay, and estimated compensation for Machine Learning Engineer - GPU Performance & Low-Latency Inference roles in New York.

View Salary Guide →

Job Title :

Machine Learning Engineer - GPU Performance & Low-Latency Inference

Job Type :

Full-Time

Job Location :

New York New York United States

Remote :

No

Jobcon Logo Job Description :

Job Description

We’re partnering with a leading global trading firm to hire a Machine Learning Engineer to make ML models run as fast as possible. You’ll work close to the hardware, squeezing latency out of real-time inference and speeding up training, in an environment where microseconds matter.

\n


\n

What you’ll do

\n
    \n
  • Build and optimize low-latency inference pipelines that deliver real-time predictions in production
  • \n
  • Profile and optimize ML workloads at the device level: memory, compute, kernels and compilers
  • \n
  • Write and tune custom GPU code (CUDA, Triton or equivalent) where libraries fall short
  • \n
  • Apply and validate low-precision techniques (FP8/INT8, quantization), knowing which optimizations stay bit-exact,
  • \n
  • Optimize batching, KV caching and serving paths for latency-critical workloads.
  • \n
  • Extend the internals of PyTorch and inference frameworks to raise performance on specific workloads
  • \n
\n


\n

What you’ll bring

\n
    \n
  • 3+ years in machine learning, focused on inference or GPU performance
  • \n
  • Hands-on experience optimizing ML at the GPU level: CUDA, Triton, C++, TensorRT or similar
  • \n
  • Experience with profiling tools (Nsight, torch.profiler) and reasoning about memory bandwidth, latency and throughput
  • \n
  • A strong grasp of modern ML architectures, attention in particular, and how they map onto hardware
  • \n
  • Production-quality Python (C++ a strong plus), with testing discipline and ownership of the systems you ship
  • \n
  • A habit of measuring before optimizing, and the judgment to know when to build versus buy
  • \n
\n


\n

Nice to have

\n
    \n
  • Experience building real-time, low-latency prediction systems in high-performance environments
  • \n
  • Experience with numerics: mixed precision, quantization and correctness at scale
  • \n
  • Contributions to open-source projects such as PyTorch, vLLM, Triton or FlashAttention
  • \n
\n


\n

No trading experience is required. What matters is deep, hands-on performance engineering and a drive to make things faster.

Jobcon Logo Position Details

Posted:

Nov 16, 2026

Reference Number:

3b65a766aaeb6671

Employment:

Full-time

City:

New York

Job Origin:

ziprecruiter

Share this job:

  • linkedin

Jobcon Logo
A job sourcing event
In Dallas Fort Worth
Aug 19, 2017 9am-6pm
All job seekers welcome!

Machine Learning Engineer - GPU Performance & Low-Latency Inference    Apply

Click on the below icons to share this job to Linkedin, Twitter!

Job Description

We’re partnering with a leading global trading firm to hire a Machine Learning Engineer to make ML models run as fast as possible. You’ll work close to the hardware, squeezing latency out of real-time inference and speeding up training, in an environment where microseconds matter.

\n


\n

What you’ll do

\n
    \n
  • Build and optimize low-latency inference pipelines that deliver real-time predictions in production
  • \n
  • Profile and optimize ML workloads at the device level: memory, compute, kernels and compilers
  • \n
  • Write and tune custom GPU code (CUDA, Triton or equivalent) where libraries fall short
  • \n
  • Apply and validate low-precision techniques (FP8/INT8, quantization), knowing which optimizations stay bit-exact,
  • \n
  • Optimize batching, KV caching and serving paths for latency-critical workloads.
  • \n
  • Extend the internals of PyTorch and inference frameworks to raise performance on specific workloads
  • \n
\n


\n

What you’ll bring

\n
    \n
  • 3+ years in machine learning, focused on inference or GPU performance
  • \n
  • Hands-on experience optimizing ML at the GPU level: CUDA, Triton, C++, TensorRT or similar
  • \n
  • Experience with profiling tools (Nsight, torch.profiler) and reasoning about memory bandwidth, latency and throughput
  • \n
  • A strong grasp of modern ML architectures, attention in particular, and how they map onto hardware
  • \n
  • Production-quality Python (C++ a strong plus), with testing discipline and ownership of the systems you ship
  • \n
  • A habit of measuring before optimizing, and the judgment to know when to build versus buy
  • \n
\n


\n

Nice to have

\n
    \n
  • Experience building real-time, low-latency prediction systems in high-performance environments
  • \n
  • Experience with numerics: mixed precision, quantization and correctness at scale
  • \n
  • Contributions to open-source projects such as PyTorch, vLLM, Triton or FlashAttention
  • \n
\n


\n

No trading experience is required. What matters is deep, hands-on performance engineering and a drive to make things faster.

Loading
Please wait..!!