Loading, Please wait..!!
Machine Learning Engineer - GPU Performance & Low-Latency Inference
-
Posted on: Nov 16, 2026
-
New York, New York
-
Onsite
-
Full-time
Curious about compensation?
Explore the historical salary trends, average pay, and estimated compensation for Machine Learning Engineer - GPU Performance & Low-Latency Inference roles in New York.
View Salary Guide →Find your match !
Job Title :
Machine Learning Engineer - GPU Performance & Low-Latency Inference
Job Type :
Full-Time
Job Location :
New York New York United States
Remote :
No
Job Description :
Job Description
We’re partnering with a leading global trading firm to hire a Machine Learning Engineer to make ML models run as fast as possible. You’ll work close to the hardware, squeezing latency out of real-time inference and speeding up training, in an environment where microseconds matter.
\nWhat you’ll do
\n- \n
- Build and optimize low-latency inference pipelines that deliver real-time predictions in production \n
- Profile and optimize ML workloads at the device level: memory, compute, kernels and compilers \n
- Write and tune custom GPU code (CUDA, Triton or equivalent) where libraries fall short \n
- Apply and validate low-precision techniques (FP8/INT8, quantization), knowing which optimizations stay bit-exact, \n
- Optimize batching, KV caching and serving paths for latency-critical workloads. \n
- Extend the internals of PyTorch and inference frameworks to raise performance on specific workloads \n
What you’ll bring
\n- \n
- 3+ years in machine learning, focused on inference or GPU performance \n
- Hands-on experience optimizing ML at the GPU level: CUDA, Triton, C++, TensorRT or similar \n
- Experience with profiling tools (Nsight, torch.profiler) and reasoning about memory bandwidth, latency and throughput \n
- A strong grasp of modern ML architectures, attention in particular, and how they map onto hardware \n
- Production-quality Python (C++ a strong plus), with testing discipline and ownership of the systems you ship \n
- A habit of measuring before optimizing, and the judgment to know when to build versus buy \n
Nice to have
\n- \n
- Experience building real-time, low-latency prediction systems in high-performance environments \n
- Experience with numerics: mixed precision, quantization and correctness at scale \n
- Contributions to open-source projects such as PyTorch, vLLM, Triton or FlashAttention \n
No trading experience is required. What matters is deep, hands-on performance engineering and a drive to make things faster.
Position Details
Posted:
Nov 16, 2026
Reference Number:
3b65a766aaeb6671
Employment:
Full-time
City:
New York
Job Origin:
ziprecruiter
Share this job:
Please wait..!!

