Explore the historical salary trends, average pay, and estimated compensation for Machine Learning Engineer (Evals and Voice Models) roles in California.
Job DescriptionJob DescriptionAircall is a unicorn AI-powered communications platform transforming how teams win with customers. For over a decade, we've believed voice builds the strongest business relationships. We bring voice, SMS, WhatsApp, and 250+ integrations together on one seamless workspace—so every customer conversation drives revenue, resolves issues faster, and builds trust. Our AI Voice Agents handle routine interactions while your team leads the conversations that matter most. Used by 24,000+ businesses worldwide. We're 600+ people from 40+ nationalities across Paris, London, Madrid, Berlin, New York, San Francisco, Seattle, Sydney, and Mexico City. We're ambitious, product-driven, and execution-focused—built for visible impact, fast decisions, and real growth. Come join us!Aircall's AI suite includes an AI Voice Agent and AI Messaging Agent that autonomously handle calls, WhatsApp, and SMS, plus AI Assist, which delivers real-time coaching, call summaries, and CRM automation for sales and support teams. We are looking for someone that can build out the evaluation foundation across all of these products and other agentic products. You'll work on voice models, agent capability evals, benchmark design, LLM-as-judge systems, failure analysis, and the infrastructure that ties it together by establishing shared metrics, test sets, and tooling to measure accuracy, resolution quality, and safety consistently across products. You will set up repeatable pipelines for regression testing and benchmarking as models and features evolve so teams can ship confidently without re-inventing evaluation methodology for each product.Key ResponsibilitiesDesign and document comprehensive evaluation frameworks for Aircall's AI agents across voice, chat and messaging.Train and fine-tune voice models (TTS, ASR, speech-to-speech) using production and synthetic data, iterating on architecture, data mix, and training strategy to improve accuracy, naturalness, and latency.Assess AI generated solutions across training pipelines, experimentation setups, debugging processes, and optimization strategies. Analyze system design decisions and identify strengths, weaknesses, and potential failure points.Design annotation guidelines and workflows for human-labeled evaluation data, and calibrate LLM-as-judge systems against human raters to ensure automated evals stay trustworthy over time.Build and maintain live quality monitoring for deployed AI agents, tracking accuracy, resolution rate, and safety signals in production, and flagging model or data drift before it impacts customers.Own the metric contract for every published AI metrics, including definition, population, grain, rollup, validity window.Build release gates, the offline regression suite each AI surface must pass before a prompt, model, or config change ships, measuring reliability across repeated trials, not just average pass rates.Build voice-specific evaluation: simulated callers across accents, languages, background noise, barge-in, DTMF, and tool failures, with latency and ASR accuracy as first-class quality metrics.Minimum QualificationsBS in Computer Science, Machine Learning, Statistics, or related field3+ years of experience in ML Engineering or Applied ML with 8+ years of overall experienceStrong experience in evaluating supervised, unsupervised, LLMs and deep learning models.Hands-on experience in failure analysis and evaluating LLMsExperience building automated evaluation systemsStrong communication skills to articulate complex technical concepts across technical and non-technical audiencesHands-on experience training or fine-tuning voice/speech models (TTS, ASR, or speech-to-speech), including data pipeline construction and experimentation.Preferred QualificationsMS / PhD in Computer Science, Machine Learning, Statistics, or related fieldExperience evaluating LLMs or agentic systems (e.g., LLM-as-a-judge, RAG evaluation)Experience with synthetic data generation and prompt engineeringExperience training or fine-tuning voice models at scale, with familiarity in synthetic data generation, model distillation, or low-latency inference optimization for production voice agents. Base salary range: $181,000—$250,000 USD Why join us?Key moment to join Aircall in terms of growth and opportunities️♀️ Our people matter, work-life balance is important at AircallFast-learning environment, entrepreneurial and strong team spirit45+ Nationalities: cosmopolite & multi-cultural mindsetCompetitive salary package & benefits DE&I Statement: At Aircall, we believe diversity, equity and inclusion – irrespective of origins, identity, background and orientations – are core to our journey. We pride ourselves on promoting active inclusion within our business to foster a strong sense of belonging for all. We're working to create a place filled with diverse people who can enrich and learn from one another. We're committed to ensuring that everyone not only has a seat at the table but is valued and respected at it by providing equal opportunities to develop and thrive. We will constantly challenge ourselves to make sure that we live up to our ambitions around diversity, equity and inclusion, and keep this conversation open. Above all else, we understand and acknowledge that we have work to do and much to learn. Want to know more about candidate privacy? Find our Candidate Privacy Notice here.We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
Machine Learning Engineer (Evals and Voice Models)
Apply
Click on the below icons to share this job to Linkedin, Twitter!
Job DescriptionJob DescriptionAircall is a unicorn AI-powered communications platform transforming how teams win with customers. For over a decade, we've believed voice builds the strongest business relationships. We bring voice, SMS, WhatsApp, and 250+ integrations together on one seamless workspace—so every customer conversation drives revenue, resolves issues faster, and builds trust. Our AI Voice Agents handle routine interactions while your team leads the conversations that matter most. Used by 24,000+ businesses worldwide. We're 600+ people from 40+ nationalities across Paris, London, Madrid, Berlin, New York, San Francisco, Seattle, Sydney, and Mexico City. We're ambitious, product-driven, and execution-focused—built for visible impact, fast decisions, and real growth. Come join us!Aircall's AI suite includes an AI Voice Agent and AI Messaging Agent that autonomously handle calls, WhatsApp, and SMS, plus AI Assist, which delivers real-time coaching, call summaries, and CRM automation for sales and support teams. We are looking for someone that can build out the evaluation foundation across all of these products and other agentic products. You'll work on voice models, agent capability evals, benchmark design, LLM-as-judge systems, failure analysis, and the infrastructure that ties it together by establishing shared metrics, test sets, and tooling to measure accuracy, resolution quality, and safety consistently across products. You will set up repeatable pipelines for regression testing and benchmarking as models and features evolve so teams can ship confidently without re-inventing evaluation methodology for each product.Key ResponsibilitiesDesign and document comprehensive evaluation frameworks for Aircall's AI agents across voice, chat and messaging.Train and fine-tune voice models (TTS, ASR, speech-to-speech) using production and synthetic data, iterating on architecture, data mix, and training strategy to improve accuracy, naturalness, and latency.Assess AI generated solutions across training pipelines, experimentation setups, debugging processes, and optimization strategies. Analyze system design decisions and identify strengths, weaknesses, and potential failure points.Design annotation guidelines and workflows for human-labeled evaluation data, and calibrate LLM-as-judge systems against human raters to ensure automated evals stay trustworthy over time.Build and maintain live quality monitoring for deployed AI agents, tracking accuracy, resolution rate, and safety signals in production, and flagging model or data drift before it impacts customers.Own the metric contract for every published AI metrics, including definition, population, grain, rollup, validity window.Build release gates, the offline regression suite each AI surface must pass before a prompt, model, or config change ships, measuring reliability across repeated trials, not just average pass rates.Build voice-specific evaluation: simulated callers across accents, languages, background noise, barge-in, DTMF, and tool failures, with latency and ASR accuracy as first-class quality metrics.Minimum QualificationsBS in Computer Science, Machine Learning, Statistics, or related field3+ years of experience in ML Engineering or Applied ML with 8+ years of overall experienceStrong experience in evaluating supervised, unsupervised, LLMs and deep learning models.Hands-on experience in failure analysis and evaluating LLMsExperience building automated evaluation systemsStrong communication skills to articulate complex technical concepts across technical and non-technical audiencesHands-on experience training or fine-tuning voice/speech models (TTS, ASR, or speech-to-speech), including data pipeline construction and experimentation.Preferred QualificationsMS / PhD in Computer Science, Machine Learning, Statistics, or related fieldExperience evaluating LLMs or agentic systems (e.g., LLM-as-a-judge, RAG evaluation)Experience with synthetic data generation and prompt engineeringExperience training or fine-tuning voice models at scale, with familiarity in synthetic data generation, model distillation, or low-latency inference optimization for production voice agents. Base salary range: $181,000—$250,000 USD Why join us?Key moment to join Aircall in terms of growth and opportunities️♀️ Our people matter, work-life balance is important at AircallFast-learning environment, entrepreneurial and strong team spirit45+ Nationalities: cosmopolite & multi-cultural mindsetCompetitive salary package & benefits DE&I Statement: At Aircall, we believe diversity, equity and inclusion – irrespective of origins, identity, background and orientations – are core to our journey. We pride ourselves on promoting active inclusion within our business to foster a strong sense of belonging for all. We're working to create a place filled with diverse people who can enrich and learn from one another. We're committed to ensuring that everyone not only has a seat at the table but is valued and respected at it by providing equal opportunities to develop and thrive. We will constantly challenge ourselves to make sure that we live up to our ambitions around diversity, equity and inclusion, and keep this conversation open. Above all else, we understand and acknowledge that we have work to do and much to learn. Want to know more about candidate privacy? Find our Candidate Privacy Notice here.We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
yes
Send Us Your Feedback
Sign
In
Please check your email. We have sent you a password reset link.
This link will expire in one hour.
Please input your account's email
Apply to Machine Learning Engineer (Evals and Voice Models)
Aircall.io, Inc. • San Francisco, California
✓
Application Started!
Redirecting you to complete your application...
Get $5 added to your wallet for registering an account. Refer others and get $1 for each successful referral. Earn while you search for jobs and redeem in gift cards!Register to search from millions of jobs across the United States. Upload your resume and get matched with top employers today!
Welcome to SnapRecruit! Your account has been created via Google.
Get $5.00 Added to Your Wallet!
Upload your resume below and apply to this job to have $5.00 added directly to your wallet, activate your account, and qualify for high-priority employer matches.
Perks & Rewards You Receive with SnapRecruit:
$5.00 Resume Bonus
Added automatically to your wallet when you apply with your resume.
$1.00 Per Referral
Earn $1.00 cash rewards for every friend who joins SnapRecruit in the US.
Earn While You Search
Get credited $0.02 - $0.05 every time you explore sponsored job listings.
Amazon eGift Cards
Redeem your earned wallet funds for Amazon eGift Cards delivered to your email.
Get $5 added to your wallet for registering anRegister to search from millions of jobs across the United States. Upload your resume and get matched with top employers today!
account.
Refers others and get $1 for each successful referral.
Earn while you search for jobs and redeem in
gift cards!
Upload your resume for a free professional assessment
Loading, Please wait..!!
Send this job to my email
Please wait..!!
Confirmation
Are you sure?
List of resumes
Confirmation
It is working for Chrome Mozilla only for now
We use essential cookies to keep SnapRecruit secure and reliable. With your permission, we also use optional analytics, AI support chat, and push-alert cookies to improve your job search experience. You can change or withdraw your choice anytime.
Privacy Policy •
Cookie Policy •
Your Privacy Rights