Post a Job

Millions Of Active Candidates

For Employers

VIEW SIMILAR JOBS →

Loading, Please wait..!!

Senior Automation Test Engineer

Posted on: Oct 28, 2026

Oxenham Group LLC
Birmingham, Alabama
Onsite
Full-time

Save this job

Apply Here APPLY LATER

Curious about compensation?

Explore the historical salary trends, average pay, and estimated compensation for Senior Automation Test Engineer roles in Alabama.

View Salary Guide →

Over 12 millions jobs on Snaprecruit.
Find your match !

VIEW SIMILAR JOBS →

Job Title :

Senior Automation Test Engineer

Job Type :

Full-Time

Job Location :

Birmingham Alabama United States

Remote :

Job Description :

View Full Description

Job Description

Our client is a fast-growing healthcare technology company building AI voice agents that answer every call for physician practices scheduling appointments, routing patients, and resolving routine requests around the clock. They are looking for a Senior Automation Test Engineer who can plug in and build rapidly, owning the test strategy and automation infrastructure for their voice AI platform and building the frameworks that make non-deterministic AI behavior measurable and regressions obvious.

Responsibilities

Design, build, and maintain automated test infrastructure across unit, integration, end-to-end, and load/performance layers for a real-time voice AI platform
Build evaluation and regression harnesses that score conversation outcomes and intent fulfillment for non-deterministic, LLM-driven agent behavior
Generate synthetic conversation data and de-identified patient scenarios to expand test coverage for rare edge cases
Develop test frameworks and tooling that enable the broader engineering team to write and run reliable tests efficiently
Define and enforce quality standards and testing best practices across engineering teams
Build simulation and regression frameworks for real-time voice and conversational AI scenarios, including heavy rules-based logic layered on top of nondeterministic AI behavior
Own test environments and CI/CD pipeline integration for automated test execution
Investigate and triage production quality issues, driving root cause analysis and prevention
Develop testing approaches that satisfy HIPAA and healthcare compliance requirements
Ramp quickly: within the first 30 days, understand the tech stack and business, and begin shaping a test suite strategy for the platform

Skills/Requirements

10 years in software quality engineering with a strong focus on test automation
Proven experience designing and owning test automation frameworks from the ground up
Proficiency in Python, TypeScript, or equivalent, with strong scripting and tooling skills
Experience with CI/CD integration (GitHub Actions, Jenkins, or equivalent)
Hands-on AI evaluation experience testing AI-powered applications, including prompt and model regression testing
Demonstrated ability to test non-deterministic conversational flows, designing assertions and evaluation criteria that tolerate variation without masking real regressions
Experience generating synthetic data at scale realistic conversation transcripts, edge-case inputs, and de-identified patient scenarios
Ability to move quickly and independently in a fast-scaling environment with minimal ramp-up time
Nice to have: experience with real-time systems, telephony, or voice platforms; LLM evaluation tooling and prompt/model versioning; evaluating speech-to-text or text-to-speech quality; load/performance testing for high-throughput systems; HIPAA-regulated environment experience; contract testing or service virtualization

Position Details

Posted:

Oct 28, 2026

Reference Number:

f6e5df54

Employment:

Full-time

City:

Birmingham

Job Origin:

ziprecruiter

Share this job: