AI Solutions QA Analyst

AI Solutions QA AnalystLocation - Remote, but candidates must reside near one of the following metro areas:Boston, New York, Wilmington, Washington, DC, Atlanta, Miami, Boca Raton, Chicago, Austin, Houston, Nashville, Dallas, Los Angeles, San Francisco, Silicon Valley, or San Diego.About the OpportunityA forward-thinking organization is building one of the most ambitious AI programs in its industry — where quality is a core driver of success, not an afterthought.As AI-powered solutions are deployed across business and operational teams, we are seeking an AI Solutions QA Analyst to own the full quality lifecycle. This role will ensure every solution released is thoroughly tested, highly reliable, and production-ready, leveraging modern QA methodologies and AI-assisted tools to move faster and more effectively than traditional testing approaches.This is a high-impact role working directly with product analysts, engineers, and QA leadership to shape how AI quality is defined and delivered in a cutting-edge environment.What You’ll DoOwn the end-to-end QA lifecycle for AI-powered solutions, including:Unit, integration, system, regression, performance, and scalability testingPartner with Analysts to design and evaluate test cases using AI-assisted tools, with a focus on:Boundary conditionsEdge casesAI output validationCollaborate with engineering teams to build and integrate automated test suites into Azure DevOps CI/CD pipelinesDefine and execute system-level QA strategies in collaboration with QA leadershipTranslate security requirements into solution-specific test scenarios alongside security teamsEvaluate AI-driven systems for:ReliabilityConsistencyStability across diverse real-world inputsMaintain comprehensive QA documentation, including:Test casesDefect trackingQuality metrics and reportingPromote a quality-first culture by embedding testing throughout the development lifecycle rather than treating it as a final checkpointWhat We’re Looking For4+ years of experience in Quality Assurance, including hands-on test execution across the full testing lifecycleStrong experience with:Unit, integration, system, and regression testingBoth manual and automated testing approachesProven ability to design tests for complex, data-driven systems, including edge cases and boundary conditionsExperience integrating automated test suites into CI/CD pipelines (Azure DevOps strongly preferred)Familiarity with performance and scalability testing concepts; experience with load testing tools is a plusStrong working knowledge of:Defect tracking systemsTest documentation practicesQA metrics and reportingDemonstrated analytical mindset, with the ability to evaluate unpredictable or emerging system behaviors (e.g., AI outputs)Bachelor’s degree in Computer Science, Information Systems, Engineering, or equivalent experiencePreferred QualificationsExperience testing AI-powered applications, including:LLM-integrated systemsPrompt-driven workflowsAgent-based architecturesFamiliarity with AI-assisted testing tools and automated test generationExposure to security testing practices in collaboration with dedicated security teamsExperience working in regulated environments (e.g., legal, financial services)Knowledge of Azure DevOps, including pipeline configuration and automated test integration