Loading, Please wait..!!
Quality Assurance Tester
-
Posted on: Nov 02, 2026
-
New York, New York
-
Onsite
-
Full-time
Curious about compensation?
Explore the historical salary trends, average pay, and estimated compensation for Quality Assurance Tester roles in New York.
View Salary Guide →
Over 12 millions jobs on Snaprecruit.
Find your match !
VIEW SIMILAR JOBS →
Find your match !
Job Title :
Quality Assurance Tester
Job Type :
Full-Time
Job Location :
New York New York United States
Remote :
No
Job Description :
Global Markets GenAI QA TesterLocation: New York, NY (2BP – onsite interview required)Duration: 18 monthsWork Arrangement: W2 contract onlyExperience: 5–10 years**W2 ONLY NO SPONSORSHIP ALLOWED**Job DescriptionBank of America is seeking a QA Tester with experience in Generative AI, test automation, and Python-based applications to support its Global Markets GenAI platform. This role will focus on validating LLM outputs, evaluating model quality, and testing the underlying AI infrastructure to ensure reliability, accuracy, and performance.The platform, Orchestra, enables secure access to external large language models (LLMs) while enforcing user authorization and controlled integrations. The ideal candidate has hands-on experience applying AI to testing, building automation solutions, and evaluating LLM responses.Key ResponsibilitiesDevelop and execute test strategies, test cases, and automated test suites for AI-powered applications and infrastructure.Evaluate LLM outputs and establish methods to measure response quality, accuracy, and reliability.Apply LLM-as-a-Judge techniques to assess model responses and identify quality issues.Develop or leverage AI agents to automate testing workflows and improve testing efficiency.Automate Python application testing using PyTest and Playwright.Use tools such as LangChain and LangSmith to inspect, trace, and evaluate model outputs.Perform database testing and data validation across MongoDB and other data platforms.Manage defects, report testing progress, and escalate risks and issues.Collaborate with developers, data scientists, product owners, and platform teams to support releases and continuous improvement.Ensure testing practices comply with enterprise security, data privacy, and governance requirements.Required Qualifications5–10 years of experience in QA, software testing, test automation, or a related technical role.Hands-on experience testing Python applications using PyTest.Experience with Playwright for test automation.Experience with MongoDB or other NoSQL databases.Practical experience using LLMs in testing, including LLM-as-a-Judge techniques or AI-agent-based test automation.Understanding of how to evaluate LLM responses and measure model quality.Familiarity with AI testing tools and frameworks such as LangChain and/or LangSmith.Strong analytical, troubleshooting, communication, and problem-solving skills.Preferred QualificationsExperience testing AI/ML applications or Generative AI platforms.Exposure to SQL, NoSQL databases, and data validation.Familiarity with Azure, Google Cloud Platform (GCP), or AWS.Experience working in Agile environments using Jira and DevOps practices.Financial services or banking industry experience.Experience collaborating with cross-functional and onshore/offshore teams.
Similar Jobs You May Be Interested In
Quality Assurance Tester 2 - TS clearance required - Washington DC area
Confidential Employer
Arlington, Virginia
Quality Assurance Tester 1 - TS clearance required to apply - Washington DC area
Confidential Employer
Arlington, Virginia
Software Quality Assurance Analyst and Tester (Planning & Reporting V&V)-Junior
Confidential Employer
Patuxent, Maryland
QA Tester / Quality Assurance Analyst - Contract (Raleigh, NC)
Confidential Employer
Raleigh, North Carolina
QA Tester | Quality Assurance | VR Gaming / Simulation | Remote options (Dallas is Ideal) | $60,000
Confidential Employer
Dallas, Texas
Senior Quality Assurance Tester
Confidential Employer
Frederick County, Maryland
Quality Assurance Tester
Confidential Employer
Redwood City, California
Quality Assurance Qa Tester
Confidential Employer
San Mateo, California
Software Quality Assurance Tester (No Remote/No Sponsorship)
Confidential Employer
Stamford, Connecticut
Quality Assurance Tester
Confidential Employer
Nashville, Tennessee
Software Quality Assurance Tester (No Remote/No Sponsorship)
Confidential Employer
East Haven, Connecticut
Software Quality Assurance Tester (No Remote/No Sponsorship)
Confidential Employer
Waterbury, Connecticut
Position Details
Posted:
Nov 02, 2026
Reference Number:
10440_4483576482
Employment:
Full-time
City:
New York
Job Origin:
APPCAST_CPC
Share this job:
Please wait..!!

