Skip to main content
Loading
loadingbar
Loading, Please wait..!!

SDET AI Lead/Senior/Junior

  • SDET AI Lead/Senior/Junior job type Posted on: Nov 18, 2026
  • SDET AI Lead/Senior/Junior experience level Apptad Inc
  • SDET AI Lead/Senior/Junior employment type New York, New York
  • SDET AI Lead/Senior/Junior employment type Onsite
  • SDET AI Lead/Senior/Junior salary Full-time

Point Apply Here APPLY LATER

Curious about compensation?

Explore the historical salary trends, average pay, and estimated compensation for SDET AI Lead/Senior/Junior roles in New York.

View Salary Guide →

Job Title :

SDET AI Lead/Senior/Junior

Job Type :

Full-Time

Job Location :

New York New York United States

Remote :

No

Jobcon Logo Job Description :

Job Role: SDET AI Lead/Senior/Junior

Job Location: NY/Alpharetta, GA- Day 1 onsite (3 days work from Office, 2 days' work from home)

Job Type: Contract

Lead role- New York

Senior Role- Alpharetta, GA

Junior Role- Alpharetta, GA

About the Role

We are seeking a Lead Software Development Engineer in Test (SDET) to lead our quality engineering strategy and build scalable, reliable, and intelligent test automation solutions. This is a hands-on leadership role combining software engineering, test architecture, mentoring, CI/CD, and artificial intelligence.

The ideal candidate has strong expertise in Python, Playwright, pytest, API testing, automation frameworks, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and AI evaluation. You will collaborate with software engineers, product managers, DevOps teams, data scientists, and AI engineers to improve quality throughout the product lifecycle.

Key Responsibilities
  • Lead the design, development, and maintenance of automation frameworks using Python, Playwright, and pytest.
  • Define test strategies covering UI, API, integration, end-to-end, regression, smoke, contract, exploratory, and non-functional testing.
  • Create scalable, maintainable, and reusable test utilities, fixtures, mocks, test data, and service virtualization.
  • Integrate automated testing into CI/CD pipelines using GitHub Actions, GitLab CI, Jenkins, or Azure DevOps.
  • Test web applications, REST and GraphQL APIs, microservices, distributed systems, databases, and event-driven architectures.
  • Review requirements, user stories, prompts, retrieval workflows, and technical designs to identify risks and define test coverage.
  • Investigate test failures, reduce flaky tests, identify root causes, and collaborate with engineering teams to resolve defects.
  • Establish coding standards, quality gates, test reporting, dashboards, metrics, and automation best practices.
  • Mentor SDETs, QA engineers, and developers in software quality, automation, and AI testing.
  • Evaluate and introduce AI-assisted testing, including test generation, intelligent test selection, failure analysis, defect prediction, and test maintenance.
  • Design testing strategies for LLM applications, including prompt validation, response quality, hallucination detection, toxicity, bias, safety, latency, cost, consistency, and reliability.
  • Test RAG systems, including document ingestion, OCR, preprocessing, chunking, metadata, embeddings, vector databases, retrieval, context assembly, grounded responses, citations, permissions, stale data, and retrieval failures.
  • Build automated evaluation frameworks using benchmark datasets, expected responses, deterministic assertions, human feedback, and model-based evaluation.
  • Promote the secure, responsible, explainable, and maintainable use of AI in testing.

Required Qualifications
  • Lead-level experience in software testing, quality engineering, or SDET roles, with the ability to own strategy and influence engineering practices.
  • Strong hands-on automation development skills using Python, Playwright, and pytest.
  • Experience designing, implementing, and maintaining automation frameworks at scale.
  • Strong knowledge of UI, API, integration, end-to-end, regression, contract, functional, and non-functional testing.
  • Experience testing web applications, APIs, microservices, and distributed systems.
  • Practical experience testing LLM-powered applications and RAG architectures.
  • Understanding of embeddings, semantic search, vector databases, context retrieval, prompt engineering, and response generation.
  • Experience creating AI test datasets, evaluation suites, prompts, expected outputs, edge cases, and quality criteria.
  • Knowledge of LLM and RAG metrics, including correctness, relevance, faithfulness, groundedness, context precision, context recall, completeness, safety, and toxicity.
  • Experience with Git, CI/CD, logging, debugging, test reporting, and Agile development.
  • Strong leadership, mentoring, communication, analytical, and problem-solving skills.

Desired Qualifications
  • Experience with Playwright tracing, screenshots, video capture, parallel execution, fixtures, and test sharding.
  • Experience with requests, Postman, Allure, JUnit, and comparable testing tools.
  • Familiarity with Ragas, DeepEval, promptfoo, LangSmith, or similar LLM evaluation frameworks.
  • Experience with OpenAI, Azure OpenAI, Anthropic, Google Vertex AI, or Amazon Bedrock.
  • Experience with Pinecone, Weaviate, Milvus, Qdrant, Chroma, pgvector, or other vector databases.
  • Knowledge of Docker, Kubernetes, AWS, Azure, GCP, SQL, NoSQL, observability, performance, accessibility, and security testing.

Nice-to-Have
  • Experience leading a quality engineering or SDET team.
  • Experience testing conversational agents, AI copilots, autonomous agents, multi-turn workflows, and tool-calling systems.
  • Familiarity with prompt injection, jailbreaks, data leakage, unauthorized retrieval, insecure output handling, and adversarial testing.
  • Experience testing machine learning models, production RAG platforms, or regulated applications.
  • Knowledge of responsible AI, privacy, security, governance, and compliance.
  • Contributions to open-source testing or AI evaluation projects.

Jobcon Logo Position Details

Posted:

Nov 18, 2026

Reference Number:

34403-5039

Employment:

Full-time

City:

New York

Job Origin:

CIEPAL_ORGANIC_FEED

Share this job:

  • linkedin

Jobcon Logo
A job sourcing event
In Dallas Fort Worth
Aug 19, 2017 9am-6pm
All job seekers welcome!

SDET AI Lead/Senior/Junior    Apply

Click on the below icons to share this job to Linkedin, Twitter!

Job Role: SDET AI Lead/Senior/Junior

Job Location: NY/Alpharetta, GA- Day 1 onsite (3 days work from Office, 2 days' work from home)

Job Type: Contract

Lead role- New York

Senior Role- Alpharetta, GA

Junior Role- Alpharetta, GA

About the Role

We are seeking a Lead Software Development Engineer in Test (SDET) to lead our quality engineering strategy and build scalable, reliable, and intelligent test automation solutions. This is a hands-on leadership role combining software engineering, test architecture, mentoring, CI/CD, and artificial intelligence.

The ideal candidate has strong expertise in Python, Playwright, pytest, API testing, automation frameworks, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and AI evaluation. You will collaborate with software engineers, product managers, DevOps teams, data scientists, and AI engineers to improve quality throughout the product lifecycle.

Key Responsibilities
  • Lead the design, development, and maintenance of automation frameworks using Python, Playwright, and pytest.
  • Define test strategies covering UI, API, integration, end-to-end, regression, smoke, contract, exploratory, and non-functional testing.
  • Create scalable, maintainable, and reusable test utilities, fixtures, mocks, test data, and service virtualization.
  • Integrate automated testing into CI/CD pipelines using GitHub Actions, GitLab CI, Jenkins, or Azure DevOps.
  • Test web applications, REST and GraphQL APIs, microservices, distributed systems, databases, and event-driven architectures.
  • Review requirements, user stories, prompts, retrieval workflows, and technical designs to identify risks and define test coverage.
  • Investigate test failures, reduce flaky tests, identify root causes, and collaborate with engineering teams to resolve defects.
  • Establish coding standards, quality gates, test reporting, dashboards, metrics, and automation best practices.
  • Mentor SDETs, QA engineers, and developers in software quality, automation, and AI testing.
  • Evaluate and introduce AI-assisted testing, including test generation, intelligent test selection, failure analysis, defect prediction, and test maintenance.
  • Design testing strategies for LLM applications, including prompt validation, response quality, hallucination detection, toxicity, bias, safety, latency, cost, consistency, and reliability.
  • Test RAG systems, including document ingestion, OCR, preprocessing, chunking, metadata, embeddings, vector databases, retrieval, context assembly, grounded responses, citations, permissions, stale data, and retrieval failures.
  • Build automated evaluation frameworks using benchmark datasets, expected responses, deterministic assertions, human feedback, and model-based evaluation.
  • Promote the secure, responsible, explainable, and maintainable use of AI in testing.

Required Qualifications
  • Lead-level experience in software testing, quality engineering, or SDET roles, with the ability to own strategy and influence engineering practices.
  • Strong hands-on automation development skills using Python, Playwright, and pytest.
  • Experience designing, implementing, and maintaining automation frameworks at scale.
  • Strong knowledge of UI, API, integration, end-to-end, regression, contract, functional, and non-functional testing.
  • Experience testing web applications, APIs, microservices, and distributed systems.
  • Practical experience testing LLM-powered applications and RAG architectures.
  • Understanding of embeddings, semantic search, vector databases, context retrieval, prompt engineering, and response generation.
  • Experience creating AI test datasets, evaluation suites, prompts, expected outputs, edge cases, and quality criteria.
  • Knowledge of LLM and RAG metrics, including correctness, relevance, faithfulness, groundedness, context precision, context recall, completeness, safety, and toxicity.
  • Experience with Git, CI/CD, logging, debugging, test reporting, and Agile development.
  • Strong leadership, mentoring, communication, analytical, and problem-solving skills.

Desired Qualifications
  • Experience with Playwright tracing, screenshots, video capture, parallel execution, fixtures, and test sharding.
  • Experience with requests, Postman, Allure, JUnit, and comparable testing tools.
  • Familiarity with Ragas, DeepEval, promptfoo, LangSmith, or similar LLM evaluation frameworks.
  • Experience with OpenAI, Azure OpenAI, Anthropic, Google Vertex AI, or Amazon Bedrock.
  • Experience with Pinecone, Weaviate, Milvus, Qdrant, Chroma, pgvector, or other vector databases.
  • Knowledge of Docker, Kubernetes, AWS, Azure, GCP, SQL, NoSQL, observability, performance, accessibility, and security testing.

Nice-to-Have
  • Experience leading a quality engineering or SDET team.
  • Experience testing conversational agents, AI copilots, autonomous agents, multi-turn workflows, and tool-calling systems.
  • Familiarity with prompt injection, jailbreaks, data leakage, unauthorized retrieval, insecure output handling, and adversarial testing.
  • Experience testing machine learning models, production RAG platforms, or regulated applications.
  • Knowledge of responsible AI, privacy, security, governance, and compliance.
  • Contributions to open-source testing or AI evaluation projects.

Loading
Please wait..!!