Skip to main content
Loading
loadingbar
Loading, Please wait..!!

Research Evaluation Specialist - Remote

  • Research Evaluation Specialist - Remote job type Posted on: Nov 11, 2026
  • Research Evaluation Specialist - Remote experience level YO AI Labs
  • Research Evaluation Specialist - Remote employment type New York, New York
  • Research Evaluation Specialist - Remote employment type Remote
  • Research Evaluation Specialist - Remote salary Full-time

Point Apply Here APPLY LATER

Curious about compensation?

Explore the historical salary trends, average pay, and estimated compensation for Research Evaluation Specialist - Remote roles in New York.

View Salary Guide →

Job Title :

Research Evaluation Specialist - Remote

Job Type :

Full-Time

Job Location :

New York New York United States

Remote :

Yes

Jobcon Logo Job Description :

Job Description

Job Description
Job Description: Research Evaluation Specialist (PhD, Master's, or Researcher)

Role Type: Contractor (Approximately 15 hours per week)
Location: Remote, Worldwide

Job Summary

We are seeking experienced Research Evaluation Specialists with advanced academic qualifications or substantial research experience to contribute to the development of rigorous evaluation materials for advanced AI models.

In this role, you will leverage your subject-matter expertise, research skills, and analytical reasoning to create challenging questions and evidence-based answers that assess AI model capabilities. You will conduct in-depth research, verify findings using authoritative sources, and refine evaluation materials to ensure accuracy, complexity, and clarity.

Prior experience in AI training, data labeling, or model evaluation is required, along with strong research expertise in your respective discipline.

Scope of Work
  1. Develop original, high-difficulty question-and-answer pairs that assess advanced AI model capabilities within your area of expertise.
  2. Research, verify, and thoroughly document answers using primary sources, scholarly literature, and authoritative references, including clear citations and supporting reasoning.
  3. Design questions that require advanced reasoning, methodological understanding, and synthesis across multiple sources while avoiding oversimplified solutions.
  4. Test questions against AI systems, identify items that do not adequately challenge the models, and iteratively increase complexity while maintaining factual accuracy.
  5. Write questions and answers with exceptional precision and clarity, eliminating ambiguity and ensuring that responses are logically defensible.
  6. Incorporate reviewer feedback and consistently follow project guidelines and quality standards.
  7. Maintain high standards of research integrity, documentation, and analytical rigor across all deliverables.
Preferred Qualifications
  1. A completed Master's or PhD, active doctoral candidacy, or equivalent research experience as a specialist, researcher, assistant professor, or professor in the relevant discipline.
  2. A demonstrated record of scholarly research, advanced subject-matter expertise, and familiarity with primary academic literature.
  3. Exceptional analytical thinking, attention to detail, and strong written communication skills in English.
  4. Experience locating, evaluating, and cross-verifying information from authoritative sources.
  5. Strong ability to work independently, manage priorities, and deliver high-quality research within a remote working environment.
  6. Expertise in developing original, challenging, and methodologically sound questions that assess deep subject-matter understanding.
  7. Required: Previous experience in AI training, data labeling, or AI model evaluation.
Compensation Structure

Compensation is output-based, with payment provided for each task that meets the project's specifications and quality requirements. The time required to complete tasks may vary depending on individual expertise and workflow.

Minimum submission requirements apply, and contributors are expected to complete a specified number of tasks each week.

Start Timeline and Availability
  • Hiring Timeline: Positions are typically filled within 48 hours.
  • Availability: Candidates should be prepared to begin promptly.
  • Project Start: Selected candidates are expected to start their first tasks within 24–48 hours of completing onboarding.
Ideal Candidate Profile

The ideal candidate combines advanced academic or research expertise with practical experience in AI training or evaluation. They can develop sophisticated evaluation questions, support answers with authoritative evidence, identify weaknesses in AI-generated responses, and deliver precise, well-documented work independently while maintaining rigorous research and quality standards.

Jobcon Logo Position Details

Posted:

Nov 11, 2026

Reference Number:

bed471cc

Employment:

Full-time

City:

New York

Job Origin:

ziprecruiter

Share this job:

  • linkedin

Jobcon Logo
A job sourcing event
In Dallas Fort Worth
Aug 19, 2017 9am-6pm
All job seekers welcome!

Research Evaluation Specialist - Remote    Apply

Click on the below icons to share this job to Linkedin, Twitter!

Job Description

Job Description
Job Description: Research Evaluation Specialist (PhD, Master's, or Researcher)

Role Type: Contractor (Approximately 15 hours per week)
Location: Remote, Worldwide

Job Summary

We are seeking experienced Research Evaluation Specialists with advanced academic qualifications or substantial research experience to contribute to the development of rigorous evaluation materials for advanced AI models.

In this role, you will leverage your subject-matter expertise, research skills, and analytical reasoning to create challenging questions and evidence-based answers that assess AI model capabilities. You will conduct in-depth research, verify findings using authoritative sources, and refine evaluation materials to ensure accuracy, complexity, and clarity.

Prior experience in AI training, data labeling, or model evaluation is required, along with strong research expertise in your respective discipline.

Scope of Work
  1. Develop original, high-difficulty question-and-answer pairs that assess advanced AI model capabilities within your area of expertise.
  2. Research, verify, and thoroughly document answers using primary sources, scholarly literature, and authoritative references, including clear citations and supporting reasoning.
  3. Design questions that require advanced reasoning, methodological understanding, and synthesis across multiple sources while avoiding oversimplified solutions.
  4. Test questions against AI systems, identify items that do not adequately challenge the models, and iteratively increase complexity while maintaining factual accuracy.
  5. Write questions and answers with exceptional precision and clarity, eliminating ambiguity and ensuring that responses are logically defensible.
  6. Incorporate reviewer feedback and consistently follow project guidelines and quality standards.
  7. Maintain high standards of research integrity, documentation, and analytical rigor across all deliverables.
Preferred Qualifications
  1. A completed Master's or PhD, active doctoral candidacy, or equivalent research experience as a specialist, researcher, assistant professor, or professor in the relevant discipline.
  2. A demonstrated record of scholarly research, advanced subject-matter expertise, and familiarity with primary academic literature.
  3. Exceptional analytical thinking, attention to detail, and strong written communication skills in English.
  4. Experience locating, evaluating, and cross-verifying information from authoritative sources.
  5. Strong ability to work independently, manage priorities, and deliver high-quality research within a remote working environment.
  6. Expertise in developing original, challenging, and methodologically sound questions that assess deep subject-matter understanding.
  7. Required: Previous experience in AI training, data labeling, or AI model evaluation.
Compensation Structure

Compensation is output-based, with payment provided for each task that meets the project's specifications and quality requirements. The time required to complete tasks may vary depending on individual expertise and workflow.

Minimum submission requirements apply, and contributors are expected to complete a specified number of tasks each week.

Start Timeline and Availability
  • Hiring Timeline: Positions are typically filled within 48 hours.
  • Availability: Candidates should be prepared to begin promptly.
  • Project Start: Selected candidates are expected to start their first tasks within 24–48 hours of completing onboarding.
Ideal Candidate Profile

The ideal candidate combines advanced academic or research expertise with practical experience in AI training or evaluation. They can develop sophisticated evaluation questions, support answers with authoritative evidence, identify weaknesses in AI-generated responses, and deliver precise, well-documented work independently while maintaining rigorous research and quality standards.

Loading
Please wait..!!