Data Annotators

Responsibilities

● Ground Truth & Subject Matter Expertise: ○ Act as the primary SME and definitive source of truth for edge cases, label ambiguities, and guideline interpretations across annotation workflows.

○ Triage, resolve, and log vendor questions promptly to keep external labeling operations moving smoothly and accurately.

● Quality Assurance & Auditing: ○ Run consistent quality audits and spot-checks on vendor deliverables against established benchmarks.

○ Track quality trends, error patterns, and deliver constructive, clear feedback to external vendor teams to drive continuous improvement.

● Documentation & Guideline Management: ○ Write, update, and organize clear, visual annotation guidelines, SOPs, and edge-case examples.

○ Manage and maintain all external-facing documentation to ensure vendors always work from up-to-date instructions.

● Tooling, Queues & Operational Support: ○ Configure, test, and maintain annotation tools, including setting up new labeling jobs, managing queues, and adjusting routing rules.

○ Pull regular quality and throughput reports to provide visibility to internal engineering stakeholders.

○ Provide light engineering support for workflow automation (e.g., generating basic scripts using AI assisted tools to clean data, run checks, or automate routine tasks).

Qualifications

● Required Experience: ○ Experience in data annotation, quality checking, or curation within data-driven or AI/ML operations.

○ Strong eye for detail and proven ability to make sound judgment calls on edge cases and ambiguous labeling tasks.

○ Clear, concise written and verbal communication skills for answering questions and explaining standards to external vendors.

○ Operational experience setting up annotation jobs, managing workflows/queues, and compiling status or quality reports.

○ 1-3 years of relevant experience, including academic research, internships, or professional roles.

BlackTree Technical Group is a full-service staffing and recruitment firm. Our team consists of an experienced group of Senior Technical Recruiters and Account Managers with over 40 years of industry experience. Our firm prides itself on our technical knowledge, innovative sourcing strategies, and most importantly, a long-standing record of proven results for both clients and job-seekers.