JOBSEARCHER

Jr. Annotation Data Engineer

AceStackDenver, COL5 SeniorOctober 24th, 2026
Role: Jr. Annotation Data Engineer
Location: RemoteFTE Only$90kExp: 2-5 Years of ExpJob Description Must Have Technical/Functional Skills:
  • Demonstrated experience in annotation quality, machine learning data operations, quality engineering, or data quality supporting physical AI, robotics, computer vision, or machine learning programs.
  • Demonstrated experience developing quality metrics and performance measurement systems.
  • Demonstrated experience working with large-scale annotation programs.
  • Demonstrated experience evaluating generated captions, reasoning outputs, or foundation model training datasets.
  • Experience defining annotation instructions, sampling plans, acceptance criteria, defect taxonomies, and audit processes.
  • Ability to analyze quality trends, identify root causes, and coordinate corrective actions.
  • Working knowledge of SQL, Python, reporting, or data analysis tools used for quality measurement.
Roles & Responsibilities:
  • Define and maintain annotation quality standards, guidelines, defect categories, and acceptance criteria.
  • Develop metrics and reporting for accuracy, consistency, completeness, rework, throughput, and reviewer agreement.
  • Design sampling, audit, escalation, and root-cause analysis processes for large-scale annotation programs.
  • Evaluate generated captions, reasoning outputs, and foundation model training datasets against defined quality criteria.
  • Analyze quality trends and partner with annotation operations, engineering, and AI teams on corrective actions.
  • Validate dataset releases and maintain traceable quality records, issue logs, and improvement plans.
  • Support calibration sessions, reviewer alignment, and continuous improvement of annotation workflows.