Web Developer - USA Remote
About the Company:
DataAnnotation works with experts to train and improve leading AI models through practical evaluation tasks and expert feedback. The company offers flexible, remote, project-based opportunities for contributors with relevant technical expertise.
About the Role:
DataAnnotation is seeking experienced software developers to evaluate AI coding agents on real engineering tasks. As a contributor, you will work with advanced AI models to complete coding tasks, assess the quality of their output, and identify weaknesses that could affect real-world software development. The role focuses on testing AI-generated code against production standards, analyzing model reasoning, and uncovering issues such as unsafe implementations, misleading test results, and incorrect solutions. Your documented findings will help improve the reliability and capabilities of future coding agents.
Key Responsibilities:
Run coding sessions with advanced AI models on real engineering tasks, including bug fixes, API enhancements, and feature development.
Evaluate AI-generated code against production-level quality standards.
Identify bugs, explain failures, and assess the quality of the model's reasoning.
Red-team coding agents to uncover unsafe code, falsely reported test successes, and incorrect solutions.
Document technical findings clearly to support AI model training and improvement.
Required Qualifications:
Professional software development experience shipping production code or substantial open-source experience.
Strong practical coding skills and the ability to evaluate software quality.
Clear written English and the ability to explain technical findings accurately.
Ability to identify bugs, assess implementation quality, and recognize potential security or reliability issues.
Education Requirements:
A formal degree is not required. Practical engineering ability and demonstrated experience are the primary considerations.
Compensation:
Potential earnings of up to $150+ per hour, depending on task difficulty and specialization.
The company states that many contributors earn an additional $100,000–$200,000 annually, while some make this work their full-time income. Actual earnings may vary based on available projects, hours worked, and individual expertise.
Location and Work Arrangement
Work Arrangement: RemoteWork Type: Flexible, project-based
Project Availability: Work is offered on an ongoing, project-based basis.