Machine Learning Evaluation Specialist

Machine Learning Evaluation Specialist (AI Training)About The RoleWhat if your years of hard-earned research expertise could directly shape the future of AI? We're looking for domain experts with deep machine learning knowledge to design evaluation challenges that push state-of-the-art AI systems to their limits.This isn't reviewing chatbot responses or rating simple outputs. You'll be crafting complex, nuanced problems that only a true domain expert could construct — and that only the most capable AI systems could hope to solve.Organization: AlignerrType: Hourly ContractLocation: RemoteCommitment: 10–40 hours/weekWhat You'll DoDesign original, expert-level machine learning problems grounded in your area of specializationBuild evaluation tasks that go far beyond standard ML pipelines — requiring genuine, deep domain knowledgeDraw from your own research experience to craft challenges that rigorously stress-test highly capable LLMsDefine precise problem statements, evaluation criteria, and gold-standard solutionsAssess AI-generated solutions for correctness, creativity, and methodological soundnessDocument problem difficulty levels, required knowledge, and expected failure modesCollaborate asynchronously with a global team of researchers and engineersWho You AreYou hold a graduate-level degree (MS or PhD preferred) in a scientific or technical field that intersects with machine learningYou have strong working knowledge of ML methods — model selection, feature engineering, evaluation metrics, and pipeline designYou're deeply familiar with active, open research problems in your fieldYou can identify precisely where general ML knowledge breaks down and specialized domain insight becomes essentialYou have experience conducting or publishing original research (highly valued)You communicate complex ideas clearly and precisely in writingYou're self-motivated and thrive working independently on intellectually demanding tasksExample Domains (Not Exhaustive)Computational biology, genomics, or bioinformaticsClimate science and environmental modelingMedical imaging and healthcare MLMaterials science and computational chemistryAstrophysics and signal processingNatural language processing for low-resource or specialized corporaRobotics, control theory, or reinforcement learning in complex environmentsFinancial modeling and quantitative analysisWhy Join UsTop-tier compensation — among the highest rates available for contract research workWork at the frontier — your contributions directly influence how next-generation AI models are measured and improvedUse your expertise meaningfully — this role is built for specialists, not generalistsFull autonomy and flexibility — set your own schedule and work from anywhereGlobal collaboration — connect asynchronously with leading researchers and engineers worldwideOngoing opportunity — strong contributors are considered for contract extensions and deeper research involvementBuild your profile — establish yourself as a recognized contributor to cutting-edge AI development and safety research