{"schemaVersion":"jobsearcher.job.v1","id":"82900e48c0e87f936d4b9a8d","url":"https://jobsearcher.com/jobs/82900e48c0e87f936d4b9a8d","canonicalUrl":"https://jobsearcher.com/jobs/82900e48c0e87f936d4b9a8d","title":"Machine Learning Evaluation Specialist","description":"About The RoleWhat if your years of hard-won research expertise could directly shape the future of AI? We're looking for machine learning domain experts to design expert-level evaluation challenges that push state-of-the-art AI systems to their limits. Your work won't sit in a drawer — it will directly influence how the next generation of AI models are measured, benchmarked, and improved.Organization: AlignerrType: Hourly ContractLocation: RemoteCommitment: 10–40 hours/weekWhat You'll DoDesign original, expert-level ML problems rooted in your specialized domain of expertiseCraft evaluation tasks that require advanced domain knowledge well beyond standard ML pipelinesDraw from your own research experience to create problems that challenge the most capable AI systems availableDefine clear problem statements, evaluation criteria, and gold-standard solutionsAssess AI-generated ML solutions for correctness, creativity, and methodological rigorDocument problem difficulty, required domain knowledge, and expected failure modesCollaborate asynchronously with a global team of researchers and engineersWho You AreGraduate-level expertise (MS or PhD preferred) in a scientific or technical domain that intersects with machine learningStrong working knowledge of ML methods — model selection, feature engineering, evaluation metrics, and pipeline designDeep familiarity with active, open research problems in your fieldAbility to identify precisely where general ML knowledge falls short and specialized domain insight becomes criticalExperience publishing or conducting original research is highly valuedExcellent written communication — able to articulate complex, nuanced problems clearly and preciselySelf-motivated and comfortable working independently on intellectually demanding tasksExample Domains (Not Exhaustive)Computational biology, genomics, or bioinformaticsClimate science and environmental modelingMedical imaging and healthcare MLMaterials science and computational chemistryAstrophysics and signal processingNatural language processing for low-resource or specialized corporaRobotics, control theory, or reinforcement learning in complex environmentsFinancial modeling and quantitative analysisWhy Join UsWork at the frontier — contribute directly to AI evaluation and safety research at the cutting edge of the fieldYour expertise finally gets its due — the niche, deep knowledge you've spent years building is exactly what this role demandsCollaborate with top minds — work asynchronously alongside researchers and engineers from leading AI labs around the worldFull autonomy and flexibility — set your own hours and work entirely on your own schedule, from anywhereHigh-impact, meaningful work — your contributions shape how the world's most advanced AI systems are tested and improvedOngoing opportunity — strong performers are considered for contract extensions and deeper research involvement","company":"Alignerr","rawCompany":"alignerr","city":"Denver","state":"CO","isRemote":false,"isActive":false,"createdAt":"2026-08-14T15:17:47.037Z","occupations":[{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"},{"code":"15-1221.00","title":"Computer and Information Research Scientists","slug":"computer-and-information-research-scientists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"}],"industries":[{"code":"541690","title":"Other Scientific and Technical Consulting Services","slug":"other-scientific-and-technical-consulting-services"},{"code":"541990","title":"All Other Professional, Scientific, and Technical Services","slug":"all-other-professional-scientific-and-technical-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Machine Learning Evaluation Specialist","description":"About The RoleWhat if your years of hard-won research expertise could directly shape the future of AI? We're looking for machine learning domain experts to design expert-level evaluation challenges that push state-of-the-art AI systems to their limits. Your work won't sit in a drawer — it will directly influence how the next generation of AI models are measured, benchmarked, and improved.Organization: AlignerrType: Hourly ContractLocation: RemoteCommitment: 10–40 hours/weekWhat You'll DoDesign original, expert-level ML problems rooted in your specialized domain of expertiseCraft evaluation tasks that require advanced domain knowledge well beyond standard ML pipelinesDraw from your own research experience to create problems that challenge the most capable AI systems availableDefine clear problem statements, evaluation criteria, and gold-standard solutionsAssess AI-generated ML solutions for correctness, creativity, and methodological rigorDocument problem difficulty, required domain knowledge, and expected failure modesCollaborate asynchronously with a global team of researchers and engineersWho You AreGraduate-level expertise (MS or PhD preferred) in a scientific or technical domain that intersects with machine learningStrong working knowledge of ML methods — model selection, feature engineering, evaluation metrics, and pipeline designDeep familiarity with active, open research problems in your fieldAbility to identify precisely where general ML knowledge falls short and specialized domain insight becomes criticalExperience publishing or conducting original research is highly valuedExcellent written communication — able to articulate complex, nuanced problems clearly and preciselySelf-motivated and comfortable working independently on intellectually demanding tasksExample Domains (Not Exhaustive)Computational biology, genomics, or bioinformaticsClimate science and environmental modelingMedical imaging and healthcare MLMaterials science and computational chemistryAstrophysics and signal processingNatural language processing for low-resource or specialized corporaRobotics, control theory, or reinforcement learning in complex environmentsFinancial modeling and quantitative analysisWhy Join UsWork at the frontier — contribute directly to AI evaluation and safety research at the cutting edge of the fieldYour expertise finally gets its due — the niche, deep knowledge you've spent years building is exactly what this role demandsCollaborate with top minds — work asynchronously alongside researchers and engineers from leading AI labs around the worldFull autonomy and flexibility — set your own hours and work entirely on your own schedule, from anywhereHigh-impact, meaningful work — your contributions shape how the world's most advanced AI systems are tested and improvedOngoing opportunity — strong performers are considered for contract extensions and deeper research involvement","datePosted":"2026-08-14T15:17:47.037Z","dateModified":"2026-08-14T15:17:47.037Z","hiringOrganization":{"@type":"Organization","name":"Alignerr","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Denver","addressRegion":"CO","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"82900e48c0e87f936d4b9a8d"},"url":"https://jobsearcher.com/jobs/82900e48c0e87f936d4b9a8d"}}