{"schemaVersion":"jobsearcher.job.v1","id":"d4dc03caeed242c82b65eb21","url":"https://jobsearcher.com/jobs/d4dc03caeed242c82b65eb21","canonicalUrl":"https://jobsearcher.com/jobs/d4dc03caeed242c82b65eb21","title":"Machine Learning Expert, AI Agent Evaluation","description":"Put your machine learning expertise to work at the frontier of AI.Bring the expertise you know best and help train AI agents that do real technical work, on your own schedule, from anywhere. You'll turn problems from your own research or professional practice into hands-on challenges that an AI agent has to solve at the command line. You'll find exactly where it breaks and set the expert standard it's measured against. Workada connects skilled contributors with meaningful AI work, and your judgment is what makes that work reliable. We're building a community of machine learning experts, both PhDs and seasoned practitioners, to test how far today's AI agents can go on real problems in machine learning. We'd like you to be part of it.What you'll doYou'll take problems from your own work and turn each one into a self-contained task that an AI agent has to solve in a Linux terminal. For every task, you'll do four things:Write a clear brief.Set up the environment with the files and tools the agent needs.Build a reference solution.Write automated checks on the final result.The strongest tasks are the ones today's best agents still get wrong. Your firsthand knowledge of where real machine learning work goes wrong is what makes them valuable. That might be a data loader that leaks the test split, a learning-rate schedule that diverges after warmup, an inference server that degrades under batching, or a GPU kernel with an off-by-one in its tiling.Specialties we're looking forIf your background is in machine learning, there's likely a place for you. We're building depth across:Training: pretraining, fine-tuning, distributed training, and optimizationInference: serving, quantization, batching, and latencyEvaluation: benchmarks, metrics, and evaluation harnessesKernels: CUDA, Triton, and performance engineeringWho we're looking forAcademics and practitioners are equally welcome. We're after your machine learning expertise and your comfort working in a terminal.RequirementsYou hold, or are completing, a PhD in machine learning, computer science, statistics, or a closely related field, or you have 5+ years of professional experience in building, training, or serving ML systems.You've done technical or computational work in your field and are comfortable in a Linux terminal: shell commands, scripting in Python or Bash, and installing packages.You've used Docker, or can pick it up quickly. We share templates and worked examples.You can draw on your own research, projects, code, or files as source material for your tasks.You have a precise, adversarial eye and enjoy finding the exact point where a workflow breaks, then explaining it cleanly.You're honest about uncertainty and write instructions that leave no room for misreading.You're based in the US or Canada.Experience with PyTorch, JAX, Hugging Face, vLLM, Triton, or CUDA is a plus.Materials you can useAlmost any part of your technical work can seed a task. For example:Scripts, code, and pipelines from your research or practiceTraining code, configs, and model checkpointsEvaluation scripts, benchmark results, and profiler tracesDatasets and the processing behind themBugs, edge cases, and failure modes you've had to debug yourselfWhat we offer$100 to $150 per hour, based on your field and experienceWeekly payouts through StripeRemote work from anywhere in the US or CanadaA flexible scheduleReal people to work with: a 30-minute interview with someone in your field, and a Lead you can reach when questions come upWe're building this group now for projects starting in the coming weeks. If you're a match, we'll share the start date for your field as soon as we have it. We won't ask you to hold time before work is ready.Ready to apply? Join a community of machine learning experts setting the standard for AI agents.Workada is an Equal Opportunity Employer. We welcome applicants from all backgrounds.","company":"Workada","rawCompany":"workada","city":"Denver","state":"CO","isRemote":false,"isActive":false,"createdAt":"2026-11-06T07:16:59.134Z","occupations":[{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"},{"code":"15-1221.00","title":"Computer and Information Research Scientists","slug":"computer-and-information-research-scientists"}],"industries":[{"code":"541690","title":"Other Scientific and Technical Consulting Services","slug":"other-scientific-and-technical-consulting-services"},{"code":"541990","title":"All Other Professional, Scientific, and Technical Services","slug":"all-other-professional-scientific-and-technical-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Machine Learning Expert, AI Agent Evaluation","description":"Put your machine learning expertise to work at the frontier of AI.Bring the expertise you know best and help train AI agents that do real technical work, on your own schedule, from anywhere. You'll turn problems from your own research or professional practice into hands-on challenges that an AI agent has to solve at the command line. You'll find exactly where it breaks and set the expert standard it's measured against. Workada connects skilled contributors with meaningful AI work, and your judgment is what makes that work reliable. We're building a community of machine learning experts, both PhDs and seasoned practitioners, to test how far today's AI agents can go on real problems in machine learning. We'd like you to be part of it.What you'll doYou'll take problems from your own work and turn each one into a self-contained task that an AI agent has to solve in a Linux terminal. For every task, you'll do four things:Write a clear brief.Set up the environment with the files and tools the agent needs.Build a reference solution.Write automated checks on the final result.The strongest tasks are the ones today's best agents still get wrong. Your firsthand knowledge of where real machine learning work goes wrong is what makes them valuable. That might be a data loader that leaks the test split, a learning-rate schedule that diverges after warmup, an inference server that degrades under batching, or a GPU kernel with an off-by-one in its tiling.Specialties we're looking forIf your background is in machine learning, there's likely a place for you. We're building depth across:Training: pretraining, fine-tuning, distributed training, and optimizationInference: serving, quantization, batching, and latencyEvaluation: benchmarks, metrics, and evaluation harnessesKernels: CUDA, Triton, and performance engineeringWho we're looking forAcademics and practitioners are equally welcome. We're after your machine learning expertise and your comfort working in a terminal.RequirementsYou hold, or are completing, a PhD in machine learning, computer science, statistics, or a closely related field, or you have 5+ years of professional experience in building, training, or serving ML systems.You've done technical or computational work in your field and are comfortable in a Linux terminal: shell commands, scripting in Python or Bash, and installing packages.You've used Docker, or can pick it up quickly. We share templates and worked examples.You can draw on your own research, projects, code, or files as source material for your tasks.You have a precise, adversarial eye and enjoy finding the exact point where a workflow breaks, then explaining it cleanly.You're honest about uncertainty and write instructions that leave no room for misreading.You're based in the US or Canada.Experience with PyTorch, JAX, Hugging Face, vLLM, Triton, or CUDA is a plus.Materials you can useAlmost any part of your technical work can seed a task. For example:Scripts, code, and pipelines from your research or practiceTraining code, configs, and model checkpointsEvaluation scripts, benchmark results, and profiler tracesDatasets and the processing behind themBugs, edge cases, and failure modes you've had to debug yourselfWhat we offer$100 to $150 per hour, based on your field and experienceWeekly payouts through StripeRemote work from anywhere in the US or CanadaA flexible scheduleReal people to work with: a 30-minute interview with someone in your field, and a Lead you can reach when questions come upWe're building this group now for projects starting in the coming weeks. If you're a match, we'll share the start date for your field as soon as we have it. We won't ask you to hold time before work is ready.Ready to apply? Join a community of machine learning experts setting the standard for AI agents.Workada is an Equal Opportunity Employer. We welcome applicants from all backgrounds.","datePosted":"2026-11-06T07:16:59.134Z","dateModified":"2026-11-06T07:16:59.134Z","hiringOrganization":{"@type":"Organization","name":"Workada","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Denver","addressRegion":"CO","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"d4dc03caeed242c82b65eb21"},"url":"https://jobsearcher.com/jobs/d4dc03caeed242c82b65eb21"}}