{"schemaVersion":"jobsearcher.job.v1","id":"dc6d31555a9f7436cb9e208d","url":"https://jobsearcher.com/jobs/dc6d31555a9f7436cb9e208d","canonicalUrl":"https://jobsearcher.com/jobs/dc6d31555a9f7436cb9e208d","title":"INFERENCE ENGINEER","description":"ABOUT THE COMPANY\r\nWe're building autonomous research agents for recursive self-improvement (multi-agent systems that propose, run, and analyze machine learning experiments). We're a small team based in San Francisco, on-site\r\nABOUT THE ROLE\r\nYou build and operate the inference systems that serve our models in production. The work spans serving infrastructure, runtime optimization, and the long tail of production infrastructure that come with running real workloads.\r\nThis is an engineering role, not a research role. You'll measure, profile, debug, and ship. You'll work alongside researchers, but your job is to make their work fast and reliable in production. Real ownership, real autonomy.\r\nWHAT YOU'LL DO\r\nBuild, operate, and harden production inference systems serving large models at high throughput\r\nOwn the performance characteristics of those systems end-to-end: throughput, latency, cost-per-token, reliability under load\r\nProfile real workloads to identify bottlenecks; ship fixes that move the metric you set out to improve\r\nImplement and integrate inference optimizations from the research team (quantization, custom kernels, scheduling improvements, memory management) into production\r\nDesign observability into the inference layer: metrics, tracing, alerting that surface regressions before users notice them\r\nRun capacity planning, autoscaling, and load testing for varied workload shapes (batch, online, mixed, agentic)\r\nDiagnose and resolve production incidents; write postmortems that turn bugs into systemic fixes\r\nWHAT WE'RE LOOKING FOR\r\nSenior ML systems engineer with 3+ years building production-grade, large-scale serving infrastructure\r\nStrong distributed systems experience; you've been on-call for systems that matter\r\nPerformance profiling and optimization fluency: you read flame graphs, you are analytical and measured before you change\r\nExperience with GPU-accelerated inference at scale (multi-GPU, multi-node, batched and streaming workloads), preferably experience with AMD GPUs\r\nFluent Python; comfortable reading and writing systems-level code in at least one of the following languages: C++, CUDA, ROCm or Triton\r\nTrack record of shipping production infrastructure, preferably surfaces serving millions of requests across diverse workloads\r\nGood written communication; you can write a runbook that someone else can follow at 3am\r\nNICE TO HAVE\r\nOpen-source contributions to inference / serving frameworks\r\nExperience with mixed cloud and on-premises deployments\r\nFamiliarity with hardware-aware optimization (memory hierarchy, NCCL/RDMA, NUMA)\r\nBackground in compilers, runtimes, or accelerator software stacks\r\nTHIS ROLE IS PROBABLY NOT FOR YOU IF\r\nYou're primarily a researcher, the work here is building, not exploring\r\nYou want to focus narrowly on one component; this role spans the stack\r\nProduction responsibility (incidents, on-call, ownership of running systems) isn't appealing\r\nJ-18808-Ljbffr","company":"Makermakerai","rawCompany":"makermakerai","city":"Millbrae","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-06-25T00:40:29.158Z","occupations":[{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1221.00","title":"Computer and Information Research Scientists","slug":"computer-and-information-research-scientists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"INFERENCE ENGINEER","description":"ABOUT THE COMPANY\r\nWe're building autonomous research agents for recursive self-improvement (multi-agent systems that propose, run, and analyze machine learning experiments). We're a small team based in San Francisco, on-site\r\nABOUT THE ROLE\r\nYou build and operate the inference systems that serve our models in production. The work spans serving infrastructure, runtime optimization, and the long tail of production infrastructure that come with running real workloads.\r\nThis is an engineering role, not a research role. You'll measure, profile, debug, and ship. You'll work alongside researchers, but your job is to make their work fast and reliable in production. Real ownership, real autonomy.\r\nWHAT YOU'LL DO\r\nBuild, operate, and harden production inference systems serving large models at high throughput\r\nOwn the performance characteristics of those systems end-to-end: throughput, latency, cost-per-token, reliability under load\r\nProfile real workloads to identify bottlenecks; ship fixes that move the metric you set out to improve\r\nImplement and integrate inference optimizations from the research team (quantization, custom kernels, scheduling improvements, memory management) into production\r\nDesign observability into the inference layer: metrics, tracing, alerting that surface regressions before users notice them\r\nRun capacity planning, autoscaling, and load testing for varied workload shapes (batch, online, mixed, agentic)\r\nDiagnose and resolve production incidents; write postmortems that turn bugs into systemic fixes\r\nWHAT WE'RE LOOKING FOR\r\nSenior ML systems engineer with 3+ years building production-grade, large-scale serving infrastructure\r\nStrong distributed systems experience; you've been on-call for systems that matter\r\nPerformance profiling and optimization fluency: you read flame graphs, you are analytical and measured before you change\r\nExperience with GPU-accelerated inference at scale (multi-GPU, multi-node, batched and streaming workloads), preferably experience with AMD GPUs\r\nFluent Python; comfortable reading and writing systems-level code in at least one of the following languages: C++, CUDA, ROCm or Triton\r\nTrack record of shipping production infrastructure, preferably surfaces serving millions of requests across diverse workloads\r\nGood written communication; you can write a runbook that someone else can follow at 3am\r\nNICE TO HAVE\r\nOpen-source contributions to inference / serving frameworks\r\nExperience with mixed cloud and on-premises deployments\r\nFamiliarity with hardware-aware optimization (memory hierarchy, NCCL/RDMA, NUMA)\r\nBackground in compilers, runtimes, or accelerator software stacks\r\nTHIS ROLE IS PROBABLY NOT FOR YOU IF\r\nYou're primarily a researcher, the work here is building, not exploring\r\nYou want to focus narrowly on one component; this role spans the stack\r\nProduction responsibility (incidents, on-call, ownership of running systems) isn't appealing\r\nJ-18808-Ljbffr","datePosted":"2026-06-25T00:40:29.158Z","dateModified":"2026-06-25T00:40:29.158Z","hiringOrganization":{"@type":"Organization","name":"Makermakerai","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Millbrae","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"dc6d31555a9f7436cb9e208d"},"url":"https://jobsearcher.com/jobs/dc6d31555a9f7436cb9e208d"}}