{"schemaVersion":"jobsearcher.job.v1","id":"da45c34490263231f372ff24","url":"https://jobsearcher.com/jobs/da45c34490263231f372ff24","canonicalUrl":"https://jobsearcher.com/jobs/da45c34490263231f372ff24","title":"Software Engineer, Production Inference (Distributed Inference)","description":"EngRadar / Thinking Machines LabSoftware Engineer, Production Inference (Distributed Inference)Thinking Machines LabThinking Machines Lab is the employer — EngRadar is a job radar, not a recruiter. We track this posting from their own careers page and send you straight there; we never handle applications or CVs.San Francisco Full-time Posted todayApply directly →About Thinking MachinesThe mission of Thinking Machines is to build AI that extends human will and judgment. We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication. We believe the future worth building is human, and we're hiring people who want to build it.About The RoleWe're hiring a Software Engineer to build and scale the distributed production inference systems that serve Inkling, Inkling-Small, and Tinker in production. You'll own the systems that turn trained models into fast, reliable, cost-efficient services — from request routing and batching to multi-node serving and GPU utilization at scale.This is a systems-heavy, production-first role. You'll work closely with research and infrastructure teams to translate rapidly evolving model architectures into serving systems that meet real-world latency, throughput, and reliability requirements, and you'll be on the front line when production inference systems need to scale, recover, or improve.What You'll DoDesign, build, and operate distributed infrastructure for large-scale model serving, including request routing, load balancing, batching, and multi-node coordinationOptimize inference latency and throughput in production, including work on KV cache management, continuous batching, speculative decoding, and quantizationBuild and maintain high-concurrency serving systems with strong uptime, low tail latency, and deep observabilityBenchmark, tune, and extend inference engines to support new model architectures as they move from research into productionPartner with research and infrastructure teams to translate emerging model designs into production-ready serving systemsBuild tooling for tracing, debugging, and resolving issues across the serving stack, from orchestration down to GPU kernelsParticipate in on-call rotation to support production inference systemsSkills & Qualifications3+ years of experience building and operating distributed systems in productionStrong systems programming skills in Python, C++, Rust, or similar languagesExperience with production infrastructure at scale: reliability, observability, and performance under real-world loadSolid understanding of networking, concurrency, and distributed systems fundamentalsPreferred QualificationsExperience with LLM inference engines such as vLLM, SGLang, or TensorRT-LLMFamiliarity with GPU programming (CUDA) or low-level performance optimizationExperience with model parallelism, tensor/pipeline parallelism, or other distributed inference techniquesTrack record of operating large-scale production systems with strict latency and uptime requirementsExperience with Kubernetes or similar orchestration systems for GPU workloadsContributions to open-source ML systems or inference infrastructure projectsLogisticsLocation: This role is based in San Francisco, CA.Compensation: Depending on background, skills and experience, the expected annual salary range for this position is $350,000 - $500,000 USD (placeholder — verify against current internal bands before publishing).Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.Benefits: Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.Posted by Thinking Machines Lab on their own careers page — you apply directly, no recruiter in between. View original / apply →More at Thinking Machines LabTResearch, General AgentsThinking Machines LabSan Francisco1mo agoTSoftware Engineer, SandboxingThinking Machines LabSan Francisco +118d agoTSite Reliability Engineer, Post TrainingThinking Machines LabSan Francisco +11mo agoTSoftware Engineer, InferenceThinking Machines LabSan Francisco +18d agoApply directly → Browse Full-Stack Jobs","company":"Engradar","rawCompany":"engradar","city":"Millbrae","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-10-14T07:56:15.982Z","occupations":[{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1221.00","title":"Computer and Information Research Scientists","slug":"computer-and-information-research-scientists"}],"industries":[{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Software Engineer, Production Inference (Distributed Inference)","description":"EngRadar / Thinking Machines LabSoftware Engineer, Production Inference (Distributed Inference)Thinking Machines LabThinking Machines Lab is the employer — EngRadar is a job radar, not a recruiter. We track this posting from their own careers page and send you straight there; we never handle applications or CVs.San Francisco Full-time Posted todayApply directly →About Thinking MachinesThe mission of Thinking Machines is to build AI that extends human will and judgment. We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication. We believe the future worth building is human, and we're hiring people who want to build it.About The RoleWe're hiring a Software Engineer to build and scale the distributed production inference systems that serve Inkling, Inkling-Small, and Tinker in production. You'll own the systems that turn trained models into fast, reliable, cost-efficient services — from request routing and batching to multi-node serving and GPU utilization at scale.This is a systems-heavy, production-first role. You'll work closely with research and infrastructure teams to translate rapidly evolving model architectures into serving systems that meet real-world latency, throughput, and reliability requirements, and you'll be on the front line when production inference systems need to scale, recover, or improve.What You'll DoDesign, build, and operate distributed infrastructure for large-scale model serving, including request routing, load balancing, batching, and multi-node coordinationOptimize inference latency and throughput in production, including work on KV cache management, continuous batching, speculative decoding, and quantizationBuild and maintain high-concurrency serving systems with strong uptime, low tail latency, and deep observabilityBenchmark, tune, and extend inference engines to support new model architectures as they move from research into productionPartner with research and infrastructure teams to translate emerging model designs into production-ready serving systemsBuild tooling for tracing, debugging, and resolving issues across the serving stack, from orchestration down to GPU kernelsParticipate in on-call rotation to support production inference systemsSkills & Qualifications3+ years of experience building and operating distributed systems in productionStrong systems programming skills in Python, C++, Rust, or similar languagesExperience with production infrastructure at scale: reliability, observability, and performance under real-world loadSolid understanding of networking, concurrency, and distributed systems fundamentalsPreferred QualificationsExperience with LLM inference engines such as vLLM, SGLang, or TensorRT-LLMFamiliarity with GPU programming (CUDA) or low-level performance optimizationExperience with model parallelism, tensor/pipeline parallelism, or other distributed inference techniquesTrack record of operating large-scale production systems with strict latency and uptime requirementsExperience with Kubernetes or similar orchestration systems for GPU workloadsContributions to open-source ML systems or inference infrastructure projectsLogisticsLocation: This role is based in San Francisco, CA.Compensation: Depending on background, skills and experience, the expected annual salary range for this position is $350,000 - $500,000 USD (placeholder — verify against current internal bands before publishing).Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.Benefits: Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.Posted by Thinking Machines Lab on their own careers page — you apply directly, no recruiter in between. View original / apply →More at Thinking Machines LabTResearch, General AgentsThinking Machines LabSan Francisco1mo agoTSoftware Engineer, SandboxingThinking Machines LabSan Francisco +118d agoTSite Reliability Engineer, Post TrainingThinking Machines LabSan Francisco +11mo agoTSoftware Engineer, InferenceThinking Machines LabSan Francisco +18d agoApply directly → Browse Full-Stack Jobs","datePosted":"2026-10-14T07:56:15.982Z","dateModified":"2026-10-14T07:56:15.982Z","hiringOrganization":{"@type":"Organization","name":"Engradar","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Millbrae","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"da45c34490263231f372ff24"},"url":"https://jobsearcher.com/jobs/da45c34490263231f372ff24"}}