{"schemaVersion":"jobsearcher.job.v1","id":"1b9e574ba9929fa19e3d06f6","url":"https://jobsearcher.com/jobs/1b9e574ba9929fa19e3d06f6","canonicalUrl":"https://jobsearcher.com/jobs/1b9e574ba9929fa19e3d06f6","title":"Inference Software Engineer","description":"About the CompanySeries A model-serving startup, ~35 people, $40M raised · San Francisco · Hybrid, 3 days · $180,000 – $280,000 + equity. They serve open-weight models for companies who can't send data to a third-party API. Their customers care about two numbers: p99 latency and cost per million tokens. This role owns both.About the RoleThe team runs a fork of vLLM across an H100 fleet. The last engineer to join cut p99 by 40% on the largest workload by reworking the scheduler. That’s the standard.ResponsibilitiesKernel-level optimisation — CUDA and Triton, custom ops, fused attentionContinuous batching, KV-cache strategy and speculative decoding, tuned per workload rather than globallyQuantisation trade-offs where the customer notices quality loss before you doMulti-GPU and multi-node serving, plus the profiling to prove any of it workedQualificationsBackgrounds that translate: HPC, graphics, embedded, compiler work, or performance engineering somewhere latency was a product requirement. Several of the strongest people in this space had never touched ML until it became a systems problem.Required SkillsFour engineers on the inference team today, going to eight this year.Pay range and compensation package$180,000 – $280,000 + equityEqual Opportunity StatementWe are committed to diversity and inclusivity.","company":"Digital Waffle","rawCompany":"digital waffle","city":"Menlo Park","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-08-27T09:02:04.190Z","occupations":[{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1299.00","title":"Computer Occupations, All Other","slug":"computer-occupations-all-other"}],"industries":[{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Inference Software Engineer","description":"About the CompanySeries A model-serving startup, ~35 people, $40M raised · San Francisco · Hybrid, 3 days · $180,000 – $280,000 + equity. They serve open-weight models for companies who can't send data to a third-party API. Their customers care about two numbers: p99 latency and cost per million tokens. This role owns both.About the RoleThe team runs a fork of vLLM across an H100 fleet. The last engineer to join cut p99 by 40% on the largest workload by reworking the scheduler. That’s the standard.ResponsibilitiesKernel-level optimisation — CUDA and Triton, custom ops, fused attentionContinuous batching, KV-cache strategy and speculative decoding, tuned per workload rather than globallyQuantisation trade-offs where the customer notices quality loss before you doMulti-GPU and multi-node serving, plus the profiling to prove any of it workedQualificationsBackgrounds that translate: HPC, graphics, embedded, compiler work, or performance engineering somewhere latency was a product requirement. Several of the strongest people in this space had never touched ML until it became a systems problem.Required SkillsFour engineers on the inference team today, going to eight this year.Pay range and compensation package$180,000 – $280,000 + equityEqual Opportunity StatementWe are committed to diversity and inclusivity.","datePosted":"2026-08-27T09:02:04.190Z","dateModified":"2026-08-27T09:02:04.190Z","hiringOrganization":{"@type":"Organization","name":"Digital Waffle","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Menlo Park","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"1b9e574ba9929fa19e3d06f6"},"url":"https://jobsearcher.com/jobs/1b9e574ba9929fa19e3d06f6"}}