{"schemaVersion":"jobsearcher.job.v1","id":"a4c8bf42432841ffff29b461","url":"https://jobsearcher.com/jobs/a4c8bf42432841ffff29b461","canonicalUrl":"https://jobsearcher.com/jobs/a4c8bf42432841ffff29b461","title":"Senior Inference Engineer","description":"As a Senior Inference Engineer at vCluster Labs, you are the first engineer we're hiring to own inference . You'll partner directly with our CTO to build the platform's inference layer from the ground up, taking models and turning them into a production-grade, query-to-response pipeline running at scale. From there, you will help lead the engineering direction of inference at vCluster, partnering with Product to shape what we build next as the space evolves.\n\nAs a Senior Inference Engineer, your role will include:\nDeploying models to production: Take LLMs and put them into production across one or more machines on GPU infrastructure, owning the full pipeline from a customer's query to the served response.\nServing frameworks: Stand up and operate serving infrastructure using vLLM, SGLang, or TensorRT-LLM.\nOptimizing for scale: Apply quantization, batching, caching, and routing to keep latency and cost in check as traffic grows.\nProgramming, not just configuring: Build real infrastructure in Python or Golang — this is an engineering role, not a research or data-science one.\nOwning the roadmap: Build the first iteration alongside our CTO, then take the lead on the inference platform and partner with Product to decide what we build next.\nThis role could be a fit for you if you bring:\nProduction LLM serving experience: You've deployed and served LLMs using vLLM, SGLang, or TensorRT-LLM, ideally at a company built around inference at scale.\nInference optimization know-how: Hands-on experience with quantization, batching, caching, and routing, not just familiarity with the terms.\nHands-on programming experience: Strong engineering skills in Python or Golang, with real production code experience.\nCommunication: Strong communication skills, explaining technical concepts clearly to both engineers and non-technical stakeholders.\nFamiliarity with containerized environments (Docker, Kubernetes)\nHands-on generative AI experience with common ML frameworks (PyTorch, Transformers)\nGood understanding of the GPU stack: CUDA, NCCL, drivers, and related libraries\nKnowledge of model architectures and fine-tuning approaches\nExperience with NVIDIA Dynamo\nAbout vCluster Labs We’re the #1 platform for AI infrastructure, trusted by the world’s fastest-growing AI cloud builders. We’re a venture-backed startup that’s raised over $28M from top-tier investors including Khosla Ventures (first investor in OpenAI, GitLab, Stripe, and DoorDash), and we’re in a hyper-growth phase looking for motivated people to join our team. Our headquarters are in San Francisco (Salesforce Tower), but our team is distributed around the globe with a remote-first culture.\nWe give AI Cloud providers and AI factories a hyperscaler-like experience on their own GPU infrastructure. Our platform runs the full stack an operator needs, from bare metal provisioning and node lifecycle management up through managed Kubernetes, Slurm, Ray, and inference clusters, so they can turn raw GPUs into cluster products they can sell in days instead of spending 12+ months building it themselves. Today we power over 100,000 GPUs and 1 million CPUs across 50+ AI clouds and Fortune 500 companies, backed by a team of 40+ infrastructure engineers who build alongside our customers rather than just shipping them software.\nWe’re the company behind vCluster, the open source technology for tenant isolation on Kubernetes, with 11,000+ GitHub stars and 40M+ tenant clusters created since 2021. Open source is part of our DNA. At KubeCon North America 2025, we launched our Infrastructure Tenancy Platform for AI, a Kubernetes-native framework built for running AI, ML, and GPU-intensive workloads anywhere, with an NVIDIA-validated reference architecture for DGX systems.\nWe offer the following benefits:\nCompetitive Salary : We offer a competitive compensation package, including equity.\nPremium Insurance : Health, dental, vision, and life Insurance, including plans for you and eligible dependents (benefits vary depending on country).\nFlexible Working Schedule: You have a doctor’s appointment or need to head to the supermarket to get groceries at 2pm? We won’t have an issue with that. To us, results matter more than clocking in and out at the same time every day.\nWorkplace Flexibility: We’re very flexible about where you work. We know things can change in life and we’re happy to adjust the work environment for you along the way.\nCulture & Values At vCluster Labs, we value and stand for:\nMake it Happen: We have a relentless bias for action and the grit to push through obstacles. We do whatever it takes to figure it out, put in the work, and ruthlessly prioritize the actions that drive measurable impact for the business.\nOwn the Outcome: We understand that our responsibility doesn't end when a task is checked off; it ends when the value is delivered. We connect our daily individual actions to the broader success of the company and our customers.\nCreate Wow: We measure success by the experience we generate, both inside and outside the company. For our customers, this means impressive speed and intuitive experiences. For our team, this means going the extra mile to support one another and to continuously drive each other to new heights.\nOpen Source, Open Mind: We are actively contributing to and maintaining open-source projects. Internally, we foster meritocracy — the strongest ideas win, no matter who or where they come from.\nBuild Tomorrow’s Standards, Intentionally: We don't just ship software; we define the state-of-the-art of tomorrow. We are fearless in tearing down old approaches to build something better, but we are disciplined in how we do it because we know our users rely on our technology to run mission-critical infrastructure platforms.\nCompensation Range:\nUnited States: $190K – $220K Offers Equity Offers Bonus\nGermany: €75K – €145K Offers Equity Offers Bonus\nGot a referral?\nIntroduce us to someone in your network who would be a great fit for one of our open roles.\n\n#J-18808-Ljbffr","company":"Vcluster","rawCompany":"vcluster","city":"Millbrae","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-10-18T03:46:11.144Z","occupations":[{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1221.00","title":"Computer and Information Research Scientists","slug":"computer-and-information-research-scientists"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Senior Inference Engineer","description":"As a Senior Inference Engineer at vCluster Labs, you are the first engineer we're hiring to own inference . You'll partner directly with our CTO to build the platform's inference layer from the ground up, taking models and turning them into a production-grade, query-to-response pipeline running at scale. From there, you will help lead the engineering direction of inference at vCluster, partnering with Product to shape what we build next as the space evolves.\n\nAs a Senior Inference Engineer, your role will include:\nDeploying models to production: Take LLMs and put them into production across one or more machines on GPU infrastructure, owning the full pipeline from a customer's query to the served response.\nServing frameworks: Stand up and operate serving infrastructure using vLLM, SGLang, or TensorRT-LLM.\nOptimizing for scale: Apply quantization, batching, caching, and routing to keep latency and cost in check as traffic grows.\nProgramming, not just configuring: Build real infrastructure in Python or Golang — this is an engineering role, not a research or data-science one.\nOwning the roadmap: Build the first iteration alongside our CTO, then take the lead on the inference platform and partner with Product to decide what we build next.\nThis role could be a fit for you if you bring:\nProduction LLM serving experience: You've deployed and served LLMs using vLLM, SGLang, or TensorRT-LLM, ideally at a company built around inference at scale.\nInference optimization know-how: Hands-on experience with quantization, batching, caching, and routing, not just familiarity with the terms.\nHands-on programming experience: Strong engineering skills in Python or Golang, with real production code experience.\nCommunication: Strong communication skills, explaining technical concepts clearly to both engineers and non-technical stakeholders.\nFamiliarity with containerized environments (Docker, Kubernetes)\nHands-on generative AI experience with common ML frameworks (PyTorch, Transformers)\nGood understanding of the GPU stack: CUDA, NCCL, drivers, and related libraries\nKnowledge of model architectures and fine-tuning approaches\nExperience with NVIDIA Dynamo\nAbout vCluster Labs We’re the #1 platform for AI infrastructure, trusted by the world’s fastest-growing AI cloud builders. We’re a venture-backed startup that’s raised over $28M from top-tier investors including Khosla Ventures (first investor in OpenAI, GitLab, Stripe, and DoorDash), and we’re in a hyper-growth phase looking for motivated people to join our team. Our headquarters are in San Francisco (Salesforce Tower), but our team is distributed around the globe with a remote-first culture.\nWe give AI Cloud providers and AI factories a hyperscaler-like experience on their own GPU infrastructure. Our platform runs the full stack an operator needs, from bare metal provisioning and node lifecycle management up through managed Kubernetes, Slurm, Ray, and inference clusters, so they can turn raw GPUs into cluster products they can sell in days instead of spending 12+ months building it themselves. Today we power over 100,000 GPUs and 1 million CPUs across 50+ AI clouds and Fortune 500 companies, backed by a team of 40+ infrastructure engineers who build alongside our customers rather than just shipping them software.\nWe’re the company behind vCluster, the open source technology for tenant isolation on Kubernetes, with 11,000+ GitHub stars and 40M+ tenant clusters created since 2021. Open source is part of our DNA. At KubeCon North America 2025, we launched our Infrastructure Tenancy Platform for AI, a Kubernetes-native framework built for running AI, ML, and GPU-intensive workloads anywhere, with an NVIDIA-validated reference architecture for DGX systems.\nWe offer the following benefits:\nCompetitive Salary : We offer a competitive compensation package, including equity.\nPremium Insurance : Health, dental, vision, and life Insurance, including plans for you and eligible dependents (benefits vary depending on country).\nFlexible Working Schedule: You have a doctor’s appointment or need to head to the supermarket to get groceries at 2pm? We won’t have an issue with that. To us, results matter more than clocking in and out at the same time every day.\nWorkplace Flexibility: We’re very flexible about where you work. We know things can change in life and we’re happy to adjust the work environment for you along the way.\nCulture & Values At vCluster Labs, we value and stand for:\nMake it Happen: We have a relentless bias for action and the grit to push through obstacles. We do whatever it takes to figure it out, put in the work, and ruthlessly prioritize the actions that drive measurable impact for the business.\nOwn the Outcome: We understand that our responsibility doesn't end when a task is checked off; it ends when the value is delivered. We connect our daily individual actions to the broader success of the company and our customers.\nCreate Wow: We measure success by the experience we generate, both inside and outside the company. For our customers, this means impressive speed and intuitive experiences. For our team, this means going the extra mile to support one another and to continuously drive each other to new heights.\nOpen Source, Open Mind: We are actively contributing to and maintaining open-source projects. Internally, we foster meritocracy — the strongest ideas win, no matter who or where they come from.\nBuild Tomorrow’s Standards, Intentionally: We don't just ship software; we define the state-of-the-art of tomorrow. We are fearless in tearing down old approaches to build something better, but we are disciplined in how we do it because we know our users rely on our technology to run mission-critical infrastructure platforms.\nCompensation Range:\nUnited States: $190K – $220K Offers Equity Offers Bonus\nGermany: €75K – €145K Offers Equity Offers Bonus\nGot a referral?\nIntroduce us to someone in your network who would be a great fit for one of our open roles.\n\n#J-18808-Ljbffr","datePosted":"2026-10-18T03:46:11.144Z","dateModified":"2026-10-18T03:46:11.144Z","hiringOrganization":{"@type":"Organization","name":"Vcluster","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Millbrae","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"a4c8bf42432841ffff29b461"},"url":"https://jobsearcher.com/jobs/a4c8bf42432841ffff29b461"}}