JOBSEARCHER

Senior Inference Engineer

Overview: As a Staff Inference Engineer at Uniphore, you will own the inference layer for an enterprise AI platform that spans voice, video, text, and structured data. You will work across research, model engineering, knowledge systems, agent, and platform teams to deliver a context-aware runtime for finetuned large language models. Your work will optimize KV-cache, prompt handling, and multi-GPU execution while ensuring reliability and cross-environment portability. This role offers hands-on impact in building scalable, secure inference at enterprise scale and shaping how agents operate in real time. You will be at the forefront of delivering production-grade AI that integrates deeply with enterpriseCompensation / Benefitsannual incentive opportunitypre-IPO stock optionsmedical, dental, vision 401(k) with matchgenerous paid time offpaid holidays and birthday leaveResponsibilitiesDesign and implement a context-aware runtime that integrates context selection, compression, and persistent task state with model executionDevelop and optimize KV-cache and prompt-prefix-cache management, including routing, retention, and offloadingImprove inference performance with speculative decoding, efficient kernels, batching, quantization, and multi-GPU executionManage lifecycle of models and adapters, enabling zero-downtime hot swapping and resource schedulingEnsure production reliability with observability, tenant isolation, overload protection, and safe rollout/rollbackBuild portable runtime integrations across GPU platforms (NVIDIA, AMD), cloud and on-premises environments, and single/multi-GPU configsKey requirements 5+ years engineering experience with 3+ years in GPU systems, distributed inference, or ML servingProduction experience with vLLM, SGLang, or TensorRT-LLMStrong Python and C++ (C++17/20) with ability to read/debug CUDA C++ or HIP kernelsHands-on GPU profiling and performance optimizationCUDA, HIP, Triton, or CUTLASS proficiencyModel inference internals: autoregressive generation, prefills vs decoding, attention architecturesSpeculative decoding and adaptive throughput tradeoffsGPU kernel development (FlashAttention-style, tiling, fusion)KV/prefix caching, fragmentation, offloading, tenant isolationContext-window management and token budgeting