Machine Learning Inference Engineer

🚨 Machine Learning Inference Engineer📍 San Francisco (On-Site)I’m working with an early-stage infrastructure company building a next-generation AI cloud — rethinking how frontier models run across heterogeneous compute environments. This team is tackling one of the hardest problems in AI infrastructure: making large-scale inference fast, reliable, and production-ready.💡 What makes this role interesting:• Work directly on LLM inference internals• Optimise latency, throughput, and memory at scale• Build serving systems across GPUs + next-gen accelerators• Hands-on with vLLM, TensorRT-LLM, and custom runtimes• Shape core infrastructure at an early-stage company🧠 You’ll:• Build end-to-end inference systems from request → runtime → response• Design batching, scheduling, and queuing systems• Improve KV cache management + memory efficiency• Debug performance bottlenecks across model, runtime, and hardware layers• Partner closely with ML, infra, and systems teams🔧 Looking for:• Experience building ML inference or model serving systems• Strong systems engineering or backend infrastructure fundamentals• Experience with scaling, distributed systems, or performance optimisation• Strong Python and/or C++ skills• Familiarity with modern inference frameworks is a plus