JOBSEARCHER

Machine Learning System Engineer

TrelloDenver, COL6 LeadOctober 16th, 2026
Machine Learning System Engineer As a ML System Engineer on the AI & ML Platform's Inference team, you will design and optimize large-scale model serving systems end-to-end. You will have the chance to own everything from distributed infrastructure (global KV cache, continuous batching, load balancing, auto-scaling) to deep low-level optimizations (GPU kernels, quantization, speculative decoding).In this role, you are expected to: Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).Optimize latency and throughput of model inference under real production workloads. Build reliable, high-concurrency serving systems that serve billions of requests reliably Benchmark, fine-tune, and accelerate inference engines. Create robust CI/CD infrastructure for seamless model deployment and inference engine updates. Partner with senior ML engineers to fine-tune and deploy open-source LLMsOn your first day, we'll expect you to have:3+ years of software engineering experience Deep low-level systems programming (C/C++ or Rust) Experience with large-scale, high-concurrent production serving. Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).It would be great, but not required if you have:1+ years of system performance optimization experience Low-level inference optimizations: GPU kernels Algorithmic inference optimizations: quantization, speculative decoding, distillation Experience with testing, benchmarking, and reliability of inference services. Experience designing and implementing CI/CD infrastructure for inference. Strong background in system optimizations: batching, caching, load balancing, parallelism.