{"schemaVersion":"jobsearcher.job.v1","id":"23fa793d1a0997b5ca29d71b","url":"https://jobsearcher.com/jobs/23fa793d1a0997b5ca29d71b","canonicalUrl":"https://jobsearcher.com/jobs/23fa793d1a0997b5ca29d71b","title":"Machine Learning System Engineer","description":"Machine Learning System EngineerAs a ML System Engineer on the AI & ML Platform's Inference team, you will design and optimize large-scale model serving systems end-to-end. You will have the chance to own everything from distributed infrastructure (global KV cache, continuous batching, load balancing, auto-scaling) to deep low-level optimizations (GPU kernels, quantization, speculative decoding).In this role, you are expected to:Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).Optimize latency and throughput of model inference under real production workloads.Build reliable, high-concurrency serving systems that serve billions of requests reliablyBenchmark, fine-tune, and accelerate inference engines.Create robust CI/CD infrastructure for seamless model deployment and inference engine updates.Partner with senior ML engineers to fine-tune and deploy open-source LLMsOn your first day, we'll expect you to have:3+ years of software engineering experienceDeep low-level systems programming (C/C++ or Rust)Experience with large-scale, high-concurrent production serving.Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).It would be great, but not required if you have:1+ years of system performance optimization experienceLow-level inference optimizations: GPU kernelsAlgorithmic inference optimizations: quantization, speculative decoding, distillationExperience with testing, benchmarking, and reliability of inference services.Experience designing and implementing CI/CD infrastructure for inference.Strong background in system optimizations: batching, caching, load balancing, parallelism.","company":"Trello","rawCompany":"trello","city":"Denver","state":"CO","isRemote":false,"isActive":false,"createdAt":"2026-10-16T14:14:13.881Z","occupations":[{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1211.00","title":"Computer Systems Analysts","slug":"computer-systems-analysts"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Machine Learning System Engineer","description":"Machine Learning System EngineerAs a ML System Engineer on the AI & ML Platform's Inference team, you will design and optimize large-scale model serving systems end-to-end. You will have the chance to own everything from distributed infrastructure (global KV cache, continuous batching, load balancing, auto-scaling) to deep low-level optimizations (GPU kernels, quantization, speculative decoding).In this role, you are expected to:Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).Optimize latency and throughput of model inference under real production workloads.Build reliable, high-concurrency serving systems that serve billions of requests reliablyBenchmark, fine-tune, and accelerate inference engines.Create robust CI/CD infrastructure for seamless model deployment and inference engine updates.Partner with senior ML engineers to fine-tune and deploy open-source LLMsOn your first day, we'll expect you to have:3+ years of software engineering experienceDeep low-level systems programming (C/C++ or Rust)Experience with large-scale, high-concurrent production serving.Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.).It would be great, but not required if you have:1+ years of system performance optimization experienceLow-level inference optimizations: GPU kernelsAlgorithmic inference optimizations: quantization, speculative decoding, distillationExperience with testing, benchmarking, and reliability of inference services.Experience designing and implementing CI/CD infrastructure for inference.Strong background in system optimizations: batching, caching, load balancing, parallelism.","datePosted":"2026-10-16T14:14:13.881Z","dateModified":"2026-10-16T14:14:13.881Z","hiringOrganization":{"@type":"Organization","name":"Trello","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Denver","addressRegion":"CO","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"23fa793d1a0997b5ca29d71b"},"url":"https://jobsearcher.com/jobs/23fa793d1a0997b5ca29d71b"}}