JOBSEARCHER

Software Engineer, GPU Inference

Overview: In this role, you will productionize and optimize Cerebras' GPU inference serving stack, spanning custom APIs, vLLM, ROCm, and rack-scale AMD GPU infrastructure. You'll drive reliability, performance, and observability across the full stack from application to hardware. The position focuses on reducing time to first token, improving throughput, and maximizing capacity efficiency in a disaggregated AI inference platform. You will work closely with cross-functional teams to transform a cutting-edge accelerator fleet into a robust production service. This is a hands-on role at the intersection of software, systems, and hardware with a meaningful impact on AI workloads at scale. Compensation / Benefitsbreakthrough AI platformopenness through open-source researchfastest AI supercomputerstartup-style vitalitynon-corporate culture
Responsibilities:
  • Productionize the GPU inference stack across APIs, model-serving workers, vLLM, PyTorch, ROCm, and GPU nodes
  • Establish deployment, upgrade, rollback, health checks, capacity management, and failure recovery for the AMD GPU fleet
  • Define service-level indicators and objectives; improve fault isolation, automated recovery, incident response, and post-incident remediation
  • Profile and optimize time to first token, throughput, tail latency, and memory and rack-level capacity under production workloads
  • Tune scheduling, continuous batching, KV-cache management, tensor/expert parallelism, and memory-aware admission control in model serving
  • Diagnose failures across application, vLLM, PyTorch, ROCm/HIP, kernels, drivers, firmware, networking and hardware
  • Build validation, regression, and benchmarking infrastructure for model quality, numerical accuracy and consistency across releases
  • Develop dashboards, tooling, and release qualification to turn investigations into repeatable engineering processes
Key requirements:
  • 5+ years of software engineering in complex production systems
  • Experience in production inference systems for large language models or similarly demanding GPU workloads
  • Strong C++ and Python, with multithreading and performance-sensitive software experience
  • Hands-on experience with high-performance model-serving frameworks (e.g., vLLM, SGLang, TensorRT-LLM, Triton)
  • Solid understanding of GPU execution, memory movement, synchronization, and profiling
  • Experience debugging distributed systems across multiple layers
  • Experience with Linux, containers, Kubernetes, observability, CI/CD, and latency-sensitive services
  • Ability to design benchmarks and translate results into production improvements
  • Strong communication and cross-functional leadership skills
  • Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, or related disciplinestrong communicationcross-functional collaborationproblem solving under pressureC++Pythonmultithreading and concurrency