Senior Software Development Engineer - LLM Inference Framework

What You Do At AMD Changes EverythingAt AMD, our mission is to build great products that accelerate next-generation computing experiences—from AI and data centers, to PCs, gaming and embedded systems. Grounded in a culture of innovation and collaboration, we believe real progress comes from bold ideas, human ingenuity and a shared passion to create something extraordinary. When you join AMD, you'll discover the real differentiator is our culture. We push the limits of innovation to solve the world's most important challenges—striving for execution excellence, while being direct, humble, collaborative, and inclusive of diverse perspectives. Join us as we shape the future of AI and beyond.Together, we advance your career.The RoleAs a senior member of the LLM inference framework team, you will be responsible for building and optimizing production-grade single-node and distributed inference runtimes for large language models on AMD GPUs. You will work at the framework and runtime layer, driving performance, scalability, and reliability, enabling tensor parallelism, pipeline parallelism, expert parallelism (MoE), and single-node or multi-node inference at scale. Your work will directly power customer-facing deployments and benchmarking platforms (e.g., InferenceMax, MLPerf, strategic partners, and cloud providers) and will be upstreamed into open-source inference frameworks such as vLLM and SGLang to make AMD a first-class platform for LLM serving.This role sits at the intersection of inference engines, distributed systems, and GPU runtime and kernel backends.The PersonYou are a systems-minded ML engineer who thinks in terms of throughput, latency, memory movement, and scheduling, not just model code. You are comfortable reading and modifying large-scale inference frameworks, debugging performance across GPUs and nodes, and collaborating with kernel, compiler, and networking teams to close end-to-end performance gaps. You enjoy working in open source and driving architecture-level improvements in inference platforms.Key ResponsibilitiesInference Framework & RuntimeArchitect and optimize distributed LLM inference runtimes based on in-house LLM engines or open-source stacks such as vLLM, SGLang, and llm-dDesign and improve TP / PP / EP (MoE) hybrid execution, including KV-cache management, attention dispatch, and token schedulingImplement and optimize multi-node inference pipelines using RCCL, RDMA, and collective-based executionPerformance & ScalabilityDrive throughput, latency, and memory efficiency across single-GPU and multi-GPU clustersOptimize continuous batching, speculative decoding, KV-cache paging, prefix caching, and multi-turn servingGPU & Backend IntegrationWork with AMD GPU libraries (AITER, HIPBLAS-LT, RCCL, ROCm runtime) to ensure inference frameworks efficiently use FP8 / FP4 GEMM and FlashAttention / MLACollaborate with compiler teams (Triton, LLVM, ROCm) to unblock framework-level performanceOpen Source & Customer EnablementUpstream features and performance fixes into vLLM, SGLang, and llm-dEnable customer PoCs and production deployments on AMD platformsBuild and maintain benchmark-grade inference pipelinesPreferred ExperienceInference Stack KnowledgeHands-on understanding of vLLM, SGLang, or similar inference stacksExperience with distributed inference scaling and a proven track record of contributing to upstream open-source projectsDeep Learning IntegrationStrong experience integrating optimized GPU performance into machine-learning frameworks (e.g., PyTorch, TensorFlow) for high-throughput and scalable inferenceKernel & Inference FrameworksStrong background in NVIDIA, AMD, or similar GPU architectures and kernel developmentSoftware EngineeringExpertise in Python and preferably experience in C/C++, including debugging, performance tuning, and test design for large-scale systemsHigh-Performance ComputingExperience running large-scale workloads on heterogeneous GPU clusters, optimizing for efficiency and scalabilityCompiler & Runtime OptimizationUnderstanding of compiler and runtime systems, including LLVM, ROCm, and GPU code generationAcademic CredentialsMaster's or PhD in Computer Science, Computer Engineering, Electrical Engineering, or a related field.