Engineering Manager, Deep Learning Inference
Overview:
As Manager, Deep Learning Inference Software, you lead a world-class team advancing AI model deployment on NVIDIA GPUs. You shape and execute the inference software strategy, partnering across compiler, libraries, and research groups to optimize end-to-end pipelines. You drive performance tuning for large-scale models and guide adoption of CUDA, Triton, CUTLASS, and multi-GPU techniques. This role blends technical leadership with strategic roadmap responsibilities, impacting real-time inference at datacenter and edge scales.
Compensation / Benefitsequitycomprehensive benefits packagebase salary and variable compensationhybrid/remote optionscareer advancement opportunities
Responsibilities:
Lead and mentor a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
Set strategy, roadmap, and delivery for NVIDIA’s inference frameworks engineering, with emphasis on Client AI.Collaborate with internal compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines across NVIDIA accelerators.
Oversee performance tuning, profiling, and optimization for large-scale LLM, multimodal, and generative AI workloads.
Guide engineers on CUDA, Triton, CUTLASS, and multi-GPU communications (NIXL, NCCL, NVSHMEM).Represent the team in roadmap and planning discussions to align with broader AI and software strategies.
Foster a culture of technical excellence, collaboration, and continuous innovation.
Key requirements:
6+ years of software development experience; 3+ years in technical leadership or engineering management.
Strong background in C/C++ software design; Python is a plus.
Hands-on GPU programming experience (CUDA, Triton, CUTLASS) and performance optimization.
Proven record of deploying or optimizing deep learning models in production environments.
Experience leading teams using Agile or collaborative software development practices.leadership and mentoringcross-functional collaborationstrategic thinkingCUDATritonCUTLASS