Senior Deep Learning Software Engineer, Inference and Model Optimization
Overview:
In this role you will advance automated inference and deployment for generative AI models, shaping the optimization stack that powers NVIDIA's TRT Model Optimizer and related platforms. You'll work across teams to implement high-performance kernels and scalable software for LLMs and diffusion models, leveraging PyTorch and the Torch 2.0 ecosystem. The role emphasizes research-informed engineering to push inference efficiency at scale and improve user experience with broad model support. Compensation / Benefitsequitycomprehensive benefits packagecompetitive salaryremote work optionscareer growth opportunitiesinclusive work environment
Responsibilities:
- Train, develop, and deploy state-of-the-art generative AI models (LLMs, diffusion) using NVIDIA's AI stack
- Leverage Torch 2.0 ecosystem to extract standardized model graph representations for automated deployment
- Develop high-performance inference optimizations (sharding, tensor/sequence parallelism, efficient attention with KV caching)
- Collaborate with cross-functional teams to implement performant kernel implementations
- Analyze and profile GPU kernel performance to identify optimization opportunities
- Drive ongoing innovation to maintain leadership in NVIDIA inference software (TRT, TRT-LLM, TRT Model Optimizer)
- Architect and design a modular, scalable platform enabling broad model support and optimizations
- Contribute to a fast-paced environment with a strong focus on software architecture and engineering
Key requirements:
- Masters, PhD, or equivalent experience in Computer Science, AI, Applied Math, or related field 8+ years of relevant deep learning work or research experience
- Excellent software design, debugging, performance analysis, and test design skills
- Strong proficiency in Python, PyTorch, and ML tools (e.g. HuggingFace)
- Strong algorithms and programming fundamentals
- Good written and verbal communication and ability to work independently and collaboratively in a fast-paced environmentstrong problem-solving skillseffective communicationindependence and collaboration in a fast-paced setting
PythonPyTorchHuggingFace