Software Engineer, GPU Performance
Overview:
In this role you will optimize GPU-based AI workloads at HeyGen to reduce latency and cost, collaborating with researchers and infrastructure engineers. You will profile with NVIDIA Nsight tools and PyTorch Profiler to identify bottlenecks, and implement high-performance kernels, batching strategies, and memory management. You will help bring optimizations into production while measuring latency, throughput, and output quality. This is a chance to work on cutting-edge AI video platforms and contribute to scalable, efficient model serving at scale. Compensation / Benefitscompetitive salary and benefits packagedynamic and inclusive work environmentprofessional growth and advancementcollaborative culture that values innovation and creativityaccess to the latest technologies and tools
Responsibilities:
- Investigate GPU utilization, kernel execution, memory bandwidth, and CPU–GPU data movement using NvTools and profilers
- Identify bottlenecks across model execution, preprocessing, and inference serving and measure impact of optimizations
- Improve performance through batching, scheduling, memory management, and better GPU utilization
- Develop or integrate high-performance GPU kernels when existing implementations limit performance
- Build benchmarks and automated checks to catch performance regressions across video workloads
- Collaborate with AI researchers and infrastructure engineers to deploy optimizations into production
- Measure effects of changes on latency, throughput, cost, and output quality
Key requirements:
- Experience optimizing GPU-based AI workloads or HPC systems
- Proficiency in Python and PyTorch or similar ML framework
- Strong curiosity about GPU hardware (memory bandwidth, cache behavior, tensor cores, CPU-GPU data movement)
- Experience using profiling tools to connect hardware behavior to application bottlenecks and validate improvements
- Ability to translate performance experiments into reliable production changes and communicate tradeoffs clearlycuriositycollaborationcommunication of complex tradeoffsCUDATritonC++ GPU programming