JOBSEARCHER

Staff Software Engineer, ML Performance, GPU

GoogleSan Jose, CAL6 LeadOctober 26th, 2026
Overview: In this role you will optimize and scale ML training and serving on Google's GPU infrastructure. You'll partner with product teams to onboard and optimize LLMs, run architecture simulations, and drive performance across fleets. You will identify bottlenecks, implement efficiency techniques, and guide hardware-aware stack improvements. This is a hands-on position focused on delivering high-performance ML workloads at scale with a strong emphasis on collaboration and impact. Compensation / Benefitsbonus targetequitybenefitscompetitive compensationcareer growth opportunitiesflexible work arrangements
Responsibilities:
  • Identify performance opportunities for LLM training and serving and drive XLA:GPU/Triton optimizations
  • Onboard, optimize, and scale LLMs and ML models on GPU hardware with product teams
  • Conduct architecture-level simulations, benchmarking, and roofline analyses to inform system design
  • Analyze fleet-wide performance and efficiency, pinpoint bottlenecks, and engineer scalable optimizations
  • Research and implement model/data efficiency techniques, tooling, and profiling mechanisms to improve workload performance and training efficiency
Key requirements:
  • Bachelor's degree or equivalent practical experience 8 years of software development experience 5 years in ML design and ML infrastructure (deployment, evaluation, data processing, debugging, fine tuning)
  • Experience with modern GPU architectures, memory hierarchies, and performance bottlenecks
  • Experience with low-level GPU programming (CUDA, Triton, CUTLASS) and performance engineering
  • Experience with modern LLMs and deployment on AI acceleratorscross-functional collaborationleadership in project teamsability to work in a matrixed organizationCUDATritonCUTLASS