Machine learning engineer II, AWS Inference Optimization

What You’ll Do: Develop model servers. Build CUDA kernels in vLLM and SGLang. Create services for GenAI optimization and benchmarking. Create or modify a mid-size application or service. Optimize large language OSS models such as Qwen, Gemma, gptoss, and others by improving inference stacks including vLLM and SGLang. Perform performance tuning for GenAI inference onAmazon SageMaker , Amazon Bedrock, andHyperpod. Implement inference optimization techniques includingSpeculative decoding , Disaggregated Inference, andCUDA kernel tuning. Own price-performance for Amazon LLM inference deployed on SageMaker, Hyperpod, and Bedrock. Deliver production solutions that leverage existing academic and industrial research or apply practical, original thinking. Deliver solutions intocustomer-facing products. Key TechnologiesvLLM, SGLang Amazon SageMaker, Amazon Bedrock Hyperpod CUDA C#, C++, Java, Perl Transformer architecture Requirements3+ yearsof non-internship professional software development experience. 2+ yearsof non-internship design or architecture experience, including design patterns and reliability and scaling of new and existing systems. 1+ yearof software development engineer or related occupational experience. 1+ yeardesigning and developing large-scale, multi-tiered, multi-threaded, embedded or distributed software applications, tools, systems, and services using C#, C++, Java, or Perl. Bachelor’s degree (or foreign equivalent) in Computer Science, Engineering, Mathematics, or a related field. Knowledge of machine learning and LLM fundamentals, including transformer architecture, training/inference lifecycles, and optimization techniques. Preferred Qualifications3+ yearsof full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations. Bachelor’s degree in computer science or equivalent. EducationBachelor’s degree (Computer Science, Engineering, Mathematics, or related field). Location and Work ModeSanta Clara, CA(onsite) CompensationUSD 165,200 - 223,600per year. BenefitsSign-on payments Restricted stock units (RSUs) Health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance, option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage) 401(k) matching Paid time off Parental leave Workplace accommodation or adjustment during the application and hiring process (including support for the interview or onboarding process) Los Angeles County ApplicantsJob duties for this position include: Work safely and cooperatively with other employees, supervisors, and staff. Adhere to standards of excellence despite stressful conditions. Communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service. Follow all federal, state, and local laws and Company policies. Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position, including the abilities and responsibilities listed above, as well as the abilities to adhere to Company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company’s reputation. Pursuant to the Los Angeles County Fair Chance Ordinance, qualified applicants with arrest and conviction records will be considered for employment.