Senior Software Development Engineer, AI/ML, AWS Neuron, Model Inference
ARCHIVED
We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.
Overview:
In this role you will help enable and optimize distributed inference for PyTorch within the Neuron SDK on AWS accelerators (Trainium and Inferentia). You will work across software and hardware teams to push performance for large models, including LLMS, while shaping the Neuron inference stack. You’ll design high-performance kernels, implement optimizations, and collaborate with customers to maximize model efficiency at scale. This is a unique chance to contribute to AI acceleration at the intersection of ML, HPC, and distributed systems.
Compensation / BenefitsHealth insurance (medical, dental, vision)401(k) matchingPaid time offParental leaveRSUsSign-on payments
Responsibilities:
Design, develop, and optimize ML models and frameworks for deployment on custom ML acceleratorsLead distributed inference support for PyTorch in the Neuron SDK and tune models for peak performance on Trainium and InferentiaCollaborate across compiler, runtime, framework, and hardware teams to optimize ML workloadsBuild infrastructure to onboard diverse models and architecturesImplement high-performance kernels and features for ML ops leveraging Neuron architectureAnalyze system-level performance across multiple generations of Neuron hardwarePerform profiling to identify bottlenecks and apply optimizations (fusion, sharding, tiling, scheduling) Conduct comprehensive testing (unit, end-to-end) with CI/CD pipelinesWork with customers to enable and optimize ML models on AWS acceleratorsEngage with open source ecosystems to ensure seamless integration and peak performance
Key requirements:
Bachelor's degree in computer science or equivalent 5+ years of professional software development experience 5+ years in design or architecture of large-scale systemsFundamentals of ML and LLMs, with experience optimizing model executionSoftware development in C++ and PythonStrong understanding of system performance, memory management, and parallel computingCollaboration across cross-functional teamsMentorship and technical leadershipStrong debugging and analytical thinkingFamiliarity with PyTorch, JIT compilation, and AOT tracingCUDA kernels or low-level ML kernelsExperience with kernel development (e.g., CUTLASS, FlashInfer)