Software Engineer, AI Performance Modeling & Co Design

What To ExpectThe AI co-design team is dedicated to developing and optimizing AI systems that can scale efficiently to thousands of compute nodes, enabling large-scale training, reinforcement learning at scale, and real-time inference for Autopilot and Optimus. Our goal is to push the boundaries of performance, power, and latency on our custom-designed AI-ASIC cluster, while delivering exceptional scalability and performance.This unique position sits at the intersection of AI research, performance modeling, and system design. You will work closely with AI engineers to optimize model architectures, leveraging network topologies to efficiently parallelize and shard models on our cluster. Additionally, you will collaborate with our hardware co-design team to model and optimize future generation performance, ensuring seamless integration of AI models and algorithms with next-generation hardware.What You'll DoDevelop and apply performance modeling techniques to analyze and optimize AI system performance, including dataflow, mapping, and sharding of AI modelsDesign and implement collective and multi-device parallel computing algorithms and architectures to accelerate AI workloadsDevelop and maintain network models and simulations to analyze and optimize AI system performance in distributed environmentsCollaborate with cross-functional teams to integrate performance modeling and optimization techniques into AI system design and development workflows Develop and maintain software tools and frameworks to support performance modeling, parallel computing, and networking development Work closely with the hardware co-design team to model and optimize future generation AI-ASIC architectures, ensuring seamless integration of AI models and algorithms with next-generation hardwareCo-design and optimize AI systems for future generations, considering emerging trends and technologies, such as 3D stacked processors, photonic interconnects, and novel memory architecturesDevelop and apply co-design methodologies to optimize AI system performance, power, and area, while ensuring scalability, reliability, and manufacturabilityWhat You'll BringBachelor's Degree in Computer Science, Computer Engineering, Electrical Engineering, proof of exceptional skills in related fields with practical software engineering experience, or equivalent experienceExperience in performance modeling, parallel computing, and networking, with a focus on AI systemsProficiency in Python and C++Experience with AI networks, such as CNNs, transformers, and diffusion model architectures, and their performance characteristicsExperience with parallel computing frameworks and libraries, such as MPI, NCCL, or GlooUnderstanding of GPU and TPU accelerators architecture, memory hierarchy, and ability to model performance and identify performance bottlenecksStrong understanding of computer networks, including network protocols, topology, and performance modelingExperience with SW/HW co-design and optimizationStrong analytical and problem-solving skills, with the ability to analyze complex systems and optimize their performanceBenefitsCompensation and BenefitsAlong with competitive pay, as a full-time Tesla employee, you are eligible for the following benefits at day 1 of hire: Medical plans > plan options with $0 payroll deduction Family-building, fertility, adoption and surrogacy benefits Dental (including orthodontic coverage) and vision plans, both have options with a $0 paycheck contribution Company Paid (Health Savings Accounts) HSA Contribution when enrolled in the High-Deductible medical plan with HSA Healthcare and Dependent Care Flexible Spending Accounts (FSA) 401(k) with employer match, Employee Stock Purchase Plans, and other financial benefits Company paid Basic Life, AD&D Short-term and long-term disability insurance (90 day waiting period) Employee Assistance Program Sick and Vacation time (Flex time for salary positions, Accrued hours for Hourly positions), and Paid Holidays Back-up childcare and parenting support resources Voluntary benefits to include: critical illness, hospital indemnity, accident insurance, theft & legal services, and pet insurance Weight Loss and Tobacco Cessation Programs Tesla Babies program Commuter benefits Employee discounts and perks programExpected Compensation$140,000 - $390,000/annual salary + cash and stock awards + benefitsPay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. The total compensation package for this position may also include other elements dependent on the position offered. Details of participation in these benefit plans will be provided if an employee receives an offer of employment., Tesla